www.lesswrong.com/posts/PLisnSFir8y5AHkmP/how-concerned-should-we-be-about-astra...
1 correction found
Looped transformers have been studied in academia since 2023.
This timeline is too late: recurrent/looped Transformer architectures were already being studied by 2018, not only since 2023.
Full reasoning
The claim dates academic study of looped transformers to 2023, but the architecture family is substantially older.
The clearest counterexample is Universal Transformers (arXiv:1807.03819), first submitted on July 10, 2018. Its abstract explicitly describes the model as a "parallel-in-time self-attentive recurrent sequence model" and presents it as a generalization of the Transformer. That is the same core idea at issue here: reusing a Transformer-style computation recurrently across depth rather than just stacking a fixed set of distinct layers.
More recent looped-transformer work explicitly acknowledges this lineage. The 2025/2026 paper Scaling Latent Reasoning via Looped Language Models says prior work ranges "from the seminal Universal Transformer (2018)" to newer recursive / latent-reasoning approaches. So even within the modern looped-transformer literature, researchers themselves trace the line back to 2018.
That means the post's date is off by about five years. A more accurate phrasing would be that recent large-language-model work on looped/recurrent-depth transformers accelerated around 2023, not that academia only started studying them then.
2 sources
- Universal Transformers
arXiv:1807.03819 was submitted on 10 Jul 2018. Its abstract states: "We propose the Universal Transformer (UT), a parallel-in-time self-attentive recurrent sequence model which can be cast as a generalization of the Transformer model."
- Scaling Latent Reasoning via Looped Language Models
The paper's introduction says prior LoopLM studies range "from the seminal Universal Transformer (2018)" to later recursive and latent-reasoning approaches, explicitly tracing looped-transformer work back to 2018.