www.lesswrong.com/posts/tEZa7PouYatK78bbb/i-am-worried-about-near-term-non-llm-a...
2 corrections found
By out of sequence, I mean that models receive random samples from the training set, instead of continuous sequences of tokens, characters, images, or audio samples.
For frontier language models, this is backwards: training usually uses sampled contiguous token sequences or chunks, not isolated random tokens. Batching/shuffling data does not mean the model is trained 'instead of' on continuous token sequences.
Full reasoning
This sentence misdescribes how modern language models are typically trained.
Frontier LLM pretraining usually samples contiguous token sequences / chunks from documents and trains on those sequences. The fact that batches are shuffled or sampled from many documents does not mean the model is trained instead of on continuous sequences. It means the model sees many different sequences during training.
Two primary-source examples:
- The GPT-3 paper states: "During training we always train on sequences of the full nctx = 2048 token context window", packing multiple documents into a sequence when needed.
- Meta's long-context LLM paper says its models are built through continual pretraining from LLaMA 2 with longer training sequences and a dataset where long texts are upsampled.
So the specific claim that models receive random samples "instead of continuous sequences of tokens" is incorrect for frontier language models. A more accurate statement would be that training usually samples many contiguous sequences from across the corpus, often in shuffled batches.
2 sources
- Language Models are Few-Shot Learners
During training we always train on sequences of the full nctx = 2048 token context window, packing multiple documents into a single sequence when documents are shorter than 2048...
- Effective Long-Context Scaling of Foundation Models | Research - AI at Meta
Our model series are built through continual pretraining from LLAMA 2 with longer training sequences and on a dataset where long texts are upsampled.
As a result, models cannot use past inputs to predict future inputs, only learn a general solution to the task they are being trained on.
Autoregressive language models are explicitly trained to use previous tokens to predict later ones. That is the core next-token-prediction objective, so saying they 'cannot use past inputs to predict future inputs' is incorrect.
Full reasoning
This is not how autoregressive language models work.
Modern LLMs are specifically trained on a next-token prediction objective: given previous tokens in a sequence, the model predicts the next token. That is exactly a method for using past inputs to predict future inputs.
Primary sources say this directly:
- OpenAI's GPT-3 announcement describes GPT-3 as an "autoregressive language model".
- The GPT-3 paper explains that training uses full token sequences, which are used for next-token prediction across the context window.
So while the author may be making a broader argument about offline training or limited persistent memory, the literal statement that models "cannot use past inputs to predict future inputs" is false for frontier language models.
2 sources
- Language models are few-shot learners | OpenAI
Specifically, we train GPT‑3, an autoregressive language model with 175 billion parameters...
- Language Models are Few-Shot Learners
During training we always train on sequences of the full nctx = 2048 token context window...