en.wikipedia.org/wiki/Large_language_model
2 corrections found
thanks in part due to the RLHF technique.
This is incorrect: GPT-3’s original few-shot prompting results did not use reinforcement learning from human feedback. RLHF was introduced later for instruction-following models like InstructGPT.
Full reasoning
The sentence attributes GPT-3's 2020 few-shot prompting ability to RLHF, but the original GPT-3 paper says the model was evaluated in the few-shot setting "without any gradient updates or fine-tuning" and with tasks specified "purely via text interaction with the model." That is standard prompting/in-context learning, not RLHF.
OpenAI's later InstructGPT work is where RLHF is explicitly introduced for instruction-following. OpenAI states that the GPT-3 API models can be prompted with carefully engineered text prompts, and then says InstructGPT is trained with reinforcement learning from human feedback. So RLHF belongs to the later alignment/fine-tuning stage, not to GPT-3's original few-shot prompting demonstrations.
2 sources
- Language Models are Few-Shot Learners
For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model.
- Aligning language models to follow instructions | OpenAI
The OpenAI API is powered by GPT-3 language models which can be coaxed to perform natural language tasks using carefully engineered text prompts... To train InstructGPT models, our core technique is reinforcement learning from human feedback (RLHF).
On the International Mathematics Olympiad qualifying exam problems, GPT-4o achieved 13% accuracy while o1 reached 83%.
This benchmark is misidentified. OpenAI reported those scores on the 2024 AIME exams, not on International Mathematical Olympiad qualifying exam problems.
Full reasoning
The stated 13% vs. 83% result is real, but the article labels the benchmark incorrectly.
OpenAI's own announcement for o1 says the comparison was on AIME: "On the 2024 AIME exams, GPT‑4o only solved on average 12% (1.8/15) of problems. o1 averaged ... 83% (12.5/15)". OpenAI separately describes AIME as a qualifier for the USA Math Olympiad (USAMO), not the International Mathematical Olympiad (IMO).
The Mathematical Association of America likewise describes AIME as the American Invitational Mathematics Examination and says top AIME scorers are invited to the USAMO or USAJMO. So the article's phrase "International Mathematics Olympiad qualifying exam problems" misstates what exam those scores came from.
2 sources
- Learning to reason with LLMs | OpenAI
We evaluated math performance on AIME... On the 2024 AIME exams, GPT-4o only solved on average 12% (1.8/15) of problems. o1 averaged 74% ... 83% (12.5/15)... A score of 13.9 places it among the top 500 students nationally and above the cutoff for the USA Mathematical Olympiad.
- MAA Invitational Competitions – Mathematical Association of America
The first in this series is the American Invitational Mathematics Exam (AIME), followed by the USA Mathematical Olympiad and Junior Mathematical Olympiad (USAMO and USAJMO)... Top-scoring participants on the AIME are invited to take the USAMO or USAJMO.