x.com/GonZalocla/status/2108151566784209028
1 correction found
they spent 3h on average per problem and tried 4000problems, that is in total 12000h of their new model.
OpenAI did not say it used 3 hours per problem. Its repo says 3 hours per result, while the model was posed about 4,000 problems, so the 12,000-hour calculation is based on a misquote.
Full reasoning
OpenAI’s own repository distinguishes results from problems.
In the README for openai/math, OpenAI writes:
- "On average, each result used three hours of ChatGPT Pro thinking compute with that model."
- "Over the course of the evaluation, the model was posed approximately 4,000 problems."
- It then says those outputs were aggregated into result families and manuscripts.
So the post changes OpenAI’s wording from per result to per problem and then multiplies 4,000 × 3 hours to get 12,000 hours.
That arithmetic is not supported by the source, because OpenAI did not say every posed problem consumed an average of 3 hours. It said the average was measured per result, a different unit. The same README also says the public catalogue contains 719 manuscripts organized into 372 families, showing that OpenAI is not equating the 4,000 attempted problems with final released results.
Because the post substitutes the wrong unit, its 12,000-hour total is a misstatement of what OpenAI reported.
1 source
- GitHub - openai/math · GitHub
How the results were produced: "On average, each result used three hours of ChatGPT Pro thinking compute with that model. Over the course of the evaluation, the model was posed approximately 4,000 problems. Aggregating the output into result families and manuscripts..." The README also says: "The current catalogue contains 719 manuscripts organized into 372 families."