x.com/CRSegerie/status/2105534432455340486/photo/1
1 correction found
scoring 100% on Exploit Gym
OpenAI’s GPT‑6 Astra launch did not report a 100% score on ExploitGym. It reported 42.4% on ExploitGym; the 100% score was on a different benchmark, ExploitBench.
Full reasoning
This appears to conflate two different cybersecurity benchmarks from OpenAI’s GPT‑6 Astra launch materials.
In OpenAI’s launch post for GPT‑6 Astra, the company says:
- ExploitBench: 100.0%
- ExploitGym: 42.4%
So the statement that the launch “bragged about scoring 100% on Exploit Gym” is not accurate. The 100% figure belongs to ExploitBench, not ExploitGym. The same page also lists a separate alignment metric called “Impossible ExploitGym” at 100.0%, which is another distinct measure and not the main ExploitGym benchmark.
Because the post specifically names Exploit Gym, readers would be misled about what OpenAI actually claimed in its launch materials.
1 source
- GPT-6 Astra: A new generation of intelligence | OpenAI
“On ExploitBench, Astra achieved a perfect score of 100%... On ExploitGym, Astra reached a 42.4% success rate...” The benchmark table likewise lists “ExploitBench 100.0%” and “ExploitGym 42.4%.”