x.com/SkyeSharkie/status/2102742337004974307
2 corrections found
The training data for RL environments is not coming from "7 figure lab employees"
This is too categorical. Frontier labs publicly say internal researchers and engineers build RL environments, evaluation data, and other training inputs for major model runs.
Full reasoning
OpenAI and Anthropic both publicly describe internal staff building the kinds of environments and task data the post says are not coming from lab employees.
- OpenAI's Research Engineer, Frontier Evals & Environments role says that team members "build the data, environments, graders, training methods, and feedback loops that shape what OpenAI's next agents can do" and specifically "Create ambitious RL environments" for major model runs.
- OpenAI's FrontierScience evaluation says it consists of "60 original research subtasks designed by PhD scientists," which is the opposite of a claim that such task data categorically does not come from highly paid lab staff.
- Anthropic's engineering documentation says its evaluations were created on top of its internal workspace, and that some tool implementations used in those evaluations were "manually written by our researchers."
Those official statements do not prove that contractors are never used anywhere in the industry. But they do directly contradict the blanket claim that RL-environment training data is not coming from lab employees. At minimum, some of it clearly is.
3 sources
- Research Engineer, Frontier Evals & Environments | OpenAI
We build the data, environments, graders, training methods, and feedback loops that shape what OpenAI's next agents can do... In this role, you'll: Create ambitious RL environments to push our models to their limits.
- Evaluating AI’s ability to perform scientific research tasks | OpenAI
FrontierScience-Research consists of 60 original research subtasks designed by PhD scientists (doctoral candidates, professors, or postdoctoral researchers).
- Writing effective tools for AI agents—using AI agents | Anthropic
Our evaluations were created on top of our internal workspace, mirroring the complexity of our internal workflows... these tools were manually written by our researchers or generated by Claude itself.
this fictional belief that labs source their own RL env tasks internally
That belief is not fictional. Public documentation from major AI labs says they do build RL/evaluation environments and tasks internally.
Full reasoning
The post calls it a "fictional belief" that labs source RL-environment tasks internally, but multiple frontier labs publicly describe doing exactly that.
- OpenAI's Frontier Evals & Environments team says it helps "build north star model environments" for major training runs and lists "Create ambitious RL environments" among the role's responsibilities.
- Anthropic's evals documentation says that, through its internal work, it has learned how to design agent evals; for complex multi-turn evals, a coding agent receives "tools, a task ... and an environment." Anthropic separately says its evaluations were built on its internal workspace, mirroring real internal projects, documents, and messages.
- OpenAI's system-card materials also describe an Internal Research Debugging Eval based on "41 real bugs from internal research experiments at OpenAI," plus tasks involving "bad environments that we found in real research experiments," which is further evidence that at least some of these tasks/environments are developed internally.
So while contractors may also be used in some parts of the AI data pipeline, it is incorrect to describe internal sourcing of RL/eval tasks as a fantasy or fiction.
4 sources
- Research Engineer, Frontier Evals & Environments | OpenAI
As a researcher working on Frontier Evals & Environments, you will help build north star model environments... In this role, you'll: Create ambitious RL environments to push our models to their limits.
- Demystifying evals for AI agents | Anthropic
Through our internal work... we've learned how to design more rigorous and useful evals for agents. For a more complex multi-turn eval, a coding agent receives tools, a task... and an environment.
- Writing effective tools for AI agents—using AI agents | Anthropic
Our evaluations were created on top of our internal workspace, mirroring the complexity of our internal workflows, including real projects, documents, and messages.
- GPT-5.5 System Card - OpenAI Deployment Safety Hub
The Internal Research Debugging Eval measures whether AI models can debug 41 real bugs from internal research experiments at OpenAI... This evaluation also includes 6 alignment auditing-related tasks... bad environments that we found in real research experiments.