www.lesswrong.com/posts/ozbbzsh49wvgfbCDp/twitch-plays-ai-interpretability-of-bo...
1 correction found
GitHub independently re-scores every submitted result before merging.
The project's own merge-procedure docs say CI re-scores only up to two new results per pull request, not every submitted result.
Full reasoning
The FAQ overstates how much GitHub CI re-checks before merge. In the repository's own merge instructions, the checks workflow is described as re-scoring "up to two of each PR's new results" on a CPU runner, not all of them.
That directly conflicts with the post's claim that GitHub "re-scores every submitted result before merging." The same repo documentation also talks about PRs adding multiple contributions/results, so the distinction matters: some submitted results can merge without being independently re-scored by CI, with further validation instead coming from post-merge QC and blind-room sampling.
So the accurate version is closer to: GitHub runs automatic checks and re-scores a limited sample of new results per PR before merge, while additional verification happens through QC and replication later.
2 sources
- moridinamael/bonsai-interp — MERGE_AGENT.md
The checks below are mechanical, and `checks` also re-scores up to two of each PR's new results on a CPU runner (`bonsai_interp.graph.replicate`)...
- moridinamael/bonsai-interp — CONTRIBUTING.md
Pick poles (`play draw --k 16` for a sweep; no claims). For each, write the label... Open a PR containing only files under `graph/contributions/`.