www.lesswrong.com/posts/SvxaKP5KdkksZPcG7/predicting-when-rl-training-breaks-cha...

The paper itself reports an exception to this blanket claim. In Section 6.2, the authors write that they "observe semantic drift in ICRL" and that models "slowly drop in monitorability when training against some orthogonal rewards with certain prompts". Appendix I.7 gives a concrete example: with the supposedly orthogonal "punish string" reward, the model reaches perfect reward immediately, but if optimization continues with an "unrestricted" prompt, monitorability still drops and the reasoning later becomes "entirely unmonitorable". The authors explicitly say this "seemingly contradict[s] our prediction" and note that the main Figure 4 result uses a different, "capped" prompt to avoid that drift.

So the broad sentence in the post — that training against orthogonal or aligned rewards kept monitorability high — is not strictly correct for all of the paper's reported experiments. A more accurate summary would be that orthogonal/aligned rewards preserved monitorability in the main reported setup, but the paper also documents orthogonal-reward runs where monitorability degraded because of semantic drift.

This sentence states the framework too strongly. The paper later documents a counterexample within its own experiments: in Appendix I.7, an in-context RL run with the supposedly orthogonal "punish string" reward shows monitorability degrading and eventually becoming "entirely unmonitorable", even though the reward was already maximized. The authors say this "seemingly contradict[s] our prediction" and explain that it is due to semantic drift from extra optimization pressure rather than reward conflict.

That means it is not correct to say that CoT will simply "stay legible otherwise". The paper's more careful position is that the aligned/orthogonal/in-conflict framework predicts the main reward-driven effect on monitorability, but it does not capture all ways monitorability can fail.