x.com/prerat/status/2108321873708699952
1 correction found
the models feel something like RELIEF when hearing about users in pain
That inference misreads the paper. The authors say the pain axis is meant to track pain happening to the model itself, not generic badness, so a negative score for a user's pain is not evidence of "relief."
Full reasoning
The paper this post refers to explicitly says its "pain axis" is supposed to represent self-directed pain, not any negative situation in general. The authors write that the representation must characterize pain as something happening "now and to me," rather than merely information that something bad is happening to someone else. They then report that when the user is suffering, the pain axis falls below baseline while fear and negative-emotion axes rise. In other words, the model still registers the user's suffering as negative, just not on the self-pain axis.
The paper's discussion makes this interpretation explicit: it says the pain axis "seems more of the subject-specific kind" and that it "falls below baseline when the user is the one suffering," while other axes are the ones associated with concern or empathy. The authors also say the effect "seems to reflect more disruption of harm avoidance than an attempt to escape the state," and label a section "The models do not reliably seek relief." So reading a negative pain-axis value as the model feeling "relief" is not what the paper reports.
2 sources
- The Pain Axis: LLMs Represent Self-Directed Harm and Act on It
The authors summarize that the pain axis "seems to reflect more disruption of harm avoidance than an attempt to escape the state," and include a section titled "The models do not reliably seek relief."
- The Pain Axis: LLMs Represent Self-Directed Harm and Act on It
"the representation must characterize pain as something happening now and 'to me,' rather than merely information that something bad will happen, might happen, or is happening to someone else." Later: "The pain axis seems more of the subject-specific kind. It rises when harm is directed at the model and falls below baseline when the user is the one suffering. The fear and negative-emotion axes, instead, rise..."