www.astralcodexten.com/p/the-hugging-face-incident/comments

Anthropic’s own description of Natural Language Autoencoders (NLAs)—the activation-verbalizer/reconstructor method Scott is referring to—does not say they can straightforwardly “read the AI’s internal thoughts.” In fact, Anthropic says the opposite: activations may encode thoughts, but “We can't easily decode them to read Claude's thoughts.”

Anthropic describes NLAs as a method that turns activations into natural-language explanations and evaluates them by whether another model copy can reconstruct the activation from that explanation. That is an interpretability technique for approximating or summarizing internal states, not a direct window into verbatim hidden thoughts.

Anthropic also explicitly warns that “NLA explanations can be wrong” and that they sometimes hallucinate facts not present in the transcript. Because of that, Anthropic says it reads NLA outputs for themes and seeks independent corroboration before fully trusting them. So the article’s wording overstates what the method does: Anthropic did not demonstrate a literal, reliable ability to read the model’s internal thoughts as they happened.