All corrections
LessWrong October 14, 2026 at 06:58 PM

www.lesswrong.com/posts/zHhGa3iQBQcdSifFL/a-critical-look-at-lean4-1

2 corrections found

1
Claim
Following this incident, the Lean FRO found seven more bugs in the kernel
Correction

Leo de Moura’s follow-up posts do not report “seven more bugs in the kernel.” His August 1 post lists six additional kernel PRs, and the later August 24 post separately breaks issues into kernel soundness bugs, runtime exploits, and other bug/hardening PRs.

Full reasoning

The linked Lean posts contradict the specific count here.

  • In Leo de Moura’s August 1, 2026 postmortem for bug #14576, he writes that Daniel Selsam at OpenAI assisted the Lean FRO and found other programming mistakes in the Lean kernel, then lists exactly six PRs: #14607, #14608, #14609, #14613, #14615, #14616.
  • In the later August 24, 2026 “kernel soundness bug hunt” post, Leo does not summarize the result as “seven more bugs in the kernel.” Instead, he separately lists:
    • 4 kernel soundness bug-fix items in official kernel code (#14613, #14616, #14806, #14807/#14843),
    • 2 runtime exploits (#14838, #14833), and
    • 5 other bug/hardening PRs (#14607, #14608, #14609, #14615, #14849).

So the claim is inaccurate on its face: the authoritative follow-up posts do not give a total of seven additional kernel bugs, and the later linked post explicitly includes some issues that were runtime problems rather than kernel bugs. The same sources also attribute the discoveries to work involving Daniel Selsam / OpenAI internal models assisting the Lean FRO, not simply “the Lean FRO found seven more bugs.”

2 sources
2
Claim
There's no spec to track because nobody knows how Lean4's type theory is supposed to behave.
Correction

That absolute claim is contradicted by public Lean projects and by Leo de Moura’s own postmortem. There are public abstract specifications and model-based checkers for Lean, and Lean’s creator explicitly wrote that the inductive constructions are “well understood.”

Full reasoning

This statement is too strong to be true.

There are public specifications and formalizations of Lean’s theory:

  • The lean4lean project publicly exposes an abstract theory layer: its README says Lean4Lean.Theory contains “the Lean metatheory and properties,” and Lean4Lean.Verify is “the proof that the Lean4Lean implementation satisfies the Lean4Lean.Theory abstract specification.” That is exactly a public spec to track against, even if the work is incomplete.
  • Leo de Moura’s August 24, 2026 postmortem says lean-inductive-models addresses concerns about whether Lean’s inductive-type theory is understood, and states: “The same derivation demonstrates that the constructions Lean uses are well understood. The soundness bugs in this area were implementation mistakes, not gaps in the theory.”

That does not mean every consistency or metatheory question is fully settled. But it does directly contradict the absolute claim that “nobody knows how Lean4’s type theory is supposed to behave” or that there is “no spec to track.” Public specs, models, and implementation-vs-theory verification efforts already exist.

2 sources
  • lean4lean README

    `lake build Lean4Lean.Theory` contains the Lean metatheory and properties. `lake build Lean4Lean.Verify` is the proof that the `Lean4Lean` implementation satisfies the `Lean4Lean.Theory` abstract specification.

  • Postmortem for the Kernel Soundness Bug Hunt - Leonardo de Moura

    This way, lean-inductive-models also addresses a concern raised on social media and in public discussions: whether the theory behind Lean's inductive types is fully understood. The same derivation demonstrates that the constructions Lean uses are well understood. The soundness bugs in this area were implementation mistakes, not gaps in the theory.

Model: OPENAI_GPT_5 Prompt: v1.16.0