www.lesswrong.com/posts/sP2Hg6uPwpfp3jZJN/lost-purposes
1 correction found
physics journals require a threshold of p<0.0001.
This overstates physics publishing norms. Physics journals do publish papers reporting effects with much weaker significance than p<0.0001, and even particle physics' famous 5-sigma rule is a discovery convention, not a blanket journal requirement.
Full reasoning
The statement is too broad and is contradicted by published papers in major physics journals.
- Journal of High Energy Physics published a 2023 paper stating that the reported polarization signal had "2.6σ" significance. Under the usual normal approximation, 2.6σ corresponds to a p-value on the order of 10^-2, which is far larger than 0.0001.
- The European Physical Journal C published a paper reporting a result with "significance of 2.8σ" and a false-alarm probability of order 10^-3, again well above 0.0001.
- CERN's own explanation of particle-physics statistics says five sigma is the commonly accepted standard for claiming discovery of new particles in particle physics. That is a context-specific discovery convention, not a general rule that "physics journals" require for publication.
So the claim is inaccurate in two ways: it treats a specialized norm from some parts of particle physics as if it applied across physics journals generally, and it gives the wrong numerical threshold. If anything, the famous particle-physics discovery convention is about 5σ (roughly p ≈ 3×10^-7 one-sided), not p<0.0001.
3 sources
- Measurement of Λ transverse polarization in e+e− collisions at √s = 3.68 − 3.71 GeV | Journal of High Energy Physics
"The significance of polarization by combining the seven energy points is found to be 2.6σ including the systematic uncertainty".
- Generalized Lomb–Scargle analysis of 36Cl decay rate measurements at PTB and BNL | The European Physical Journal C
"the minimum FAP is at a frequency of 12.65/year, corresponding to a FAP of O(10^-3), with a significance of 2.8σ."
- Why do physicists mention 'five sigma' in their results? | CERN
"Five sigma is generally the accepted value for statistical significance for finding new particles within the Standard Model" and "In most areas of science that use statistical analysis, the five-sigma threshold seems overkill."