www.lesswrong.com/posts/Z2RWoyJp9j2yEfbHN/the-hessian-rank-bounds-the-learning-c...
1 correction found
Z(n)nd12+d−d13≥
This exponent appears to be a typo: after the stated change of variables, the prefactor should scale like n^{d1/2 + (d−d1)/4}, not n^{d1/2 + (d−d1)/3}.
Full reasoning
In the proof, the author defines the change of variables
- (w'(1)=n^{1/2}w(1)) in (d_1) dimensions, and
- (w'(2)=n^{1/4}w(2)) in (d-d_1) dimensions.
A standard multivariable change-of-variables formula says the integration measure picks up the absolute value of the Jacobian determinant; for a scaling by (n^{a}) in (k) dimensions, volumes scale by (n^{ak}). Equivalently, if you solve for the old variables ((w(1)=n^{-1/2}w'(1)), (w(2)=n^{-1/4}w'(2))), then
[
dw = n^{-d_1/2}, n^{-(d-d_1)/4}, dw'.
]
So when you “rearrange and substitute” after that change of variables, the power of (n) that moves to the left-hand side should be (n^{d_1/2 + (d-d_1)/4}), not (n^{d_1/2 + (d-d_1)/3}).
This is also consistent with the rest of the post, which states the intended bound (\lambda \le d_1/2 + (d-d_1)/4) (denominator 4), and later displays the corresponding limit with a denominator 4. That makes the lone denominator 3 in this intermediate step best explained as a typographical error.
What to change: replace the (d-d_1) term’s “/3” with “/4” in that displayed inequality.
1 source
- Jacobian matrix and determinant — Wikipedia
“The absolute value of the Jacobian determinant … gives us the factor by which the function f expands or shrinks volumes … this is why it occurs in the general substitution rule.” (and it “arises as a multiplicative factor within the integral”).