www.lesswrong.com/posts/8YnHuN55XJTDwGPMr/a-gentle-introduction-to-sparse-autoen...

The post expands the acronym MLP as “multi-layer perceptions,” but in machine learning MLP is the standard abbreviation for multilayer perceptron, a type of feedforward neural network.

This isn’t a matter of terminology preference: the “P” in MLP refers to perceptron, the historical name for a (simple) neural network unit/network, not “perception(s).”

This claim is false in basic linear algebra.

• Orthogonality implies linear independence (for nonzero vectors), but the reverse direction does not hold.
• A simple counterexample in (\mathbb{R}^2): ((1,0)) and ((1,1)) are linearly independent (neither is a scalar multiple of the other) but not orthogonal (their dot product is (1)).

Jim Hefferon’s open linear algebra textbook explicitly states that “not every basis … has mutually orthogonal vectors” and gives an explicit basis for (\mathbb{R}^2) whose members “are not orthogonal.” Since a basis is (by definition) linearly independent, that directly contradicts the post’s statement that linear independence (for n vectors in n-dimensional space) requires orthogonality.

The post dates the creation of autoencoders to the 1990s, but published work describing autoencoder-style networks for compression/dimensionality reduction exists earlier.

A 1988 paper by Bourlard & Kamp discusses multilayer perceptrons in auto-association mode specifically as candidates for “data compression or dimensionality reduction.” This is the core autoencoder setup (learn an internal representation that reconstructs the input). That places autoencoder-style methods in the literature before the 1990s.

A later historical review (“Autoencoders reloaded,” 2022) explicitly discusses the 1988 work and notes the terminology shift: “Auto-associative multilayer perceptrons are now called autoencoders.” Together, these sources directly contradict the claim that autoencoders were created in the 1990s.