en.wikipedia.org/wiki/Inception_(deep_learning_architecture)
1 correction found
The team solved it by using two "auxiliary classifiers"
Follow-up work from the Inception authors says this explanation was likely wrong: the auxiliary classifiers did not improve early convergence and instead acted mainly as regularizers.
Full reasoning
The later Inception-v3 paper revisits the role of auxiliary classifiers and explicitly says the original explanation was "most likely misplaced." It reports that auxiliary classifiers did not result in improved convergence early in training, which is what you would expect if they were solving vanishing gradients. Instead, the authors conclude that the auxiliary classifiers act as a regularizer.
So while GoogLeNet/Inception v1 did include two auxiliary classifiers, describing them as the thing that "solved" the vanishing-gradient problem is not supported by the authors' later analysis of how those branches actually behaved during training.
1 source
- Rethinking the Inception Architecture for Computer Vision
Interestingly, we found that auxiliary classifiers did not result in improved convergence early in the training... this means that original the hypothesis of [20] that these branches help evolving the low-level features is most likely misplaced. Instead, we argue that the auxiliary classifiers act as regularizer.