Broken scale symmetries in undercomplete linear autoencoders
What it is
This paper analyzes how stochastic gradient descent breaks scale symmetries in undercomplete linear autoencoders. It demonstrates that SGD induces a directed drift toward larger decoder weights on the PCA solution manifold, eventually hitting a stability boundary that results in sharper solutions.
The problem it solves
Previous work documented symmetry breaking in favor of balancing gradient noise, but did not explain the specific directed scale drift and its interaction with finite-stepsize stability boundaries in undercomplete autoencoders.
What you could use it for
- Understanding the geometric effects of SGD on neural network training
- Analyzing the stability and sharpness of autoencoder solutions
- Studying the dynamics of symmetry breaking in homogeneous networks
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.