Entropy-SGD: Biasing gradient descent into wide valleys

Keyphrases

• Energy Landscape 100%
• Gradient Descent 100%
• Generalization Error 66%
• Local Entropy 66%
• Langevin Dynamics 33%
• State-of-the-art Techniques 33%
• Optimization Algorithm 33%
• Recurrent Neural Network 33%
• Hessian 33%
• Entropy-based 33%
• Zero Eigenvalue 33%
• Local Geometry 33%
• Nested Loops 33%
• Convolutional Neural Network 33%
• Flat Region 33%
• Training Time 33%
• Positive Eigenvalues 33%
• Negative Eigenvalues 33%
• Local Extrema 33%
• Large Flat 33%
• Uniform Stability 33%
• Training Deep Neural Networks 33%

Computer Science

• Eigenvalue 100%
• Gradient Descent 100%
• Generalization Error 100%
• Objective Function 50%
• Optimization Algorithm 50%
• Nested Loop 50%
• Recurrent Neural Network 50%
• Deep Neural Network 50%
• Convolutional Neural Network 50%

Mathematics

• Local Entropy 100%
• Eigenvalue 50%
• Objective Function 50%
• Neural Network 50%
• Local Geometry 50%
• Positive Eigenvalue 50%
• Negative Eigenvalue 50%
• Deep Neural Network 50%