The goal of this project is to summarize the various experiences in the neural network training process. When we train the network, we always have to face two major problems. One is the design of the network structure, and the other is the training configuration of the network, which is hyperparameter tuning
- Vectorization
- Preprocessing
- Data Normalization
- zero mean
- unit variance
- Feature Scaling
- Imbalanced Data
- Missing Data
- Data Normalization
- Problem
- Overfitting
- Data
- Collect more Data
- Dimensionality Reduction
- Data Augmentation
- Model
- Dropout
- Early Stopping
- Batch Normalization
- Weight Regularizers
- Reducing network size
- Data
- Gradient Vanishing/Exploding
- Vanishing
- Initialization Weight
- ReLU
- Exploding
- Gradient Clipping
- Vanishing
- Reproducible Results
- Overfitting
- Automatic
- NASnet
- MNASnet
- SNAS
- EAT-NAS
- Genetic CNN
- Hierarchical Representations
- Manual
-
Automatic
-
Manual