Data augmentation in machine learning

Description

Exploring the impact of data augmentation in machine learning models

How to run

This project uses pipenv for dependency management (pip install --user pipenv) (tutorial: https://www.jetbrains.com/help/pycharm/pipenv.html)

# clone project
git clone git@github.com:alexhernandezgarcia/data-augmentation.git
cd data-augmentation

# install both dev and prod (default) dependencies
pipenv install --dev

# activate environment created pipenv
pipenv shell


# to install a new packagae
pipenv install <PACKAGE_NAME>

Train model with a experiment configuration from configs/experiment/

# train on CPU (Default)
python src/train.py experiment=<YOUR EXPERIMENT CONFIG> trainer=cpu
# example
python src/train.py experiment=moons_experiment.yaml trainer=cpu


# train on GPU
python src/train.py trainer=gpu
python src/train.py experiment=moons_experiment.yaml trainer=cpu

You can override any parameter from command line like this

python src/train.py experiment=moons_experiment.yaml datamodule.dataset.n_samples=2000

You can create multi-run of an experiment using Hydra multi-run direction (-m or --multi-run)

# now Hydra with create 8 different experiment runs that will each have a different value for the n_samples parameter
python src/train.py -m experiment=moons_experiment.yaml datamodule.dataset.n_samples=20,50,100,250,500,1000,5000,10000

This process can also be scripted (bash or python shell scripts) and saved in scripts/ directory for an example checkout scripts/moons_run.sh or scripts/schedule.sh

A WandB dashboard created using scripts/moons_run.sh can be seen at Moons_Experiment_Dashboard

Project Structure

The directory structure of new project looks like this:

├── configs                   <- Hydra configuration files
│   ├── callbacks                <- Callbacks configs
│   ├── datamodule               <- Datamodule configs
│   ├── debug                    <- Debugging configs
│   ├── experiment               <- Experiment configs
│   ├── extras                   <- Extra utilities configs
│   ├── hparams_search           <- Hyperparameter search configs
│   ├── hydra                    <- Hydra configs
│   ├── local                    <- Local configs
│   ├── logger                   <- Logger configs
│   ├── model                    <- Model configs
│   ├── paths                    <- Project paths configs
│   ├── trainer                  <- Trainer configs
│   │
│   ├── eval.yaml             <- Main config for evaluation
│   └── train.yaml            <- Main config for training
│
├── data                   <- Project data
│
├── logs                   <- Logs generated by hydra and lightning loggers
│
├── notebooks              <- Jupyter notebooks. Naming convention is a number (for ordering),
│                             the creator's initials, and a short `-` delimited description,
│                             e.g. `1.0-jqp-initial-data-exploration.ipynb`.
│
├── scripts                <- Shell scripts
│
├── src                    <- Source code
│   ├── datamodules              <- Lightning datamodules
│   ├── models                   <- Lightning models
│   ├── tasks                    <- Different scenarios, like training, evaluation, etc.
│   ├── utils                    <- Utility scripts
│   │
│   ├── eval.py                  <- Run evaluation
│   └── train.py                 <- Run training
│
├── tests                  <- Tests of any kind
│
├── .env.example              <- Example of file for storing private environment variables
├── .gitignore                <- List of files ignored by git
├── .pre-commit-config.yaml   <- Configuration of pre-commit hooks for code formatting
├── Makefile                  <- Makefile with commands like `make train` or `make test`
├── pyproject.toml            <- Configuration options for testing and linting
├── requirements.txt          <- File for installing python dependencies
├── setup.py                  <- File for installing project as a package
└── README.md

Workflow

Basic workflow

1. Write your PyTorch Lightning module (see models/mnist_module.py for example)

2. Write your PyTorch Lightning datamodule (see datamodules/mnist_datamodule.py for example)

3. Write your experiment config, containing paths to model and datamodule

4. Run training with chosen experiment config:

   python src/train.py experiment=experiment_name.yaml

---

Logs

Hydra creates new output directory for every executed run.

Default logging structure:

├── logs
│   ├── task_name
│   │   ├── runs                        # Logs generated by single runs
│   │   │   ├── YYYY-MM-DD_HH-MM-SS       # Datetime of the run
│   │   │   │   ├── .hydra                  # Hydra logs
│   │   │   │   ├── csv                     # Csv logs
│   │   │   │   ├── wandb                   # Weights&Biases logs
│   │   │   │   ├── checkpoints             # Training checkpoints
│   │   │   │   └── ...                     # Any other thing saved during training
│   │   │   └── ...
│   │   │
│   │   └── multiruns                   # Logs generated by multiruns
│   │       ├── YYYY-MM-DD_HH-MM-SS       # Datetime of the multirun
│   │       │   ├──1                        # Multirun job number
│   │       │   ├──2
│   │       │   └── ...
│   │       └── ...
│   │
│   └── debugs                          # Logs generated when debugging config is attached
│       └── ...

You can change this structure by modifying paths in hydra configuration.

---

This project is built upon the lightning-hydra-template. For a thorough explanation of all the concepts and practices please follow that repository.