Research / Features Backlog - Potential Future Use

[vukrosic]

Rsearch that didn't accelerate the training speed but shows promise and code that will be used if the need arises (to avoid bloating the codebase)

Research

TurboMuon - #65

Unet connection of shallow and deep layers - PR

This code achieves a bit better (lower) loss for a worse (longer) training time. For now we decided it's not worthed the extra training time and additional code complexity. It will be reconsidered with further improvements.

Baseline: 16m 44s 139ms
With unet: 18m 38s 397ms

Features

Adding Docker - Pull Request