transformer

Building a transformer from scratch from the popular research paper "Attention Is All You Need" in Python

In progress:

1. Input Embeddings
2. Positional Encodings
3. Layer Normalization
4. Feed Forward
5. Multi-Head Attention
6. Residual Connection
7. Encoder
8. Decoder
9. Linear Layer
10. Transformer
11. Task overview
12. Tokenizer
13. Dataset
14. Training loop
15. Validation loop
16. Attention visualization