- The architecture for this model is heavily inspired by Google's paper: Attention is all you need.
- It is a decoder-only Auto Regressive model built from scratch using
pytorch. - This model is trained on a Shakespeare's verse.
You can run it from my deployed huggingface's SPACE 💖
- Be sure to include text formatted in Shakespeare's style.
- Or you could just use a new line character for a start and see what the GPT generates!