ALBERT-pytorch-implementation

This is a ALBERT Model written using torch only without huggingface transformer library.

• As it is a code that implements only the model structure, the training code is not provided. Therefore, in order for you to actually train the model, you have to train the model by writing the train code yourself.
• I wrote the variable names in detail to help you study.
• If you want to set the same as the library environment in which I developed this code, you can use pip3 install -r requirements.txt
• You can do a simple test by running the ALBERT-testing.ipynb or ALBERT-testing.py code.

files

• ALBERT.py
• Transformer.py
• MultiHeadAttention.py
• Embeddings.py
• ALBERT-testing.ipynb
• ALBERT-testing.py
• requirements.txt