An implementation of GPT2 Byte Pair Encoding encoder/decoder in golang. Generally very fast, bottlenecked by the regex engine for whitespace separation.
brandonrjacobs/gpt_bpe
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
Releases
No releases published
Languages
- Go 88.7%
- C 6.2%
- Python 2.4%
- CMake 1.3%
- Other 1.4%