Tokenizer Request: larger variants

[kooshi]

The README suggests that multiple variants would be available, but there's only the 1024 version.
https://huggingface.co/datasets/willdepueoai/parameter-golf/tree/main/datasets/datasets

Can we get official tokenizers and datasets up to 16k?

[clarkkev]

I made a sp4096 vocab and got good results with it! The tokenizer and pre-tokenized documents are on HuggingFace. See my README for download/re-creation commands.