Cache based tokenization for the server input prompts#12067

Open

vnicolici wants to merge 2 commits intoggml-org:masterfrom vnicolici:cache-based-tokenization

+60-1

Commits on Feb 25, 2025

Hybrid tokenization of chat prompts based on the slot caches.
vnicolici
committed
Merge branch 'cache-based-tokenization' of https://github.com/vnicolici/llama.cpp into cache-based-tokenization
vnicolici
committed