CAG with Ollama and vLLM

[farisalasmary]

Hi @hhhuang,

I wounder if there is a way to adapt CAG in Ollama and vLLM? have you tried them?
Also, is it easy to do it in both?

Thanks

[abdesselam-benameur]

I'm looking for the same thing, that would be interesting if we could implement this in frameworks like these

[guneetsinghchatha]

Maybe play around with the dev mode in LMstudio