Research Spike: using cached content when processing #122
Description
When adding non deterministic AI workflows into the course we open the user experience to be different for every student. Also, when processing the same content over and over we are paying for the llm to process the same tokens different times. There might be ways to address both concerns with some caching capabilities at the orchestrator or processor level.
This task is a first pass to investigate and design how such a feature could work.