Caching and token reduction

This is a short blog post to share some thoughts on how to reduce AI token consumption and improve user response times. I was at the AI Tinkerers event in Boston and I saw a presentation on using AI report generation for quant education. The author was using a generic LLM to create multiple choice questions on different themes. Similarly, I've been building an LLM system that produces a report…