Making my local LLM voice assistant faster and more scalable with RAG

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

If you read my previous blog post, you probably already know that I like my smart home open-source and very local, and that certainly includes any voice assistant I may have. If you watched the video demo, you have probably also found out that it’s… slow. Trust me, I did too. Prefix caching helps, but it feels like cheating. Sure, it’ll look amazing in a demo, but as soon as I…

Read on johnthenerd.com ↗