Putting the Open back in AI: RAG

Continuing our theme of running large language models (LLM) locally on your PC for free i ; in this post we build upon the local AI chatbot from part 1, improving the responses by retrieving data to make it more knowledgeable. To do this, we use a technique called retrieval augmented generation (RAG), that adds data to the prompt before it goes to the model. Conventional wisdom says RAG is a magic…