Recent content on Haystack
Live Last read · last published · next check
This notebook demonstrates how to setup breakpoints in a Haystack pipeline. In this case, we will set up break points in a hybrid retrieval-augmented generation (RAG) pipeline. The pipeline combines BM25 and embedding-based retrieval methods, then uses a transformer-based reranker and an LLM to generate answers. Install packages %%bash pip install haystack-ai>=2.16.0 pip install…
Level : Advanced Time to complete : 20 minutes Components Used : Agent , ChatPromptBuilder , InMemoryBM25Retriever OpenAIChatGenerator Archived Components Used : InMemoryChatMessageStore , ChatMessageRetriever , ChatMessageWriter Prerequisites : You need an OpenAI API Key Goal : After completing this tutorial, you'll have learned how to incorporate a conversational history into a RAG pipeline to…
Evaluation measures performance using metrics like precision, recall, and relevancy, providing a clear picture of your pipeline's strengths and weaknesses using LLMs or ground-truth labels. Evaluating RAG systems can help understand performance bottlenecks and optimize one component at a time, for example, a Retriever or a prompt used with a Generator. Here's a step-by-step guide explaining what…
by Tuana Celik ( LI , Twitter/X ) This is part one of the Advanced Use Cases series: 1️⃣ Extract Metadata from Queries to Improve Retrieval cookbook & full article 2️⃣ Query Expansion & the full article 3️⃣ Query Decomposition cookbook & full article 4️⃣ Automated Metadata Enrichment In this cookbook, you'll learn how to implement query expansion for RAG. Query expansion consists of asking an LLM…
In this cookbook we will show how to calculate a hallucination risk based on the research paper LLMs are Bayesian, in Expectation, not in Realization and this GitHub repo, https://github.com/leochlon/hallbayes . In this notebook, we'll use the OpenAIChatGenerator from haystack-experimental . ⚠️ Archived package : haystack-experimental is archived and no longer maintained. This experiment was not…
Notebook by Kacper Łukawski 🚀 Part of Haystack 3.0 Launch Week : five days of new drops (July 20–24). In this notebook, we build a fully local, fully async agent that uses a skill to change how it reports back, and a custom bash tool to actually use the machine it runs on. Haystack's Agent can be given skills - folders of instructions the agent reads on demand, in the same SKILL.md format used by…
Level : Advanced Time to complete : 25 minutes Components/Packages Used : agent-pack-haystack ( create_advanced_rag_agent , create_deep_research_agent ), Agent , InMemoryDocumentStore Prerequisites : Haystack 3.0 or later, an OpenAI API key , and a Tavily API key (free tier available) Goal : After completing this tutorial, you'll understand what Agent Pack is and why it exists, and you'll have run…
Notebook by Bilge Yücel 🚀 Part of Haystack 3.0 Launch Week : five days of new drops (July 20–24). Every call to Agent.run() returns metadata alongside the agent's reply. In this cookbook you'll use that metadata, and Agent hooks , to enforce soft and hard budget policies. You'll: Build a simple agent with a custom tool and inspect step_count , token_usage , and tool_call_counts . Implement a…
Level : Advanced Time to complete : 20 minutes Components Used : Agent , OpenAIChatGenerator Prerequisites : You need an OpenAI API Key Goal : After completing this tutorial, you'll have learned how to implement human-in-the-loop workflows in Haystack agents using confirmation strategies, create custom confirmation policies, and control tool execution approval flows. Overview This tutorial…
⭐️ Highlights Haystack 3.0 is a major release for building production-grade agents with full control and flexibility . It ships a wave of new capabilities: a more capable Agent with hooks and first-class skills, built-in run introspection, first-class async for serving, a leaner core, and safer pipeline loading. A few small, intentional breaking changes come with it but our Migration Guide and…
Today, we're happy to announce the release of a new major version: Haystack 3.0 🎉 Haystack is an open-source AI orchestration framework for building production-grade agents with full control and flexibility, and 3.0 is the release where agents move to the center of the framework . On top of a lighter core, this release ships a wave of agentic capabilities: first-class skills, hooks to control the…
Level : Advanced Time to complete : 20 minutes Components Used : InMemoryDocumentStore , SentenceTransformersDocumentEmbedder , SentenceTransformersTextEmbedder , InMemoryEmbeddingRetriever , ChatPromptBuilder , OpenAIChatGenerator , Agent Prerequisites : You must have an OpenAI API Key and be familiar with creating pipelines Goal : After completing this tutorial, you will have learned how to…
⭐️ Highlights 📦 Slimming down Haystack core ahead of 3.0 This release begins the migration of many components out of haystack core and into dedicated integration packages, in preparation for Haystack 3.0. Components with heavy or optional dependencies — including all SentenceTransformers embedders and rankers, the Hugging Face API components, the legacy Generator s, TikaDocumentConverter ,…
Level : Beginner Time to complete : 15 minutes Components Used : InMemoryDocumentStore , DocumentLanguageClassifier , MetadataRouter , DocumentWriter , TextLanguageRouter , DocumentJoiner , InMemoryBM25Retriever , ChatPromptBuilder , OpenAIChatGenerator Goal : After completing this tutorial, you'll have learned how to build a Haystack pipeline to classify documents based on the (human) language…
The Model Context Protocol (MCP) is the connective tissue of the modern AI stack. If you are building AI agents or production RAG systems, understanding how MCP works with Haystack is one of the most direct ways to make your applications composable, reusable, and reachable from the tools your users already live in. This blog post explains what MCP is, why it matters, and the different ways you can…
Notebook by Stefano Fiorucci This notebook shows how to create a multilingual Retrieval Augmented Generation application, starting from a podcast. 🧰 Stack : Haystack LLM framework OpenAI Whisper model for audio transcription Qdrant vector database multilingual embedding model: multilingual-e5-large multilingual LLM: Mistral Small Installation %% capture ! pip install - U mistral - haystack…
🐛 Bug Fixes Fixed the Agent exiting prematurely under the default exit_conditions=["text"] . The agent now only stops when the last message is an assistant message with non-empty text (or when no tool invoker is configured). Previously, if the LLM produced an invalid tool call that was discarded, the resulting assistant message with empty text and no tool calls would trigger an exit, preventing…
Level : Beginner Time to complete : 10 minutes Components Used : InMemoryDocumentStore , SentenceTransformersDocumentEmbedder , SentenceTransformersTextEmbedder , InMemoryEmbeddingRetriever , ChatPromptBuilder , and a ChatGenerator such as OpenAIChatGenerator , MistralChatGenerator , or TransformersChatGenerator . Prerequisites : Access to a large language model, either an API key from a provider…
Level : Beginner Time to complete : 15 minutes Goal : After completing this tutorial, you'll have learned how to build an indexing pipeline that will preprocess files based on their file type, using the FileTypeRouter . 💡 (Optional): After creating the indexing pipeline in this tutorial, there is an optional section that shows you how to create a RAG pipeline on top of the document store you just…
Level : Intermediate Time to complete : 20 minutes Concepts and Components Used : @super_component , Pipeline , DocumentJoiner , SentenceTransformersTextEmbedder , InMemoryBM25Retriever , InMemoryEmbeddingRetriever , SentenceTransformersSimilarityRanker Goal : After completing this tutorial, you'll have learned how to create custom SuperComponents using the @super_component decorator to simplify…
Level : Intermediate Time to complete : 10 minutes Components Used : InMemoryDocumentStore , InMemoryEmbeddingRetriever , SentenceTransformersDocumentEmbedder , SentenceTransformersTextEmbedder Goal : After completing this tutorial, you'll have learned how to embed metadata information while indexing documents, to improve retrieval. ⚠️ Note of caution: The method showcased in this tutorial is not…
Level : Intermediate Time to complete : 20 minutes Components Used : SentenceTransformersDocumentImageEmbedder , ImageFileToDocument , DocumentToImageContent , DocumentTypeRouter , LLMDocumentContentExtractor Prerequisites : You need an OpenAI API Key Goal : After completing this tutorial, you'll have learned how to index and retrieve images using Haystack and build a RAG pipeline that can answer…
Level : Beginner Time to complete : 15 minutes Components Used : TransformersExtractiveReader , InMemoryDocumentStore , InMemoryEmbeddingRetriever , DocumentWriter , SentenceTransformersDocumentEmbedder , SentenceTransformersTextEmbedder Goal : After completing this tutorial, you'll have learned how to build a Haystack pipeline that uses an extractive model to display where the answer to your…
Level : Intermediate Time to complete : 15 minutes Components Used : DocumentSplitter , SentenceTransformersDocumentEmbedder , InMemoryDocumentStore , InMemoryBM25Retriever , InMemoryEmbeddingRetriever , and SentenceTransformersSimilarityRanker Prerequisites : None Goal : After completing this tutorial, you will have learned about creating a hybrid retrieval and when it's useful. Overview Hybrid…
Level : Intermediate Time to complete : 15 minutes Components Used : InMemoryDocumentStore , InMemoryEmbeddingRetriever , ChatPromptBuilder , OpenAIChatGenerator , DocumentMRREvaluator , FaithfulnessEvaluator , SASEvaluator Prerequisites : You must have an API key from an active OpenAI account as this tutorial is using the gpt-4o-mini model by OpenAI: https://platform.openai.com/api-keys Goal :…
Level : Intermediate Time to complete : 15 minutes Components Used : TransformersZeroShotTextRouter , TransformersTextRouter , InMemoryDocumentStore , SentenceTransformersDocumentEmbedder , SentenceTransformersTextEmbedder , InMemoryEmbeddingRetriever , InMemoryBM25Retriever Goal : After completing this tutorial, you will have learned about TransformersZeroShotTextRouter and TransformersTextRouter…
Notebook by David Batista This is part one of the Advanced Use Cases series: 1️⃣ Extract Metadata from Queries to Improve Retrieval & the full article 2️⃣ Query Expansion cookbook & full article 3️⃣ Query Decomposition cookbook & the full article 4️⃣ Automated Metadata Enrichment In this notebook, we'll discuss how to implement a custom component, QueryMetadataExtractor , that extracts entities…
Notebook by David S. Batista This notebook shows how to use LLMMetadataExtractor , we will use a arge Language Model to perform metadata extraction from a Document. Setting Up ! uv pip install haystack - ai ! uv pip install sentence - transformers - haystack Initialize LLMMetadataExtractor Let's define what kind of metadata we want to extract from our documents, we wil do it through a LLM prompt,…
Notebook by Stefano Fiorucci In this notebook, I do some experiments on embedding meaningful metadata to improve Document retrieval. %% capture ! pip install wikipedia haystack - ai sentence - transformers - haystack rich import rich Load data from Wikipedia We are going to download the Wikipedia pages related to some bands, using the python library wikipedia . These pages are converted into…
When building applications with LLMs, writing effective prompts is a long process of trial and error. Often, if you switch models, you also have to change the prompt. What if you could automate this process? That's where DSPy comes in - a framework designed to algorithmically optimize prompts for Language Models. By applying classical machine learning concepts (training and evaluation data,…
Notebook by Madeeswaran Kannan & Mathis Lucka In this notebook, you'll learn how to use the AsyncPipeline and async-enabled components to build and execute a Haystack pipeline in an asynchronous environment. It's based on this short Haystack tutorial , so it would be a good idea to familiarize yourself with it before we begin. A further prerequisite is working knowledge of cooperative scheduling…
📚 This cookbook has an accompanying article with a complete walkthrough "Optimizing Retrival with HyDE" In this coookbook, we are building Haystack components that allow us to easily incorporate HyDE into our RAG pipelines, to optimize retrieval. To learn more about HyDE and when it's useful, check out our guide to Hypothetical Document Embeddings (HyDE) Install Requirements ! pip install…
Colab by Tuana Celik - ( LI & Twitter ) Quick guide to building Question Answering on the web with a Mistral AI model and Haystack. We use mistral-small-latest via the Mistral AI API and the MistralChatGenerator from the official Mistral integration for Haystack. Use the MistralChatGenerator to query the model on its own Add the generator to a full RAG Pipeline (on the web) Install dependencies !…
Notebook by Bilge Yucel ( LI & X (Twitter) ) In this example, we'll create a RAG application using prompting techniques in Anthropic's Prompt Engineering Guide . This application will use Anthropic Claude 3 models and Haystack to extract relevant quotes from given documents and generate an answer based on extracted quotes. 📚 Useful Sources: Docs: AnthropicChatGenerator Integration: Anthropic…
In their Llama 3.2 collection , Meta released two small yet powerful Language Models. In this notebook, we'll use the 3B model to build an Agentic Retrieval Augmented Generation application . 🎯 Our goal is to create a system that answers questions using a knowledge base focused on the Seven Wonders of the Ancient World. If the retrieved documents don't contain the answer, the application will…
In this notebook, you'll learn how to use AstraDB as a data source in your Haystack pipelines. Prerequisites You'll need an OpenAPI key to follow along. (Haystack is model-agnostic so feel free to use a different one if you'd prefer!) You'll need the following variables in order to use the Haystack extension. The following tutorials will show you how to create an AstraDB database, and save these…
This notebook demonstrates how to set up breakpoints within an Agent component in a Haystack pipeline. Breakpoints can be placed either on the chat_generator or on any of the tools used by the Agent . This guide showcases both approaches. The pipeline features an Agent acting as a database assistant, responsible for extracting relevant information and writing it to the database. Install packages…
In this notebook, we introduce the features we've developed for tool/function calling support in Haystack. We refactored the ChatMessage dataclass, to be more flexible and future-proof. We introduced some new dataclasses: ToolCall , ToolCallResult , and Tool . We added support for tools in the OpenAIChatGenerator and other Chat Generators. We introduced the ToolInvoker component, to actually…
by Bilge Yucel ( X , Linkedin ) In this cookbook, we walk through the Evaluators in Haystack, create an evaluation pipeline and try different Evaluation Frameworks like FlowJudge . 📚 Useful Resources: Article: Benchmarking Haystack Pipelines for Optimal Performance Evaluation Walkthrough Evaluation tutorial Evaluation Docs haystack-evaluation repo 📺 Watch Along ! pip install haystack - ai…
by Tuana Celik: Twitter , LinkedIn , Tilde Thurium: Twitter , LinkedIn and Silvano Cerza: LinkedIn This is a notebook showing how you can use Gemini + Vertex AI with Haystack. To use Gemini models on the Gemini Developer API with Haystack, check out our documentation . Gemini is Google's newest model. You can read more about its capabilities here . Install dependencies As a prerequisite, you need…
Evaluating the responses of Language Models and LLM-based applications often involves using model-based metrics that do not require ground truth labels. Large proprietary models like GPT-4 and Claude 3 Opus are frequently employed as evaluators and demonstrate a good correlation with human evaluations. However, relying on closed models poses several challenges: fairness: the training data of these…
Simple RAG example on the Oscars using Llama 3.1 open models and the Haystack LLM framework . Installation ! pip install transformers - haystack sentence - transformers - haystack bitsandbytes Authorization you need an Hugging Face account you need to accept Meta conditions here: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct and wait for the authorization import getpass , os os .…
📚 This cookbook has an accompanying article with a complete walkthrough " Level up Your RAG Application with Speaker Diarization " LLMs excel with text data, answering complex questions without manual reading or searching. When dealing with audio or video, providing transcription is key. Transcription captures spoken content of the audio or video, but in multi-speaker recordings, it may miss…
Notebook by Stefano Fiorucci and Tuana Celik We are going to build a nice Retrieval Augmented Generation pipeline for Rock music, using the 🏗️ Haystack LLM orchestration framework and a good LLM: 💬 Zephyr 7B Beta (fine-tuned version of Mistral 7B V.01 that focuses on helpfulness and outperforms many larger models on the MT-Bench and AlpacaEval benchmarks) Install dependencies wikipedia is needed…
Level : Beginner Time to complete : 10 minutes Components Used : TransformersChatGenerator , ChatPromptBuilder Prerequisites : None Goal : After completing this tutorial, you'll understand how to serialize and deserialize between YAML and Python code. Overview 📚 Useful Documentation: Serialization Serialization means converting a pipeline to a format that you can save on your disk and load later.…
Level : Advanced Time to complete : 20 min Components Used : TransformersChatGenerator Goal : Apply TurboQuant KV cache compression to a local LLM and measure its memory and throughput impact with Haystack. Overview Every time an LLM generates a token, it reads and writes a key-value (KV) cache - a growing table of intermediate activations that lets the model attend to previous tokens without…
Level : Beginner Time to complete : 15 minutes Components Used : Agent , OpenAIChatGenerator , SerperDevWebSearch , ComponentTool , PipelineTool Prerequisites : You must have an OpenAI API Key and a SerperDev API Key Goal : After completing this tutorial, you'll have learned how to create an Agent that can use both components and pipelines as tools to answer questions and perform tasks. Overview…
Level : Advanced Time to complete : 20 minutes Components Used : Agent , DuckduckgoApiWebSearch , OpenAIChatGenerator , DocumentWriter Prerequisites : You need an OpenAI API Key , and a Notion Integration set up beforehand Goal : After completing this tutorial, you'll have learned how to build a multi-agent system in Haystack where each agent is specialized for a specific task. Overview…
Level : Intermediate Time to complete : 10 minutes Components Used : ConditionalRouter , SerperDevWebSearch , ChatPromptBuilder , OpenAIChatGenerator Prerequisites : You must have an OpenAI API Key and a Serper API Key for this tutorial Goal : After completing this tutorial, you'll have learned how to create an agentic RAG pipeline with conditional routing that can fallback to websearch if the…
Notebook by Bilge Yucel In this notebook, we will use Ollama , Gemma2 and Haystack to build a self-reflecting agent that can leverage web resources to augment its self-reflection and decision-making capabilities. 📚 Useful Sources Docs Tutorials Install Dependencies ! pip install - U ollama - haystack serperdev - haystack trafilatura 'nltk>=3.9.1' git + https :// github . com / deepset - ai /…
![](https://haystack.deepset.ai/cookbook/prompt_optimization_with_dspy/"https://raw.githubusercontent.com/stanfordnlp/dspy/main/docs/images/DSPy8.png")
![](https://haystack.deepset.ai/cookbook/llama32_agentic_rag/"https://img-cdn.inc.com/image/upload/w_1280,ar_16:9,c_fill,g_auto,q_auto:best/images/panoramic/meta-llama3-inc_539927_dhgoal.webp")
![](https://haystack.deepset.ai/cookbook/prometheus2_evaluation/"https://haystack.deepset.ai/images/haystack-ogimage.png")
![](https://haystack.deepset.ai/cookbook/llama3_rag/"https://img-cdn.inc.com/image/upload/w_1280,ar_16:9,c_fill,g_auto,q_auto:best/images/panoramic/meta-llama3-inc_539927_dhgoal.webp")