Building a Decentralized RAG with Blockchain Verification & Source Code

A production-grade Retrieval-Augmented Generation (RAG) platform with document integrity secured by an Ethereum smart contract registry and local IPFS storage.

Why Do We Need Blockchain in RAG?

In standard Retrieval-Augmented Generation (RAG) applications, documents are parsed, chunked, embedded, and stored in a vector database. When a user asks a question, the system retrieves semantically relevant chunks and feeds them to the LLM to generate an answer.

This architecture has a critical vulnerability: Vector Database Poisoning & Tampering.

The Vulnerability

If an attacker gains write access to the vector database or document storage (such as a local file folder, S3 bucket, or SQL database), they can alter document contents, inject false data, or modify system instructions. When the RAG engine queries the database, it retrieves this compromised text and sends it to the LLM. The LLM, unaware of the tampering, will answer the user’s question with false information, citing the compromised document as a source.

The Solution: Blockchain Verification

A blockchain provides a tamper-proof, decentralized, and immutable ledger. In our RAG system, we use the blockchain as a cryptographically secure root of trust:

1. Immutable Registration: When a document is uploaded, we compute its unique SHA256 checksum and record it on the Ethereum blockchain via a smart contract (DocumentRegistry.sol), bound to the document’s IPFS Content Identifier (CID). Once written, this transaction is permanent and cannot be modified or forged.

2. On-Retrieval Audits (Zero-Trust): When a chunk is retrieved for a query, the backend fetches the original document bytes from storage and computes the SHA256 hash. It calls the blockchain registry to check:

   • Does this document’s CID exist on-chain?

   • Does the computed SHA256 match the recorded hash on the blockchain?

3. Refusal to Answer: If the hashes do not match (meaning the document has been tampered with or poisoned), the verifier immediately flags it. The system discards the chunk, and if no verified context remains, the LLM refuses to answer, protecting the user from database poisoning.

Why IPFS + Blockchain?

Storing large documents (such as PDFs or text files) directly on the blockchain is extremely inefficient and cost-prohibitive due to the high gas costs of on-chain storage.

Instead, we use a hybrid model:

• Off-Chain Storage (IPFS): IPFS (InterPlanetary File System) uses Content Addressing. A file is identified by its Content Identifier (CID), a cryptographic hash of its contents. If a single character changes, the CID changes.

• On-Chain Indexing (Ethereum): The blockchain contract stores only the small metadata records (IPFS CID, SHA256 hash, owner address, block timestamp, version).

This design provides decentralized, secure storage with lightweight, inexpensive blockchain validation.

Core Libraries & Technologies Used

We utilize a modern stack of libraries to build this robust environment:

Web3 & Smart Contracts

• Solidity (v0.8.24): The contract programming language used to write DocumentRegistry.sol. It defines the mapping schema and handles access controls (checking that only the owner can delete or update a registered document).

• Hardhat: A Node.js development environment for Ethereum. We use it to:

  • Compile smart contracts to generate the ABI.

  • Spin up a local EVM network (npx hardhat node) running on port 8545 to test transactions without paying real gas.

  • Automate contract deployments using script runners.

• Web3.py (v7.16): The Python adapter library. Our FastAPI backend uses it to connect to the Hardhat JSON-RPC node over HTTP. Web3.py handles transaction building, account credential signing, gas limit estimations, transaction receipt waiting, and contract view calls.

Artificial Intelligence & Database

• Ollama: A lightweight local LLM execution engine. We use it to:

  • Run the nomic-embed-text:latest model to generate 768-dimensional vector embeddings of text chunks.

  • Run the llama3.2:1b model to execute local reasoning and stream responses.

• ChromaDB: An AI native vector database. It stores semantic chunks alongside metadata indexes and provides fast cosine-similarity lookups (<2s latency) without requiring external cloud databases.

Backend, Frontend, and Testing

• FastAPI: The high-performance Python web framework used to expose API routes (/upload, /query, /documents, /verify/{cid}, /document). It manages CORS, multi-part form file uploads, JWT token issuance, and JSON/text response streaming.

• Streamlit: A Python framework for building interactive user interfaces. It runs on port 8501, managing user login states, system connection indicators, file uploads, chat queries, and displays detailed log outputs.

• PyPDF2: Extracts raw text from binary PDF layouts.

• Playwright: A cross-browser testing library. It automates Chrome in headless or headful mode to run end-to-end tests, filling forms, uploading files, and making assertions to automatically verify the app’s health.

Github link: https://github.com/lamhotsiagian/llm-blockchain

Folder Structure

llm-blockchain/
├── README.md                  # Detailed startup and run commands
├── requirements.txt           # Python application dependencies
├── .env                       # Environment configurations
├── config.yaml                # RAG parameter setup
├── blockchain/                # Smart Contract & Hardhat project
│   ├── contracts/
│   │   └── DocumentRegistry.sol
│   ├── scripts/
│   │   └── deploy.js
│   ├── hardhat.config.js
│   └── package.json
├── src/                       # Source codebase
│   ├── api/
│   │   └── main.py            # FastAPI endpoints
│   ├── auth/
│   │   └── jwt.py             # JWT token helpers
│   ├── blockchain/
│   │   └── client.py          # Web3.py wrapper client
│   ├── chunking/
│   │   └── chunker.py         # Sliding window text chunking
│   ├── config/
│   │   └── config.py          # Config registry loader
│   ├── ingestion/
│   │   └── extractor.py       # PDF/TXT parser
│   ├── ipfs/
│   │   └── client.py          # IPFS connection & mock storage
│   ├── llm/
│   │   └── client.py          # Ollama LLM client
│   ├── vectordb/
│   │   └── client.py          # ChromaDB persistent client
│   ├── verifier/
│   │   └── verifier.py        # Hash verification logic
│   └── app.py                 # Streamlit client portal
└── tests/                     # Integration tests
    ├── generate_seed_data.py  # programmatically generate seed docs
    ├── seed.py                # Upload seed data via API
    └── test_e2e.py            # Playwright E2E browser test

Read more