RAG_Tutorial.md

Understood. You're asking for the tutorial document to be updated with diagrams and examples for all the concepts mentioned in the diagrams within the provided document. This will involve breaking down each flow chart and component diagram.

Here's the revised tutorial document, incorporating diagrams and examples for each major concept presented graphically in your original document:

---

Building Production-Ready Multi-Stage RAG Systems: A Student's Guide

Introduction to RAG Systems

Imagine you're asking a question to an AI, and it needs to find information from a vast library of documents to give you a good answer. This is what a Retrieval Augmented Generation (RAG) system does. It combines two main parts:

1. Retrieval: Finding relevant information from a large dataset (like a library).
2. Generation: Using a large language model (LLM) to generate a human-like answer based on the retrieved information.

This tutorial will explain how RAG systems have evolved from simple "two-stage" systems to more advanced "multi-stage" architectures for better accuracy.

The Problem: Why Simple RAG Isn't Always Enough

Traditional RAG systems often face a challenge: how to bridge the "semantic gap." This means that sometimes, the words in your question don't perfectly match the words in the documents that contain the answer, even if they mean the same thing.

• Dense Retrievers (Bi-encoders): These are good at understanding the meaning of your query and finding documents that are semantically similar, even if the exact keywords aren't present. They are fast but might miss very specific information.
• Cross-encoders: These are very accurate at judging how relevant a document is to your query, but they are much slower because they compare the query and document more deeply. This makes them expensive for searching through a huge number of documents.

Our solution uses a "cascade architecture," which means it uses these different approaches in a sequence to get the best of both worlds.

System Architecture Overview

Let's break down how a multi-stage RAG system works, from getting documents ready to answering your questions.

Here's a high-level overview of the entire RAG system, showing the major pipelines:

graph TB

    subgraph "Ingestion Pipeline"
        RAW[Raw Documents] --> CHUNK[Intelligent Chunking<br/>- Sliding Window<br/>- Semantic Boundaries<br/>- Metadata Preservation]
        CHUNK --> EMBED[Embedding Generation<br/>Qwen3-Embedding]
        EMBED --> VECTORDB[(Vector Database<br/>PostgreSQL + pgvector)]
        VECTORDB --> INDEX[HNSW Index<br/>m=16, ef=200]
    end

    subgraph "Query Pipeline"
        QUERY[User Query] --> QEMBED[Query Embedding<br/>Qwen3-Embedding]
        QEMBED --> STAGE1[Stage 1: ANN Search<br/>Top-K Retrieval]
        STAGE1 --> STAGE2[Stage 2: Reranking<br/>BGE-Reranker Cross-Encoder]
        STAGE2 --> CONTEXT[Context Selection<br/>Diversity + Relevance]
        CONTEXT --> GEN[Generation<br/>DeepSeek/Llama3/GPT-4]
        GEN --> RESPONSE[Final Response]
    end

    subgraph "Feedback Loop"
        RESPONSE --> METRICS[Quality Metrics<br/>- Relevance Score<br/>- Answer Accuracy<br/>- User Feedback]
        METRICS --> FINETUNE[Model Fine-tuning<br/>Continuous Improvement]
        FINETUNE --> EMBED
    end

Loading

1. Ingestion Pipeline (Getting Documents Ready)

Before a RAG system can answer questions, it needs to process and store all the documents.

flowchart TD

    START([Document Upload]) --> VALIDATE{Document<br/>Valid?}
    VALIDATE -->|No| ERROR[Return Error]
    VALIDATE -->|Yes| DETECT[Detect Document Type<br/>PDF/DOCX/TXT/HTML]

    DETECT --> EXTRACT[Extract Text<br/>+ Metadata]
    EXTRACT --> CLEAN[Text Cleaning<br/>- Remove special chars<br/>- Normalize whitespace<br/>- Fix encoding]

    CLEAN --> LANG{Language<br/>Detection}
    LANG --> SEGMENT[Sentence Segmentation<br/>using spaCy/NLTK]

    SEGMENT --> CHUNK_STRATEGY{Choose<br/>Chunking<br/>Strategy}
    CHUNK_STRATEGY -->|Technical Docs| SEMANTIC[Semantic Chunking<br/>- Heading-based<br/>- Topic modeling]
    CHUNK_STRATEGY -->|Narrative Text| SLIDING[Sliding Window<br/>- 512 tokens<br/>- 50 token overlap]
    CHUNK_STRATEGY -->|Structured Data| ENTITY[Entity-based<br/>Chunking]

    SEMANTIC --> CHUNKS[Document Chunks]
    SLIDING --> CHUNKS
    ENTITY --> CHUNKS

    CHUNKS --> BATCH{Batch<br/>Size?}
    BATCH -->|< 32| SINGLE[Single Processing]
    BATCH -->|>= 32| PARALLEL[Parallel Processing]

    SINGLE --> EMBED_GEN[Generate Embeddings<br/>Qwen3-0.6B]
    PARALLEL --> EMBED_GEN

    EMBED_GEN --> QUALITY{Quality<br/>Check}
    QUALITY -->|Pass| STORE[(Store in pgvector)]
    QUALITY -->|Fail| RETRY{Retry<br/>Count?}
    RETRY -->|< 3| EMBED_GEN
    RETRY -->|>= 3| LOG_ERROR[Log Error<br/>Manual Review]

    STORE --> UPDATE_INDEX[Update HNSW Index]
    UPDATE_INDEX --> CACHE_WARM[Warm Cache<br/>Popular Documents]
    CACHE_WARM --> COMPLETE([Ingestion Complete])

    style START fill:#e1f5e1
    style COMPLETE fill:#e1f5e1
    style ERROR fill:#ffe1e1
    style LOG_ERROR fill:#ffe1e1

Loading

• Raw Documents: This is your initial collection of information (e.g., PDFs, web pages, internal reports).
• Intelligent Chunking: Large documents are broken down into smaller, manageable "chunks." This is important because LLMs have a limited "context window" (how much text they can process at once).
  • Example: Sliding Window Chunking
    • Original Text: "The quick brown fox jumps over the lazy dog. The dog then woke up and barked loudly. A nearby cat was startled by the noise."
    • Chunk 1 (5 words, 2 word overlap): "The quick brown fox jumps"
    • Chunk 2: "fox jumps over the lazy"
    • Chunk 3: "over the lazy dog. The"
    • (This ensures context is not lost at chunk boundaries)
  • Example: Semantic Chunking (Heading-based)
    • If a document has sections like "1. Introduction", "2. Methodology", "3. Results", "4. Conclusion", each of these sections could become a chunk.
• Embedding Generation: Each chunk is converted into a numerical representation called an "embedding." This process uses a Dense Embedding Model (like Qwen3-Embedding) which captures the meaning of the text.
  • Example: The sentence "The cat sat on the mat" might be converted into an embedding vector like [0.1, 0.5, -0.2, 0.8, ...]. A semantically similar sentence, like "A feline rested on the rug," would have a very similar embedding vector.
• Vector Database (PostgreSQL + pgvector): These embeddings (vectors) are stored in a special database that can quickly find similar vectors. PostgreSQL with the pgvector extension is a common choice.
  • Example: When you store embeddings in pgvector, it looks like this in a table:

    CREATE TABLE documents (
        id BIGSERIAL PRIMARY KEY,
        content TEXT NOT NULL,
        embedding VECTOR(1024) NOT NULL
    );
    INSERT INTO documents (content, embedding) VALUES
    ('The capital of France is Paris.', '[0.1, 0.2, ..., 0.9]');
    

• HNSW Index: To speed up searches in the vector database, an HNSW (Hierarchical Navigable Small World) index is used. Think of it as an organized map that helps find similar embeddings very quickly.
  • Example: An HNSW index organizes vectors into layers. Top layers have fewer, broadly connected points, allowing for quick "jumps" to the general area of interest. Lower layers have denser, more precise connections for fine-grained searching once the general area is found. This is much faster than checking every single vector.

    CREATE INDEX idx_documents_embedding_hnsw ON documents
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 200, ef =