RAGMaster

A complete, readable Retrieval-Augmented Generation (RAG) chat app — built to learn from.

FastAPI + React · local embeddings · hybrid retrieval & reranking · streaming answers with citations · RAGAS evaluation

---

Summary

RAGMaster is an end-to-end reference implementation of a RAG chatbot. You point it at a folder of documents (PDF / Markdown / text); it chunks and indexes them with local embeddings, retrieves the most relevant passages for each question using a hybrid retriever, augments the prompt with that context, and generates a grounded, cited answer — streamed token-by-token to a minimal React UI. A RAGAS harness lets you measure quality as you tweak the pipeline.

Model-agnostic by design. The LLM layer speaks the OpenAI chat-completions protocol, so it runs on OpenRouter (one key for Claude, Gemini, Llama, Qwen, …), OpenAI, DeepSeek, Mistral, Qwen/DashScope, Moonshot (Kimi), Groq, Google's OpenAI-compat endpoint, or a fully local model via Ollama / LM Studio. Switching providers is configuration, not code.

Status: an educational reference. It's a single-user demo, not a hardened multi-tenant product — see Security notes.

            ┌──────────────┐   ┌──────────────┐   ┌──────────────┐   ┌────────────┐
  question →│  RETRIEVAL   │ → │  AUGMENT     │ → │  GENERATION  │ → │  answer +  │
            │ hybrid+rerank│   │ prompt+cite  │   │ stream (SSE) │   │  citations │
            └──────────────┘   └──────────────┘   └──────────────┘   └────────────┘
                                       ▲
                          EVALUATION (RAGAS) measures the whole loop

---

Walkthrough

The four stages below mirror the code, so you can read the repo top-to-bottom.

Retrieval

Goal: given a question, find the handful of passages most likely to contain the answer.

How RAGMaster does it:

1. Load & chunk (backend/pdf_ingestion.py, backend/chunking.py) — documents are loaded (PDF / MD / TXT) and split with a structure-aware chunker that follows the Markdown heading tree, never splits mid-sentence, and targets a token budget with overlap (default 320 tokens, 48 overlap). Each chunk keeps its heading breadcrumb so citations are precise.

2. Embed locally (backend/rag_service.py) — chunks are embedded with all-MiniLM-L6-v2 via sentence-transformers, on-device. No API key, no cost for indexing or search.

3. Store (backend/vector_store.py) — vectors + text + metadata go into a persistent ChromaDB collection.

4. Hybrid search + rerank (backend/retrieval.py) — for each query:

   • Dense vector search (ChromaDB) captures semantic similarity.
   • Sparse BM25 keyword search catches exact terms (names, codes).
   • The two rankings are fused with Reciprocal Rank Fusion (score = Σ 1/(k+rank), k=60).
   • A cross-encoder reranker (ms-marco-MiniLM-L-6-v2) re-scores the top candidates for a precision boost, returning the final top-k.

   Every stage is a config flag, so you can ablate it: USE_HYBRID, USE_RERANKER, TOP_K, CANDIDATE_K, RRF_K.

Augmentation

Goal: turn the retrieved passages + the question into a prompt that forces a grounded, citable answer.

How RAGMaster does it (backend/rag_service.py):

• Numbered context — each retrieved passage is labelled [1], [2], … with its source heading, so the model can cite specific passages.
• System instruction — answer only from the provided context, cite with bracketed numbers, and say so when the answer isn't present (reduces hallucination).
• Conversation history — for multi-turn threads, the most recent message pairs are prepended (THREAD_HISTORY_LENGTH).
• Optional HyDE (USE_QUERY_TRANSFORM) — the model first writes a short hypothetical answer, and that is embedded instead of the raw question; this often improves recall for vague queries.

Generation

Goal: produce the answer with any LLM, and stream it back with its sources.

How RAGMaster does it:

• One OpenAI-compatible client (backend/llm_provider.py) — configured purely via LLM_API_KEY / LLM_BASE_URL / LLM_MODEL. Swap providers without touching code (see the provider table).
• Streaming (POST /chat/stream) — tokens are streamed over Server-Sent Events so the UI types the answer out live; POST /chat is the blocking variant.
• Citations & trace — responses include the source passages used (heading, relevance score, preview) and a step-by-step "thinking" trace the UI can show.
• Caching — an LRU cache (RESPONSE_CACHE_SIZE) returns instant answers for repeated questions; the frontend memoizes too.

Evaluation

Goal: know whether a change to chunking, retrieval, or the prompt actually improves answers — not just vibes.

How RAGMaster does it (backend/eval/):

• A golden set (golden_set.json) of Q&A over the sample docs, including an unanswerable item the model must abstain on.
• RAGAS metrics: faithfulness (grounded, no hallucination), answer relevancy, and context precision — judged by the same provider you configured (any OpenAI-compatible model), scored with local embeddings.
• A CLI (python -m eval.run_eval) and an interactive Streamlit dashboard (streamlit run eval/dashboard.py).

cd backend
pip install -r requirements-eval.txt
python -m eval.run_eval            # score the built-in question set

---

Features

• Full RAG pipeline — load (PDF/MD/TXT) → chunk → embed → store → retrieve → augment → generate.
• Hybrid retrieval — dense + BM25 fused with RRF, then cross-encoder reranking.
• Local embeddings — all-MiniLM-L6-v2 on-device; indexing/search need no key.
• Any LLM provider — one OpenAI-compatible client, configured via .env.
• Streaming answers — token-by-token over SSE.
• Grounded with citations — answers cite the numbered passages they used.
• Conversation threads — multi-turn chat with per-thread history.
• Optional HyDE — hypothetical-document query transformation.
• Caching — backend LRU + frontend memoization.
• Feedback — thumbs up/down stored for later analysis.
• Evaluation — RAGAS harness + Streamlit dashboard.
• Production touches — rate limiting, optional API-key auth, CORS, request-ID logging, Docker.

Architecture

Backend (/backend) — FastAPI + Uvicorn

File	Responsibility
main.py	API routes, middleware, auth, rate limiting
rag_service.py	the end-to-end pipeline
retrieval.py	hybrid retriever (dense + BM25 + RRF + rerank)
vector_store.py	persistent ChromaDB collection
chunking.py / pdf_ingestion.py	chunking + multi-format loading
llm_provider.py	client for any OpenAI-compatible endpoint
database.py / models.py	async SQLAlchemy, ChatHistory + Feedback

Frontend (/frontend) — React 18 + Vite src/components/Chat.jsx handles the conversation flow, SSE streaming, source chips, history search, and feedback, talking to the backend through the Vite dev proxy (/api → http://localhost:8000). See FOLDER_STRUCTURE.md for the full tree.

Quickstart (Docker)

cp backend/.env.example backend/.env   # then edit and add your LLM_API_KEY
docker compose up --build

• Frontend → http://localhost:3000
• Backend → http://localhost:8000

The default sample_docs/ knowledge base is mounted and indexed on first startup.

Quickstart (manual)

Prerequisites: Python 3.10+, Node.js 18+.

# 1) Backend
cd backend
python -m venv venv
venv\Scripts\activate          # Windows
# source venv/bin/activate     # macOS/Linux
pip install -r requirements.txt
cp .env.example .env           # then edit .env (see Configuration below)
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# 2) Frontend (separate terminal)
cd frontend
npm install
npm run dev

Open http://localhost:3000.

On first run the backend downloads the embedding + reranker models (a few hundred MB) and indexes sample_docs/ — give it a minute.

Helper that boots both at once (Git Bash / WSL / macOS / Linux): ./scripts/start.sh.

Configuration — choose any LLM provider

Set three values in backend/.env:

LLM_API_KEY=your_api_key_here
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_MODEL=openai/gpt-4o-mini

Then pick a provider (full preset list in backend/.env.example):

Provider	LLM_BASE_URL	Example LLM_MODEL
OpenRouter (gateway)	https://openrouter.ai/api/v1	anthropic/claude-3.5-sonnet, google/gemini-2.0-flash-001, qwen/qwen-2.5-72b-instruct
OpenAI	https://api.openai.com/v1	gpt-4o-mini
Anthropic (Claude)	https://api.anthropic.com/v1/	claude-3-5-sonnet-latest
Google Gemini	https://generativelanguage.googleapis.com/v1beta/openai/	gemini-2.0-flash
DeepSeek	https://api.deepseek.com/v1	deepseek-chat
Mistral	https://api.mistral.ai/v1	mistral-small-latest
Qwen / DashScope	https://dashscope-intl.aliyuncs.com/compatible-mode/v1	qwen-plus
Moonshot (Kimi)	https://api.moonshot.cn/v1	moonshot-v1-8k
Groq	https://api.groq.com/openai/v1	llama-3.3-70b-versatile
Ollama (local, free)	http://localhost:11434/v1	llama3.1

Embeddings always run locally — the key is used only for generation. Other knobs (retrieval, chunking, caching, DB, auth, CORS, rate limit) are documented inline in .env.example.

Knowledge base

The default corpus lives in sample_docs/. To use your own:

1. Drop .pdf, .md, or .txt files into sample_docs/ (or point DOCUMENTS_DIR at another folder).
2. Restart the backend, or call POST /reindex to rebuild the index.

Don't commit copyrighted or private documents. design/, leadership/, documents/, and corpus/ are gitignored as drop-zones for private corpora.

API endpoints

Method	Endpoint	Description
GET	/health	Readiness: chunks loaded, vector DB ready.
GET	/history	Recent chat exchanges (optionally ?thread_id=).
POST	/chat	Run the full RAG pipeline, return a complete answer.
POST	/chat/stream	Stream the answer token-by-token (SSE).
POST	/reindex	Rebuild the index from the documents directory.
POST	/ingest	Alias of /reindex for multi-doc setups.
POST	/feedback	Record thumbs up/down on an answer.

/chat expects { "message": "...", "thread_id": "..." } and returns { "response", "logs", "sources", "thread_id" }.

Tests

Backend unit tests cover pure logic (chunking, retrieval fusion, ingestion) — no API key or model download needed:

cd backend
pip install -r requirements-dev.txt
python -m pytest

Security notes

This is a learning/demo project. Before exposing it publicly:

• Auth is optional and off by default. Set API_KEY in .env to require an X-API-Key header on every request.
• Chat history is shared, not per-user. /history returns recent exchanges for everyone — there is no user isolation. Don't treat it as multi-tenant.
• CORS is restricted to CORS_ORIGINS — set these for production.
• Never commit secrets. backend/.env is gitignored; .env.example ships placeholders only.

Contributing

Issues and PRs welcome — it's meant to be read, forked, and extended. Keep the code small and well-commented, and prefer reusing the existing utilities. See CONTRIBUTING.md for setup, tests, and ground rules (notably: never commit secrets or copyrighted documents).

License

MIT.