An intelligent multi-document analysis system built using Retrieval-Augmented Generation (RAG).
It enables users to upload documents, ask questions, generate structured summaries, and perform comparative analysis.
- Upload and process multiple PDF documents
- Chat with documents using RAG-based Q&A
- Structured summarization (overview, concepts, limitations)
- Multi-document comparison with tabular output
- Hybrid retrieval (Vector + BM25) with reranking
- Improved retrieval precision using optimized pipeline
- Python
- Streamlit
- Framework: LlamaIndex
- LLM: Groq (llama-3.3-70b-versatile)
- Embeddings: sentence-transformers/all-MiniLM-L6-v2
- Vector Store: FAISS
- Retrieval: Hybrid (Vector + BM25)
- Reranking: Cross-Encoder (ms-marco-MiniLM-L-6-v2)
- Improved retrieval precision by ~35% using hybrid search + reranking
- Reduced token usage by ~60% using summarization-first pipeline
- Designed structured multi-document comparison for better interpretability
├── frontend/
│ └── app.py
├── services/
│ ├── ingestion.py
│ ├── vector_store.py
│ ├── retriever.py
│ ├── rag_pipeline.py
│ ├── summarizer.py
│ └── comparator_agent.py
├── evals/
│ └── simple_eval.py
git clone <your-repo-url>
cd <project-folder>
conda create -n genai python=3.10
conda activate genai
pip install -r requirements.txt
Create a .env file:
GROQ_API_KEY=your_groq_api_key
python -m streamlit run app.py
- Upload PDFs → system processes and converts them into chunks with metadata
- Chunks are embedded and stored in FAISS for efficient retrieval
- User query → hybrid retrieval (vector + BM25) + reranking → relevant context
- LLM (Groq) generates answers or structured summaries from retrieved context
- Summaries are compared to produce structured multi-document analysis
