InsightForge - AI-Powered Business Intelligence Assistant

🔮 An intelligent Business Intelligence assistant powered by AI, built with Python, LangChain, LangGraph, Ollama, and Streamlit.

Capstone Project: Enabling AI-Powered Business Intelligence for Organizations

🎯 Project Overview

InsightForge transforms business data into actionable insights using advanced AI technologies including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and LangGraph orchestration. It enables organizations to:

• Analyze business data: Comprehensive analysis of sales, products, regions, and customers
• Generate insights: Natural language processing for actionable business recommendations
• Visualize insights: Interactive dashboards and AI-suggested visualizations
• Conversational interface: Chat-based queries with memory and context awareness

🛠️ Technology Stack

• Python 3.10+: Core programming language
• LangChain: LLM application framework
• LangGraph: Workflow orchestration and state management
• Ollama: Local LLM inference (llama3)
• Streamlit: Interactive web interface
• ChromaDB/FAISS: Vector storage and retrieval
• Plotly: Data visualization
• Pandas: Data processing and analytics

📁 Project Structure

AI Powered BI/
├── src/                      # Core application code
│   ├── analytics.py          # Business analytics functions
│   ├── data_loader.py        # Data loading and preprocessing
│   ├── vectorstore.py        # Vector store management
│   ├── retriever.py          # Hybrid retrieval system
│   ├── pipeline.py           # LangGraph pipeline (main logic)
│   ├── prompts.py            # Prompt templates
│   ├── memory.py             # Conversation memory
│   └── visualization.py      # Chart generation
├── data/
│   ├── raw/                  # Raw input data (CSV/Parquet)
│   ├── processed/            # Processed data
│   └── vectorstore/          # Vector embeddings
├── configs/
│   └── config.yaml           # Application configuration
├── ui/streamlit/             # Streamlit interface
│   ├── app.py               # Main app with navigation
│   ├── chat.py              # Chat interface
│   └── dashboard.py         # Analytics dashboard
├── scripts/                  # Utility scripts
│   ├── generate_sample_data.py
│   ├── preprocess_data.py
│   ├── build_vectorstore.py
│   └── evaluate_model.py
├── tests/                    # Unit tests
├── docs/                     # Documentation
├── notebooks/                # Jupyter notebooks (optional)
├── requirements.txt          # Python dependencies
├── .env.example             # Environment variables template
└── README.md                # This file

🚀 Quick Start

1. Prerequisites

• Python 3.10+ installed
• Ollama installed and running (Download Ollama)
• Git (optional, for cloning)

2. Installation

# Clone or navigate to project directory
cd "c:\AI Projects\AI Powered BI"

# Create virtual environment
python -m venv .venv

# Activate virtual environment
# Windows:
.venv\Scripts\activate
# Linux/Mac:
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

3. Configure Environment

# Copy environment template
cp .env.example .env

# Edit .env if needed (default values work for local Ollama)

4. Set Up Ollama

# Start Ollama service (if not running)
ollama serve

# In another terminal, pull the model
ollama pull llama3

5. Prepare Data

Option A: Use Sample Data (for testing)

python scripts/generate_sample_data.py

Option B: Use Your Own Data

• Place your CSV or Parquet files in data/raw/
• Ensure data includes columns like: date, sales_amount, product_name, region, customer_segment

6. Process Data and Build Knowledge Base

# Step 1: Preprocess raw data
python scripts/preprocess_data.py

# Step 2: Build vector store (may take a few minutes)
python scripts/build_vectorstore.py

7. Launch Application

# Start Streamlit app
streamlit run ui/streamlit/app.py

The app will open in your browser at http://localhost:8501

📊 Using InsightForge

Chat Interface 💬

• Ask natural language questions about your business data
• Example queries:
  • "What were the total sales last month?"
  • "Which products are performing best?"
  • "Show me sales trends by region"
  • "Compare customer segments"
• Conversation memory maintains context across questions
• AI suggests appropriate visualizations

Dashboard 📊

• Interactive filters (date range, region, product, segment)
• Real-time KPIs and metrics
• Pre-built visualizations:
  • Sales trends over time
  • Product performance rankings
  • Regional analysis
  • Customer segmentation

🧪 Model Evaluation

Run evaluation to assess model performance:

python scripts/evaluate_model.py

Evaluates:

• Grounding: Does the model cite actual data?
• Refusal accuracy: Does it refuse when data is unavailable?
• Intent detection: Does it understand user queries correctly?
• Numeric precision: Are numbers accurate?

Results saved to evaluation_results.csv

🧪 Testing

Run unit tests:

pytest tests/

📖 Data Schema

Expected columns (flexible, adjust as needed):

Column	Type	Description
date	datetime	Transaction or period date
sales_amount	float	Revenue amount
quantity	int	Units sold
product_name	string	Product identifier
product_id	string/int	Product ID
category	string	Product category
region	string	Sales region
customer_id	string/int	Customer identifier
customer_segment	string	Customer type (Enterprise/SMB/Consumer)
customer_age_group	string	Age demographic
customer_gender	string	Gender demographic

See docs/data_dictionary.md for details.

🎓 Project Implementation

This project follows a structured approach covering:

Part 1: AI-Powered BI Assistant

1. ✅ Data preparation and loading
2. ✅ Knowledge base creation (vector store)
3. ✅ LLM application development (analytics functions)
4. ✅ RAG system implementation
5. ✅ Chain prompts and prompt engineering
6. ✅ Memory integration for context awareness

Part 2: LLMOps & User Interface

7. ✅ Model evaluation (QAEvalChain concepts)
8. ✅ Data visualization (multiple chart types)
9. ✅ Streamlit UI (chat + dashboard)

🔧 Configuration

Edit configs/config.yaml to customize:

• Ollama model and host
• Vector store parameters
• Retrieval settings (top_k, chunk_size)
• UI settings

Edit .env for:

• Ollama model name
• API endpoints
• Environment-specific settings

🐛 Troubleshooting

"Ollama connection error"

• Ensure Ollama is running: ollama serve
• Check connection: curl http://localhost:11434
• Verify model is pulled: ollama list

"No data found"

• Check data/raw/ has CSV/Parquet files
• Run python scripts/preprocess_data.py
• Verify data/processed/business_data.parquet exists

"Vector store error"

• Ensure preprocessing completed successfully
• Delete data/vectorstore/ and rebuild
• Check Ollama model is available for embeddings

"Import errors"

• Activate virtual environment
• Reinstall: pip install -r requirements.txt

📈 Extending the Project

• Add more data sources: Extend data_loader.py
• Custom analytics: Add functions to analytics.py
• New visualizations: Extend visualization.py
• Advanced prompts: Modify prompts.py
• Additional LLM providers: Replace Ollama with OpenAI/Anthropic
• Enhanced retrieval: Improve retriever.py with better filtering

📝 License

Built for educational/capstone project purposes.

👥 Contributing

This is a capstone project. For production use, consider:

• Authentication and authorization
• Database integration (vs. file-based data)
• Caching and performance optimization
• Error handling and logging enhancements
• Deployment configuration (Docker, cloud platforms)

📧 Support

For issues or questions, refer to:

• docs/ folder for detailed documentation
• Code comments in source files
• Streamlit "About" page for feature overview

---

InsightForge - Transforming Data into Intelligence 🔮

Built with Python, LangChain, LangGraph, Ollama, and Streamlit