A sophisticated multi-expert AI system that orchestrates multiple LLM providers (OpenAI, Anthropic, Google) through a three-stage council process to deliver comprehensive, well-reasoned answers with domain-specific intelligence.
Live Demo | Healthcare | Finance
┌─────────────────────────────────────────────────────────────┐
│ User Query │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 1: Divergent Answers (Parallel) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ OpenAI │ │ Anthropic │ │ Google Gemini │ │
│ │ GPT-4 │ │ Claude │ │ Pro │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 2: Peer Review & Ranking │
│ Each expert reviews and ranks all answers │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 3: Final Synthesis │
│ Best-ranked expert synthesizes consensus │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Final Response │
│ With citations (ICD-10 / GAAP / IFRS) │
└─────────────────────────────────────────────────────────────┘
- Three-Stage Council Process: Divergent answers → Peer review → Final synthesis
- Domain-Specific Intelligence: Healthcare (ICD-10 codes) & Finance (GAAP/IFRS standards)
- Semantic Search: Vector embeddings for intelligent knowledge retrieval
- Real-Time Streaming: Watch the council deliberate in real-time
- Admin Dashboard: Analytics, performance tracking, and query history
- Redis Caching: High-performance caching for faster responses
- Rate Limiting: Protection against abuse with configurable limits
| Category | Technology |
|---|---|
| Framework | Next.js 15 (App Router) |
| Language | TypeScript 5 |
| Database | PostgreSQL (Neon) |
| ORM | Drizzle ORM |
| Cache | Redis (Upstash) |
| LLM Providers | OpenAI, Anthropic, Google AI |
| Vector Search | OpenAI Embeddings |
| Styling | Tailwind CSS |
| Deployment | Vercel |
- Node.js 18+
- Docker (for local development)
- API keys for LLM providers
- Clone the repository
git clone https://github.com/middesurya/llm-council.git
cd llm-council- Install dependencies
npm install- Start local services (PostgreSQL + Redis)
npm run services:start- Configure environment
cp .env.example .env
# Edit .env and add your API keys- Run database migrations
npm run db:migrate- Generate vector embeddings (optional)
npx tsx scripts/populate-embeddings.ts- Start the development server
npm run dev- Open your browser Navigate to http://localhost:3000
- Go to neon.tech and create a free account
- Create a new project
- Copy the connection string (looks like
postgresql://user:pass@ep-xxx.us-east-2.aws.neon.tech/dbname?sslmode=require)
- Go to upstash.com and create a free account
- Create a new Redis database
- Copy the connection URL (use the one that starts with
rediss://)
- Click the "Deploy with Vercel" button above, or:
npm i -g vercel
vercel-
Add environment variables in Vercel dashboard:
DATABASE_URL- Your Neon PostgreSQL connection stringREDIS_URL- Your Upstash Redis URLADMIN_PASSWORD- A secure admin passwordOPENAI_API_KEY- Your OpenAI API keyANTHROPIC_API_KEY- Your Anthropic API keyGOOGLE_API_KEY- Your Google AI API keyENABLE_CACHE- Set totrue
-
Run database setup:
npx tsx scripts/setup-db.tsSee .env.production.example for all production configuration options.
| Variable | Required | Description |
|---|---|---|
DATABASE_URL |
Yes | PostgreSQL connection string |
REDIS_URL |
No | Redis connection string (caching) |
ADMIN_PASSWORD |
Yes | Admin dashboard password |
OPENAI_API_KEY |
Yes* | OpenAI API key |
ANTHROPIC_API_KEY |
Yes* | Anthropic API key |
GOOGLE_API_KEY |
Yes* | Google AI API key |
*At least one LLM provider key is required.
| Endpoint | Method | Description |
|---|---|---|
/api/health |
GET | Health check (DB + Redis status) |
/api/council/query |
POST | Submit query to council |
/api/council/query/stream |
POST | Submit query with streaming |
/api/status |
GET | LLM provider status |
/api/admin/data/dashboard |
GET | Dashboard analytics |
{
"status": "healthy",
"timestamp": "2024-01-01T00:00:00.000Z",
"version": "0.1.0",
"uptime": 3600,
"checks": {
"database": { "status": "up", "latency": 5 },
"redis": { "status": "up", "latency": 2 }
}
}llm-council/
├── app/ # Next.js app directory
│ ├── healthcare/ # Healthcare domain UI
│ ├── finance/ # Finance domain UI
│ ├── admin/ # Admin dashboard
│ └── api/ # API routes
│ ├── health/ # Health check endpoint
│ ├── council/ # Council query endpoints
│ └── admin/ # Admin API routes
├── src/
│ ├── lib/
│ │ ├── llm/ # LLM orchestration
│ │ ├── knowledge/ # Domain knowledge bases
│ │ ├── cache/ # Redis caching layer
│ │ ├── db/ # Database schema & client
│ │ ├── security/ # Rate limiting, validation
│ │ └── observability/ # Logging, metrics
│ └── config/
│ └── domains.ts # Domain-specific prompts
├── scripts/ # Utility scripts
│ ├── setup-db.ts # Database setup
│ ├── populate-embeddings.ts # Generate embeddings
│ └── migrate.ts # Run migrations
└── docs/ # Documentation
npm run dev # Start development server
npm run build # Build for production
npm run start # Start production server
npm run lint # Run ESLint
npm run db:migrate # Run database migrations
npm run db:generate # Generate migration files
npm run services:start # Start Docker services
npm run services:stop # Stop Docker servicesAsk medical questions and receive responses with ICD-10 code citations:
Query: "What causes high blood sugar?"
Response includes:
- Type 2 Diabetes [ICD-10: E11]
- Hyperglycemia [ICD-10: R73.9]
- Red flags for diabetic ketoacidosis
- Source: ADA Standards of Care
Ask accounting questions with GAAP/IFRS standard citations:
Query: "How do I recognize revenue?"
Response includes:
- ASC 606 / IFRS 15 five-step model
- Section references (ASC 606-10-25-1)
- Implementation guidance
- Source: FASB Accounting Standards Codification
- Overall Citation Accuracy: 73.9%
- Finance Domain Accuracy: 100%
- Healthcare Domain Accuracy: 61.1%
- Source URL Inclusion: 87.5%
- Cache Hit Rate: 80%+ (with Redis enabled)
- Phase 1: Core Council System
- Phase 2: Domain-Specific Intelligence
- Phase 3: Semantic Search & Citations
- Phase 4: Admin Dashboard & Analytics
- Phase 5: Redis Caching & Rate Limiting
- Phase 6: User Accounts & History
- Phase 7: A/B Testing Framework
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
- Built with Claude Code - Anthropic's AI CLI tool
- Inspired by Andrej Karpathy's LLM Council
Disclaimer: This system provides educational information only. For medical, financial, or legal advice, please consult qualified professionals.