Rosetta

Rosetta analyzes a code repository — a local path or a GitHub URL — and generates accurate, structured documentation from it. Instead of asking an LLM to read raw source files, Rosetta first performs full static analysis (language detection, AST parsing, cross-file call-graph construction, dependency extraction) and only then hands the structured result to an LLM, whose job is limited to turning verified facts into readable prose. The goal is a documentation tool that understands a codebase rather than one that guesses at it.

Features

• Multi-language static analysis — Python source is parsed with the ast module; JavaScript/TypeScript/TSX is parsed with Tree-sitter. Both produce the same structured shape (classes, functions, imports, API endpoints, constants), so the rest of the pipeline is language-agnostic.
• Cross-file call graph — resolves function calls across file boundaries using import analysis, producing a real dependency graph (which function calls which, and from where) rather than a flat file list.
• Automatic diagram generation — PlantUML component diagrams and Mermaid flowcharts are generated directly from the parsed repository structure and call graph.
• Framework-aware extraction — detects FastAPI/Flask-style route decorators and SQLAlchemy/Pydantic/Django-style model base classes automatically, without configuration.
• Pluggable LLM backends — generation can run against a local Ollama model (e.g. llama3.2:3b) for a fully offline setup, or against the Gemini API for stronger synthesis on larger repositories.
• Job history and persistence — every parse/generate run is tracked as a Project and GenerationJob in a database, with the generated output stored as a GeneratedDoc and retrievable later.
• GitHub repository support — accepts a GitHub URL directly; clones it via GitPython, with an automatic fallback to downloading the repository as a zip archive if git clone is blocked or redirected by network restrictions.

How It Works

Client → API layer (FastAPI) → Static analysis engine → Prompt builder → LLM provider → Stored document

1. API layer (routes.py) resolves the input — a local path or a GitHub URL — and hands it to the analysis engine.
2. Static analysis engine (repository_parser.py, orchestrating everything under parser/) walks the repository, detects languages, builds the directory tree, extracts project metadata (requirements.txt, pyproject.toml, package.json), parses every source file, builds the cross-file call graph, and generates PlantUML/Mermaid diagrams. All of this runs without any LLM involvement.
3. Prompt builder (PromptBuilder in services.py) formats the structured analysis into a filled prompt template, ranking files by importance so the most relevant parts of a large repository are prioritized.
4. LLM provider (LLMService in services.py) sends the prompt to Ollama or Gemini and returns the generated Markdown.
5. The result is persisted as a GeneratedDoc tied to a GenerationJob, and returned to the client.

Tech Stack

Backend

• Python, FastAPI, Uvicorn
• Pydantic (request/response validation)
• SQLAlchemy (persistence layer)
• GitPython + requests (repository cloning/download)

Frontend

• React, TypeScript
• Tailwind CSS
• Axios
• react-markdown (rendering generated documentation)

Static Analysis

• Python ast module (Python source parsing)
• Tree-sitter (tree-sitter, tree-sitter-languages) for JavaScript/TypeScript/TSX parsing

AI / LLM

• Ollama (local inference, default model llama3.2:3b)
• Google Gemini API (gemini-2.5-flash)

Project Structure

rosetta/
├── backend/          FastAPI application: routes, config, LLM + prompt services
│   ├── config.py
│   ├── main.py
│   ├── routes.py
│   └── services.py
├── frontend/         React + TypeScript client
│   ├── public/
│   └── src/
├── models/           SQLAlchemy database models and engine setup
│   ├── database.py
│   └── db_models.py
├── parser/           The static analysis engine
│   ├── ast_parser.py          Python AST parsing
│   ├── js_ts_parser.py        Tree-sitter based JS/TS/TSX parsing
│   ├── call_graph.py          Cross-file call graph construction
│   ├── diagram_generator.py   PlantUML + Mermaid generation
│   ├── language_detector.py   Language detection by extension/shebang
│   ├── tree_generator.py      Directory tree + repo summary
│   ├── repository_parser.py   Orchestrates the full analysis pipeline
│   └── tests/                 Parser unit tests and fixtures
├── prompts/          LLM prompt templates
├── generate.py       CLI entry point for local generation
└── requirements.txt

Prerequisites

• Python 3.x
• Node.js (for the frontend)
• Ollama running locally or a Gemini API key, for documentation generation

Installation

Backend

git clone https://github.com/<your-username>/rosetta.git
cd rosetta
python -m pip install -r requirements.txt

Frontend

cd frontend
npm install

Configuration

Create a .env file in the project root:

# Database
DATABASE_URL=sqlite:///./rosetta.db

# Ollama (local LLM)
OLLAMA_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2:3b
OLLAMA_NUM_CTX=8192

# Gemini (optional, for stronger generation)
GEMINI_API_KEY=your-api-key-here

# App
APP_HOST=0.0.0.0
APP_PORT=8000
DEBUG=false
DOCS_OUTPUT_DIR=rosetta_docs

If DATABASE_URL is unreachable or misconfigured, the backend automatically falls back to a local SQLite database.

Usage

Start the backend

uvicorn main:app --host 0.0.0.0 --port 8000

Start the frontend

cd frontend
npm start

Or generate documentation directly from the CLI, no server required:

python generate.py /path/to/some/repo

API Reference

Method	Path	Handler	Purpose
GET	/	root()	Health check / API status
POST	/parse	parse_repo()	Run static analysis on a repository and store it as a Project
POST	/generate	generate_docs()	Run full analysis + LLM generation, returns a Markdown document
GET	/jobs/{job_id}	get_job_status()	Check the status of a generation job
GET	/docs/{job_id}	get_generated_doc()	Retrieve the generated document for a completed job
GET	/projects	list_projects()	List all previously analyzed projects
GET	/history	get_history()	List past generation jobs
DELETE	/history/{job_id}	delete_history_item()	Delete a single job from history
DELETE	/history	delete_history()	Clear all generation history
DELETE	/projects/{project_id}	delete_project()	Delete a project and its associated jobs/docs

Data Model

Two layers of models exist in the codebase, serving different purposes:

• Request schemas (ParseRequest, GenerateRequest in routes.py) — Pydantic models validating incoming API request bodies. Not persisted.
• Persisted models (models/db_models.py) — SQLAlchemy ORM models stored in the database:
  • Project — a repository that has been analyzed
  • GenerationJob — a single parse/generate run against a Project
  • GeneratedDoc — the resulting Markdown output for a completed job

Testing

# Parser tests (Python + JS/TS parsing)
python -m pytest

# Frontend tests
cd frontend
npm test

Roadmap

The static analysis engine and knowledge graph are designed as the foundation for a broader goal: an interactive repository explorer where documentation is one output among several, alongside a visual, clickable call graph and per-function relationship views. The parsing and graph-building pipeline described above already produces the data needed for this; the interactive UI is in active development.