LokBhasha

Understand Marathi government documents in clear English, then localize them into other Indian languages with Lingo.dev.

LokBhasha is a Cloudflare-hosted document localization workflow for public-service communication. It accepts Marathi circulars and notices, detects government terminology from a curated 19k glossary, produces canonical English through Lingo.dev, and localizes that English into user-selected Indian languages.

Experience	Link
Demo video	YouTube walkthrough
Live app	lokbhasha.pages.dev
Live API health check	lokbhasha-api.dpratik3005.workers.dev/health
Current TestSprite report	testsprite-mcp-test-report.md

Table of Contents

• Why LokBhasha
• Product Flow
• Screens
• TestSprite-Guided Quality Work
• Architecture
• Glossary and Lingo Model
• Local Setup
• Public API
• Cloudflare Deployment
• Repository Notes

Why LokBhasha

Government documents often reach citizens in language that is formal, domain-heavy, and slow to interpret. LokBhasha is designed to preserve official terminology where accuracy matters while making the document understandable and portable across languages.

Principle	Implementation
Preserve official terms	Cloudflare D1 detects matches from the curated government glossary before localization.
Keep translation authoritative	Lingo.dev handles Marathi to English and English to Indian-language localization.
Avoid noisy output	Users choose when to generate translations, explanations, and action items.
Make quality visible	The UI shows glossary hits, setup status, quality metadata, and baseline comparison details.

Product Flow

1. Upload a PDF or paste Marathi text.
2. Prepare PDFs in the browser with embedded-text extraction and OCR fallback.
3. Detect government glossary matches from the Cloudflare D1 term store.
4. Localize Marathi into canonical English through Lingo.dev.
5. Generate optional Indian-language outputs, explanations, or action items on demand.
6. Review glossary context, Lingo setup context, quality context, and side-by-side results.

Screens

Live sample flow	Result view

TestSprite-Guided Quality Work

TestSprite was used as the primary product-testing driver for the Cloudflare version of LokBhasha. The workflow covered live sample flow, PDF analysis, pasted-text analysis, result persistence, optional localized outputs, details-page quality panels, glossary surfaces, and validation states.

Evidence	Result	What it improved
Round 1 baseline run	5/6 passed	Exposed an upload-flow reliability gap in a critical user path.
Round 2 validation run	7/7 passed	Confirmed the upload and optional-output flow fixes after Cloudflare migration polish.
Final full-suite run (TC001-TC020)	20/20 passed	Validated end-to-end stability for current frontend + backend flows.
Current report	Open report	Consolidates generated tests and latest run outcomes.

This testing cycle directly drove product fixes in upload flow handling, result-route and empty-state behavior, and baseline/optional-output flow reliability.

Run the TestSprite MCP server

Set TESTSPRITE_API_KEY in the MCP server environment, then start the server:

npx @testsprite/testsprite-mcp@latest server

Architecture

Marathi PDF or text
        |
        v
Next.js static frontend on Cloudflare Pages
        |
        v
Cloudflare Worker API
        |
        +--> Cloudflare D1 glossary detection
        |
        +--> Lingo.dev localization
        |
        v
Canonical English, glossary context, optional translations, explanations, and actions

Layer	Responsibility
frontend/	Next.js UI exported as static assets for Cloudflare Pages.
cloudflare-api/	Cloudflare Worker API for analyze, enrich, glossary status, setup, and quality routes.
cloudflare-api/migrations/0001_glossary.sql	D1 seed generated from the active government glossary.
dict/19k.json	Curated government glossary source.
analyze/	Shared localization logic and legacy Node harness kept during migration.
backend/	Legacy extraction reference kept only until Cloudflare migration cleanup is complete.

Glossary and Lingo Model

LokBhasha uses a government-specific glossary instead of a generic large dictionary.

Concern	Owner
Active glossary source	dict/19k.json
Runtime term detection	Cloudflare D1
Translation authority	Lingo.dev
Current glossary role	Marathi terminology detection and UI context
Current Lingo path	Marathi to English, then English to selected Indian languages

This separation is intentional: D1 quickly finds important terms inside the document, while Lingo.dev performs the localization work.

Local Setup

Prerequisites

Requirement	Notes
Node.js	Version 22+
npm	Version 10+
Cloudflare	Account access and Wrangler authentication for deploys
Lingo.dev	API key and engine ID
TestSprite	API key for MCP-based testing

Install dependencies

cd cloudflare-api
npm ci

cd ../frontend
npm ci

cd ../analyze
npm ci

Configure environment

Use .env.example and cloudflare-api/.dev.vars.example as references.

Variable	Used by
NEXT_PUBLIC_API_BASE_URL	Frontend API routing
LINGODOTDEV_API_KEY	Worker localization calls
LINGODOTDEV_ENGINE_ID	Lingo.dev engine selection
LINGODOTDEV_TARGET_LOCALES	Optional Indian-language output list
GLOSSARY_LAST_SYNC_AT	Glossary status surface
GLOSSARY_REMOTE_TERM_COUNT	Glossary status surface
TESTSPRITE_API_KEY	TestSprite MCP server

Build the D1 glossary seed

cd cloudflare-api
npm run build:d1

This writes cloudflare-api/migrations/0001_glossary.sql.

Run Locally

Start the Worker API:

cd cloudflare-api
npm run build:d1
npx wrangler dev

Start the frontend in a second terminal:

cd frontend
npm run dev

Default local URLs:

Service	URL
Frontend	http://localhost:3000
Worker API	http://127.0.0.1:8787

Public API

Endpoint	Purpose
GET /health	Worker health check.
POST /analyze	Accepts Marathi text prepared by the frontend, detects glossary hits, localizes to canonical English through Lingo.dev, and returns the core result.
POST /enrich	Generates optional outputs the user asks for, such as selected target languages or explanation text.
GET /glossary-status	Returns glossary package, D1 readiness, and last known Lingo sync metadata.
GET /lingo-setup	Returns a read-only summary of the active Lingo setup.
GET /quality-summary	Returns glossary and setup quality metadata used by the result UI.
POST /quality/baseline-compare	Compares the current canonical result with the same request without glossary hints.

Cloudflare Deployment

The GitHub deploy workflow in .github/workflows/deploy.yml is shaped around Cloudflare Pages, Cloudflare Workers, and D1.

Job	Action
Deploy Cloudflare API	Install Worker dependencies, build the D1 seed, apply the remote D1 migration, deploy the Worker.
Deploy Cloudflare Pages Frontend	Install frontend dependencies, build the static Next.js export, deploy frontend/out.
Deployment Summary	Report job-level outcomes.

Deploy the Worker API manually

cd cloudflare-api
npm ci
npm run build:d1
npx wrangler d1 execute GLOSSARY_DB --remote --file migrations/0001_glossary.sql
npx wrangler deploy

Deploy the frontend manually

cd frontend
npm ci
npm run build
npx wrangler pages deploy out --project-name lokbhasha

Current public Cloudflare endpoints:

Service	URL
Frontend	https://lokbhasha.pages.dev
API	https://lokbhasha-api.dpratik3005.workers.dev

Repository Notes

• master is the shipped branch for this repository.
• Feature work should stay in isolated git worktrees before review and merge.
• Glossary management remains Lingo-authoritative; D1 is the runtime detection store.
• analyze/ and backend/ remain in the repo as migration support until the Cloudflare cutover cleanup is complete.