EveryTongue: your language deserves a translator

Train a neural translator for ANY language from a spreadsheet. No ML experience required.

Google Translate supports ~130 languages. Humans speak ~7,000. Thousands of languages, many of them endangered, have zero translation tooling, even when communities have already collected dictionaries and phrase books.

EveryTongue closes that gap: give it a spreadsheet of sentence pairs, get back a working neural translator with a web UI.

pip install everytongue

everytongue train --data pairs.xlsx --src-code spa_Latn --tgt-code kek_Latn
everytongue serve --model ./models/spa-kek

That's it. A spreadsheet in, a translator out.

Architecture

Interactive/exportable version: docs/assets/architecture.html.

The flagship demo: Q'eqchi'

Q'eqchi' is a Mayan language spoken by ~800,000 people in Guatemala and Belize. It is not in Google Translate. It is not in NLLB-200. For machine translation purposes, it barely exists.

The examples/qeqchi folder contains ~660 Spanish ↔ Q'eqchi' sentence pairs. Train your own Q'eqchi' translator in minutes:

everytongue train --data examples/qeqchi/pairs.json \
  --src-col spanish --tgt-col qeqchi \
  --src-code spa_Latn --tgt-code kek_Latn \
  --similar-to spa_Latn

If your language has a spreadsheet like this one, you can do the same today.

Why this works when naive fine-tuning fails

Fine-tuning a translation model on a tiny dataset usually produces garbage (Xk'a'a'a'a'a'..., ask us how we know). EveryTongue bakes in the low-resource recipe:

Problem	What EveryTongue does
Base model never saw your language	Starts from NLLB-200 (200-language multilingual prior), not a bilingual model
Language code doesn't exist	Registers a new language token, warm-starts its embedding from the closest known language (--similar-to)
Tiny dataset overfits instantly	Real train/val/test split, early stopping, best-checkpoint restore
Degenerate repetition at inference	Beam search + no_repeat_ngram_size
BLEU is meaningless for morphologically rich languages	chrF++ as the headline metric
Wasteful fixed-length padding	Dynamic per-batch padding

Commands

Command	What it does
everytongue train	Fine-tune a translator on your sentence pairs
everytongue translate	Translate text, a file, or stdin
everytongue eval	Score a model (chrF++ / BLEU) on held-out data
everytongue serve	Launch a local web UI (pip install 'everytongue[serve]')

Train the reverse direction by swapping the codes:

everytongue train --data pairs.xlsx --src-code kek_Latn --tgt-code spa_Latn --similar-to spa_Latn

Python API

from everytongue import Translator

t = Translator("./models/spa-kek")
t.translate(["Hay una araña en la cabeza del niño"])
# ["Wan jun aj am sa' xjolom li ch'ina'al"]

Data format

Any of CSV / TSV / Excel / JSON / JSONL with two text columns:

spanish	qeqchi
Lo mismo	A'an aj wi'
Araña	Aj am

EveryTongue cleans it for you: strips whitespace, drops empties and duplicates, filters over-long rows, and warns when the dataset is too small to expect good results.

How much data do you need? It trains from as few as 20 pairs, but quality scales with data: a few hundred pairs → useful for words and stock phrases; a few thousand → genuinely usable translations.

Language codes

Use NLLB-200 codes (spa_Latn, eng_Latn, quy_Latn...). If your language isn't in the list, invent a code (ISO 639-3 + script, e.g. kek_Latn for Q'eqchi') and pass --similar-to with the closest language NLLB knows, a related language, or the dominant contact language your speakers also use.

Hardware

Works on CPU, Apple Silicon (MPS), and NVIDIA GPUs. A few hundred pairs fine-tune in minutes on a free Colab T4, and overnight on a laptop CPU is fine too.

Roadmap

• Back-translation augmentation from monolingual corpora
• One-click Colab notebook
• HuggingFace Hub upload (everytongue push) + community model zoo
• Pivot-language data augmentation
• Quantized export for offline/mobile use

Contributing

The most valuable contribution is training a model for a language that has nothing. Open an issue with your language and dataset size, we'll help you get it working, and feature it here.

Language	Pairs	chrF++	Trained by
Q'eqchi' (kek)	656	,	flagship example
your language here

License

MIT, because language tools should belong to the communities that need them.