Natural Language → Domain-Specific Commands — production-ready framework for transforming natural language (Polish + English) into shell, SQL, Docker, Kubernetes, browser automation, and desktop GUI commands.
pip install nlp2cmd[all]
nlp2cmd cache auto-setup # Setup Playwright browsers
# Single commands
nlp2cmd "uruchom usługę nginx" # → systemctl start nginx
nlp2cmd "znajdź pliki większe niż 100MB" # → find . -type f -size +100MB
# Multi-step browser automation
nlp2cmd -r "otwórz openrouter.ai, wyciągnij klucz API i zapisz do .env"
# Canvas drawing with video recording
nlp2cmd -r "wejdź na jspaint.app i narysuj biedronkę" --video webm
# Web form filling
nlp2cmd -r "otwórz https://example.com/kontakt i wypełnij formularz i wyślij"Complex queries are automatically detected and decomposed into executable action plans:
nlp2cmd -r "otwórz tab w firefox wyciągnij klucz API z OpenRouter i zapisz do .env"
# → domain=multi_step, 7 steps: navigate → check_session → click → wait → extract_key → prompt_secret → save_envThe pipeline routes queries through:
- Cache — instant lookup of previously seen multi-step patterns
- ComplexQueryDetector — regex-based multi-step detection (~0.1ms)
- ActionPlanner — rule-based decomposition for known services (10 API providers)
- KeywordIntentDetector — single-command keyword matching (1615 templates)
| Domain | Examples |
|---|---|
| Shell | find, ls, grep, ps, du, df, tar, chmod, systemctl |
| Docker | container, image, compose, volume, network |
| SQL | SELECT, INSERT, CREATE, JOIN, window functions |
| Kubernetes | kubectl, pods, deployments, services, helm |
| Browser | Playwright automation, form filling, site exploration |
| Git | commit, branch, merge, rebase, stash, tag |
| DevOps | systemctl, Ansible, Terraform, CI/CD, nginx, cron |
| API | curl, httpie, wget, REST, GraphQL |
| FFmpeg | video/audio conversion, streaming, recording |
| Package Mgmt | apt, pip, npm, yarn, snap, flatpak, brew |
| Desktop | Firefox tabs, Thunderbird, xdotool/ydotool, window mgmt |
| Canvas | jspaint.app drawing (shapes, ladybug, text) |
| Remote | SSH, SCP, rsync, tmux, VPN |
| Data | jq, csvkit, awk, sed, sqlite, pandas |
| IoT | Raspberry Pi GPIO, MQTT, sensors |
| RAG | ChromaDB, Qdrant, embeddings, LangChain |
100% accuracy with Qwen-Coder-3B, 24,355× speedup. Template pipeline + pre-warm cache eliminate 31% of queries without LLM.
from nlp2cmd.generation.evolutionary_cache import EvolutionaryCache
cache = EvolutionaryCache()
r = cache.lookup("znajdź pliki PDF większe niż 10MB") # template: ~15ms, cached: ~0.015ms
r = cache.lookup("znajdz pliki PDF wieksze niz 10MB") # typo → similarity hit!Every command result is automatically validated by a local LLM (qwen2.5:3b via Ollama). If the validator marks a result as fail, a cloud LLM (OpenRouter) suggests an improved command and optionally patches patterns.json/templates.json so the pipeline learns from the mistake.
Execute command → stdout/stderr
↓
LLM_VALIDATOR (local, ~0.5s)
query + command + output → pass/fail + score + reason
↓ fail?
LLM_REPAIR (OpenRouter cloud)
full context → improved_command + JSON patches
↓
Retry + update data files
# Example output after command execution:
llm_validator: pass
score: 0.90
reason: Found 1 camera with RTSP on the local network.
model: qwen2.5:3bConfiguration in .env:
LLM_VALIDATOR_ENABLED=true # enabled by default
LLM_VALIDATOR_MODEL=qwen2.5:3b # local Ollama model
LLM_REPAIR_ENABLED=true # cloud repair on fail
LLM_REPAIR_MODEL=qwen/qwen-2.5-coder-32b-instructTest suite: python3 examples/08_llm_validation/test_validator.py — 15 test cases, 100% accuracy with qwen2.5:3b.
Complex multi-step browser automation uses a schema-driven feedback loop — each step is validated, failures are classified, and repairs are escalated until a solution is found.
Step Schema (declarative)
↓
Execute step → Validate result (pre/post)
↓ failed?
Classify failure:
schema_error → wrong selector/URL → Page analysis (DOM scan)
page_state_error → redirect/CAPTCHA/modal → Navigate to correct page
data_error → not logged in / no key → Login flow / create key
handling_error → browser crash / timeout → Retry / escalate
↓
Repair escalation:
1. Rule-based ~0ms (selector alternatives)
2. Page analysis ~50ms (DOM scan for elements)
3. Local LLM ~500ms (qwen2.5:3b diagnosis)
4. Cloud LLM ~2s (OpenRouter 32B repair)
↓
Retry with repaired params (max 5 attempts)
Provider-agnostic: works across HuggingFace, OpenRouter, Anthropic, GitHub, Groq, and any SaaS site without hardcoded URLs — PageAnalyzer dynamically finds API key sections by scanning navigation links.
Test suite: python3 examples/08_llm_validation/test_feedback_loop.py — 19 test cases (failure classification + page analysis + multi-provider), 100% accuracy.
Native Polish NLP with 87%+ accuracy: lemmatization, fuzzy matching, diacritic normalization.
nlp2cmd "zainstaluj vlc" # → sudo apt-get install vlc
nlp2cmd "pokaż procesy zużywające pamięć" # → ps aux --sort=-%mem | head -10
nlp2cmd "znajdź pliki .log starsze niż 2 dni" # → find . -type f -name "*.log" -mtime +2Control desktop applications through natural language — works on both X11 and Wayland:
nlp2cmd -r "otwórz tab w firefox, wyciągnij klucz API z OpenRouter i zapisz do .env"
nlp2cmd -r "wejdź na jspaint.app i narysuj biedronkę" --video webm
nlp2cmd -r "otwórz Thunderbird i sprawdź pocztę"
nlp2cmd -r "zminimalizuj wszystko"Supported API services for key extraction: OpenRouter, Anthropic, OpenAI, Groq, Mistral, DeepSeek, Together, GitHub, HuggingFace, Replicate.
Record entire browser automation sessions with video output:
# Full session recording (WebM format)
nlp2cmd -r "otwórz tab w firefox wyciągnij klucz API z OpenRouter i zapisz do .env" --video webm --execute-web
# Short video clips (3-10 seconds)
nlp2cmd -r "wejdź na jspaint.app i narysuj koło" --video webm --duration 5Demo Recording: nlp2cmd-operouter-token.mp4 - Firefox automation with API key extraction
nlp2cmd -r "otwórz https://example.com/kontakt i wypełnij formularz i wyślij"
nlp2cmd -r "wejdź na strona-docelowa.pl i wypełnij formularz kontaktu"
# → Automatically explores site, finds contact form, fills with .env datanlp2cmd -r "wejdź na https://katalog-stron.pl/firmy, znajdź strony www firm i zapisz do firmy.csv"
# → Navigates profiles, extracts external websites, saves to CSVQuery → [Cache] → [ComplexDetector] → [ActionPlanner] → [KeywordDetector] → [TemplateGenerator]
↓ ↓
ActionPlan (multi-step) SingleCommand
↓ ↓
PipelineRunner ←────────────┘
├── ShellExecutionMixin
├── BrowserExecutionMixin
├── DesktopExecutionMixin
└── PlanExecutionMixin
PipelineRunner is split into 4 mixins (~4400 LOC total) for maintainability:
| Module | Lines | Responsibility |
|---|---|---|
pipeline_runner.py |
170 | Core class, mixin composition |
pipeline_runner_shell.py |
217 | Shell execution, safety policies |
pipeline_runner_browser.py |
1561 | DOM/DQL, multi-action browser automation |
pipeline_runner_desktop.py |
389 | xdotool/ydotool/wmctrl desktop control |
pipeline_runner_plans.py |
2277 | Multi-step ActionPlan execution |
# Full installation
pip install nlp2cmd[all]
# Specific components
pip install nlp2cmd[browser,nlp] # Web automation + Polish NLP
pip install nlp2cmd[sql,shell] # Database + system commands
# Desktop automation tools
nlp2cmd-install-desktop
# or: make install-desktop
# Setup Playwright browsers
nlp2cmd cache auto-setup# Generate command
nlp2cmd "pokaż pliki użytkownika" # → find $HOME -type f
nlp2cmd --dsl sql "pokaż użytkowników" # → SELECT * FROM users
# Execute immediately (run mode)
nlp2cmd -r "list files" --auto-confirm
# Multi-step with video
nlp2cmd -r "wejdź na jspaint.app i narysuj biedronkę" --video webm
# Full session recording
nlp2cmd -r "otwórz tab w firefox wyciągnij klucz API z OpenRouter i zapisz do .env" --video webm --execute-web
# Interactive mode
nlp2cmd --interactive
# Web automation
nlp2cmd web-schema extract https://example.com
nlp2cmd web-schema history --stats
# Cache management
nlp2cmd cache info
nlp2cmd cache auto-setup
nlp2cmd cache full-clear --yes
# Service mode (HTTP API)
nlp2cmd service --host 0.0.0.0 --port 8000
# Diagnostics
nlp2cmd doctor --fix
nlp2cmd --show-decision-tree "zapytanie"
nlp2cmd --show-schemafrom nlp2cmd.generation.pipeline import RuleBasedPipeline
pipeline = RuleBasedPipeline()
# Single command
result = pipeline.process("pokaż pliki w katalogu")
print(result.command) # ls -la .
# Multi-step (auto-detected)
result = pipeline.process("otwórz openrouter.ai, wyciągnij klucz API i zapisz do .env")
print(result.domain) # multi_step
print(result.action_plan) # ActionPlan with 7 steps# Execute action plan
from nlp2cmd.pipeline_runner import PipelineRunner
runner = PipelineRunner()
result = runner.execute_action_plan(plan, dry_run=False)pytest tests/ -v # All 1543+ tests
pytest tests/unit/ -v # Unit tests
pytest tests/e2e/ -v # End-to-end tests
pytest --cov=nlp2cmd --cov-report=html # With coverage| Document | Description |
|---|---|
| Project Structure | Architecture and module organization |
| Installation Guide | Setup instructions |
| User Guide | Complete usage tutorial |
| CLI Reference | CLI documentation |
| Python API | Python API guide |
| Examples Guide | Examples overview |
| Keyword Detection | Intent detection pipeline |
| Web Schema Guide | Browser automation |
| Cache Management | Caching system |
| Desktop Automation | Desktop GUI control |
| Canvas Drawing | jspaint.app drawing |
| Firefox Sessions | Session injection |
| Evolutionary Cache | Self-learning cache |
| Stream Protocols | SSH, RTSP, VNC, etc. |
| Service Mode | HTTP API service |
| Contributing | Development guidelines |
Apache License 2.0 - see LICENSE for details.
Created by Tom Sapletta - tom@sapletta.com