Use TTA.dev ModelRouterPrimitive for intelligent local/cloud model selection

[theinterneti]

Summary

TTA has an open issue (#294) to validate llama3 8B for the 7,500-token context profile. TTA.dev now ships ModelRouterPrimitive — it automatically routes LLM calls to the right model (local Ollama, Groq, OpenAI, etc.) based on a task profile, with fallback chains. TTA can use this instead of hardcoding model choices per pipeline stage.

What ModelRouterPrimitive provides

from ttadev.primitives.llm import ModelRouterPrimitive, TaskProfile

router = ModelRouterPrimitive(
    tiers=[
        # Try local llama3 first (free, fast, private)
        {"provider": "ollama", "model": "llama3:8b", "max_tokens": 7500},
        # Fallback to Groq if local is unavailable
        {"provider": "groq", "model": "llama3-8b-8192"},
        # Final fallback to cloud
        {"provider": "openai", "model": "gpt-4o-mini"},
    ]
)

# IPA uses a 'narrative' profile — router picks appropriate tier
result = await router.execute(
    TaskProfile(task_type="narrative", token_budget=6000),
    context
)

Key capabilities:

• Automatic tier selection: picks cheapest model that fits the token budget
• Health-aware routing: skips unavailable providers (Ollama down → Groq)
• Per-task profiles: IPA, WBA, NGA can each declare different requirements
• Model benchmarks: built-in benchmark data for quality/cost tradeoffs
• Hardware detection: knows if local GPU can handle a given model

Why this matters for TTA

TTA #294 is trying to manually validate one model config. ModelRouter makes model selection a first-class concern — each pipeline stage declares what it needs and the router handles the rest. When the local model fails validation, the system transparently routes to cloud without code changes.

This also directly addresses the cost/quality tradeoff for TTA: short NPC dialogue → local llama3; complex world reconciliation (#300) → cloud model.

Adoption steps

1. Add ttadev dependency (already declared)
2. Configure router tiers in TTA's config.py or .env
3. Replace hardcoded model= params in IPA, WBA, NGA with TaskProfile calls
4. Monitor routing decisions in Langfuse (Adopt TTA.dev Langfuse integration: trace all LLM calls and agent tool use #360) — traces show which tier was selected

Related

• TTA Local model validation: 7,500-token context profile for llama3 8B #294 local model validation
• TTA Adopt TTA.dev Langfuse integration: trace all LLM calls and agent tool use #360 Langfuse (model selection visible per trace)
• TTA.dev ttadev/primitives/llm/model_router.py
• TTA.dev ttadev/primitives/llm/task_selector.py