Skip to content

Implement Tier 2 LLM-based intent routing #22

@mingshl

Description

@mingshl

Originally created by @mingshl in mingshl/opensearch-agent-server#19

Description

Implement LLM-based intent detection in the router to classify user messages when page context alone is insufficient.

Current state

Currently the router only does:

  • Tier 1: Exact page context match
  • Tier 3: Fallback agent

Desired behavior

  • Tier 2: When page context doesn't match any agent, analyze the user's message with a lightweight LLM call to determine which agent is best suited
  • Example: user asks about search relevance from the home page → route to ART agent

Considerations

  • Should be fast (small model, cached decisions)
  • Fallback gracefully if LLM call fails
  • Configurable: can be disabled in favor of deterministic routing only

References

  • Router: src/orchestrator/router.py
  • Registry: src/orchestrator/registry.py

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions