Skip to content

server: accept agent cache hints#82

Open
bittoby wants to merge 1 commit into
GeniePod:mainfrom
bittoby:dynamo-152-runtime-agent-hints
Open

server: accept agent cache hints#82
bittoby wants to merge 1 commit into
GeniePod:mainfrom
bittoby:dynamo-152-runtime-agent-hints

Conversation

@bittoby
Copy link
Copy Markdown

@bittoby bittoby commented May 23, 2026

Refs GeniePod/genie-claw#152.

Summary

  • parse top-level conversation_id plus nvext.agent_hints.session_id
  • accept priority, OSL, speculative-prefill, and ephemeral cache TTL metadata
  • sanitize session ids before using them for persistent KV sessions
  • echo accepted non-streaming hint metadata under jetson.agent_hints

Verification

  • git diff --check origin/main..HEAD
  • cmake -S . -B /tmp/genie-ai-runtime-check -DJLLM_BUILD_SERVER=ON could not complete on this Mac host because CUDA/nvcc is not installed: Failed to find nvcc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant