Skip to content

[Bug]: Semantic Cache guardrail can be saved/deployed without required embedding config, crashing the entire proxy #3417

Description

@randilt

Please select the area the issue is related to

AI Workspace, Gateway

Please select the aspect the issue is related to

Aspect/UX (User experience, flows, usability, clarity), Aspect/Configuration (Config files, settings, env vars, defaults)

Description

The Semantic Cache guardrail's Console UI only exposes similarityThreshold, jsonPath, and streamingJsonPath.

Per semantic cache policy's own docs (https://wso2.com/api-platform/policy-hub/policies/semantic-cache), this policy also requires embedding provider / vector DB settings (embedding_provider_dimension, embedding_provider_api_key, etc.) set in the gateway's config.toml — but the UI never mentions this, and lets you save/deploy without it, with no warning.

At runtime, the policy fails to build (missing config keys), and this crashes policy chain construction for the entire proxy — not just caching-related traffic. Every route/method on the proxy returns a generic 500 Internal Server Error with no useful detail; the real cause is only visible in gateway logs.

Steps to Reproduce

  1. Create an App LLM Proxy on a self-hosted gateway with no embedding provider / vector DB config in config.toml.
  2. Guardrails & Policies → Add → Semantic Cache → leave all fields at default → Save.
  3. Deploy — succeeds with no errors/warnings.
  4. Invoke any resource on the proxy:
   curl -k -X POST "https://<gateway-host>:8443/<proxy-context>/<any-path>" \
     -H "X-API-Key: <valid proxy key>" -H "Content-Type: application/json" -d '{...}'

{"error":"Internal Server Error"}, HTTP 500, on every route/method.
5. Gateway runtime logs show:

   level=ERROR msg="Failed to build policy chain for route, skipping" ... error="... failed to resolve config for key \"embeddingDimension\": ... CEL evaluation error: no such key: embedding_provider_dimension"
   level=ERROR msg="Policy chain not found for route, returning 500" route=POST|/<proxy-context>/*|*

repeated for every route/method on the proxy.

Severity Level of the Issue

Severity/Blocker (Core functionality is broken and consumer is blocked. Need immediate attention)

Environment Details (with versions)

  • ghcr.io/wso2/api-platform/ai-workspace:1.1.0-SNAPSHOT
  • ghcr.io/wso2/api-platform/gateway-runtime:1.1.0
  • ghcr.io/wso2/api-platform/gateway-controller:1.1.0
  • Self-hosted gateway, local Docker setup, macOS host
  • curl 8.7.1

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions