From 145f07aa37514e656c7c41b6fded685026a07cc5 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 21:44:15 +0000 Subject: [PATCH 1/4] Update stacklok/toolhive to v0.35.0 Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- .github/upstream-projects.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/upstream-projects.yaml b/.github/upstream-projects.yaml index 2b47be52..c0b36c46 100644 --- a/.github/upstream-projects.yaml +++ b/.github/upstream-projects.yaml @@ -44,7 +44,7 @@ projects: - id: toolhive repo: stacklok/toolhive - version: v0.34.0 + version: v0.35.0 # toolhive is a monorepo covering the CLI, the Kubernetes # operator, and the vMCP gateway. It also introduces cross- # cutting features that land in concepts/, integrations/, From 4af271672b74288e5588cc17688febd7408c5f61 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 13 Jul 2026 21:45:16 +0000 Subject: [PATCH 2/4] Refresh reference assets for toolhive v0.35.0 --- static/api-specs/toolhive-api.yaml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/static/api-specs/toolhive-api.yaml b/static/api-specs/toolhive-api.yaml index b70a631a..cc1db034 100644 --- a/static/api-specs/toolhive-api.yaml +++ b/static/api-specs/toolhive-api.yaml @@ -498,6 +498,13 @@ components: $ref: '#/components/schemas/github_com_stacklok_toolhive_pkg_authserver.DCRUpstreamConfig' identity_from_token: $ref: '#/components/schemas/github_com_stacklok_toolhive_pkg_authserver.IdentityFromTokenRunConfig' + insecure_allow_http: + description: |- + InsecureAllowHTTP permits plain-HTTP authorization and token endpoint URLs + for this upstream. Only for in-cluster development environments (e.g. an + OAuth2 provider served over HTTP in a kind cluster) where TLS is not + available. Never set this in production. + type: boolean redirect_uri: description: |- RedirectURI is the callback URL where the upstream IDP will redirect after authentication. @@ -552,6 +559,13 @@ components: ClientSecretFile is the path to a file containing the OAuth 2.0 client secret. Mutually exclusive with ClientSecretEnvVar. Optional for public clients using PKCE. type: string + insecure_allow_http: + description: |- + InsecureAllowHTTP permits a plain-HTTP issuer URL and HTTP discovery + endpoints for this upstream. Only for in-cluster development environments + (e.g. Dex served over HTTP in a kind cluster) where TLS is not available. + Never set this in production. + type: boolean issuer_url: description: |- IssuerURL is the OIDC issuer URL for automatic endpoint discovery. From 7f5c8b438220296a57af2d046cc743dcd3aacc8a Mon Sep 17 00:00:00 2001 From: "claude[bot]" <41898282+claude[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 21:54:40 +0000 Subject: [PATCH 3/4] Document rate limit observability metrics Adds the three new OpenTelemetry metrics (decisions, Redis errors, and Redis Lua check latency) introduced in toolhive v0.35.0. Covers attribute sets, counting semantics, and example PromQL on the rate limiting guide, with cross-references from the vMCP telemetry guide and the observability concept page. Co-authored-by: Unknown --- docs/toolhive/concepts/observability.mdx | 9 +++ docs/toolhive/guides-k8s/rate-limiting.mdx | 61 +++++++++++++++++++ .../guides-vmcp/telemetry-and-metrics.mdx | 18 ++++++ 3 files changed, 88 insertions(+) diff --git a/docs/toolhive/concepts/observability.mdx b/docs/toolhive/concepts/observability.mdx index c8f987c7..ddae4914 100644 --- a/docs/toolhive/concepts/observability.mdx +++ b/docs/toolhive/concepts/observability.mdx @@ -227,6 +227,15 @@ monitoring backend operations, workflow executions, and optimizer performance. For details, see the [vMCP telemetry guide](../guides-vmcp/telemetry-and-metrics.mdx). +### Rate limit metrics + +When [rate limiting](../guides-k8s/rate-limiting.mdx) is configured on an +MCPServer or VirtualMCPServer, ToolHive emits per-bucket decision, Redis error, +and Redis Lua check latency metrics so you can watch for rejections and detect +fail-open periods when Redis is unreachable. See +[Observe rate limit activity](../guides-k8s/rate-limiting.mdx#observe-rate-limit-activity) +for metric names, attributes, and example queries. + ## Trace context propagation ToolHive supports two methods of trace context propagation: diff --git a/docs/toolhive/guides-k8s/rate-limiting.mdx b/docs/toolhive/guides-k8s/rate-limiting.mdx index 1b2f2112..cffdbdd4 100644 --- a/docs/toolhive/guides-k8s/rate-limiting.mdx +++ b/docs/toolhive/guides-k8s/rate-limiting.mdx @@ -234,6 +234,65 @@ A request must pass both the server-level vMCP limit and the per-tool limit (if defined). Limits apply to the vMCP aggregator and are independent from any limits configured on the backend MCPServers it routes to. +## Observe rate limit activity + +When telemetry is enabled on the MCPServer or VirtualMCPServer (via a +[`MCPTelemetryConfig`](./telemetry-and-metrics.mdx) referenced by +`telemetryConfigRef`), the rate limiter emits OpenTelemetry metrics for each +Redis bucket check. Use them to watch for rejections, spot Redis health issues, +and track how long the atomic Lua check takes. + +All three metrics carry `namespace` and `server` attributes identifying the +resource that owns the bucket. Prometheus appends `_total` to counter names and +`_seconds` to the latency histogram (with the standard `_bucket`, `_sum`, and +`_count` series suffixes). + +| Metric | Type | Additional attributes | +| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------- | +| `toolhive_rate_limit_decisions` | Counter | `decision` (`allowed` or `rejected`), `scope` (`shared` or `per_user`), `operation_type` (`server` or `tool`) | +| `toolhive_rate_limit_redis_errors` | Counter | `error_type` (`timeout`, `connection`, `auth`, or `other`) | +| `toolhive_rate_limit_check_latency` | Histogram | (none) | + +Counting semantics for `toolhive_rate_limit_decisions`: + +- An **allowed** request increments the counter once for every applicable + bucket. For example, a request that satisfies both a shared and a per-user + bucket increments the counter twice. +- A **rejected** request increments only for the first bucket rejected by the + atomic Redis check, so a single rejection is never double-counted across + scopes. +- Requests with no applicable bucket do not increment the counter. This includes + discovery methods that pass through unconditionally (`tools/list`, + `prompts/list`, `initialize`, `ping`) and unauthenticated requests against a + server that only has per-user buckets configured. + +The `toolhive_rate_limit_check_latency` histogram records the duration of each +Redis Lua call, whether the call succeeds or fails. Rate limit enforcement +[fails open](#how-rate-limiting-works) when Redis is unreachable, so watch +`toolhive_rate_limit_redis_errors` alongside decisions to catch periods where +limits stopped applying. + +Example PromQL queries against a Prometheus scrape of the server's `/metrics` +endpoint: + +```promql +# Rejection rate across all buckets, by server +sum by (namespace, server) ( + rate(toolhive_rate_limit_decisions_total{decision="rejected"}[5m]) +) + +# 95th-percentile Redis check latency +histogram_quantile( + 0.95, + sum by (le) (rate(toolhive_rate_limit_check_latency_seconds_bucket[5m])) +) + +# Redis errors by class. Non-zero timeout or connection rates warrant attention. +sum by (error_type) ( + rate(toolhive_rate_limit_redis_errors_total[5m]) +) +``` + :::info[Per-tool limits with the optimizer] When the [tool discovery optimizer](../guides-vmcp/optimizer.mdx) is enabled, @@ -281,6 +340,8 @@ spec: ## Next steps +- [Telemetry and metrics](./telemetry-and-metrics.mdx) to enable OpenTelemetry + metrics collection for the metrics above - [Token exchange](./token-exchange-k8s.mdx) to configure token exchange for upstream service authentication - [CRD reference](../reference/crds/index.mdx) for complete field definitions diff --git a/docs/toolhive/guides-vmcp/telemetry-and-metrics.mdx b/docs/toolhive/guides-vmcp/telemetry-and-metrics.mdx index 5e28c91d..2d38093b 100644 --- a/docs/toolhive/guides-vmcp/telemetry-and-metrics.mdx +++ b/docs/toolhive/guides-vmcp/telemetry-and-metrics.mdx @@ -167,6 +167,24 @@ These metrics track workflow execution across backends: | `toolhive_vmcp_workflow_errors` | Counter | Total workflow execution errors | | `toolhive_vmcp_workflow_duration` | Histogram | Duration of workflow executions | +### Rate limit metrics + +When [rate limiting](../guides-k8s/rate-limiting.mdx) is enabled on the +VirtualMCPServer, these metrics track token bucket decisions and the health of +the Redis backend that stores the buckets. Each metric carries `namespace` and +`server` attributes identifying the resource: + +| Metric | Type | Description | +| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `toolhive_rate_limit_decisions` | Counter | Bucket decisions. Additional attributes: `decision` (`allowed` or `rejected`), `scope` (`shared` or `per_user`), `operation_type` (`server` or `tool`) | +| `toolhive_rate_limit_redis_errors` | Counter | Redis errors during rate limit checks. Additional attribute: `error_type` (`timeout`, `connection`, `auth`, or `other`) | +| `toolhive_rate_limit_check_latency` | Histogram | Duration of the atomic Redis Lua rate limit check, in seconds (each check, including failures) | + +For the counting semantics (allowed increments per bucket, rejected only for the +first rejected bucket) and example PromQL queries, see +[Observe rate limit activity](../guides-k8s/rate-limiting.mdx#observe-rate-limit-activity) +in the rate limiting guide. + ### Optimizer metrics When the vMCP optimizer is enabled, these metrics track tool-finding and From b53136f396f583c0d9a644a4c3f11a8e884670ce Mon Sep 17 00:00:00 2001 From: "claude[bot]" <41898282+claude[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 21:58:51 +0000 Subject: [PATCH 4/4] Tighten rate limit observability wording --- docs/toolhive/concepts/observability.mdx | 6 +++--- docs/toolhive/guides-k8s/rate-limiting.mdx | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/toolhive/concepts/observability.mdx b/docs/toolhive/concepts/observability.mdx index ddae4914..7d0e5237 100644 --- a/docs/toolhive/concepts/observability.mdx +++ b/docs/toolhive/concepts/observability.mdx @@ -230,9 +230,9 @@ For details, see the ### Rate limit metrics When [rate limiting](../guides-k8s/rate-limiting.mdx) is configured on an -MCPServer or VirtualMCPServer, ToolHive emits per-bucket decision, Redis error, -and Redis Lua check latency metrics so you can watch for rejections and detect -fail-open periods when Redis is unreachable. See +MCPServer or VirtualMCPServer, ToolHive emits metrics for bucket decisions, +Redis errors, and Redis Lua check latency so you can watch for rejections and +detect fail-open periods when Redis is unreachable. See [Observe rate limit activity](../guides-k8s/rate-limiting.mdx#observe-rate-limit-activity) for metric names, attributes, and example queries. diff --git a/docs/toolhive/guides-k8s/rate-limiting.mdx b/docs/toolhive/guides-k8s/rate-limiting.mdx index cffdbdd4..0b4fcadb 100644 --- a/docs/toolhive/guides-k8s/rate-limiting.mdx +++ b/docs/toolhive/guides-k8s/rate-limiting.mdx @@ -243,9 +243,9 @@ Redis bucket check. Use them to watch for rejections, spot Redis health issues, and track how long the atomic Lua check takes. All three metrics carry `namespace` and `server` attributes identifying the -resource that owns the bucket. Prometheus appends `_total` to counter names and -`_seconds` to the latency histogram (with the standard `_bucket`, `_sum`, and -`_count` series suffixes). +resource that owns the bucket. Prometheus appends `_total` to counter names. The +latency histogram is exported with the `_seconds` unit suffix and the standard +`_bucket`, `_sum`, and `_count` series suffixes. | Metric | Type | Additional attributes | | ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------- | @@ -269,8 +269,8 @@ Counting semantics for `toolhive_rate_limit_decisions`: The `toolhive_rate_limit_check_latency` histogram records the duration of each Redis Lua call, whether the call succeeds or fails. Rate limit enforcement [fails open](#how-rate-limiting-works) when Redis is unreachable, so watch -`toolhive_rate_limit_redis_errors` alongside decisions to catch periods where -limits stopped applying. +`toolhive_rate_limit_redis_errors` alongside decisions to detect fail-open +periods. Example PromQL queries against a Prometheus scrape of the server's `/metrics` endpoint: @@ -341,7 +341,7 @@ spec: ## Next steps - [Telemetry and metrics](./telemetry-and-metrics.mdx) to enable OpenTelemetry - metrics collection for the metrics above + collection and export for these metrics - [Token exchange](./token-exchange-k8s.mdx) to configure token exchange for upstream service authentication - [CRD reference](../reference/crds/index.mdx) for complete field definitions