Update stacklok/toolhive to v0.35.0

.github/upstream-projects.yaml

@@ -44,7 +44,7 @@ projects:
 
   - id: toolhive
     repo: stacklok/toolhive
-    version: v0.34.0
+    version: v0.35.0
     # toolhive is a monorepo covering the CLI, the Kubernetes
     # operator, and the vMCP gateway. It also introduces cross-
     # cutting features that land in concepts/, integrations/,

docs/toolhive/concepts/observability.mdx

@@ -227,6 +227,15 @@ monitoring backend operations, workflow executions, and optimizer performance.
 For details, see the
 [vMCP telemetry guide](../guides-vmcp/telemetry-and-metrics.mdx).
 
+### Rate limit metrics
+
+When [rate limiting](../guides-k8s/rate-limiting.mdx) is configured on an
+MCPServer or VirtualMCPServer, ToolHive emits metrics for bucket decisions,
+Redis errors, and Redis Lua check latency so you can watch for rejections and
+detect fail-open periods when Redis is unreachable. See
+[Observe rate limit activity](../guides-k8s/rate-limiting.mdx#observe-rate-limit-activity)
+for metric names, attributes, and example queries.
+
 ## Trace context propagation
 
 ToolHive supports two methods of trace context propagation:

docs/toolhive/guides-k8s/rate-limiting.mdx

@@ -234,6 +234,65 @@ A request must pass both the server-level vMCP limit and the per-tool limit (if
 defined). Limits apply to the vMCP aggregator and are independent from any
 limits configured on the backend MCPServers it routes to.
 
+## Observe rate limit activity
+
+When telemetry is enabled on the MCPServer or VirtualMCPServer (via a
+[`MCPTelemetryConfig`](./telemetry-and-metrics.mdx) referenced by
+`telemetryConfigRef`), the rate limiter emits OpenTelemetry metrics for each
+Redis bucket check. Use them to watch for rejections, spot Redis health issues,
+and track how long the atomic Lua check takes.
+
+All three metrics carry `namespace` and `server` attributes identifying the
+resource that owns the bucket. Prometheus appends `_total` to counter names. The
+latency histogram is exported with the `_seconds` unit suffix and the standard
+`_bucket`, `_sum`, and `_count` series suffixes.
+
+| Metric                              | Type      | Additional attributes                                                                                         |
+| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------- |
+| `toolhive_rate_limit_decisions`     | Counter   | `decision` (`allowed` or `rejected`), `scope` (`shared` or `per_user`), `operation_type` (`server` or `tool`) |
+| `toolhive_rate_limit_redis_errors`  | Counter   | `error_type` (`timeout`, `connection`, `auth`, or `other`)                                                    |
+| `toolhive_rate_limit_check_latency` | Histogram | (none)                                                                                                        |
+
+Counting semantics for `toolhive_rate_limit_decisions`:
+
+- An **allowed** request increments the counter once for every applicable
+  bucket. For example, a request that satisfies both a shared and a per-user
+  bucket increments the counter twice.
+- A **rejected** request increments only for the first bucket rejected by the
+  atomic Redis check, so a single rejection is never double-counted across
+  scopes.
+- Requests with no applicable bucket do not increment the counter. This includes
+  discovery methods that pass through unconditionally (`tools/list`,
+  `prompts/list`, `initialize`, `ping`) and unauthenticated requests against a
+  server that only has per-user buckets configured.
+
+The `toolhive_rate_limit_check_latency` histogram records the duration of each
+Redis Lua call, whether the call succeeds or fails. Rate limit enforcement
+[fails open](#how-rate-limiting-works) when Redis is unreachable, so watch
+`toolhive_rate_limit_redis_errors` alongside decisions to detect fail-open
+periods.
+
+Example PromQL queries against a Prometheus scrape of the server's `/metrics`
+endpoint:
+
+```promql
+# Rejection rate across all buckets, by server
+sum by (namespace, server) (
+  rate(toolhive_rate_limit_decisions_total{decision="rejected"}[5m])
+)
+
+# 95th-percentile Redis check latency
+histogram_quantile(
+  0.95,
+  sum by (le) (rate(toolhive_rate_limit_check_latency_seconds_bucket[5m]))
+)
+
+# Redis errors by class. Non-zero timeout or connection rates warrant attention.
+sum by (error_type) (
+  rate(toolhive_rate_limit_redis_errors_total[5m])
+)
+```
+
 :::info[Per-tool limits with the optimizer]
 
 When the [tool discovery optimizer](../guides-vmcp/optimizer.mdx) is enabled,
@@ -281,6 +340,8 @@ spec:
 
 ## Next steps
 
+- [Telemetry and metrics](./telemetry-and-metrics.mdx) to enable OpenTelemetry
+  collection and export for these metrics
 - [Token exchange](./token-exchange-k8s.mdx) to configure token exchange for
   upstream service authentication
 - [CRD reference](../reference/crds/index.mdx) for complete field definitions

docs/toolhive/guides-vmcp/telemetry-and-metrics.mdx

@@ -167,6 +167,24 @@ These metrics track workflow execution across backends:
 | `toolhive_vmcp_workflow_errors`     | Counter   | Total workflow execution errors |
 | `toolhive_vmcp_workflow_duration`   | Histogram | Duration of workflow executions |
 
+### Rate limit metrics
+
+When [rate limiting](../guides-k8s/rate-limiting.mdx) is enabled on the
+VirtualMCPServer, these metrics track token bucket decisions and the health of
+the Redis backend that stores the buckets. Each metric carries `namespace` and
+`server` attributes identifying the resource:
+
+| Metric                              | Type      | Description                                                                                                                                            |
+| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `toolhive_rate_limit_decisions`     | Counter   | Bucket decisions. Additional attributes: `decision` (`allowed` or `rejected`), `scope` (`shared` or `per_user`), `operation_type` (`server` or `tool`) |
+| `toolhive_rate_limit_redis_errors`  | Counter   | Redis errors during rate limit checks. Additional attribute: `error_type` (`timeout`, `connection`, `auth`, or `other`)                                |
+| `toolhive_rate_limit_check_latency` | Histogram | Duration of the atomic Redis Lua rate limit check, in seconds (each check, including failures)                                                         |
+
+For the counting semantics (allowed increments per bucket, rejected only for the
+first rejected bucket) and example PromQL queries, see
+[Observe rate limit activity](../guides-k8s/rate-limiting.mdx#observe-rate-limit-activity)
+in the rate limiting guide.
+
 ### Optimizer metrics
 
 When the vMCP optimizer is enabled, these metrics track tool-finding and

static/api-specs/toolhive-api.yaml

@@ -498,6 +498,13 @@ components:
           $ref: '#/components/schemas/github_com_stacklok_toolhive_pkg_authserver.DCRUpstreamConfig'
         identity_from_token:
           $ref: '#/components/schemas/github_com_stacklok_toolhive_pkg_authserver.IdentityFromTokenRunConfig'
+        insecure_allow_http:
+          description: |-
+            InsecureAllowHTTP permits plain-HTTP authorization and token endpoint URLs
+            for this upstream. Only for in-cluster development environments (e.g. an
+            OAuth2 provider served over HTTP in a kind cluster) where TLS is not
+            available. Never set this in production.
+          type: boolean
         redirect_uri:
           description: |-
             RedirectURI is the callback URL where the upstream IDP will redirect after authentication.
@@ -552,6 +559,13 @@ components:
             ClientSecretFile is the path to a file containing the OAuth 2.0 client secret.
             Mutually exclusive with ClientSecretEnvVar. Optional for public clients using PKCE.
           type: string
+        insecure_allow_http:
+          description: |-
+            InsecureAllowHTTP permits a plain-HTTP issuer URL and HTTP discovery
+            endpoints for this upstream. Only for in-cluster development environments
+            (e.g. Dex served over HTTP in a kind cluster) where TLS is not available.
+            Never set this in production.
+          type: boolean
         issuer_url:
           description: |-
             IssuerURL is the OIDC issuer URL for automatic endpoint discovery.