add docs and client use case for external-filter

Signed-off-by: rayneLi <[email protected]>

Author: rayne-Li

docs/source/features/gateway-plugins.rst

@@ -194,6 +194,38 @@ To set up rate limiting, add the user header in the request, like this:
     If rate limit support is required, ensure this `user` header is always set in the request. if you do not need rate limit, you do not need to set this header.
 
 
+External Filter
+===============
+The ``external-filter`` header is evaluated **after** the routing strategy selects the optimal Pod set.  allows users to dynamically restrict the target Pods using Kubernetes ``labelSelector`` expressions.
+
+The header value follows the Kubernetes label selector syntax:
+
+- ``key=value``
+- ``key in (a, b)``
+- ``key!=value``
+- comma-separated selector list
+
+For label selector syntax reference:
+https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
+
+.. code-block:: bash
+   curl -v http://${ENDPOINT}/v1/completions \
+      -H "Content-Type: application/json" \
+      -H "routing-strategy: random" \
+      -H "external-filter: environment=production,tier=frontend" \
+      -d '{
+            "model": "deepseek-r1-distill-llama-8b",
+            "prompt": "San Francisco is a",
+            "max_tokens": 128,
+            "temperature": 0
+          }'
+
+.. note::
+    1. Filtering happens **after** the routing strategy. It never changes which Pods are considered “optimal” by the routing strategy.
+    2. It only reduces the eligible Pod set by applying extra label constraints.
+    3. Same as `no target pod`, If the filter eliminates all Pods, the request will fail with ``no ready pods for routing``.
+    4. ``external-filter`` is optional. When omitted, no extra filtering is applied.
+
 Headers Explanation
 --------------------
 
@@ -216,7 +248,8 @@ Target Headers & General Error Headers
      - Specifies the destination pod selected by the routing algorithm. Useful for verifying routing decisions.
    * - ``routing-strategy``
      - Defines the routing strategy applied to this request. Ensures correct routing logic is followed.
-
+   * - ``external-filter``
+     - Provides a generic and pluggable mechanism to further filter candidate Pods after routing.
 
 Routing & Error Debugging Headers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

pkg/plugins/gateway/gateway_req_body.go

@@ -74,7 +74,19 @@ func (s *Server) HandleRequestBody(ctx context.Context, requestID string, req *e
 		headers = buildEnvoyProxyHeaders(headers, HeaderModel, model)
 		klog.InfoS("request start", "requestID", requestID, "requestPath", requestPath, "model", model, "stream", stream)
 	} else {
-		// external filter in header
+		// external-filter in header, value is k8s labelSelector format https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
+		// like: environment=production,tier=frontend; environment in (production, qa)
+		// full request example:
+		// curl -v http://${ENDPOINT}/v1/completions \
+		//    -H "Content-Type: application/json" \
+		//    -H "routing-strategy: random" \
+		//    -H "external-filter: key1=value1,key2=value2" \
+		//    -d '{
+		//        "model": "deepseek-r1-distill-llama-8b",
+		//        "prompt": "San Francisco is a",
+		//        "max_tokens": 128,
+		//        "temperature": 0
+		//    }'
 		externalFilter := routingCtx.ReqHeaders[HeaderExternalFilter]
 		klog.InfoS("found external filter", "filter", externalFilter)
 		targetPodIP, err := s.selectTargetPod(routingCtx, podsArr, externalFilter)