feat(flowcontrol): add foundational types

Introduces the foundational packages for the new Flow Controller
component.

This change includes:
- The top-level README outlining the motivation, high-level
  architecture, and component pillars.
- The `types` package, which defines the core data contracts, request
  lifecycle interfaces, error-handling vocabulary, and final outcome
  enums for the entire module.

This foundational PR establishes the core concepts and data models upon
which the rest of the Flow Controller implementation will be built.

Author: LukeAVanDrie

pkg/epp/flowcontrol/README.md

@@ -0,0 +1,116 @@
+# Flow Control Module
+
+## Introduction
+
+In a multi-tenant, heterogeneous inference serving environment, managing diverse SLOs and fairness requirements is
+critical. Today, the serving stack often relies on a simple "best-effort" or FIFO (First-In, First-Out) basis for
+handling requests. This is insufficient and leads to significant problems:
+
+* **Head-of-Line Blocking**: A long-running, low-priority request can block short, high-priority requests, violating
+  SLOs.
+* **Lack of Predictability**: Without proper queuing and prioritization, it's impossible to provide predictable latency
+  guarantees to different tenants.
+* **Inability to Handle Saturation**: Under heavy load, the system has no graceful way to manage overload, leading to
+  cascading failures instead of controlled degradation.
+
+The Flow Controller is a sophisticated library designed to solve these problems. It acts as a crucial gatekeeper that
+decides *if* and *when* a request should proceed to be scheduled. Its primary mission is to enable predictable, fair,
+and efficient utilization of shared backend resources by enforcing prioritization, applying fairness policies, managing
+request queuing under saturation, and orchestrating displacement (the eviction of lower-priority queued items to make
+space for higher-priority ones).
+
+It is designed for extensibility, allowing custom logic for policies and queuing mechanisms to be plugged into a robust,
+high-performance orchestration engine.
+
+### Role in the Gateway API Inference Extension
+
+Within the Gateway API Inference Extension's Endpoint Picker (EPP), the Flow Controller acts as a crucial gatekeeper
+between the Routing and Scheduling layers. It decides *if* and *when* a request, already assigned to a logical flow
+(e.g., a specific workload or tenant), should proceed to be scheduled onto a backend resource. It is the primary
+mechanism for managing diverse SLOs, ensuring fairness among competing workloads, and maintaining system stability under
+high load.
+
+### High Level Architecture
+
+The following diagram illustrates the high-level dependency model and request flow for the system. It shows how
+concurrent client requests are managed by the central `FlowController`, which in turn relies on a set of decoupled
+components to make its decisions. Each component package in this module will contain its own more detailed architectural
+diagrams.
+
+```mermaid
+graph LR
+    %% Style Definitions
+    classDef default fill:#fff,stroke:#333,stroke-width:1.5px,color:#000;
+    classDef client fill:#dcfce7,stroke:#333;
+    classDef system_entry fill:#fef9c3,stroke:#333;
+    classDef downstream_ok fill:#dbeafe,stroke:#333;
+    classDef downstream_err fill:#fee2e2,stroke:#333;
+
+    %% Client Goroutines (Fan-In)
+    subgraph Client Goroutines
+        direction TB
+        R1(Goroutine 1);
+        R2(Goroutine N);
+    end
+
+    %% Flow Control System
+    subgraph Flow Control System
+        C{Flow Controller Engine};
+
+        subgraph Internal Interactions
+            direction LR
+            D(Ports) -- "abstracts state" --> E(Flow Registry);
+            D -- "abstracts load" --> SD(Saturation Detector);
+            E -- "configures" --> F(Framework);
+            F -- "defines" --> P(Plugins: Queues & Policies);
+        end
+
+        C -- "Orchestrates via<br>abstractions" --> D;
+    end
+
+    %% Downstream Actions (Fan-Out)
+    subgraph Downstream Actions
+       direction TB
+       A1(Outcome: Dispatched<br>Proceed to Scheduler);
+       A2(Outcome: Rejected<br>Return Error);
+    end
+
+    %% Connections
+    R1 -- "calls & blocks" --> C;
+    R2 -- "calls & blocks" --> C;
+    C -- "unblocks 'goroutine 1'" --> A1;
+    C -- "unblocks 'goroutine N'" --> A2;
+
+    %% Apply Classes
+    class R1,R2 client;
+    class C system_entry;
+    class A1 downstream_ok;
+    class A2 downstream_err;
+    class D,E,F,P,SD default;
+```
+
+## Architectural Pillars
+
+The Flow Controller framework is built on several key components that work in concert. This architecture is designed to
+be highly modular and scalable, with clear separation of concerns. For a deep dive into the specific design choices and
+their justifications, please refer to the detailed documentation within the relevant sub-packages.
+
+1.  **The `FlowController` Engine (`./controller`)**: The central, sharded orchestrator responsible for the main request
+    processing loop. It manages a pool of workers that distribute incoming requests, apply policies, and dispatch
+    requests to the backends. Its design focuses on high throughput and backpressure.
+
+2.  **Pluggable `Policy` Framework (`./framework`)**: This defines the core interfaces for all pluggable logic. It
+    features a two-tier policy system for `InterFlow` (decisions *between* different flows) and `IntraFlow`
+    (decisions *within* a single flow) logic, covering both request dispatch and displacement.
+
+3.  **Extensible `SafeQueue` System (`./framework`)**: This defines the `framework.SafeQueue` interface for
+    concurrent-safe request storage. It uses a `QueueCapability` system that allows for diverse and extensible queue
+    implementations (e.g., FIFO, Priority Heap) while maintaining a stable interface.
+
+4.  **The `FlowRegistry` (`./registry`, `./ports`)**: This is the stateful control plane of the system. It manages the
+    configuration and lifecycle of all flows, policies, and queues. It presents a sharded view of its state to the
+    `FlowController` workers to enable parallel operation with minimal lock contention.
+
+5.  **Core Types and Service Ports (`./types`, `./ports`)**: These packages define the foundational data structures
+    (e.g., `FlowControlRequest`), errors, and service interfaces that decouple the engine from its dependencies,
+    following a "Ports and Adapters" architectural style.

pkg/epp/flowcontrol/types/README.md

@@ -0,0 +1,33 @@
+# Flow Control Core Types
+
+This package defines the fundamental data structures, interfaces, and errors that form the vocabulary of the Flow
+Control system. It establishes the core concepts of the request lifecycle and its final, reportable outcomes.
+
+## Request Lifecycle Interfaces
+
+A request's journey through the Flow Controller is represented by a series of interfaces that define its state as it
+moves through the system:
+
+1.  **`FlowControlRequest`**: The initial, "raw" contract for an incoming request. It carries the essential data
+    provided by the client, such as its `FlowID` and `ByteSize`.
+2.  **`QueueItemAccessor`**: The internal, enriched, and read-only view of a request once it has been accepted by the
+    controller. This interface is the primary means by which policy plugins inspect items.
+3.  **`QueueItemHandle`**: An opaque, queue-specific handle to a queued item. The controller uses this handle to perform
+    targeted operations, such as removing a specific item, without needing to know the queue's internal implementation
+    details.
+
+## Final State Reporting: Outcomes and Errors
+
+The final state of every request is reported using a combination of a `QueueOutcome` enum and a corresponding `error`.
+This provides a clear, machine-inspectable way to understand the result.
+
+* **`QueueOutcome`**: A concise enum summarizing the final result (e.g., `QueueOutcomeDispatched`,
+  `QueueOutcomeRejectedCapacity`, `QueueOutcomeEvictedDisplaced`). This is ideal for metrics.
+
+* **Errors**: For any non-dispatch outcome, a specific sentinel error is returned. These are nested to provide detailed
+  context:
+    * `ErrRejected`: The parent error for any request rejected *before* being enqueued.
+    * `ErrEvicted`: The parent error for any request removed *after* being enqueued for reasons other than dispatch.
+
+Callers of `FlowController.EnqueueAndWait()` can first use `errors.Is()` to check for the general class of failure
+(`ErrRejected` or `ErrEvicted`), and then unwrap the error to find the specific cause (e.g., `ErrQueueAtCapacity`).

pkg/epp/flowcontrol/types/errors.go

@@ -0,0 +1,78 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package types
+
+import (
+	"errors"
+)
+
+// --- High Level Queue Outcome Errors ---
+var (
+	// ErrRejected is a sentinel error indicating a request was rejected by the Flow Controller *before* being formally
+	// enqueued. Errors returned by `FlowController.EnqueueAndWait()` that signify pre-queue rejection will wrap this
+	// error.
+	// Callers should use `errors.Is(err, ErrRejected)` to check for this general class of failure.
+	ErrRejected = errors.New("request rejected pre-queue")
+
+	// ErrEvicted is a sentinel error indicating a request was removed from a queue *after* being successfully enqueued,
+	// but for reasons other than successful dispatch (e.g., TTL expiry, displacement).
+	// Errors returned by `FlowController.EnqueueAndWait()` that signify post-queue eviction will wrap this error.
+	// Callers should use `errors.Is(err, ErrEvicted)` to check for this general class of failure.
+	ErrEvicted = errors.New("request evicted from queue")
+)
+
+// --- Pre-Enqueue Rejection Errors ---
+// Errors that can occur before a request is formally added to a `framework.SafeQueue`.
+// When returned by `FlowController.EnqueueAndWait()`, these specific errors will typically be wrapped by `ErrRejected`.
+var (
+	// ErrNilRequest indicates that a nil `types.FlowControlRequest` was provided.
+	ErrNilRequest = errors.New("FlowControlRequest cannot be nil")
+
+	// ErrFlowIDEmpty indicates that a flow ID was empty when one was required.
+	ErrFlowIDEmpty = errors.New("flow ID cannot be empty")
+
+	// ErrQueueAtCapacity indicates that a request could not be enqueued because queue capacity limits were met and
+	// displacement (if applicable) failed to make space.
+	ErrQueueAtCapacity = errors.New("queue at capacity and displacement failed to make space")
+)
+
+// --- Post-Enqueue Eviction Errors ---
+// Errors that occur when a request, already in a `framework.SafeQueue`, is removed for reasons other than dispatch.
+// When returned by `FlowController.EnqueueAndWait()`, these specific errors will typically be wrapped by `ErrEvicted`.
+var (
+	// ErrTTLExpired indicates a request was evicted from a queue because its effective Time-To-Live expired.
+	ErrTTLExpired = errors.New("request TTL expired")
+
+	// ErrContextCancelled indicates a request was evicted because its associated context (from
+	// `FlowControlRequest.Context()`) was cancelled. This error typically wraps the underlying `context.Canceled` or
+	// `context.DeadlineExceeded` error.
+	ErrContextCancelled = errors.New("request context cancelled")
+
+	// ErrDisplaced indicates a request was evicted from a queue because it was chosen as a victim by a displacement
+	// policy to make space for another request.
+	ErrDisplaced = errors.New("request displaced")
+)
+
+// --- General FlowController Errors ---
+// General runtime errors for the Flow Controller.
+var (
+	// ErrFlowControllerShutdown indicates that an operation could not complete or an item was evicted because the Flow
+	// Controller is shutting down or has stopped.
+	// When returned by `FlowController.EnqueueAndWait()`, this will be wrapped by `ErrRejected` (if rejection happens
+	// before internal queuing) or `ErrEvicted` (if eviction happens after internal queuing).
+	ErrFlowControllerShutdown = errors.New("FlowController is shutting down")
+)

pkg/epp/flowcontrol/types/flow.go

@@ -0,0 +1,33 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS ßIS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package types defines the core data structures and service contracts for the Flow Controller system. It establishes
+// the "vocabulary" of the system, including the request lifecycle interfaces, final outcomes, and standard error types.
+package types
+
+// FlowSpecification defines the configuration of a logical flow, encapsulating its identity and registered priority.
+//
+// A FlowSpecification acts as the registration key for a flow within the Flow Registry.
+type FlowSpecification interface {
+	// ID returns the unique name or identifier for this flow (e.g., model name, tenant ID), corresponding to the value
+	// from `FlowControlRequest.FlowID()`.
+	ID() string
+
+	// Priority returns the numerical priority level currently associated with this flow within the Flow Registry.
+	//
+	// Convention: Lower numerical values indicate higher priority.
+	Priority() uint
+}

pkg/epp/flowcontrol/types/outcomes.go

@@ -0,0 +1,93 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package types
+
+import "strconv"
+
+// QueueOutcome represents the high-level final state of a request's lifecycle within the Flow Controller.
+//
+// It is returned by `FlowController.EnqueueAndWait()` along with a corresponding error. This enum is designed to be a
+// low-cardinality label ideal for metrics, while the error provides fine-grained details for non-dispatched outcomes.
+type QueueOutcome int
+
+const (
+	// QueueOutcomeDispatched indicates the request was successfully processed by the Flow Controller and unblocked for
+	// the caller to proceed.
+	// The associated error from `FlowController.EnqueueAndWait()` will be nil.
+	QueueOutcomeDispatched QueueOutcome = iota
+
+	// --- Pre-Enqueue Rejection Outcomes (request never entered a `framework.SafeQueue`) ---
+	// For these outcomes, the error from `FlowController.EnqueueAndWait()` will wrap `ErrRejected`.
+
+	// QueueOutcomeRejectedCapacity indicates rejection because queue capacity limits were met and displacement (if
+	// applicable) failed to make space.
+	// The associated error will wrap `ErrQueueAtCapacity` (and `ErrRejected`).
+	QueueOutcomeRejectedCapacity
+
+	// QueueOutcomeRejectedOther indicates rejection for reasons other than capacity before the request was formally
+	// enqueued.
+	// The specific underlying cause can be determined from the associated error (e.g., a nil request, an unregistered
+	// flow ID, or a controller shutdown), which will be wrapped by `ErrRejected`.
+	QueueOutcomeRejectedOther
+
+	// --- Post-Enqueue Eviction Outcomes (request was in a `framework.SafeQueue` but not dispatched) ---
+	// For these outcomes, the error from `FlowController.EnqueueAndWait()` will wrap `ErrEvicted`.
+
+	// QueueOutcomeEvictedTTL indicates eviction from a queue because the request's effective Time-To-Live expired.
+	// The associated error will wrap `ErrTTLExpired` (and `ErrEvicted`).
+	QueueOutcomeEvictedTTL
+
+	// QueueOutcomeEvictedContextCancelled indicates eviction from a queue because the request's own context (from
+	// `FlowControlRequest.Context()`) was cancelled.
+	// The associated error will wrap `ErrContextCancelled` (which may further wrap the underlying `context.Canceled` or
+	// `context.DeadlineExceeded` error) (and `ErrEvicted`).
+	QueueOutcomeEvictedContextCancelled
+
+	// QueueOutcomeEvictedDisplaced indicates eviction from a queue to make space for another request due to a
+	// displacement policy.
+	// The associated error will wrap `ErrDisplaced` (and `ErrEvicted`).
+	QueueOutcomeEvictedDisplaced
+
+	// QueueOutcomeEvictedOther indicates eviction from a queue for reasons not covered by more specific eviction
+	// outcomes.
+	// The specific underlying cause can be determined from the associated error (e.g., a controller shutdown while the
+	// item was queued), which will be wrapped by `ErrEvicted`.
+	QueueOutcomeEvictedOther
+)
+
+// String returns a human-readable string representation of the QueueOutcome.
+func (o QueueOutcome) String() string {
+	switch o {
+	case QueueOutcomeDispatched:
+		return "Dispatched"
+	case QueueOutcomeRejectedCapacity:
+		return "RejectedCapacity"
+	case QueueOutcomeRejectedOther:
+		return "RejectedOther"
+	case QueueOutcomeEvictedTTL:
+		return "EvictedTTL"
+	case QueueOutcomeEvictedContextCancelled:
+		return "EvictedContextCancelled"
+	case QueueOutcomeEvictedDisplaced:
+		return "EvictedDisplaced"
+	case QueueOutcomeEvictedOther:
+		return "EvictedOther"
+	default:
+		// Return the integer value for unknown outcomes to aid in debugging.
+		return "UnknownOutcome(" + strconv.Itoa(int(o)) + ")"
+	}
+}