feat: check request cancellation before batch assembly

[pskiran1]

What does the PR do?

The PR checks TRITONBACKEND_RequestIsCancelled for each request in Run() and, when every request in the batch has been cancelled, responds and returns without submitting any work to the engine.

TensorRT issues a batch as a single asynchronous submission — one enqueueV3, a cudaGraphLaunch replay, or a multi-device fan-out — so there is no intermediate point at which partial work can be abandoned. The only actionable checkpoint is before the batch is assembled, which is where this check sits.

Scope is deliberately whole-batch only. Dropping a subset of requests from a forming batch would require re-deriving the batch size, shape tensor values and optimization profile selection, and is not worth the complexity unless profiling shows a meaningful rate of mid-batch cancellations.

Related PRs

• core: feat: check request cancellation before backend dispatch core#522 — check request cancellation before backend dispatch
• server: test: check request cancellation before backend dispatch server#8953 — L0_request_cancellation coverage

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge with no outstanding actionable issue identified.

Summary

• Queries cancellation only until a live request or query failure is encountered.
• Returns cancelled responses without enqueueing fully cancelled batches.
• Excludes implicit-state models so required state updates still occur.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Receive request batch] --> B{Uses implicit state?}
  B -- Yes --> E[Assemble and execute batch]
  B -- No --> C{All requests cancelled?}
  C -- No --> E
  C -- Query error --> E
  C -- Yes --> D[Send cancelled responses]
  D --> F[Return without TensorRT submission]

Loading

[yinggeh]

Is all_requests_cancelled a descriptive name for the value assigned here?

[pskiran1]

The initial value represents whether execution may be skipped, not whether all requests are already known to be cancelled. I renamed all_requests_cancelled to skip_batch_execution for better naming convention.