feat(SPEC-DFLASH2): W7 — async scheduling survives an Eagle-type speculator, drafts riding in the output (#1824)

[localai-bot]

The engine forced synchronous scheduling whenever a speculator was configured
(model_loader.cpp carried !resolved_spec_config_.has_value() && on the
async resolution), a deferral SPEC-MTP I5d recorded in
.agents/specs/mtp-spec-decode.md §2.8. Upstream keeps async scheduling ON for
the Eagle-type family — "dflash" (which DFlash2 rides) and every MTP type
included (vllm/config/vllm.py:1064-1112, speculative.py:60-65 @
555967922) — and at c1 spec decode (~360 steps for 2048 tokens) the forced
sync arm serializes every host-side scheduling cost into each step: the largest
named host-side divergence in the #1574 speed gap.

This wave (SPEC-DFLASH2 W7, spec .agents/specs/spec-decode-async-scheduling.md,
committed first) ports the draft-in-output flow and flips the enable to
upstream's method predicate:

• SpeculativeConfig::async_scheduling_compatible() mirrors the pin's family
  check: mtp/dflash/dspark/eagle/eagle3 stay async; host ngram and
  draft_model (refused at the pin) force sync.
• AsyncScheduler takes the SpeculativeConfig and assigns
  spec_token_ids = [-1] * num_spec_tokens_to_schedule after each schedule
  (async_scheduler.py:24-25,43-45); SchedulerOutput gains
  num_spec_tokens_to_schedule and num_invalid_spec_tokens.
• Scheduler::update_draft_token_ids_in_output (scheduler.py:2072-2107)
  trims the worker's drafts to the scheduled count, pads a short row with -1,
  and records the invalid tail; EngineCore::step_with_batch_queue's
  deferred-grammar branch calls it (core.py:718-731).
• EngineCore::post_step skips the out-of-band pull under async scheduling
  (core.py:617), read off a new Scheduler::async_scheduling() virtual — the
  resolution product IS the scheduler class.
• The rejection rollback skips a request draining stale frames
  (async_tokens_to_discard, scheduler.py:1670-1675), and preemption clears
  spec_token_ids (:1217-1218).
• The WORKER half, two deliberate adaptations with upstream anchors recorded in
  the spec (§Design D3/D4): the placeholder fill is host-side (our propose
  materializes drafts on the host, so the runner patches a local copy of the
  scheduled map from its own pending_drafts_), and the computed-token
  correction is exact-structural (num_tokens_no_spec - 1) where upstream's is
  optimistic-with-GPU-correction (gpu_model_runner.py:1356-1396) — our
  rejection result is host-visible in time, upstream's is not.
• runner_supports_async() is split from async_input_combine_: async
  SCHEDULING capability no longer carries the spec veto, while the
  non-draft-aware device input-combine keeps it (I5e). Spec steps keep the
  synchronous host sampler.

Red first, all captured before the implementation:

• test_mtp_depth "W7 (Speculators force synchronous scheduling; upstream keeps async ON for Eagle-type methods (dflash included) #1824)" cases failed behaviorally on the pre-W7 tree:
  CHECK(eng.async_scheduling_enabled()) read false with an MTP speculator,
  max_concurrent_batches read 1 == 2 (2 cases / 3 assertions red).
• The scheduler/engine suites failed to compile naming exactly the missing
  surface: Scheduler has no member update_draft_token_ids_in_output,
  SchedulerOutput has no member num_spec_tokens_to_schedule /
  num_invalid_spec_tokens, no AsyncScheduler ctor taking a
  SpeculativeConfig.

Green, CPU tier:

• test_mtp_depth 10/10 (123): a spec engine resolves async ON, ngram stays
  sync, and sync/async emit IDENTICAL tokens through BOTH production fronts
  (depth-1 LLMEngine::step and AsyncLLM's depth-2 step_with_batch_queue)
  on the synthetic GDN-hybrid model with a real MTP head, with the depth
  witnesses asserted in every arm.
• test_engine_core_proc 15/15 (121): the placeholder / worker-fill contract
  at the ModelRunnerBase seam — under async the engine never pulls
  out-of-band (take_calls == 0), the scheduler ships -1 placeholders, and
  the stub's fill emits the sync flow's exact token streams.
• test_async_scheduler 9/9 (86) incl. the ported
  test_no_placeholder_underflow_on_discarded_spec_frame and a steady-state
  placeholder/rollback arithmetic case over acceptance 0/1/k.
• test_scheduler 38/38 (459) incl. update_draft_token_ids_in_output
  trim/pad/skip/replace semantics.
• test_speculative_mtp_depth 5/5 (27) incl. the method truth table.
• Full ctest: 600/601 passed (3 checkpoint-gated skips); the one red,
  test_cpu_threadpool, is the known parallel-ctest load flake and passes
  serially (9/9, 19602 assertions). scripts/agent-preflight.sh green except
  the known-flaky test_cpu_x86_llamacpp_floor (test_cpu_x86_llamacpp_floor: the contended-leg case is load-dependent — at loadavg 63 the harness exits NO_QUIET_WINDOW (4) instead of GIVING_UP (2), so the guarantee goes untested and reads as a failure of whatever diff is in flight #618, the recorded
  exception).

Mutations (each applied, focused suite red, restored):

• M1 (reachability): revert the enable line to the pre-W7
  !resolved_spec_config_.has_value() && form — test_mtp_depth red, 2
  cases / 3 assertions (the production call site IS the flip).
• M2: delete num_output_placeholders -= num_rejected — test_async_scheduler
  red, 5 assertions (placeholder budget drifts).
• M3: delete the async_tokens_to_discard == 0 guard — the ported underflow
  case red (num_output_placeholders read -4 == 1).
• M4: discard the worker fill's result (splice the -1 map verbatim) —
  test_mtp_depth red, 5 cases (every spec engine arm).
• M5: delete the computed-token correction write — test_mtp_depth red at
  exactly the depth-2 arm's token identity (:876), the arm the correction
  exists for; depth-1 stays green because its rollback lands before the next
  schedule.
• M6: delete update_draft_token_ids_in_output's trim — test_scheduler red,
  5 assertions.
• M7: delete the AsyncScheduler placeholder assignment —
  test_async_scheduler red (3 assertions) AND test_engine_core_proc red
  (2 assertions).

Every mutation compiled clean, was run, and the tree was restored and
verified byte-for-byte against pre-mutation sha256 sums before the next.

Owed, recorded in the spec's ## Owed:

• A1: the GPU c1 TPOT A/B (async-ON vs VT_ASYNC_SCHED=0, same binary, Qwen3.8-27B three-way at each engine's best: one checkpoint ours/vLLM/SGLang all serve, and three of the four recorded NVFP4 blockers are properties of the unsloth artifact rather than the format #1574
  workload) — operator-run under an rc lease; no speed number is claimed
  here.
• A2: the device-resident async spec sampler (upstream's overlap on spec
  steps: device rejection sampling, device-resident drafts, the draft-aware
  combine, the optimistic correction).
• A3: structured output × spec × async (the grammar validate_tokens arm,
  deferred with the multi-row bitmask).

Closes #1824.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]