Track model-internal reasoning-state signals (attention, activations)

[randomvariable]

Candidate

A family of 2026 papers controls reasoning using model-internal signals — attention
distributions, hidden-state activations, residual-stream directions — rather than logits or
generated text. Reported results are strong, and the family is architecturally distinct
enough from #27 / #28 / #29 to be tracked separately.

• arXiv:2606.15070 — ASAG, "Stop When Further Reasoning Won't Help: Attention-State
  Adaptive Generation". Training-free and plug-and-play. Infers reasoning state from
  attention distributions and adapts the generation strategy. Reports +3.2% average
  accuracy while cutting generated tokens by ~40% on Qwen3-8B across nine benchmarks,
  on DeepSeek-R1-Distill and Qwen3 families. Accuracy up and tokens down is a stronger
  claim than anything else surveyed and warrants scrutiny.
• arXiv:2607.18100 — fine-grained reasoning control via activation steering,
  specifically targeting self-loops. Intervenes with steering vectors that generalize
  across state pairs.
• arXiv:2607.17188 — PUMA, phase-momentum alignment. Training-free tiered diagnostic
  using latent velocity and tortuosity plus entropic uncertainty. Explicitly motivated by
  the failure of information-theoretic approaches ("deceptive convergence", where low
  uncertainty masks hallucination) and of post-hoc latent analysis.
• arXiv:2607.21433 — linear probes on hidden-state activations predicting reasoning
  non-convergence. Weak on its own (layer-20 activations at token 150 give AUC 0.608
  ±0.080; sweep-level permutation test p=0.063, which the authors state their sample size
  cannot confirm) but carries one finding that matters: activation probes consistently
  outperform behavioural baselines derived from token entropy and repetition statistics.

That last point is independently useful. It is direct evidence against the feature set used
by the CUSUM decoding monitors (arXiv:2607.11317, arXiv:2607.20129) evaluated and rejected
in #27, and it suggests that if trajectory-health monitoring is ever worth doing, internals
are where the signal actually lives.

Why this is filed as tracking, not implementation

The blocker is architectural, not empirical.

vLLM's sampling path receives logits. Everything in #27 and #28 works because logits are
already there. This family needs data that the serving path does not currently expose:

1. Attention distributions are not materialized. FlashAttention and paged-attention
   kernels compute attention without ever writing the full attention matrix to memory —
   that is precisely why they are fast. ASAG needs attention distributions per step.
   Extracting them means either a slow path that materializes attention, or a kernel-level
   reduction that computes the specific statistic ASAG needs inline. Both are significant
   work, and the first would likely cost more throughput than the 40% token reduction wins
   back. This must be quantified before anything else.
2. Hidden-state activations require a model-runner hook. Reading layer-20 activations
   mid-decode means a per-layer tap in the forward pass, plumbed to the sampler, per
   request, under continuous batching. [Core][Feature] Observation Plugin for Intercepting & Routing on Activations vllm-project/vllm#37002 (upstream, open) proposes an observation
   plugin for intercepting activations — that is the natural substrate, and its fate should
   be tracked before anything is built here.
3. Steering vectors are model-specific artifacts. Activation steering needs vectors
   derived per model, and often per layer. Shipping that upstream means shipping or deriving
   artifacts, which is a different proposition from a heuristic and may be disqualifying on
   its own.
4. The strongest claim is the least verified. ASAG's simultaneous accuracy gain and 40%
   token reduction is an outlier against every other result surveyed, where the consistent
   pattern is a token/accuracy tradeoff. Treat as unverified until replicated. If it
   replicates it is the most valuable result in this area; if it does not, that closes the
   most expensive line of work here cheaply.

What to do now

Track, do not build. Concretely:

1. Watch [Core][Feature] Observation Plugin for Intercepting & Routing on Activations vllm-project/vllm#37002 (observation plugin for activations) upstream. If it lands, the substrate
   for the hidden-state members of this family exists and the calculus changes.
2. Offline replication of ASAG's headline number on recorded traces from a quantized model
   — is the accuracy gain real, and does it survive quantization? This is cheap and settles
   whether the expensive integration question is worth asking.
3. Quantify the cost of exposing the specific attention statistic ASAG needs, versus the
   throughput won back by generating 40% fewer tokens. If the ratio is unfavourable the
   whole attention branch closes.
4. Do not pursue steering vectors unless 2 and 3 both come back positive; the artifact
   problem makes it the least upstreamable member of the family.

Promotion criteria

Promote to an implementation issue only when all hold:

• ASAG's accuracy gain replicates offline on at least one quantized reasoning model,
  with a paired test, not just directionally.
• A concrete extraction path exists for the required internal signal that does not
  regress decode throughput by more than the token reduction wins back — measured, not
  argued.
• No per-model artifact required, or a maintainable plan for producing one.
• Upstream has, or plausibly will have, a supported hook — [Core][Feature] Observation Plugin for Intercepting & Routing on Activations vllm-project/vllm#37002 or a successor.

If ASAG fails to replicate, close this issue and record the negative result, keeping the
2607.21433 finding about activation probes beating entropy/repetition baselines as the
durable takeaway.

Related

• Reasoning-block marker penalty to reduce overthinking under quantized serving #27 — reasoning-block marker penalty (logits layer; where the rejected CUSUM monitors
  were also evaluated)
• Reserve output budget for the answer so reasoning never exhausts max_tokens #28 — answer budget reserve (logits layer, deadline-driven)
• Evaluate convergence-signal early exit for reasoning chains #29 — text-signal early exit (output layer)
• [Core][Feature] Observation Plugin for Intercepting & Routing on Activations vllm-project/vllm#37002 — observation plugin for intercepting activations (upstream, open)