Skip to content

Stop delivering steering messages the user removed from the queue - #342370

Merged
Karthik Nadig (karthiknadig) merged 2 commits into
mainfrom
karthiknadig/bug-swarm/withdraw-removed-steering
Oct 18, 2026
Merged

Karthik Nadig (karthiknadig) merged 2 commits into
mainfrom
karthiknadig/bug-swarm/withdraw-removed-steering

Conversation

@karthiknadig

@karthiknadig Karthik Nadig (karthiknadig) commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Refs #336466

Problem

In an Agent Host chat with the Copilot agent (the chat view or the Agents window), you can send a steering message while the agent works, then remove it with Remove from Queue. The removal does not stick: the agent still receives the message, and it appears in the transcript as its own request.

Root cause

Two layers let a removed steering message through.

  1. The host refuses the removal. Since agentHost: preserve multiple pending steering messages #340669, AgentService rejects a client's ChatPendingMessageRemoved for a steering message while a turn is active ("Cannot change a steering message while it is being processed"). No provider could take back a message it had already been given, so the message stays queued on the host: agentService.ts#L11101-L11107.
  2. Nothing withdraws the message from the SDK. The queue drain hands each steering message to the provider as soon as it is set (queueDrainContribution.ts#L161-L193). On removal it only clears its own bookkeeping (#L136-L141). CopilotAgentSession.sendSteering sends the message with mode: 'immediate' and discards the message ID that send() returns. It also keeps the entry it uses to recognize the SDK's user.message echo (copilotAgentSession.ts#L5587-L5627). When the echo arrives, _takeMatchingPendingSteering still matches the text and starts a turn for it (#L7853-L7855, #L2320-L2340). The SDK provides session.rpc.queue.withdrawMessage({ messageId, expectedPrompt }) for this case, but nothing calls it.

Fix

  • Provider contract. IAgent gets an optional withdrawSteeringMessage(chat, messageId). The host accepts removal of an in-flight steering message only when the session's provider implements it. Codex and Claude do not implement it, so they keep today's rejection. Editing an in-flight steering message is still rejected for every provider.
  • Queue drain. When a client removes a steering message that was already handed to the provider, the queue drain calls withdrawSteeringMessage. Removals the host makes itself, after the agent consumed a message or on cancel, do not trigger a withdrawal.
  • Copilot session. CopilotAgentSession keeps the SDK message ID and the exact prompt it sent. withdrawSteering:
    • drops a message that is still being prepared (MCP reconciliation or attachments), so it is never sent;
    • otherwise calls rpc.queue.withdrawMessage and forgets the message only when the SDK confirms that it removed it;
    • keeps today's behavior if the model already received the message (removed: false). The message still appears as its own turn, so the transcript matches what the model saw, and the agent host logs that it could not be withdrawn.
  • Matching the SDK echo. The SDK's user.message echo carries the message ID that send() returned. The Copilot session now matches the echo to a pending steering message by that ID first, and falls back to matching by text only when the echo or the pending message has no ID yet. Without this, two identical steering messages could be mixed up while one of them was being withdrawn: the removed one showed as a turn and the delivered one stayed pending.

Withdrawing a steering message that the runtime already took for the running turn returns removed: false and does not interrupt that turn. The SDK interrupts a turn only to withdraw the message that started it.

This PR covers the first scenario in #336466, where a removed message is still sent. It does not cover the second scenario, where a message is sent while it is being edited. #336895 addresses that scenario by disabling editing of pending steering messages, so this PR uses "Refs" instead of "Fixes".

Before / after

No GIFs. Recording the real Copilot agent was not possible because the test GitHub sign-in kept being revoked, so the agent could not run a turn. The proof is the deterministic tests below. They drive the same code paths with the existing fake SDK session and fake providers.

How to test

Manual check, signed in, with the Copilot agent in the chat view or the Agents window:

  1. Ask for something slow, for example "Run Start-Sleep 40 in the terminal, then say DONE."
  2. While it runs, type "Also say BANANA" and press Enter to send it as a steering message.
  3. Hover the steering row and select Remove from Queue.
  4. Expected: the row disappears and stays gone, no "Also say BANANA" request appears, and the reply does not mention BANANA. The agent host log shows Withdrew steering message.

Automated:

scripts\test.bat --run src/vs/platform/agentHost/test/node/copilotAgentSession.test.ts --run src/vs/platform/agentHost/test/node/chatContributions.test.ts --run src/vs/platform/agentHost/test/node/agentService.test.ts --grep "withdraw|the user removes|removed steering|remove a steering message mid-turn"

What the new tests check

Test On main With this PR
AgentService: a user can remove a steering message mid-turn when the provider can withdraw it The host rejects the removal with "Cannot change a steering message while it is being processed", and the message stays queued. The removal is accepted, the queue is empty, and the provider is asked to withdraw the message.
Queue drain: withdraws steering the user removes mid-turn, but not steering the agent consumed The provider is never asked to withdraw anything. Only the message the user removed is withdrawn. A message the agent already consumed is not.
Copilot session: a steering message the user removes before the model reads it is withdrawn and never becomes a turn No way to withdraw exists (withdrawSteering is not a function). The fake SDK then delivers the message, as the real runtime would. The SDK receives withdrawMessage with the ID and prompt it was sent, and no turn starts for the message.
Copilot session: a steering message the user removes while it is still being prepared is never sent Same as above. Nothing is sent to the SDK, and no turn starts.
Copilot session: a removed steering message the model already received still shows as its own turn Same as above. When the SDK reports that it could not remove the message, the echo still opens a turn, so the transcript matches what the model saw.
Copilot session: an identical steering message delivered while another is being withdrawn becomes its own turn Same as above. Two steering messages have the same text. While the first one's withdrawal is still waiting for the SDK, the SDK delivers the second one. The second one becomes the turn, and nothing is left pending.

Validation

Check Command Result
New tests on main @ dc811b77cd1 (fix reverted, tests kept) the scripts\test.bat ... --grep command above ❌ 6 failing, as expected. The host rejects the removal and the message stays queued; the queue drain never withdraws; the Copilot session has no withdraw path.
New tests with this PR same ✅ 6 passing
All Agent Host unit tests scripts\test.bat --runGlob "vs/platform/agentHost/test/**/*.test.js" ✅ 14305 passing, 82 pending, 0 failing
Type check npm run typecheck-client ✅ no errors
Lint npm run eslint -- <the 9 changed .ts files> ✅ no problems
Real-model check — ⏭️ skipped: the test GitHub sign-in kept being revoked

The Agent Host test run unsets COPILOT_HOME. One existing test (isolates peer runtimes, preserves local adoption identity and refuses revoked prepared work) compares the SDK storage path against process.env, so it fails whenever COPILOT_HOME is set, as it is inside a Copilot CLI session. That test does not involve steering.

This PR and #342373, which runs late steering messages as the next turn, both touch sendSteering and the queue drain contribution. Whichever merges second needs a small rebase.


Found with automated exploratory testing and code reading; fixed with deterministic unit tests. Draft until reviewed by a human.

Removing a pending steering message during an Agent Host turn was refused
by the host, and nothing withdrew the message from the Copilot SDK, so it
was still delivered as its own turn. Let providers that can withdraw steering
accept the removal, and have Copilot withdraw it from the SDK steering queue
(or skip sending it if it is still being prepared). A message the model
already received is kept and still shown as its own turn.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 18, 2026 07:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An echo arriving during withdrawal can recreate a removed identical-text request and leave the consumed message queued.

1 open issue
What changed in this PR

Adds provider-supported withdrawal of removed steering messages, addressing the removal scenario in #336466 while leaving editing restrictions separate.

Changes:

  • Allows mid-turn removal for providers with withdrawal support.
  • Withdraws Copilot SDK messages while preserving already-delivered messages in the transcript.
  • Adds regression tests and documents the behavior.
File Description
src/​vs/​platform/​agentHost/​test/​node/​mockAgent.ts Adds optional withdrawal mock.
src/​vs/​platform/​agentHost/​test/​node/​copilotAgentSession.test.ts Tests preparation, withdrawal, and already-delivered messages.
src/​vs/​platform/​agentHost/​test/​node/​chatContributions.test.ts Tests client-only withdrawal dispatch.
src/​vs/​platform/​agentHost/​test/​node/​agentService.test.ts Tests provider-supported mid-turn removal.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotAgentSession.ts Tracks SDK sends and implements withdrawal.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotAgent.ts Forwards withdrawal to the chat session.
src/​vs/​platform/​agentHost/​node/​chatContributions/​queueDrain/​queueDrainContribution.ts Requests withdrawal for client removals.
src/​vs/​platform/​agentHost/​node/​agentService.ts Gates removal on provider support.
src/​vs/​platform/​agentHost/​common/​agent.ts Defines the optional withdrawal contract.
src/​vs/​platform/​agentHost/​AGENTS.md Documents steering withdrawal behavior.

Review effort: Balanced

Comment thread src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts
When two steering messages had the same text and one was being withdrawn,
the echo for the other one could be matched to the withdrawn message. That
showed the removed message as a turn and left the delivered one pending.
The SDK echo carries the id that send returned, so prefer an exact id match
and only fall back to matching by text for messages without a known id.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@karthiknadig
Karthik Nadig (karthiknadig) marked this pull request as ready for review October 18, 2026 19:09
@karthiknadig Karthik Nadig (karthiknadig) added the copilot-cli-agent Background Agent related features/bugs label Oct 18, 2026
@karthiknadig
Karthik Nadig (karthiknadig) merged commit 564f458 into main Oct 18, 2026
49 checks passed
@karthiknadig
Karthik Nadig (karthiknadig) deleted the karthiknadig/bug-swarm/withdraw-removed-steering branch October 18, 2026 23:03
@vs-code-engineering vs-code-engineering Bot added this to the 1.143.0 milestone Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

copilot-cli-agent Background Agent related features/bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants