feat: extended protocol for conversation-based agents experience support #368

volodkevych · 2025-03-14T19:15:34Z

Problem

We need to extend chat protocol to support agent-specific flow.

Solution

Protocol and server interface were extended to support:

'aws/chat/sendChatUpdate' - for pushing async updates from server to client
'aws/chat/fileClick' - for opening file or executing related file action, if provided
'aws/selectWorkspaceItem' - to give possibility to prompt customer select another workspace folder in IDE if current is too big
'aws/openFileDiff' - to trigger open file differences in editor

ChatOptions QuickActionCommand were extended:

disabled was removed - disabling quick actions within quick action tabs needs to be re-modelled. Current approach will not work as chat options from different servers are merged in Flare runtimes. No real client is using this field now.

OpenTabParams params were extended to give possibility to configure contents and state of the tab if a new tab is requested.

Other minor changes:
fileList and prompts supported in chat message responses.
placeholderText can be set.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

runtimes/runtimes/chat/encryptedChat.test.ts

kmile · 2025-03-17T15:52:57Z

runtimes/protocol/chat.ts

@@ -81,3 +90,9 @@ export const followUpClickNotificationType = new ProtocolNotificationType<Follow
 export const openTabRequestType = new ProtocolRequestType<OpenTabParams, OpenTabResult, never, void, void>(
    OPEN_TAB_REQUEST_METHOD
 )
+export const chatUpdateNotificationType = new ProtocolNotificationType<ChatUpdateParams, void>(


Does this update existing chat messages in the current tab or add new ones?

Add and update - depends on messageId provided.

Ok! No deletion.

runtimes/protocol/chat.ts

kmile · 2025-03-17T15:54:47Z

runtimes/runtimes/base-runtime.ts

@@ -122,6 +127,7 @@ export const baseRuntime = (connections: { reader: MessageReader; writer: Messag
        onChatPrompt: handler => lspConnection.onRequest(chatRequestType.method, handler),
        onEndChat: handler => lspConnection.onRequest(endChatRequestType.method, handler),
        onQuickAction: handler => lspConnection.onRequest(quickActionRequestType.method, handler),
+        onQuickActionTrigger: handler => lspConnection.onNotification(quickActionNotificationType.method, handler),


There is a request onQuickAction, and a notification onQuickActionTrigger. Why we need both is a bit unclear.

And the type of onQuickActionTrigger is quickActionNotification.

Same as above.

kmile · 2025-03-17T15:56:19Z

runtimes/protocol/workspace.ts

+    SelectWorkspaceItemResult,
+} from './lsp'
+
+export const selectWorkspaceItemRequestType = new ProtocolRequestType<


What;s the difference between this and showDocument?

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#window_showDocument

The intent and protocol is different. We need to get selected items back in server.

FYI: I think we usually run into issues when we use ProtocolRequestType and therefore usually use the AutoParameterStructuresProtocolRequestType

What kind of issues?

When would the client send this? What is the use case?

kmile · 2025-03-17T15:58:22Z

runtimes/runtimes/base-runtime.ts

@@ -134,6 +140,8 @@ export const baseRuntime = (connections: { reader: MessageReader; writer: Messag
        onSourceLinkClick: handler => lspConnection.onNotification(sourceLinkClickNotificationType.method, handler),
        onFollowUpClicked: handler => lspConnection.onNotification(followUpClickNotificationType.method, handler),
        openTab: params => lspConnection.sendRequest(openTabRequestType.method, params),
+        chatUpdate: params => lspConnection.sendNotification(chatUpdateNotificationType.method, params),


The wording of this seems to be reversed compared to the others:

onLinkClick

openFileDiff

showMessage

openTab

chatUpdate

should that be updateChat? or sendChatUpdate?

I agree with Kamiel, updateChat and sendChatUpdate feel easier to understand

types/chat.ts

kmile · 2025-03-17T16:06:11Z

types/chat.ts

@@ -120,14 +132,18 @@ export interface QuickActions {
    quickActionsCommandGroups: QuickActionCommandGroup[]
 }

+export interface TabData {
+    placeholderText?: string
+    messages: ChatMessage[]


New messages? Replace matched messages? Replace entire tab?

this is an update with new or updated messages - depends on messageId

^ should that insight/info be a docstring comment on the field here (or somewhere)?

#387

kmile · 2025-03-17T16:07:14Z

types/workspace.ts

@@ -0,0 +1,24 @@
+import { URI, TextDocument } from './lsp'
+
+export const SELECT_WORKSPACE_ITEM_REQUEST_METHOD = 'aws/selectWorkspaceItem'


Is this coming from the client or the server? In what situation will this be used?

Server to client, to request customer request to select folder if workspace folder is too big.

ege0zcan · 2025-03-19T12:45:41Z

types/workspace.ts

+    items: WorkspaceItem[]
+}
+
+export interface OpenFileDiffParams {


In this case originalFileUri is the initial version of the file, and isDeleted specificies if the original file was deleted and fileContent contains the new/updated file contents, is this understanding correct?

ege0zcan · 2025-03-19T12:48:12Z

runtimes/runtimes/standalone.ts

@@ -319,6 +321,9 @@ export const standalone = (props: RuntimeProps) => {
                    onDidDeleteFiles: params => lspConnection.workspace.onDidDeleteFiles(params),
                    onDidRenameFiles: params => lspConnection.workspace.onDidRenameFiles(params),
                    onUpdateConfiguration: lspServer.setUpdateConfigurationHandler,
+                    selectWorkspaceItem: params =>
+                        lspConnection.sendRequest(selectWorkspaceItemRequestType.method, params),


nit: Do we have to specify selectWorkspaceItemRequestType.method? I remember if we just passed selectWorkspaceItemRequestType it also worked and I remember when you pass the .method which is a string, you lose some benefits that come with typescript

Same applies to openFileDiff

I've seen different versions used throughout the codebase. If you know what it impacts exactly, I can change.

ege0zcan · 2025-03-19T13:15:48Z

runtimes/server-interface/lsp.ts

+        selectWorkspaceItem: (
+            handler: RequestHandler<
+                SelectWorkspaceItemParams,
+                SelectWorkspaceItemResult | undefined | null,


Isn't this second field for partial result? Does this mean select workspace item will support partial results?

ege0zcan · 2025-03-19T13:24:49Z

types/chat.ts

+    data?: TabData
+}
+
+export type FileAction = 'accept-change' | 'reject-change' | string


what is the purpose of string here? Is it for future expansion?

I'll remove it - whatever is in the string needs to be anyway supported by mynah-ui so better to keep it explicit.

types/chat.ts

ege0zcan · 2025-03-19T13:29:23Z

types/chat.ts

 export interface OpenTabResult extends TabEventParams {}
+
+export interface TabState {


what does it mean for a tab to be in progress and cancellable?

"cancellable" - practically means stop button will be shown
"inProgress" - it's being worked on by server, so no input should be accepted and progress bar shown

…t and options for open tab

ege0zcan · 2025-03-19T15:28:36Z

runtimes/server-interface/lsp.ts

@@ -136,6 +139,10 @@ export type Lsp = {
        onDidDeleteFiles: (handler: NotificationHandler<DeleteFilesParams>) => void
        onDidRenameFiles: (handler: NotificationHandler<RenameFilesParams>) => void
        onUpdateConfiguration: (handler: RequestHandler<UpdateConfigurationParams, void, void>) => void
+        selectWorkspaceItem: (
+            handler: RequestHandler<SelectWorkspaceItemParams, SelectWorkspaceItemResult | undefined | null, void>


why do we allow for undefined or null as a result?

Customer could click cancel in the dialog popup, something needs to be returned in this case.

What operations trigger this dialog popup? Is there another (existing or new) request that this relates to?

Followup action called "Choose another folder in your workspace". This is needed to request customer select a folder for context.

kmile · 2025-03-19T16:02:19Z

runtimes/README.md

+
+| Description                          | Method                            | Params                         | Method type                                                                                                                     | Response Type   |
+| ------------------------------------ | --------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------- | --------------- |
+| Request to select workspace item (folder, file) with the selected items returned | `aws/selectWorkspaceItem`         | `SelectWorkspaceItemParams`                   | [Request](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#requestMessage) Server to Client           | `SelectWorkspaceItemResult`    |


I don't understand what this means. Does this return the file contents? Show it on screen?

This returns URI to selected folder/file back to LSP server. I do not think we need to return any contents as LSP server has access to contents if needed. The purpose of this method is to give customer possibility to select.

justinmk3 · 2025-03-20T17:28:07Z

types/chat.ts

+
+export interface TabState {
+    inProgress?: boolean
+    cancellable?: boolean


/** Server is working, should show progress bar, should not accept input. */ inProgress?: boolean /** Should show stop button. */ cancellable?: boolean

volodkevych force-pushed the agents-through-chat-protocol branch from f6d6870 to 46f881c Compare March 14, 2025 19:20

volodkevych commented Mar 14, 2025

View reviewed changes

runtimes/runtimes/chat/encryptedChat.test.ts Outdated Show resolved Hide resolved

kmile reviewed Mar 17, 2025

View reviewed changes

runtimes/protocol/chat.ts Outdated Show resolved Hide resolved

kmile reviewed Mar 17, 2025

View reviewed changes

types/chat.ts Outdated Show resolved Hide resolved

kmile reviewed Mar 17, 2025

View reviewed changes

types/chat.ts Outdated Show resolved Hide resolved

kmile reviewed Mar 17, 2025

View reviewed changes

volodkevych marked this pull request as ready for review March 18, 2025 10:27

volodkevych requested a review from a team as a code owner March 18, 2025 10:27

ege0zcan reviewed Mar 19, 2025

View reviewed changes

types/chat.ts Outdated Show resolved Hide resolved

ege0zcan reviewed Mar 19, 2025

View reviewed changes

volodkevych added 11 commits March 19, 2025 15:31

feat: interface for reporting error from chat-client to hosting clien…

16b22b2

…t and options for open tab

feat: add sendChatUpdate to chat protocol

ca4c656

feat: types for workspace protocol

88356ba

feat: added chat and workspace protocol to runtimes

4499daa

feat: protocol changes for openTab and renamings

1edb709

feat: added quickAction notification for async flows

08317ab

chore: format the changes

a612f0d

feat: added support for prompts

f52d358

feat: added unit tests

e7b6517

fix: remove TabData from QuickAction command config

f816e4f

chore: updated README with new protocol methods

b662593

volodkevych added 8 commits March 19, 2025 15:33

fix: rollback triggerQuickAction notification

00de1a1

fix: rollback triggerQuickAction notification

72acdc2

chore: add more descrition to readme

717b786

fix: removed unused async flag from quick actions

f003317

fix: re-add placeholder in quick actions as it's used in some agents

fb74893

chore: improved comments

32f7b7e

fix: handle review comments

1a6e58e

feat: renamed method for chat updates

e7e4a9f

volodkevych force-pushed the agents-through-chat-protocol branch from a61df5f to e7e4a9f Compare March 19, 2025 14:34

ege0zcan reviewed Mar 19, 2025

View reviewed changes

kmile reviewed Mar 19, 2025

View reviewed changes

justinmk3 mentioned this pull request Mar 19, 2025

feat: added agent protocol #323

Closed

ege0zcan approved these changes Mar 19, 2025

View reviewed changes

volodkevych merged commit 0bc496e into main Mar 20, 2025
3 checks passed

volodkevych deleted the agents-through-chat-protocol branch March 20, 2025 10:17

github-actions bot mentioned this pull request Mar 20, 2025

chore(release): release packages from branch main #384

Merged

justinmk3 reviewed Mar 20, 2025

View reviewed changes

justinmk3 mentioned this pull request Mar 20, 2025

docs: conversation-based agents #387

Open

		@@ -0,0 +1,24 @@
		import { URI, TextDocument } from './lsp'

		export const SELECT_WORKSPACE_ITEM_REQUEST_METHOD = 'aws/selectWorkspaceItem'

		export interface OpenTabResult extends TabEventParams {}

		export interface TabState {

feat: extended protocol for conversation-based agents experience support #368

feat: extended protocol for conversation-based agents experience support #368

Uh oh!

Conversation

volodkevych commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

License

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kmile Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kmile Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justinmk3 Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

volodkevych commented Mar 14, 2025 •

edited

Loading

kmile Mar 17, 2025 •

edited

Loading

kmile Mar 17, 2025 •

edited

Loading

justinmk3 Mar 20, 2025 •

edited

Loading

justinmk3 Mar 20, 2025 •

edited

Loading