Skip to content

Update MCP response to improve agent's answers #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
vinamra28 wants to merge 1 commit into
base: main
Choose a base branch
from
+5 −1
Draft

Update MCP response to improve agent's answers #319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions internal/mcp/tools/get_chunks.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ func RegisterGetChunksForEmbeddings(s *mcp.Server, k8sClient *k8sclient.Client,
Description: `Search for relevant text chunks in a pipeline's data product using vector cosine similarity. Returns top matching chunks for the given query.
If pipeline_name is not known, call list_unstructured_data_pipelines_for_user first and follow the instructions in its response.
If the returned chunks are not sufficient to answer the user's question, you may call get_processed_document with the same pipeline_name and the file_id from the top matching chunk to retrieve the full processed document for more context.
If no matching chunks are found, it could be because the user does not have access to the original source file — suggest they request access to it.
Always cite sources: include file_id in your answer so the user can visit the source file. For Google Drive sources, provide the URL as https://drive.google.com/file/d/<file_id>/view.
Comment on lines +48 to +49

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Use neutral guidance for empty search results.

The tools return no access-status signal. An empty result can mean no relevant match or missing source access, so the agent cannot identify access as the cause.

  • internal/mcp/tools/get_chunks.go#L48-L49: Present missing access as one possible cause instead of defaulting to an access request.
  • skills/unstructured-data-mcp-skill.md#L14-L17: State the same uncertainty, correct the grammar, and capitalize Google Drive and URL.
📍 Affects 2 files
  • internal/mcp/tools/get_chunks.go#L48-L49 (this comment)
  • skills/unstructured-data-mcp-skill.md#L14-L17
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/mcp/tools/get_chunks.go` around lines 48 - 49, Update the
empty-result guidance in internal/mcp/tools/get_chunks.go lines 48-49 to present
missing access as one possible cause, not the default explanation or an
automatic request for access; retain the requirement to cite file_id and provide
Google Drive URLs in the specified format. Apply the same uncertainty guidance
in skills/unstructured-data-mcp-skill.md lines 14-17, correcting the grammar and
capitalizing “Google Drive” and “URL”.

Source: Linters/SAST tools

On error: report the exact error to the user and STOP. Do NOT retry with other pipelines.
Comment on lines +48 to 50

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Make citation requirements conditional on source-bearing success responses.

The current “always cite” rules conflict with error and empty-result behavior. These paths can return no verified file_id, so agents may fabricate a citation.

  • internal/mcp/tools/get_chunks.go#L48-L50: Require citations only when a successful result returns a file_id; keep errors citation-free.
  • internal/mcp/tools/get_processed_document.go#L46-L47: Do not treat an echoed request file_id as proof that document retrieval succeeded.
  • skills/unstructured-data-mcp-skill.md#L15-L17: Require citations only when source content and a returned file_id are available.
📍 Affects 3 files
  • internal/mcp/tools/get_chunks.go#L48-L50 (this comment)
  • internal/mcp/tools/get_processed_document.go#L46-L47
  • skills/unstructured-data-mcp-skill.md#L15-L17
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/mcp/tools/get_chunks.go` around lines 48 - 50, Make citation
guidance conditional on verified source-bearing success: in
internal/mcp/tools/get_chunks.go lines 48-50, require a citation only when the
successful result includes a file_id and keep error responses citation-free; in
internal/mcp/tools/get_processed_document.go lines 46-47, do not treat an echoed
request file_id as proof of successful retrieval; in
skills/unstructured-data-mcp-skill.md lines 15-17, require citations only when
source content and a returned file_id are both available.

On follow-up: if the user is not satisfied, ask them which pipeline to search. Do NOT automatically try other pipelines.`,
}, func(ctx context.Context, _ *mcp.CallToolRequest, args getChunksArgs) (*mcp.CallToolResult, any, error) {
Expand Down
1 change: 1 addition & 0 deletions internal/mcp/tools/get_processed_document.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ func RegisterGetProcessedDocument(s *mcp.Server, k8sClient *k8sclient.Client) {
Name: "get_processed_document",
Description: `Retrieve the processed document output for a given file_id from a pipeline's DocumentProcessor stage Snowflake table.
If pipeline_name is not known, call list_unstructured_data_pipelines_for_user first and follow the instructions in its response.
Always cite sources: include file_id in your answer so the user can visit the source file. For Google Drive sources, provide the URL as https://drive.google.com/file/d/<file_id>/view.
On error: report the exact error to the user and STOP. Do NOT retry with other pipelines.`,
}, func(ctx context.Context, _ *mcp.CallToolRequest, args getProcessedDocumentArgs) (*mcp.CallToolResult, any, error) {
username := ""
Expand Down
3 changes: 2 additions & 1 deletion skills/unstructured-data-mcp-skill.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ You have access to an MCP server that lets you search documents and knowledge ba
## Key Rules

- Only use pipeline names returned by `list_unstructured_data_pipelines_for_user`. Never guess.
- Ground every answer in the returned content. If the data doesn't contain the answer, say so.
- Ground every answer in the returned content. If the data doesn't contain the answer, say so. Also, if matching chunks aren't found, it can be due to user doesn't have access to the original file to which they might have to request access.
- If a tool call fails, tell the user there was a technical issue. Don't interpret errors as content.
- One pipeline per search call. If spanning multiple pipelines, make separate calls and attribute answers clearly.
- Give citations at all times so that user can visit the source file for reference. In case of google drive, provide the url as well. URL can be formed as follows: `https://drive.google.com/file/d/<file_id>/view`, where, file_id is the file ID returned from the `get_chunks_for_embeddings` and `get_processed_document` tool calls.
Loading