fix: move Python venv out of agent artifact upload path to restore MCP telemetry observability

[Copilot]

The daily-performance-summary workflow was flagged as missing MCP telemetry (gateway.jsonl / rpc-messages.jsonl) — but both files ARE generated. The root cause: shared Python setup components created a venv at /tmp/gh-aw/agent/venv, which is unconditionally included in the compiler-generated agent artifact upload path (/tmp/gh-aw/agent/). This inflated the artifact to ~253 MB / ~19,448 files (numpy, pandas, matplotlib, scipy + all deps), causing observability report downloads to time out before locating the MCP telemetry files.

Changes

• shared/trending-charts-simple.md, shared/python-nlp.md, shared/python-dataviz.md: Changed Python venv location from /tmp/gh-aw/agent/venv → /tmp/gh-aw/python/venv

/tmp/gh-aw/python/ is not in the artifact upload path list, so the venv is no longer uploaded. Agent artifacts for affected workflows drop from ~253 MB back to a few MB of actual log files.

- if [ ! -d /tmp/gh-aw/agent/venv ]; then
-   python3 -m venv /tmp/gh-aw/agent/venv
+ # Use /tmp/gh-aw/python/venv to avoid polluting the agent artifact upload path (/tmp/gh-aw/agent/)
+ if [ ! -d /tmp/gh-aw/python/venv ]; then
+   python3 -m venv /tmp/gh-aw/python/venv
  fi
- echo "/tmp/gh-aw/agent/venv/bin" >> "$GITHUB_PATH"
+ echo "/tmp/gh-aw/python/venv/bin" >> "$GITHUB_PATH"

All 268 workflows recompiled. Affects any workflow importing these shared components.

• Fixes [deep-report] Daily Project Performance Summary Generator (MCP Scripts) missing gateway.jsonl / rpc-messages.jsonl artifacts #49365

[pelikhan]

@copilot they are in the agent artifact

[github-actions]

Thanks for opening this PR on behalf of the core team! 👋 The investigation summary is clear and well-documented. Here's what looks solid:

• On-topic fix — addressing the critical observability gap (missing MCP telemetry artifacts) flagged in issue [deep-report] Daily Project Performance Summary Generator (MCP Scripts) missing gateway.jsonl / rpc-messages.jsonl artifacts #49365
• Clear scope — narrowly focused on relocating Python venv paths across three workflow files to reduce artifact bloat
• Strong investigation — root cause analysis showing the 253 MB artifact issue is well-documented

Since this is still a WIP draft (0 changes committed), here's what's next:

1. Complete the checklist — implement the three workflow file updates (shared/trending-charts-simple.md, shared/python-nlp.md, shared/python-dataviz.md) to move venv from /tmp/gh-aw/agent/venv to /tmp/gh-aw/python/venv
2. Regenerate locks — run make recompile to update lock files
3. Validate — test that the artifact size reduction eliminates the timeout issue
4. Mark ready — remove the [WIP] prefix once tests pass

Complete the Python venv relocation fix:

1. Update `/tmp/gh-aw/agent/venv` references to `/tmp/gh-aw/python/venv` in:
   - shared/trending-charts-simple.md
   - shared/python-nlp.md
   - shared/python-dataviz.md

2. Run `make recompile` to regenerate workflow lock files

3. Validate the artifact size reduction by re-running the affected workflows and confirming:
   - gateway.jsonl and rpc-messages.jsonl artifacts are present
   - artifact download completes successfully
   - artifact size is significantly reduced

4. When validation passes, remove [WIP] from the PR title and mark ready for merge

Generated by ✅ Contribution Check · auto · 49.9 AIC · ⌖ 5.03 AIC · ⊞ 8.7K · ◷

[Copilot]

@copilot they are in the agent artifact

Fixed in commit fix: update remaining /tmp/gh-aw/agent/venv references in dataflow and nlp workflows. The initial fix missed two files:

• dataflow-pr-discussion-dataset.md: Had its own python3 -m venv /tmp/gh-aw/agent/venv step plus 8 references to /tmp/gh-aw/agent/venv/bin/python3 in the prompt body — all updated to /tmp/gh-aw/python/venv.
• copilot-pr-nlp-analysis.md: Had a stale reference to /tmp/gh-aw/agent/venv/bin/python3 in the prompt instructions (the shared python-nlp.md was already fixed, but the wrapper workflow still pointed to the old path).

All 268 workflows recompiled cleanly. No /tmp/gh-aw/agent/venv references remain in any .md or .lock.yml files.

[github-actions]

🔍 Triage Summary

Field	Value
Category	bug
Risk	medium
Priority	medium
Score	45/100 (impact 25 + urgency 15 + quality 5)
Recommended action	defer
Batch	—

Moves Python venv out of artifact upload path to fix MCP telemetry visibility. Draft, no CI, touches shared components.

Automated triage — see the triage report for full context.

Generated by 🔧 PR Triage Agent · auto · 40.3 AIC · ⌖ 4.01 AIC · ⊞ 8K · ◷

[pelikhan]

@copilot update instructions or built prompts to specify where the user python venv should be created

[Copilot]

@copilot update instructions or built prompts to specify where the user python venv should be created

Done in commit fix: update instructions to specify Python venv location (/tmp/gh-aw/python/venv). Two instruction/documentation files updated:

• .github/aw/charts.md: Replaced pip install --user in both Option A and Option B examples with the canonical venv setup at /tmp/gh-aw/python/venv, matching the pattern already used in the shared components.
• .github/workflows/daily-agentrx-trace-optimizer.md: Updated the agent prompt to create the AgentRx venv at /tmp/gh-aw/python/agentrx/.venv instead of /tmp/gh-aw/agent/agentrx/.venv, with an inline note explaining why.

All 268 workflows recompiled cleanly.

[Copilot]

Pull request overview

Moves Python virtual environments outside the agent artifact path, preventing oversized uploads from obscuring MCP telemetry.

Changes:

• Relocates virtual environments to /tmp/gh-aw/python/.
• Updates workflow prompts and shared components to use the new paths.
• Regenerates affected workflow lock files.

Show a summary per file

File	Description
.github/aw/charts.md	Updates chart setup examples.
.github/workflows/shared/trending-charts-simple.md	Relocates charting venv.
.github/workflows/shared/python-nlp.md	Relocates NLP venv.
.github/workflows/shared/python-dataviz.md	Relocates visualization venv.
.github/workflows/dataflow-pr-discussion-dataset.md	Relocates DataFlow venv and references.
.github/workflows/dataflow-pr-discussion-dataset.lock.yml	Recompiles DataFlow setup.
.github/workflows/daily-agentrx-trace-optimizer.md	Updates AgentRx venv instructions.
.github/workflows/daily-agentrx-trace-optimizer.lock.yml	Refreshes compiled metadata.
.github/workflows/copilot-pr-nlp-analysis.md	Updates runtime Python path.
.github/workflows/copilot-pr-nlp-analysis.lock.yml	Recompiles NLP setup and prompt.
.github/workflows/weekly-issue-summary.lock.yml	Recompiles visualization setup.
.github/workflows/stale-repo-identifier.lock.yml	Recompiles charting setup.
.github/workflows/python-data-charts.lock.yml	Recompiles visualization setup.
.github/workflows/prompt-clustering-analysis.lock.yml	Recompiles NLP and charting setup.
.github/workflows/portfolio-analyst.lock.yml	Recompiles visualization setup.
.github/workflows/org-health-report.lock.yml	Recompiles visualization setup.
.github/workflows/github-mcp-structural-analysis.lock.yml	Recompiles visualization setup.
.github/workflows/detection-analysis-report.lock.yml	Recompiles charting setup.
.github/workflows/daily-security-observability.lock.yml	Recompiles Python setup components.
.github/workflows/daily-repo-chronicle.lock.yml	Recompiles visualization setup.
.github/workflows/daily-performance-summary.lock.yml	Removes the venv from its agent artifact path.
.github/workflows/daily-news.lock.yml	Recompiles visualization setup.
.github/workflows/daily-issues-report.lock.yml	Recompiles NLP and visualization setup.
.github/workflows/daily-experiment-report.lock.yml	Recompiles charting setup.
.github/workflows/daily-code-metrics.lock.yml	Recompiles visualization setup.
.github/workflows/copilot-session-insights.lock.yml	Recompiles visualization setup.
.github/workflows/audit-workflows.lock.yml	Recompiles charting setup.
.github/workflows/api-consumption-report.lock.yml	Recompiles charting setup.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 28/28 changed files
• Comments generated: 0
• Review effort level: Balanced