Skip to content

test: nudge team leader to coordinate in team room#1015

Open
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:fix/test-21-leader-dm-nudge
Open

test: nudge team leader to coordinate in team room#1015
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:fix/test-21-leader-dm-nudge

Conversation

@RerankerGuo

@RerankerGuo RerankerGuo commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • wait for the asynchronously injected Team Leader context before test-18 reads the Leader AGENTS.md
  • when test-21 sees Leader activity only in the Leader DM, send one corrective nudge in the Team Room and mention the Leader there
  • keep the existing runtime-error and Team Room coordination assertions intact

Root causes observed in CI

  1. test-18 could read Leader AGENTS.md before TeamReconciler's context write became visible in MinIO.
  2. The Leader sometimes acknowledged the assignment in its DM but kept doing the work itself. A corrective DM message also stayed in that DM; placing the one-shot nudge in the Team Room gives the Leader the correct coordination context.

Temporary dependency

This branch is temporarily stacked on #985. The previous #1015 run's only two
failures were SHARD_A test-06-multi-worker: in both logs the Manager never
created Bob. #985 replaces that LLM-driven setup with deterministic controller
setup and is fully green across all 10 integration matrix jobs.

Relative to #985, this PR still changes only:

  • tests/test-18-team-config-verify.sh
  • tests/test-21-team-project-dag.sh

After #985 merges, this PR automatically reduces to those two files.

Validation

  • bash -n tests/test-06-multi-worker.sh tests/test-18-team-config-verify.sh tests/test-21-team-project-dag.sh
  • git diff --check against both current main and test: make multi-worker bob setup deterministic #985
  • previous targeted SHARD_C (test-18) and SHARD_D (test-21) jobs passed
  • refreshed CI run 29250335473: all builds and all 10 integration matrix jobs passed, including both previously failing SHARD_A variants and SHARD_D/test-21

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=copaw / wk=copaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr-2 / mgr=copaw / wk=copaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

📊 CI Metrics Report

Summary

Metric Current Baseline Change
LLM Calls 71 81 -10 ↓ -12.3%
Input Tokens 2091854 2803871 -712017 ↓ -25.4%
Output Tokens 11068 16791 -5723 ↓ -34.1%
Total Tokens 2102922 2820662 -717740 ↓ -25.4%

By Role

Role Metric Current Baseline Change
🧠 Manager LLM Calls 56 68 -12 ↓ -17.6%
Input Tokens 1730078 2502214 -772136 ↓ -30.9%
Output Tokens 8747 13725 -4978 ↓ -36.3%
Total Tokens 1738825 2515939 -777114 ↓ -30.9%
🔧 Workers LLM Calls 15 13 +2 ↑ +15.4%
Input Tokens 361776 301657 +60119 ↑ +19.9%
Output Tokens 2321 3066 -745 ↓ -24.3%
Total Tokens 364097 304723 +59374 ↑ +19.5%

Per-Test Breakdown

Test Mgr Calls Wkr Calls Δ Calls Mgr In Wkr In Mgr Out Wkr Out Δ Tokens Trend
02-create-worker 6 0 -6 ↓ -50.0% 157765 0 775 0 -200082 ↓ -55.8% ✅ improved
03-assign-task 9 6 0 — 0% 231375 144494 1338 1037 -95412 ↓ -20.1% — unchanged
04-human-intervene 14 0 +1 ↑ +7.7% 357903 0 1796 0 -73299 ↓ -16.9% ⚠️ regressed
05-heartbeat 6 0 -1 ↓ -14.3% 187574 0 963 0 -86715 ↓ -31.5% ✅ improved
06-multi-worker 21 9 -4 ↓ -11.8% 795461 217282 3875 1284 -262232 ↓ -20.5% ✅ improved

Trends

3 test(s) improved (fewer LLM calls)
⚠️ 1 test(s) regressed (more LLM calls)


Generated by HiClaw CI on 2026-07-13 13:07:56 UTC


📦 Download debug logs & test artifacts

@RerankerGuo
RerankerGuo force-pushed the fix/test-21-leader-dm-nudge branch from cc32546 to ac100d7 Compare July 10, 2026 06:49
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=copaw / wk=hermes)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@RerankerGuo
RerankerGuo force-pushed the fix/test-21-leader-dm-nudge branch from ac100d7 to 702f12c Compare July 13, 2026 05:14
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=openclaw / wk=openclaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@RerankerGuo
RerankerGuo force-pushed the fix/test-21-leader-dm-nudge branch from 685ac4b to c86d3d6 Compare July 14, 2026 11:02
@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=copaw / wk=copaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=openclaw / wk=openclaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=openclaw / wk=openclaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=copaw / wk=hermes)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=copaw / wk=copaw)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=copaw / wk=hermes)

Commit: c86d3d6
Workflow run: #1648

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@RerankerGuo

Copy link
Copy Markdown
Contributor Author

Fresh independent reproduction after the repository secret recovered: #1051 base-branch job 87604983224 reached test-21 and passed 43/44 assertions, but the Leader replied only in its DM (I need to include the roomId for the dev worker.) and never posted the Team Room assignment. The test then failed on the exact DM-only coordination path this PR handles with a one-shot Team Room nudge. This failure is unrelated to #1051’s GitHub token normalization and provides current-main evidence for #1015.

@shiyiyue1102

Copy link
Copy Markdown
Collaborator

Test stabilization is useful, but the current stacked structure creates more maintenance cost than the test-only value justifies. #985, #1015, and #1007 share or depend on setup changes, and those commits are also leaking into unrelated PR diffs. Please flatten this into one independent test-only branch, or land the dependency first and then rebase each remaining PR so every diff contains only its own change. Also document the intended merge order if the PRs must stay separate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants