feat: update e2e tests to properly work with async mode#840
Open
feat: update e2e tests to properly work with async mode#840
Conversation
- Modified setup() to start a per-test daemon with isolated HOME, config, and Unix domain sockets for async processing - Modified teardown() to gracefully shut down daemon and clean up - Wrapped git() and git-ai() shell helpers to pass daemon env vars (socket paths, trace2, force-TTY, timeout) - Set init.defaultBranch=main in global gitconfig before git init - Added wait_for_note() helper for polling authorship notes after non-commit operations (rebase, cherry-pick, merge --squash) - Inserted wait_for_note calls after all rebase operations in tests - Updated CI workflows to use GIT_AI_ASYNC_MODE=true Closes #839 Co-Authored-By: Sasha Varlamov <sasha@sashavarlamov.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
These workflows run shell scripts that don't start a daemon or set force-TTY/timeout env vars, so async mode would race. Only the BATS e2e tests (e2e-tests.yml) have proper daemon infrastructure. Co-Authored-By: Sasha Varlamov <sasha@sashavarlamov.com>
Add reusable daemon helper scripts (start-async-daemon.sh, stop-async-daemon.sh) that handle daemon lifecycle in CI: - Create isolated daemon home with async_mode config - Probe /usr/bin/git to avoid pointing at the proxy symlink - Export env vars (GIT_AI_TEST_FORCE_TTY, POST_COMMIT_TIMEOUT_MS, socket paths) to GITHUB_ENV for subsequent steps - Graceful shutdown with force-kill fallback Update install-scripts-local.yml: - Flip GIT_AI_ASYNC_MODE to true at workflow level - Add Start/Stop async daemon steps around Unix test steps - Override GIT_AI_ASYNC_MODE to false for Windows job (no Unix-domain socket support) Update nightly-agent-integration.yml: - Flip GIT_AI_ASYNC_MODE to true at workflow level - Add Start/Stop async daemon steps in both Tier 1 and Tier 2 jobs Co-Authored-By: Sasha Varlamov <sasha@sashavarlamov.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables async mode (
GIT_AI_ASYNC_MODE: "true") across all three CI workflows by starting a per-test/per-job daemon and configuring the wrapper to poll for authorship notes.BATS e2e tests (
tests/e2e/user-scenarios.bats):setup()creates an isolatedTEST_DAEMON_HOME(overriding$HOME) with async-mode config, starts the daemon viagit-ai bg run, and polls for Unix domain sockets to appeargit()andgit-ai()shell wrappers now pass daemon socket paths, trace2 config, force-TTY, and a 30s post-commit timeoutteardown()sendsbg shutdown, waits up to 2s, thenkill -9as fallback, restores$HOME, and cleans up both temp dirswait_for_note()helper polls for authorship notes (up to 20s) after non-commit operations like rebase where the wrapper doesn't auto-pollinit.defaultBranch mainset in global gitconfig beforegit init(required because isolated HOME loses the system default)Reusable daemon helper scripts (
scripts/nightly/):start-async-daemon.sh: creates isolated daemon home with async-mode config, probes/usr/bin/gitand/usr/local/bin/gitto avoid accidentally using the git-ai proxy symlink, starts daemon in background, polls for sockets (10s), exports all env vars toGITHUB_ENVfor subsequent workflow stepsstop-async-daemon.sh: gracefulbg shutdownvia control socket, 2s wait, force-kill fallback, daemon home cleanupCI workflow changes:
e2e-tests.yml:GIT_AI_ASYNC_MODEflipped to"true"install-scripts-local.yml:GIT_AI_ASYNC_MODEflipped to"true"with Start/Stop daemon steps around Unix test steps; Windows job overrides to"false"(no Unix-domain socket support)nightly-agent-integration.yml:GIT_AI_ASYNC_MODEflipped to"true"with Start/Stop daemon steps in both Tier 1 (hook wiring + synthetic checkpoint) and Tier 2 (live agent + attribution verification) jobsCloses #839
Review & Testing Checklist for Human
source start-async-daemon.shin one step and must stay alive for subsequent steps. The PID is persisted viaGITHUB_ENVfor the stop step. Verify background processes are not killed between steps on Ubuntu and macOS runners.test-synthetic-checkpoint.sh,verify-synthetic-attribution.sh, etc.) are not modified — they depend onGIT_AI_TEST_FORCE_TTYandGIT_AI_POST_COMMIT_TIMEOUT_MSbeing set in the environment viaGITHUB_ENV. Confirm these are correctly inherited by thebashsubprocesses that run the scripts.start-async-daemon.shprobes/usr/bin/gitthen/usr/local/bin/gitto avoid the proxy symlink. Verify this finds the correct git binary onmacos-latestrunners (Xcode CLT vs Homebrew).git_hooks_enabled: falsein daemon config doesn't prevent authorship note generation — the daemon should use trace2 events rather than git hooks, but verify this is the correct code path in async mode.install-scripts-localandnightly-agent-integrationworkflows manually on this branch (applyintegrationlabel to the PR) and verify all jobs pass. Forinstall-scripts-local, check both Ubuntu and macOS Unix jobs succeed and Windows jobs still pass with sync mode.Notes
task lintandtask format:checkpassinstall-scripts-local.ymlremain on sync mode (GIT_AI_ASYNC_MODE: "false") because Unix-domain sockets are unavailableGIT_AI_TEST_FORCE_TTY=1+GIT_AI_POST_COMMIT_TIMEOUT_MS=30000);wait_for_note()is only needed after rebase/squash operations in the BATS testswait_for_note20s — confirm these are sufficient under loadLink to Devin session: https://app.devin.ai/sessions/85ba6a1442ec430b807e35f9d66c23dc
Requested by: @svarlamov