Skip to content

fix(cookbook): activate local Windows venv in bash runner - #5734

Merged
o3LL merged 1 commit into
odysseus-dev:devfrom
zoomdbz:fix/local-windows-bash-venv-prefix-v2
Aug 18, 2026
Merged

fix(cookbook): activate local Windows venv in bash runner#5734
o3LL merged 1 commit into
odysseus-dev:devfrom
zoomdbz:fix/local-windows-bash-venv-prefix-v2

Conversation

@zoomdbz

@zoomdbz zoomdbz commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Local Windows Cookbook serve/download tasks run through detached Git Bash, while the frontend builds PowerShell venv activation prefixes such as & C:\path\Scripts\Activate.ps1. The primary Serve and Download emitters previously left those paths unquoted. POSIX parsing then stripped backslashes or split paths containing spaces, so the selected venv never activated.

This PR:

  • Quotes venv activation paths in the primary Serve and Download emitters.
  • Converts quoted and unquoted local-Windows PowerShell venv prefixes to Git Bash source /c/.../Scripts/activate commands before the existing env-prefix validator runs.
  • Reuses the existing _git_bash_path converter and rejects unsupported or unsafe input shapes.
  • Uses a linear parser instead of a user-controlled regular expression.
  • Adds regression coverage for quoted and unquoted paths, spaces, forward slashes, pass-through behavior, frontend emitters, route scoping, and long whitespace input.

Remote Windows, POSIX, and conda prefixes remain unchanged. Supersedes #4894, which was closed after dev was force-updated without absorbing this fix.

Target branch

  • This PR targets dev, not main. All PRs land in dev; main is curated by the maintainer at each release.

Linked Issue

Part of #2710; supersedes #4894

Type of Change

  • Bug fix (non-breaking - fixes a confirmed issue)
  • New feature (non-breaking - adds new behavior)
  • Breaking change (changes or removes existing behavior)
  • Refactor / cleanup (behavior unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs - this is not a duplicate.
  • This PR targets dev
  • My changes are limited to the scope described above - no unrelated refactors or whitespace changes mixed in.
  • I actually ran the app (uvicorn app:app) and verified the primary native Windows Download and Serve paths.
  • I did not run the app/runtime validation and stated that gap in How to Test. Leave this unchecked when the app-run box above is checked.

How to Test

Native Windows runtime validation was performed on rebased local head 254f0aff. Its change patch has the same Git patch ID as PR head 85ad875c.

  1. Booted an isolated native Windows app instance with uvicorn app:app on 127.0.0.1:7099.
  2. Configured Local for the Windows venv at G:\AI\odysseus\venv.
  3. Ran a model download through the primary Download path; hf-internal-testing/tiny-random-gpt2 completed and the API returned 200.
  4. Ran a primary Serve task using python -m pip --version; pip resolved from G:\AI\odysseus\venv\Lib\site-packages and the process exited 0.
  5. Stopped the isolated validation app. The daily-driver app and existing model processes were not touched.

Automated validation performed:

  • Focused Windows env-prefix regression selection: 15 passed, 69 deselected, 2 warnings.
  • Related Cookbook suites: 25 passed, 2 warnings.
  • Python compilation and node --check for both changed JavaScript modules passed.
  • GitHub Actions full pytest suite: 5696 passed, 4 skipped, 8 warnings.
  • CodeQL, Trivy, dependency review, secret scan, and workflow security checks passed.

Residual gap: G:\AI\odysseus\.runtime-validation\Venv With Space was configured in the native Windows UI, but that second path was not executed end-to-end. Quoted and unquoted paths containing spaces remain covered by the focused regression suite.

Visual / UI changes - REQUIRED if you touched anything that renders

No rendered layout, CSS, icon, typography, or component behavior changed. static/js/cookbookRunning.js and static/js/cookbookDownload.js only quote command payload values before requests are sent.

  • Screenshot or short clip of the change in the running app, attached below.
  • Style match: no styles, colors, fonts, spacing, layout, or rendered components changed.
  • No new component patterns. The change only reuses the existing shared PowerShell quoting helper.
  • I am not an LLM agent submitting a bulk PR. This is a focused single-bug fix with targeted regression coverage.

Screenshots / clips

Native Windows Cookbook runtime validation

The capture shows the primary Download task completed and the primary Serve task resolving pip from the configured local Windows venv before exiting 0. GitHub-hosted evidence source.

@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 24, 2026
@zoomdbz

zoomdbz commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

CI note: the Python tests job is failing during collection before this PR's Cookbook code runs: src/agent_loop.py uses Any without importing it. I opened #5735 as a separate one-line fix for that upstream dev break. Once that lands, this PR can be rebased cleanly.

@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch from 56004fd to cd57f07 Compare July 27, 2026 16:54
@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch from cd57f07 to 6b48a97 Compare August 13, 2026 21:11

@o3LL o3LL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 6b48a97.

The direction is right and the guard is correctly scoped, but I don't think this
fix reaches the paths it's aiming at. Details below.

Findings

P0 issue: the conversion never fires on the main serve and download paths

  • Problem: _local_windows_bash_env_prefix recovers the Windows path with
    shlex.split(ep, posix=True). That only works when the frontend single-quoted
    the path. Three emitters quote through _psQuote, but the two primary ones
    don't:

    • static/js/cookbookRunning.js:1974 in _launchServeTask — the main serve
      launcher, called from the Serve panel (cookbookServe.js:3712), every retry
      and auto-fix path, and the pip-reinstall shortcut.
    • static/js/cookbookDownload.js:541 in _runModelDownload — the model
      Download button.

    Both emit '& ' + envPath with no quoting at all. In posix mode shlex treats
    the backslashes as escapes, so parts[1] comes back as
    C:UsersdomodysseusvenvScriptsActivate.ps1, the [/\\]Activate\.ps1$ guard
    fails to match, and the function returns ep unchanged.

    Running the exact call-site expression
    (_safe_env_prefix(_local_windows_bash_env_prefix(ep))) on this branch:

    & 'C:\Users\dom\odysseus\venv\Scripts\Activate.ps1'   (_psQuote emitters)
      -> [ -f "/c/Users/dom/odysseus/venv/Scripts/activate" ] && source "/c/Users/dom/odysseus/venv/Scripts/activate" || true
    
    & C:\Users\dom\odysseus\venv\Scripts\Activate.ps1     (cookbookRunning.js:1974, cookbookDownload.js:541)
      -> & 'C:UsersdomodysseusvenvScriptsActivate.ps1'
    
  • Impact: On local Windows, launching a serve from the Serve panel or
    starting a download from the Download button still writes a PowerShell line
    into the bash runner — with the path corrupted on top. That is the same
    "venv never activates, packages look missing" symptom #2710 reports, so the
    bug survives this PR on the two flows most people use. The three flows that
    do get fixed (dep install, quick-download box, download retry) are the
    secondary ones.

  • Ask: Quote at the two emitters so they match the other three
    ('& ' + _psQuote(...)), and also make the helper tolerant of the unquoted
    form rather than depending on the caller — it sits in front of the
    _safe_env_prefix validator, so it shouldn't assume its input is well-formed.
    A raw-string match on ^&\s*'?(.+?\.ps1)'?$ before falling back to shlex
    would cover both shapes.

  • Location: routes/cookbook_helpers.py:1212,
    static/js/cookbookRunning.js:1974, static/js/cookbookDownload.js:541

P1 issue: an unquoted path containing a space fails the request outright

  • Problem: Same root cause, different symptom. With
    & C:\Users\dom\My Envs\venv\Scripts\Activate.ps1, posix shlex yields three
    tokens, so _safe_env_prefix raises HTTPException(400, "Invalid env_prefix")
    and the launch never starts. Confirmed on this branch.

  • Impact: Pre-existing rather than introduced here, but it lives on exactly
    the code path this PR is fixing and any venv under C:\Users\<name>\My Documents\... or a "Program Files"-style directory hits it. Quoting at the
    emitters (finding 1) removes it; leaving the emitters alone leaves a hard 400
    behind after this merges.

  • Location: routes/cookbook_helpers.py:1212

P2 suggestion: this is the fourth copy of the drive-letter conversion

  • Problem: _git_bash_path() at routes/cookbook_helpers.py:57 already does
    C:\... -> /c/... including the backslash rewrite, is platform-independent
    (so it works under pytest on Linux/macOS), and is already used further up the
    same file at line 170. core/platform_compat.py:219 has another copy, which
    cookbook_routes.py already imports from. This adds a fourth, inline.

  • Impact: Four implementations of one Windows-path rule means four places to
    touch if the mount-root assumption ever needs to change, and they already
    disagree — the new one rewrites a UNC path to //nas/share/... while
    _git_bash_path returns it untouched with backslashes intact.

  • Ask: Reuse _git_bash_path and do the Activate.ps1 -> activate rename
    around it. If UNC support is the reason for the local copy, please extend
    _git_bash_path instead so there's one converter.

  • Location: routes/cookbook_helpers.py:1220-1224, routes/cookbook_helpers.py:57

P2 issue (test): the one test covers only the shape that already worked

  • Problem: The new test asserts the quoted, drive-lettered, space-free case.
    Not covered: the unquoted form the two main emitters actually send (which
    would have caught finding 1), spaces, forward-slash input, UNC, and every
    pass-through case — source /home/.../activate, conda activate,
    eval "$(conda shell.bash hook)" && ..., None, "". The PR body's claim
    that remote Windows keeps the PowerShell path is also unasserted.

  • Impact: The suite passes while the primary flow stays broken, which is how
    finding 1 got this far.

  • Ask: Add the unquoted-input case, plus a handful of pass-through
    assertions so a future edit can't silently start rewriting POSIX or conda
    prefixes.

  • Location: tests/test_cookbook_helpers.py:111

P3 nit (non-blocking): a wrong venv path now fails silently

  • Problem: After conversion, _safe_env_prefix wraps the result as
    [ -f "<path>" ] && source "<path>" || true, so a typo'd or moved venv
    activates nothing and the run continues against the system interpreter with no
    message in the log.

  • Impact: That is indistinguishable from the bug this PR fixes, and #2710 is
    a report of exactly that confusion. It matches existing POSIX behaviour, so
    it's not a regression — just a missed chance to make the next report easier.

  • Ask: Optional: || echo "venv not found at <path>" on the miss branch.

  • Location: routes/cookbook_helpers.py:1204

Open Questions

  • question (scope, non-blocking): Local Windows conda has the same problem —
    cookbookRunning.js:1976 and cookbookDownload.js:543 emit
    conda activate C:\path\to\env unquoted, and _safe_env_prefix's conda branch
    passes it straight into the bash runner. Deliberately out of scope for a
    venv-only fix, which is fine, but is that a follow-up you're planning?

  • question (non-blocking): /c/ is the Git-for-Windows and MSYS2 mount root,
    and find_bash() normally lands on Git Bash, so this is right in practice. It
    does resolve shutil.which("bash") first though, and only rejects the WSL /
    Store stubs — a Cygwin bash earlier on PATH would want /cygdrive/c. Same
    assumption as the existing helpers, so I'm not asking for a change; flagging it
    as a known edge if Cookbook-on-Windows reports keep coming in.

  • question: Did you get to run this on the Windows host? The "I actually ran
    the app" box is unticked and "How to Test" reads as instructions rather than a
    result. I ask because finding 1 means a launch from the Serve panel or the
    Download button would still have failed, so a real end-to-end run should have
    surfaced it.

Validation

  • Ran: macOS, Python 3.11.15, worktree at refs/pull/5734/head (6b48a97).
    Full suite python -m pytest -q: 2 failed, 5037 passed, 4 skipped in
    139.44s
    — both failures are the known environment-dependent ones
    (test_workspace_confine.py::test_glob_confined_e2e, /tmp resolving to
    /private/tmp; test_integration_api_call_ssrf.py::test_real_socket_falls_back_from_dead_first_to_live_second,
    real-socket timing) and neither touches this change. The subset from your
    "How to Test": 94 passed, 1 skipped. compileall on both touched Python files:
    clean. Booted the app (uvicorn app:app on 127.0.0.1:7099) — /api/health,
    / and /cookbook all 200. Then drove
    _safe_env_prefix(_local_windows_bash_env_prefix(ep)) — the literal expression
    at both call sites — over the six payload shapes the five frontend emitters
    produce; that's where findings 1 and 2 come from.

  • Not run: No native Windows host here, so nothing exercised find_bash(),
    _launch_local_detached, or an actual Git Bash source. The /c/ mount root
    and the activation itself are unverified end-to-end — my evidence is
    string-level plus reading the call sites. I also didn't run the remote-Windows
    path; I confirmed by reading that local_windows = IS_WINDOWS and not remote
    and that remote Windows takes the separate ps_lines branch above
    cookbook_routes.py:2136, so it never reaches the changed line. No JS in the
    diff, so no node --check.

  • Residual risk: Low for what the diff changes — it's guarded, and POSIX and
    remote Windows are provably untouched, so merging it can't break anything that
    works today. The risk is that it reads as fixing #2710 without fixing it on
    the two primary flows, and the issue gets closed on that basis.

PR Hygiene

  • Target/template/checks: Targets dev, one focused change, no drive-by
    edits, no UI so the screenshot requirement is correctly N/A. Title passes
    Conventional Commits and the description check is green; ready for review is
    on. Part of #2710 is the right linkage — #2710 is broader than this and
    shouldn't auto-close. The one gap is the unticked "I actually ran the app" box
    (see Open Questions). Your CI note about src/agent_loop.py is stale now —
    #5735 merged on 2026-07-27 and the run on 2026-08-13 is green.

  • Related, duplicate, or superseding context: #4894 is closed unmerged
    (2026-07-23), so superseding it is accurate. Searched open PRs for
    cookbook/env_prefix and cookbook/windows/venv overlap — this is the only one.
    The branch is 5 commits behind dev and nothing on dev has touched
    cookbook_routes.py, cookbook_helpers.py or tests/test_cookbook_helpers.py
    since the base, so it still rebases clean and GitHub reports it mergeable.

@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch from 6b48a97 to 01c1192 Compare August 16, 2026 16:49
@github-actions github-actions Bot added needs runtime validation Runtime validation not attested — tick the app-run box after running it, or state the gap needs visual evidence UI-sensitive change without an attested screenshot or clip from the running app and removed ready for review Description complete — ready for maintainer review labels Aug 16, 2026
Comment thread routes/cookbook_helpers.py Fixed
@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch from 01c1192 to 85ad875 Compare August 16, 2026 16:55
@zoomdbz

zoomdbz commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in 85ad875 and rebased onto current dev.

Addressed the review findings:

  • Serve and Download now PowerShell-quote venv activation paths before sending env_prefix.
  • Local Windows Git Bash conversion handles quoted and unquoted paths, spaces, and forward slashes; it reuses _git_bash_path.
  • Replaced the uncontrolled-data regex with a linear parser after CodeQL flagged it.
  • Added regression coverage for supported formats, pass-through behavior, frontend emitters, route scoping, and long whitespace input.

Validation: focused regression suite 15 passed; Python and JavaScript syntax passed; full CI, CodeQL, and Trivy are green.

Ready for re-review.

@zoomdbz

zoomdbz commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

@o3LL Runtime and visual evidence are now posted.

Native Windows validation on rebased local head 254f0aff exercised the primary Download and Serve paths. The change has the same Git patch ID as PR head 85ad875c.

  • The primary Download path completed hf-internal-testing/tiny-random-gpt2 and returned 200.
  • The primary Serve path ran python -m pip --version, resolved pip from G:\AI\odysseus\venv\Lib\site-packages, and exited 0.
  • The screenshot and exact validation scope are in the PR description.
  • The separate native run from Venv With Space was not completed; I stated that residual gap instead of claiming it. The focused regression suite covers quoted and unquoted paths containing spaces.

The requested emitter, parser, converter-reuse, and regression changes remain in 85ad875c. Re-requesting review.

@github-actions github-actions Bot added ready for review Description complete — ready for maintainer review and removed needs runtime validation Runtime validation not attested — tick the app-run box after running it, or state the gap needs visual evidence UI-sensitive change without an attested screenshot or clip from the running app labels Aug 17, 2026
@zoomdbz

zoomdbz commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

@o3LL Additional native-Windows reproduction confirms the exact base-branch failure this PR fixes.

While validating #6106 against current dev, a real Cookbook Direct Download correctly selected the saved Local venv profile, then the generated Git Bash task failed at line 16 with:

syntax error near unexpected token `&'
& 'G:\AI\odysseus\venv\Scripts\Activate.ps1'

That is the PowerShell activation prefix reaching the Bash runner unchanged. #5734 converts this path before Bash execution. The reproduction used hf-internal-testing/tiny-random-gpt2; no model server or GPU process started.

The requested emitter quoting, tolerant quoted/unquoted parsing, shared _git_bash_path reuse, and expanded pass-through coverage are already present in the current PR head. The local rebased head remains patch-equivalent and ready for the contributor's manual branch update.

@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch 3 times, most recently from 0931c38 to f0e09d9 Compare August 17, 2026 18:04
@zoomdbz
zoomdbz force-pushed the fix/local-windows-bash-venv-prefix-v2 branch from f0e09d9 to a0e61a4 Compare August 17, 2026 18:12

@o3LL o3LL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at a0e61a45, rebased onto 032967af.

All five findings from 6b48a97 are addressed, and two of them better than I asked
for: the helper now accepts the unquoted form itself rather than depending on the
emitters, and the regex I suggested was replaced with a linear parser after CodeQL
flagged the backtracking.

The Serve capture answers the one thing I couldn't check from macOS. I was unsure
whether sourcing CPython's bash activate would survive PATH="$VIRTUAL_ENV/"Scripts":$PATH"
under Git Bash, since POSIX splitting on : should break that string apart. pip ... from G:\AI\odysseus\venv\Lib\site-packages says it resolves correctly in practice,
which is the fact the whole approach rests on. Thanks for going and getting it.

Validation

  • Ran: re-verified the delta end-to-end on macOS — the real _psQuote output from
    the running page piped into the backend helper, round-tripping paths with spaces and
    apostrophes; injection attempts (;, &&, $(...), embedded newline) all land on 400.
    Full suite green apart from two known macOS environment failures. Confirmed working on a Windows host with Git Bash
  • Residual risk: the venv path containing a space is covered by the regression tests
    but not by a native run, as you noted yourself.

a0e61a45 is patch-identical to 85ad875c — same five blobs — so the rebase doesn't
change what I reviewed.

@o3LL
o3LL merged commit 43682d4 into odysseus-dev:dev Aug 18, 2026
23 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants