Skip to content

feat(windows): unblock the import path and cover it in CI - #454

Merged
gloryfromca merged 29 commits into
mainfrom
feat/windows-support
Sep 24, 2026
Merged

gloryfromca merged 29 commits into
mainfrom
feat/windows-support

Conversation

@gloryfromca

@gloryfromca gloryfromca commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

Makes EverOS run on native Windows as far as the unit suite can see, and puts
a CI job behind that claim.

import fcntl at module scope in core/persistence/locking.py was the
single import-time blocker. It sits on core.persistence.__init__, which
every persistence path imports, so even everos --help died with ImportError
before printing anything:

cli/main.py:16 -> commands/cascade.py:43 -> component/embedding/__init__.py:33
  -> component/utils/config_hints.py:11 -> core/persistence/__init__.py:36
  -> core/persistence/locking.py:24  ->  ImportError: No module named 'fcntl'

portalocker is already a dependency (the OME single-engine guard uses it) and
dispatches to fcntl.flock on POSIX / LockFileEx on Windows, so no new
dependency and no change in POSIX semantics.

With the import path open, the first Windows CI run surfaced what was behind
it: 12 failed / 2538 passed. Three were real defects, the rest were POSIX
assumptions in tests. All fixed here; the third run is 2550 passed / 4
skipped
, identical to Linux.

Defect Where Fix
file:// uris never worked on Windows — unquote leaves the leading-slash drive form /C:/x memory/extract/parser/mapping.py urllib.request.url2pathname, the stdlib converter; a no-op beyond percent-decoding on POSIX
Text IO without encoding= follows the locale (cp1252 on Windows); TUI demo frames raised UnicodeEncodeError on box-drawing glyphs 7 sites in tui/demo/readme_media.py and infra/persistence/lancedb/__init__.py explicit utf-8. The markdown layer already passed it everywhere, so memory data was never at risk
SO_REUSEADDR means the opposite on Windows — a second socket may bind a port the probe holds, so the benchmark fleet's hold-until-spawn excluded nobody benchmarks/run.py SO_EXCLUSIVEADDRUSE where it exists

Things that looked like blockers and were not: lancedb ships a win_amd64
wheel; portalocker already branches on os.name; watchdog's Observer
dispatches to WindowsApiObserver; uvloop is excluded on win32 by uvicorn's
markers; raise_signal(SIGTERM) in the cascade worker is only reachable under
the API lifespan, where uvicorn's handler is installed on Windows too.

Plus a Windows / WSL2 install guide, with a Chinese mirror, and
scan_interval_seconds becomes a real [cascade] setting (reviewer note 5).

Coverage for the two subsystems most exposed to a different OS — the
cascade watcher and the LanceDB store — 20 new unit tests that run on all
three CI platforms and assert on outcomes, never event order (reviewer note
6). Writing them surfaced one real watcher defect, fixed in fd3c9da: a late
modified for a path already unlinked overwrote the deleted row and
resurrected a gone file until the scanner's next sweep. FSEvents reorders;
the guard now records disk truth.

Then the branch was run on a stock Windows 11 Enterprise machine — no VC++
redistributable, corporate image, nothing preinstalled — and it did not start:
import greenlet → DLL load failed while importing _greenlet. greenlet sits
under SQLAlchemy's async engine, so every SQLite call was dead. Of 27 compiled
packages it was the only one to fail; its wheel links msvcp140.dll and does
not ship it. windows-latest has the redistributable preinstalled, so CI is
structurally unable to catch this. Fixed in d91702a with no manual step
(reviewer note 7): a Windows-only msvc-runtime dependency puts the DLLs in
sys.prefix, and everos/__init__.py registers that directory with the DLL
loader before anything else imports. On the same machine afterwards:
everos init and everos cascade status (opens SQLite) exit 0, and all 23
new tests pass — including the six real ReadDirectoryChangesW observer tests.

Area

  • Architecture method
  • Benchmark
  • Use case
  • Documentation
  • Developer experience
  • CI, build, or release

Verification

CI on this branch (latest run):
  unit tests (Windows)         2573 passed, 4 skipped     <- all 23 new tests ran there, msvc-runtime
                               installed by marker; same skips as Linux/macOS (benchmark datasets absent)
  unit tests / (3.13)          pass
  integration tests / (3.13)   pass
  lint, package, links, commit messages, PR title   pass
  integration tests (Milvus)   FAIL -- upstream: `pull access denied for minio/minio`.
                               Also failing on PR #453; last green on main was 2026-09-08.
                               Not touched by this branch.

Local (macOS):
  make ci                      exit 0 (lint -> test -> integration -> package)
  unit                         2576 passed, 4 skipped  (2550 before the 26 new tests)
  integration                  183 passed, 5 skipped, 7 deselected

Windows branches that macOS cannot reach, proven another way:
  url2pathname  -- called nturl2path.url2pathname directly (what url2pathname
                   resolves to on Windows): three drive-letter cases round-trip;
                   the old unquote gave `/C:/...` for each.
  import path   -- meta_path finder blocking POSIX-only stdlib: all 11 EverOS
                   layers import clean (9 of 11 failed before the lock change).

Mutation pass on the new tests: 13 source mutations (guards removed, branches
inverted, fallback constants moved, Observer.schedule() dropped) each turned
their target tests red on the claimed symptom; sources restored byte-identical.
The three Lance probes (handle release after close(), prune with an open
reader, MAX_PATH) cannot be reddened on POSIX -- they exist to be run on
windows-latest and a red there is a finding, not a test bug.

The one timing test touched (test_rebuild_runs_periodically) had its window
widened from 0.2s to 0.6s; the >= 3 assertion is unchanged. One TUI scroll
test got an extra pilot.pause() on each side of scroll_home; assertions
unchanged. Neither is reproducible on macOS.

Stock Windows 11 Enterprise box (no VC++ redistributable), branch at 9c0fe18:
uv python install 3.12 + uv sync --frozen ok (msvc-runtime pulled in by marker)
import everos, greenlet 3.5.3 <- via the hook
import greenlet (control, no everos first) DLL load failed <- the pre-fix state
everos init --root exit 0
everos cascade status --root exit 0 <- opens SQLite via the console script
the 23 new tests 23 passed
full unit suite 2573 passed, 4 skipped
full integration suite (stubs, no key) 183 passed, 5 skipped (Milvus, no URI), 7 deselected (slow/live)
incl. test_cascade_fsevents_repro (os.replace bursts vs the real watcher) and
test_memorize_concurrent_session_lock (cross-process lock under LockFileEx)

Then the loop the Windows guide is really about, on that same machine:
everos server start --root (detached, dummy LLM key -> Tier 1) /health 200 in 2s
write users/admin/episodes/episode-2026-09-23.md with a marker token
6s later: cascade_worker_processed kind=episode upserted=1 <- ReadDirectoryChangesW, for real
everos cascade status: done 1 / pending 0 / failed 0
POST /api/v2/memory/search method=keyword query= 200, the episode, score 0.86
So on a stock Windows machine: install -> start -> edit a file -> search it,
with no installer, no administrator step, no manual prerequisite.

Then the slow/live selection, on the same machine, with real OpenRouter +
DeepInfra keys (never persisted there):
-m "slow or live_llm" 28 passed, 1 skipped, 0 failed, exit 0, 9m05s
the skip is test_real_sigint_during_phase_await (os.kill(pid, SIGINT) is
TerminateProcess on Windows -- it killed the pytest process on the first try)
Getting there fixed three things in the suite itself (commits 167473d, 89ac544):
tests/e2e/test_reflection_e2e.py could not be collected since #425 on main
(bare import adapters), which aborted every live run everywhere; the
multimodal e2e hand-built file://{path}; the rename watcher test asserted
the source leg on the watcher alone where ReadDirectoryChangesW can split the
RENAMED pair across reads (now asserted after one sweep).

And a soak (the LanceDB soak harness, ported to Windows: psutil instead of
/proc, Win32_Process.Create instead of systemd, in-process RSS/disk caps
instead of cgroup) is running on that machine as this is written: 10 h at
the ceiling preset, two concurrent cascade sync storms, disk capped at
80 GB. Its dry run surfaced the writer defect fixed in 47b96eb (reviewer
note 8). Results land in a follow-up.

Still not verified: the editor-saved edit above was written by a script
(VS Code writes CRLF; the watcher tests cover in-place and atomic saves, the
live loop has not yet seen one). One machine so far. The guide still says
native Windows is not yet supported; that sentence gets revisited with the
soak result.
The WSL2 guide's networking claims (localhost forwarding, \\wsl$\ inotify)
are from documented WSL mechanics, not measured.

Checklist

  • I kept the change scoped to the relevant area.
  • I am opening this from a separate branch, not pushing directly to main.
  • I updated docs, examples, or setup notes when behavior changed.
  • I added or updated tests when the change affects behavior.
  • I did not commit secrets, .env files, dependency folders, or generated output.
  • Active relative links in Markdown files resolve.

Notes for Reviewers

  1. AlreadyLocked is not an OSError subclass, so the contention branch
    stopped catching BlockingIOError. Getting that wrong turns a normal wait
    into an unhandled exception.
  2. portalocker's POSIX branch is fcntl.flock, not lockf — record-lock
    semantics ("closing any fd for the file drops all locks") would have been a
    real regression.
  3. The Windows CI job calls uv directly, not make — the image has no
    usable make. uv sync --frozen works because the lockfile is universal
    (sys_platform == 'win32' markers, pywin32 present).
  4. SO_EXCLUSIVEADDRUSE is selected by hasattr, not a platform string —
    src/ has zero sys.platform checks and this keeps it that way.
  5. scan_interval_seconds is now a [cascade] setting (last commit). It
    was a CascadeConfig dataclass default that from_settings() never read,
    while docs/cascade_runbook.md told operators to "drop the scan interval to
    ~2.5 s" — advice that needed a code change to follow. It matters most on the
    mounts this PR documents: a Windows directory bound into WSL2 or a container
    delivers no filesystem events, so the scanner is the only path an md edit
    takes to the index and its interval is the edit-to-searchable latency.
    The exact-set test on CascadeSettings.model_fields grows by one; its own
    rule (deadlines stay constants, cadences may move) classifies this as a
    cadence. The forwarding test asserts on the scanner's _interval, not on
    CascadeConfig, for the same reason the existing four assert on the worker.
  6. The watcher tests read md_change_state, not events. inotify, FSEvents
    and ReadDirectoryChangesW disagree about what an editor's save looks like
    (Windows reports os.replace over an existing file as REMOVED + RENAMED, so
    on_deleted fires for a path that still exists). Asserting on the final row
    is what lets one test hold on all three. The six real-Observer tests wait
    up to 15 s for delivery; a passing run takes ~1 s, the deadline only bites
    when nothing arrives -- which is exactly the /mnt/c failure the guide
    warns about. If test_deep_memory_root_still_stores_and_counts reds on
    Windows, the fix is documentation (LongPathsEnabled) or a startup check,
    not the test.
  7. msvc-runtime is a supply-chain decision, stated plainly. Its PyPI
    metadata has no author or homepage and the license field reads
    "Proprietary" (Microsoft's redistributable terms, repackaged). It ships
    per-CPython win32/amd64/arm64 wheels and tracks MSVC versions (14 releases,
    14.29 → 14.44); uv.lock pins it by hash and the marker keeps it off POSIX
    entirely. The alternative is telling every Windows user to install the
    redistributable by hand, which is the manual step this PR exists to remove.
    Two things the hook does not cover: (a) a host application that imports
    sqlalchemy before everos — sqlalchemy imports greenlet eagerly, so the
    registration comes too late; the test suite hit exactly this and
    tests/conftest.py now imports everos first. Covering arbitrary import
    order needs a startup .pth shipped in the wheel — a packaging change,
    left as a separate decision. (b) Nothing; the feature-detected
    getattr(os, "add_dll_directory", None) keeps src/ at zero
    sys.platform checks and makes the hook a no-op on POSIX and on Windows
    machines that already have the redistributable.
  8. writer.py now retries the staging swap through a Windows sharing
    violation
    (47b96eb). The soak dry run's load feeder died 62 s in with
    PermissionError: [WinError 5] from os.replace(tmp, target) -- the
    target was open in another process at that instant (cascade worker
    reading it, Defender scanning it). The server's own markdown writer is the
    same primitive at the same call, so /add would have failed the same way
    under load. _replace_with_retry retries only PermissionError with
    exponential backoff (8 attempts, ~2.5 s), re-raises unchanged after the
    budget, never retries other errors; each attempt is one atomic
    os.replace. POSIX never raises here, so behaviour there is unchanged and
    no platform check was needed. Three tests fake os.replace and pin the
    behaviour on every platform; each was mutation-checked. The retry fired
    within the first minute of the real soak (REPLACE_RETRY_OK after 1 retries), so the condition is real and frequent on Windows.

🤖 Generated with Claude Code

Native Windows verification (2026-09-23/24, stock Windows 11 Enterprise laptop)

Machine: Intel Core Ultra 7 155H (22 threads), 32 GB, no Visual C++ Redistributable, Python 3.12 via uv. Everything below ran on that box, not in CI.

What Result
pip install everos → everos init → everos server start works, no manual step (msvc-runtime supplies the C++ runtime greenlet needs; everos/__init__.py registers sys.prefix with the DLL loader)
unit / integration / live-LLM suites 2576 passed / 4 skipped · 183 passed / 5 skipped · 28 passed / 1 skipped
Four memory kinds on a Tier 3 server (real LLM + embedding + rerank) episode 5 · profile 1 · agent case 1 · agent skill 1 produced and read back via /get and /search
Python range 3.12 / 3.13 / 3.14 regular: unit + integration green unchanged; ≤3.11 refused (requires-python, PEP 695 in src/); 3.14t: no lancedb wheel
10 h soak (feeder ~1 md rewrite/s + 0.5 search/s + 12 % /add, two concurrent everos cascade sync processes, md fuzz) 78 264 entries · 16 328 searches · 2 264 /add · storms 1 754 + 1 769 runs, 0 errors · integrity CLEAN (all tables open, schemas verify, no entry lost) · RSS 224 MB → plateau ~2.3 GB from hour 3 (max 2.9 GB) · LanceDB dir peak 6.7 GB, floor 2.7 GB, live 437 MB · disk net +6.6 GB at peak, returned · os.replace sharing violations: 49 retried, 0 gave up

Fixed along the way (all in this PR): POST /add with an orphan role: "tool" row → 422 instead of 500 (06509b2); a bare ManualTick at a business-event strategy → not_dispatched instead of a dead-lettered AttributeError (5692baf), and the follow-up so a manual trigger still reaches a Cron strategy like reflect_episodes (c99db49).

Found by the soak, not Windows-specific, tracked outside this PR:

  • Two processes running the cascade on the same tree (server worker + CLI cascade sync) can each insert the same row: after 10 h, 1 158 / 1 219 / 1 241 duplicate global ids in episode / atomic_fact / foresight (~4.5 % extra rows, same md_path, same content, updated_at 14 s apart). No corruption; counts and disk inflate.
  • Request latency under sustained write load: search p50 0.5 s in hour 0 → 3.2 s in hour 9 (p95 up to 7 s), five /health 5 s timeouts, while the same query on the idle server takes 12–15 ms and raw Lance FTS < 100 ms. py-spy: the event-loop thread spends ~85 % of its busy time in lancedb to_list(), half of it in import pytz failing on every timestamp value (pyarrow tries pytz first, the failed import is not cached, each attempt walks sys.path; Defender makes every stat expensive on Windows).
  • atomic_fact version cleanup lost the commit race against the CLI processes for ~45 min (health reported it as designed; it recovered).

Related PRs: #455 (API doc clarifications found by driving the server the way the Claude Code plugin does), #456 (knowledge document deleted before it was indexed came back).

CI note: integration tests (Milvus 2.6) is red on every branch since the minio/minio Docker Hub image went away; unrelated to this PR.

Adversarial review round (2026-09-24)

Two read-only reviewers attacked this branch (product code; tests / CI / docs). No blocker. Fixed in b75e755 + ab33efe:

  • DLL hook only probed sys.prefix; a pip install that falls back to a per-user install puts the wheel's data files under site.getuserbase(), where the hook was a no-op. Both bases are probed now, with a test.
  • Comments that were not true: the locking module said LockFileEx (portalocker's default Windows locker is msvcrt.locking, same semantics); the replace-retry budget is ~2.5 s, not ~5 s (also wrong in 47b96eb's body); POSIX can raise PermissionError on an immutable target.
  • Each os.replace retry now logs at debug level — the soak's 49 retries were the harness writer's; the product path was unobservable.
  • scan_interval_seconds gains gt=0; docs/api.md's not_dispatched sentence names the fifth gate.
  • The two watcher save tests asserted only "not deleted / still pending", which the first write already satisfied — with the handler stubbed out they passed. They now assert the row's mtime advanced.
  • Guide: the unit number was a macOS figure (now the CI windows-latest count, 2583 / 4); "pip install" was not the path run (a uv checkout was — the wheel path is being verified on the box now and the sentence will be restored with that evidence); "no entry lost" excludes the deliberately malformed fuzz files; 3.13 / 3.14 are attributed to CI / macOS; the intro no longer contradicts the native section; LongPathsEnabled and the wheel-only msvc-runtime risk are stated.

Left for the maintainers: make unit tests (Windows) a required check after merge (command in .work_context/windows_support/status.md); decide whether msvc-runtime shipping wheels only, per CPython minor, is acceptable (a newer Python cannot install EverOS on Windows until upstream publishes a wheel).

zhanghui and others added 7 commits September 23, 2026 18:05
`import fcntl` at module scope was the single import-time blocker for
running EverOS on Windows. It sits on `core.persistence.__init__`, which
every persistence path imports, so even `everos --help` died with
ImportError before printing anything.

portalocker is already a dependency (the OME single-engine guard uses
it). It dispatches to `fcntl.flock` on POSIX and `LockFileEx` on
Windows, so POSIX semantics are unchanged: same whole-file advisory
lock, same release on process exit. Note it is `flock`, not `lockf` —
no POSIX record-lock semantics creep in.

`AlreadyLocked` is not an OSError subclass, so the contention branch
catches it instead of BlockingIOError.

Verified: the existing cross-process contention tests (spawn context,
which is what Windows uses) pass, and blocking `fcntl` behind an import
hook lets every EverOS layer import cleanly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Windows had no CI coverage at all, so "it should work" was the strongest
claim available. The job mirrors the Linux unit job on windows-latest.

It calls uv directly rather than through make, because the Windows image
has no usable make. `uv sync --frozen` resolves there: the lockfile is
universal, carrying sys_platform == 'win32' resolution markers and
pywin32 for portalocker's Win32 locker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers the preflight that tells you whether this machine needs a reboot,
the two-phase scripted install, and the one failure that is silent: a
memory root on /mnt/c gets no filesystem events, so the cascade watcher
starts, logs normally, and sees nothing.

Also documents why LibreOffice has to be the Linux build inside the
distro, and why 0.0.0.0 is not the fix for a localhost that will not
connect — EverOS ships no auth of its own, and how far 0.0.0.0 reaches
depends on the WSL networking mode.

Ships a Chinese mirror alongside, which the CJK language-policy gate
allows for `*.zh.md` translated mirrors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first `unit-windows` run came back 12 failed / 2538 passed, which split
into three real defects and four tests carrying POSIX assumptions.

Defects:

- `file://` uris never worked on Windows. `unquote(parsed.path)` leaves the
  uri's leading-slash drive form `/C:/x`, which is a different path from
  `C:\x`. `urllib.request.url2pathname` is the stdlib converter for exactly
  this and is a no-op beyond percent-decoding on POSIX.
- Text written without an explicit encoding follows the locale, which is
  cp1252 on Windows, so the TUI demo's box-drawing glyphs raised
  UnicodeEncodeError. Fixed at all seven `read_text`/`write_text` sites
  that lacked one; the markdown layer already passed utf-8 everywhere, so
  memory data itself was never at risk.
- `SO_REUSEADDR` means the opposite thing on Windows: it lets a second
  socket bind a port the probe already holds. The benchmark fleet's
  hold-until-spawn therefore excluded nobody and handed two concurrent runs
  the same port -- the exact race its docstring says the bind is there to
  close. Windows spells that exclusivity `SO_EXCLUSIVEADDRUSE`.

Tests: build file uris with `Path.as_uri()` instead of f-string
concatenation, set `USERPROFILE` alongside `HOME` (expanduser reads the
former on Windows), `re.escape` a path used as a regex, and `json.dumps` a
path going into a JSON-valued env var -- a Windows path is not a valid JSON
string literal, `\U` is an invalid escape.

One timing test was widened, not weakened: Windows' ~15ms timer granularity
stretches the 50ms rebuild interval to ~110ms, so the observation window
grew from 0.2s to 0.6s while the assertion stays at >= 3 sweeps.

Verified: full unit suite 2550 passed / 4 skipped on macOS, unchanged from
before. The Windows branch of the uri fix was exercised directly through
`nturl2path.url2pathname`, which is what `url2pathname` resolves to there:
all three drive-letter cases round-trip, where the old `unquote` produced
`/C:/...`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second `unit-windows` run: 3 failed / 2547 passed.

Two failures moved rather than vanished. The previous commit made the TUI
demo write its SVG frames as utf-8, so the failure shifted from write
(UnicodeEncodeError) to the test's own `read_text()` with no encoding,
which follows the locale and is cp1252 on Windows -- byte 0x81 has no
mapping there. Read back with the encoding the product writes.

The third is a race the slower runner exposes. `_record_line` calls
`scroll_end(animate=False)` after `Static.update()`, and Textual applies
that scroll after the layout refresh the update triggers. After 16 lines a
single `pilot.pause()` does not always drain every deferred scroll, so one
can land after `scroll_home` and re-pin the panel to the bottom, which is
what `assert 13 == 0` shows: 13 is max_scroll_y. Pause twice on each side
of the scroll so the queue is empty before and after; the assertions are
unchanged.

Not reproducible on macOS (39/39 TUI tests pass before and after); the
race is reasoned from the widget's deferral, not observed locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The guide closed by saying native Windows had no CI coverage. After this
branch it has: the unit suite passes 2550/2550 on windows-latest. Say so,
and say exactly what is still missing before it can be called supported --
the integration suite does not run there, and no one has run a full
`everos serve` on a Windows machine end to end.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`scan_interval_seconds` was a `CascadeConfig` dataclass default that
`from_settings()` never read: no TOML key, no env var. The cascade runbook
nonetheless told operators to "drop the scan interval to ~5 s" on WSL2 and
network mounts -- advice that needed a code change to follow.

It matters most on exactly those mounts. A Windows directory bound into
WSL2 or a container delivers no filesystem events, so the watcher sees
nothing and the scanner is the only path an md edit takes to the index;
its interval is then the edit-to-searchable latency. Every sweep also
stats every md file, which over a slow mount is the cost of shortening it.
That is a write-volume-dependent cadence by the settings module's own
rule, so it moves alongside the four `optimize_*` cadences.

The exact-set test on `CascadeSettings.model_fields` grows by this field.
Its rule is that deadlines stay constants and cadences may move; this is a
cadence. The forwarding test gains a fifth env var and asserts on the
scanner's own `_interval`, not on `CascadeConfig` -- the same reason the
existing four assert on the worker.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gloryfromca
gloryfromca marked this pull request as ready for review September 23, 2026 11:46
zhanghui and others added 12 commits September 23, 2026 20:23
`_Handler.on_modified` never checked whether the path still existed. On
FSEvents, which coalesces and reorders, a create followed by an unlink
inside its latency window can deliver the ``modified`` leg last; the
upsert then overwrote the ``deleted`` row with ``modified`` / mtime 0.0,
and the state table resurrected a file that was no longer on disk until
the scanner's next sweep (up to 30 s) noticed. Not data loss -- the worker
fails to read the file and the scanner reconciles -- but a wrong terminal
state and a spurious failed row.

Surfaced by the new real-observer test `test_unlink_enqueues_deleted`,
which failed 2 in 6 runs on macOS with the row sitting at
``'modified'/'pending'``. `on_deleted` and `on_moved` already guard the
mirror case (a deletion for a path that still exists); this closes the
other direction in `_enqueue`, the single exit all four callbacks share.
Disk is the truth: an ``added`` / ``modified`` for a path that is gone is
recorded as ``deleted``.

Pinned deterministically by
`test_modified_for_a_gone_path_is_recorded_as_deleted` in the next
commit; removing the guard turns it red on that exact symptom.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The watcher's unit coverage was two pure helpers; the four `_Handler`
callbacks, both drop paths in `_enqueue`, `start()` creating the root, and
the observer -> SQLite delivery path had none. The integration test that
exercised them does not run on Windows CI, which only runs `tests/unit`,
so on the one platform where the backend differs (ReadDirectoryChangesW)
nothing had ever been asserted.

`test_watcher_events.py` -- 16 tests, asserting on the `md_change_state`
row, never on event order, so one test holds on inotify, FSEvents and
Win32 alike. Ten call the callbacks directly (the LanceDB-wipe guard in
`on_deleted`, both `on_moved` guards, both drops, the swallowed upsert
failure, `ensure()` on start). Six run a real `Observer` on the memory
root: new file, in-place save, temp-file + `os.replace` over an existing
target (Windows reports that as REMOVED + RENAMED, so `on_deleted` fires
for a path that still exists), unlink, rename within root.

`test_platform_filesystem.py` -- four tests for what NTFS changes and
POSIX never shows: the husk sweep continuing past a refused `rmdir`
(sharing violation), table files releasable after `close()` so the
memory root can be deleted or moved, prune completing while a second
handle reads the table, and a memory root deep enough to cross MAX_PATH.

Every test that a POSIX mutation can reach was shown red by one: 13
mutations of `watcher.py` / `repository.py` (guards removed, branches
inverted, fallback constants moved, `schedule()` dropped) each turned
their target tests red on the claimed symptom, sources restored
byte-identical afterwards. The three Lance probes for handle release,
open-reader prune and path length cannot be reddened on POSIX; they
exist to be run on windows-latest.

One assertion was reworked during that pass. "No row" for a non-kind
path could not distinguish a clean drop from an exception swallowed
downstream (the `except` in `_enqueue_async` dereferences `spec.name`
itself, so a `None` spec fails silently into the unretrieved future); the
test now records what the guard hands downstream instead.

Full unit suite on macOS: 2570 passed, 4 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Found on a stock Windows 11 Enterprise box while running the branch there:
`import greenlet` fails with `DLL load failed while importing _greenlet`.
System32 has none of the VC++ runtime DLLs and the VC++ 2015-2022 x64
Redistributable is not installed. greenlet is a C++ extension whose wheel
does not bundle `msvcp140.dll`; uv's managed Python ships only the
`vcruntime140*.dll` pair. SQLAlchemy's async engine depends on greenlet,
so every SQLite call dies and `everos serve` cannot start.

pyarrow and lancedb import fine on the same machine because their wheels
bundle the runtime, which is why the four LanceDB tests passed there while
all fifteen SQLite-backed watcher tests errored. windows-latest has the
redistributable preinstalled, so CI is green and cannot catch this.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On a stock Windows machine `pip install everos` followed by `everos server
start` died before serving anything: `import greenlet` -> `DLL load failed
while importing _greenlet`. greenlet sits under SQLAlchemy's async engine,
so every SQLite call was dead with it. Of the 27 compiled packages in the
environment it was the only one to fail -- the rest either bundle the MSVC
C++ runtime in their wheel or do not need it; greenlet's wheel links
`msvcp140.dll` and does not ship it, and a stock Windows does not have it.

Two pieces make the install self-sufficient, with no installer and no
administrator rights:

- A Windows-only dependency, `msvc-runtime; sys_platform == 'win32'`. It
  places the runtime DLLs in `sys.prefix` and `sys.prefix\Scripts`.
- `everos/__init__.py` registers those directories with the DLL loader via
  `os.add_dll_directory` before anything else in the package is imported.
  Placement alone is not enough: a venv's `python.exe` is a launcher (uv's
  is a trampoline), so the loader's "application directory" is the base
  interpreter's and the DLLs next door are never searched. Verified on a
  Windows 11 Enterprise box with no VC++ redistributable: without the call
  greenlet fails, with it greenlet imports. Handles are kept in a
  module-level list -- a collected handle unregisters its directory again.

Feature-detected (`getattr(os, "add_dll_directory", None)`), keeping the
codebase's zero `sys.platform` checks. On POSIX, and on Windows with the
redistributable already installed, the hook is a no-op.

Supply-chain note, stated rather than buried: `msvc-runtime` on PyPI has no
author or homepage metadata and its license field reads "Proprietary" (the
Microsoft redistributable terms it repackages). It ships per-CPython
win32/amd64/arm64 wheels and tracks MSVC versions (14 releases, 14.29 to
14.44). uv.lock pins it by hash. The alternative -- telling every Windows
user to install the redistributable by hand -- is the manual step this
change exists to remove.

The three unit tests exercise the factored function with a fake prefix and a
recorded `add_dll_directory`, so they hold on every platform; each was
shown red by a mutation (isfile check removed, None-guard removed, Scripts
dir dropped). The guides now say the runtime is handled, not that it must
be installed.

Full unit suite on macOS: 2573 passed, 4 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On the stock Windows box the runtime hook worked from the command line
(`import everos, greenlet` loads greenlet 3.5.3) yet the 15 SQLite-backed
watcher tests still errored under pytest. sqlalchemy imports greenlet at
import time and records the failure; the first line of the test module is
`from sqlmodel import ...`, so greenlet was already broken by the time the
package init ran.

conftest.py is loaded before any test module, so importing everos there
first makes the whole unit suite exercise the shipped runtime the way the
`everos` console script does. The comment names what this does not fix: a
host application that imports sqlalchemy before everos. That needs the
registration to run at interpreter start (a .pth), which is a packaging
change and a separate decision.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Running `-m "slow or live_llm"` on the Windows box surfaced three problems
in the suite itself, none in the product.

1. `tests/e2e/test_reflection_e2e.py` aborted collection of the whole run
   (`ModuleNotFoundError: No module named 'adapters'`), so zero tests
   executed. It imported `benchmarks.run` as a package, but since #425 the
   runner imports its siblings bare (`import adapters`) and only resolves
   with `benchmarks/` itself on sys.path -- the unit tests that use it
   already do that dance; this file now mirrors them. Verified against
   two detached worktrees: origin/main fails, e8612b9 (v1.3.0, before
   #425) collects. Not Windows-specific; CI never collects this file.
   The same refactor moved the prompts into `adapters/locomo.py`, added a
   `config` parameter to `_build_context`, and a `{current_date_line}`
   placeholder to ANSWER_PROMPT; the script body is adapted to those by
   signature. It has no `def test_` -- it is a manual driver with a
   `test_` filename -- and the adapted body has not been executed here;
   that needs a Tier 2 server and a LoCoMo run.

2. `test_real_sigint_during_phase_await_returns_130_with_resume_hint`
   sends itself `os.kill(os.getpid(), signal.SIGINT)`. On Windows that
   call is TerminateProcess with the signal number as exit code, so the
   test killed the pytest process mid-run: 27 of 29 had run, the summary
   and every traceback were lost, and `pytest_exit=2` was the literal
   SIGINT value, not pytest's "interrupted". Skipped on win32 with the
   mechanism in the reason.

3. `test_add_html_file_uri_parsed_into_buffer` built its uri as
   `f"file://{doc}"`, which on Windows is `file://C:\...` -- drive letter
   in the host slot. Same fix as the unit tests in f7a31c7: `Path.as_uri()`.

Collection of the live selection is now clean (29 selected, 0 errors);
the default `test_backfill_flags.py` selection still passes locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`test_rename_within_root_moves_the_row` failed once on windows-latest
(2572 passed, 1 failed): after `os.rename(a, b)` the row for `a` sat at
``added`` for the full 15 s while `b` was ``added`` as expected.

ReadDirectoryChangesW reports a rename as RENAMED_OLD then RENAMED_NEW and
watchdog pairs them only when both arrive in the same read -- the pairing
variable is local to one `queue_events` call and starts empty. Split across
two reads, the OLD leg is dropped and the watcher never sees the source
path. The scanner is the system's answer to exactly this: a state row whose
path is gone from disk is re-emitted as ``deleted`` on the next sweep.

The test asserted more than the watcher promises on Windows. It now asserts
the destination leg on the watcher, then runs one `scan_once` and asserts
the source leg -- the contract the system actually offers, on every
backend. Nothing was loosened: both legs are still required.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The soak dry run on Windows killed its load feeder 62 s in with
`PermissionError: [WinError 5]` from `os.replace(tmp, target)`: the target
was open in another process at that instant (the cascade worker reading it,
or Defender scanning the file it had just written). On Windows a file
another process holds open cannot be replaced; on POSIX it always can. The
server's own markdown writer is the same primitive at `writer.py:158`, so
under the same conditions `/add` would have failed the same way.

`_replace_with_retry` retries only `PermissionError`, with exponential
backoff (8 attempts, ~5 s of patience), and re-raises unchanged after the
budget -- nothing is swallowed, and any other `OSError` is not retried at
all. Each attempt is still a single atomic `os.replace`. On POSIX the first
attempt succeeds, so behaviour there is unchanged; no platform check.

Three tests fake `os.replace` so the behaviour is pinned on every platform:
transient refusal retried and the write lands (with the backoff sequence
asserted), persistent refusal propagates after exactly the budget with the
staging file cleaned up, and a non-permission error is not retried. Each
was shown red by a mutation (retry bypassed, `except OSError`, budget check
removed).

Full unit suite: 2576 passed, 4 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`dispatch(strategy_filter=...)` skipped the subscription lookup, so a bare
`ManualTick` from `POST /api/v2/ome/trigger` reached strategies declared
`Immediate(on=[AgentCaseExtracted])` / `Immediate(on=[SkillClusterUpdated])`.
Their handlers read `event.agent_id` → AttributeError → dead_letter. Found
while driving the four memory kinds on the Windows box.

The filter now keeps the name restriction but still requires the strategy to
subscribe to `type(event)`; a manual tick at a business-event strategy comes
back as `not_dispatched`. Strategies that list `ManualTick` are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A `role: "tool"` message without `tool_call_id` passed DTO validation and
only failed deep in extraction (`_boundary._to_conversation_item` raises
ValueError, retried as a transient LLM parse error, then 500). Refuse it at
the DTO with a `model_validator`; `docs/api.md` states the requirement.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5692baf made `dispatch(strategy_filter=...)` require a subscription to
`type(event)`, which also refused the bare ManualTick that
`POST /ome/trigger {"name": "reflect_episodes", "force": true}` sends at a
Cron strategy — the documented way to run reflection on demand
(docs/reflection.md). Caught by the agent-sim driver against a live server;
the unit tests only covered `Immediate(on=[ManualTick])` strategies.

A ManualTick at a Cron strategy is accepted: CronTick and ManualTick carry
the same single field, so the handler cannot read anything the tick lacks.
Idle strategies still have to subscribe (IdleTick has fields ManualTick does
not). Dispatcher and route tests pin the Cron case.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native-Windows section said the configuration was not yet supported
and listed what was missing. Both gaps are closed on a stock Windows 11
machine: the integration and live-LLM suites ran, a Tier 3 server produced
and served all four memory kinds, and a 10-hour soak with two concurrent
CLI sync processes ended with the index intact. The section now states the
numbers, the Python range, and the three Windows specifics an operator will
meet (sharing violations on replace, Windows Search on the memory root,
split rename events). The two soak findings that are not Windows-specific
(duplicate rows under cross-process sync, latency under write load) are
named so the section does not overclaim. Chinese mirror updated in step.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zhanghui and others added 2 commits September 24, 2026 12:33
Two tests read files with `Path.read_text()` and no encoding, which on
Windows means the console codepage. The files hold an em dash, and on a
zh-CN machine (cp936 / GBK) that byte sequence is an illegal multibyte
sequence: both tests fail with `UnicodeDecodeError`. CI's `windows-latest`
is en-US (cp1252), where the same bytes decode to the wrong characters
without raising, so CI never saw it; the soak laptop did. Explicit UTF-8,
matching what the markdown layer already does everywhere.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zhanghui and others added 2 commits September 24, 2026 13:51
Adversarial review of this branch's product code. The DLL hook looked only
under `sys.prefix`; `pip install` without write access to site-packages
(Python under Program Files) falls back to a per-user install whose data
files land under `site.getuserbase()`, where the hook was a no-op and
`import greenlet` still failed. Both bases are probed now, with a test.

Smaller corrections from the same review: the locking module claimed
`LockFileEx` while portalocker's default Windows locker is `msvcrt.locking`
(same semantics, wrong name); the replace-retry budget is ~2.5 s, not the
~5 s the comment and the commit body said; POSIX can raise PermissionError
on an immutable target, so "never" was wrong; each retry now logs at debug
level so the product path is observable (the soak's 49 retries were the
harness's); `scan_interval_seconds` gains `gt=0` (0 would spin the scanner
with no interval); the api.md `not_dispatched` sentence names the fifth
gate. The two watcher save tests now assert the saved row's mtime advanced,
so a lost event fails them — with the handler stubbed out they used to pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review of the guide against the evidence: the unit-suite number was a
macOS figure, "pip install" was not the path exercised (a uv checkout
was), "no entry lost" ignored the deliberately malformed files, the 49
replace retries belonged to the harness's writer, and the 3.13 / 3.14
claims came from Linux CI and macOS. Each sentence now says what ran on
which machine. The intro no longer contradicts the native section, the
docs index row matches, and two things an operator will meet are added:
LongPathsEnabled (both test machines had it on, a stock install does not)
and msvc-runtime shipping wheels only, per CPython minor version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The wheel built from this branch was installed with pip into a plain venv
on the Windows box (dependencies resolved from PyPI), imported, and started
a server that answered /health, /add and /search. The one step between
install and start is an LLM api_key in everos.toml: the server refuses to
start without it, and `everos init` says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On Windows the spawn-started child re-imports the package, and pulling in
lancedb behind Defender takes 15 s or more, so the 5 s readiness wait failed
three of the four cross-process tests on the soak box (idle run: 3 failed,
2587 passed). The readiness wait is not what these tests measure; every
timing assertion starts after the child signals ready. Raise only that wait.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gloryfromca
gloryfromca enabled auto-merge (squash) September 24, 2026 08:54
zhanghui and others added 2 commits September 24, 2026 17:05
The page was still called 'Running EverOS on Windows (WSL2)' although the
native install is verified and covered by its own CI job. Rename it to
docs/windows.md, retitle, and open with the two ways in the order a reader
should try them; the WSL2 walkthrough body is unchanged. References updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gloryfromca
gloryfromca merged commit ca6ea7d into main Sep 24, 2026
11 checks passed
@gloryfromca
gloryfromca deleted the feat/windows-support branch September 24, 2026 09:15
This was referenced Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant