Skip to content

fix: read the rollback position before the write retry loop#126

Merged
michael-denyer merged 1 commit into
masterfrom
fix/assoc-writer-tell-rollback
Jul 26, 2026
Merged

fix: read the rollback position before the write retry loop#126
michael-denyer merged 1 commit into
masterfrom
fix/assoc-writer-tell-rollback

Conversation

@michael-denyer

Copy link
Copy Markdown
Owner

A latent bug in IncrementalAssocWriter._write_buf, found while clearing that file's
pyrefly baseline entries. 107 → 100 entries, and the write path no longer has a failure
mode that bypasses its own error handling.

The bug

The rollback position was read inside the try-block it protects:

for attempt in range(1 + len(_RETRY_BACKOFF)):
    try:
        pos = self._file.tell()      # <-- if this raises...
        self._file.write(buf)
        ...
    except OSError as e:
        last_error = e
        try:
            self._file.seek(pos)     # <-- ...pos was never assigned

If tell() raised — ESPIPE on a pipe or FIFO, a bad fd — pos was unbound, so the rollback
raised UnboundLocalError. That is not an OSError, so it fell straight through the
handler that was supposed to deal with it:

  • no retry
  • no seek/truncate rollback
  • no _cleanup_partial()
  • escaped write()'s documented Raises: OSError contract

and the caller got this instead of the disk error:

UnboundLocalError: cannot access local variable 'pos' where it is not associated with a value

Reproduced by making tell() raise OSError(ESPIPE).

The fix

tell() runs once, before the loop. That is correct rather than merely convenient:

  • The position is the same on every attempt — a failed write is truncated back to it — so
    nothing is lost by not re-reading it per attempt.
  • A failure there means nothing was written, so there is no partial state to roll back,
    and letting the OSError propagate is the honest signal. The context manager still
    deletes the partial output on the way out (asserted in the test).

The handle is also bound to a local with an explicit open check, rather than dereferencing
self._file five times behind a guard that lives in three separate callers. A private method
that dereferences an optional attribute shouldn't depend on every caller remembering.

Baseline

That is what clears this file's 7 entries — all of them were this one method reaching
through the optional attribute. 107 → 100.

Verification

Regression test written first and confirmed failing against the old code
(test_write_buf_tell_failure_surfaces_as_oserror), in the style the file already uses for
its other injected-I/O-failure tests:

E   UnboundLocalError: cannot access local variable 'pos' ...
src/jamma/lmm/io.py:186: UnboundLocalError
Check Result
pytest tests/test_incremental_writer.py 29 passed (28 existing + 1 new)
pytest tests/ 2242 passed, 10 skipped, 3 xfailed
All 4 LMM modes × both backends, .assoc.txt byte-identical — this is the hot write path
ruff check / ruff format --check clean
pyrefly check --baseline on 3.11.13/numpy 2.4.6 and 3.13.5/numpy 2.5.1 0 errors both

Note for the next baseline pass

tests/test_incremental_writer.py holds the single largest remaining cluster at 23
entries
— and they are all one root cause, not 23: writer._file is TextIO | None, and
every test that injects an I/O failure does writer._file.write = .... One small helper that
returns the narrowed handle collapses the whole cluster. The new test here already avoids
adding a 24th, so this PR grows the baseline by nothing.

`IncrementalAssocWriter._write_buf` read its rollback position inside the
try-block it protects:

    for attempt in ...:
        try:
            pos = self._file.tell()
            self._file.write(buf)
            ...
        except OSError as e:
            self._file.seek(pos)     # pos unbound if tell() was what failed

If `tell()` raised, `pos` was never assigned, so the rollback raised
`UnboundLocalError`. That is not an `OSError`, so it bypassed the retry, the
rollback, and `_cleanup_partial`, and escaped `write()`'s documented
`Raises: OSError` contract with a misleading message. Reproduced with an
ESPIPE from `tell()`:

    UnboundLocalError: cannot access local variable 'pos' where it is not
    associated with a value

`tell()` now runs once before the loop. The position is the same on every
attempt, since a failed write is truncated back to it, so nothing is lost by
not retrying it — and a failure there means nothing was written, so there is
no partial state to roll back. The context manager still deletes the partial
output on the way out.

The file handle is also bound to a local with an explicit open check, instead
of dereferencing `self._file` five times behind a guard that lives in three
separate callers. That clears the seven pyrefly baseline entries in this file
(107 -> 100), which were all this method reaching through the optional
attribute.

Regression test added first and confirmed failing against the old code:
`test_write_buf_tell_failure_surfaces_as_oserror`.

`pytest tests/` — 2242 passed, 10 skipped, 3 xfailed. All four LMM modes on
both backends still produce byte-identical .assoc.txt output.
@michael-denyer
michael-denyer merged commit 2848404 into master Jul 26, 2026
12 checks passed
@michael-denyer
michael-denyer deleted the fix/assoc-writer-tell-rollback branch July 26, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant