Repository navigation
fix(tests): decode export-cache Git paths as UTF-8 - #13269
Conversation
MarcZX10
left a comment
There was a problem hiding this comment.
Tests run and reviewed — looks good
The explicit UTF-8 decoder fixes the path regression: under a CP1252-decoder emulation and a Unicode temporary root, the PR head passes while the base fails two tests.
What I ran
Reviewed head 35f00deb91f5749b297ab3ec39b35dba19f8385f against PR base/current main 3d19f3fed31be38d7196ea1bd74e29ab6cc3c9a4; a local merge preview was clean. Environment: Linux x86_64, Python 3.11.2, Git 2.39.5. I ran the changed suite in isolated worktrees pinned to the PR head and base. For the locale regression check, a temporary sitecustomize.py shim forced CP1252 only for captured text subprocesses that did not specify an encoding; LC_ALL=C.UTF-8 kept filesystem paths UTF-8-capable, and TMPDIR pointed to /tmp/AnyPS5-测试.
python3 tools/pr_precheck.py --repo boykopovar/AnyPS5 --pr 13269
python3 tools/check_conventions.py --base 3d19f3fed31be38d7196ea1bd74e29ab6cc3c9a4 --head 35f00deb91f5749b297ab3ec39b35dba19f8385f
python3 tools/check_test_registration.py --scripts
python3 -m unittest tools.tests.test_pr_overlap_export_cache
env LC_ALL=C.UTF-8 TMPDIR=/tmp/AnyPS5-测试 PYTHONPATH=/tmp/anyps5-cp1252-emulation python3 -m unittest tools.tests.test_pr_overlap_export_cacheThe final two commands were run separately in worktrees at the PR head and base. In the temporary shim, subprocess.run adds encoding="cp1252" only when text=True, capture_output=True, and no encoding was passed.
Results
| Check | Result |
|---|---|
| Changed Python suite | Passed: 7 tests at the PR head, including with a Unicode temp root. |
| CP1252 regression control | Discriminated: base failed 2 tests with corrupted paths; PR head passed all 7. |
| Conventions and PR template | Passed in the no-build precheck. |
| Test registration | Passed: all 222 test scripts are registered. |
| Patch whitespace | Passed. |
| Current-main merge preview | Passed cleanly; tree 01a977976aca5fd25418a932953de02a67ebfcfd. |
| No-build precheck | Core applicable checks passed. Its optional test-quality helper was absent from the isolated PR checkout, and its default-locale negative control did not emulate CP1252; I separately ran the targeted control above. |
| Hosted CI | Eligibility, conventions, intake, and labels were still queued at the final snapshot; no hosted result is claimed. |
Unedited local test output
PR head, with the CP1252 decoder emulation:
.......
----------------------------------------------------------------------
Ran 7 tests in 0.297s
OK
Base, with the same emulation:
E...E..
======================================================================
ERROR: test_a_cache_from_other_parsers_is_ignored (tools.tests.test_pr_overlap_export_cache.ExportCacheTests.test_a_cache_from_other_parsers_is_ignored)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/AnyPS5-pr13269-base/tools/tests/test_pr_overlap_export_cache.py", line 92, in test_a_cache_from_other_parsers_is_ignored
cache = json.loads(self.cache.read_text(encoding="utf-8"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1059, in read_text
with self.open(mode='r', encoding=encoding, errors=errors) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1045, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/AnyPS5-测试/tmplha18832/.git/pr_overlap-exports.json'
======================================================================
ERROR: test_an_unreadable_cache_is_rebuilt (tools.tests.test_pr_overlap_export_cache.ExportCacheTests.test_an_unreadable_cache_is_rebuilt)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/AnyPS5-pr13269-base/tools/tests/test_pr_overlap_export_cache.py", line 100, in test_an_unreadable_cache_is_rebuilt
self.cache.write_text("{not json", encoding="utf-8")
File "/usr/lib/python3.11/pathlib.py", line 1079, in write_text
with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1045, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/AnyPS5-测试/tmpbvbtag1w/.git/pr_overlap-exports.json'
----------------------------------------------------------------------
Ran 7 tests in 0.280s
FAILED (errors=2)
Negative control
| Assertion | Change to code under test | Did the new head fail? |
|---|---|---|
| The cache path is derived from Git output without mojibake when the parent locale decoder is CP1252 and the temp root contains non-ASCII characters. | Ran the base version without explicit UTF-8, then the PR version with encoding="utf-8". |
Base: yes, 2 FileNotFoundErrors. PR head: no, all 7 tests pass. |
Findings
Blocking
- None found in this focused test-only change.
Non-blocking
- The CP1252 case was emulated on Linux; I did not run native Windows Python.
Sources
Not verified
- Native Windows execution and the hosted CI jobs, which were queued at the final snapshot. The author’s separate Python 3.14.7 report was not independently run.
|
Linux run at this PR's exact head Arch Linux, kernel 7.2.9, Ryzen 9 7950X, g++ (GCC) 16.2.1 20260810, glibc 2.44, x86-64. Release,
|
|
Qualification update — 2026-10-30, repeat 2 Actual Windows check at exact published Evidence: https://github.com/JJenkx/AnyPS5/releases/download/linux-validation-20261023/owned-current-windows-20261030-repeat2.tar.gz, SHA256 e423c97373758bf59239e02c138b74f35f1a45488793bc92878a4c836f94a584. AI-assisted: yes (Codex). AI-assisted: yes (Codex). |
What
The export-cache test helper decodes Git's UTF-8 output with the parent's locale. Under CP1252, a temporary directory containing
éorЖis misdecoded bygit rev-parse --absolute-git-dir, soself.cachepoints at the wrong path and two tests raiseFileNotFoundError.Request UTF-8 explicitly in this one test helper, matching production
pr_overlap.git. This is the additional Python fixture issue reported alongside #13083; C++ scratch-file handling is tracked separately in #13104.Tested
Linux, Python 3.14.7, pinned main
a1782bdc02c68599830868a7005fd5bced4b7894against this one-line change. The existing seven-test suite uses synthetic local Git repositories.éЖNo skips. CP1252 is modeled at the parent subprocess decoder on Linux; native Windows execution remains untested here. Git fixtures use empty templates with hooks disabled. Conventions and whitespace checks pass.
Unmodified upstream overlap checker: all 1,470 open PRs checked against main
2f3fa0f7dc3ce81028c89c7ec17a3010f37c9eb0; no shared files or exports.Linux qualification logs, source pins and overlap report (SHA-256
368a5cdbe83686c1401f9c46ba3576a04c60d88c624b147e12e809a93cd94bf9; upload/download verified).Qualification update — 2026-10-30, repeat 2
Actual Windows check at exact published
35f00deb91f5749b297ab3ec39b35dba19f8385f: Windows 11 Enterprise 10.0.22621, Ryzen 9 5950X, Python 3.15 with-X utf8=0/PYTHONUTF8=0. The unchanged seven export-cache fixture tests pass at each ASCII, é and Ж TEMP path: 21/21. Restoring only the old locale-decoding helper (text=Truein place of explicit UTF-8) keeps ASCII green but reproduces two FileNotFoundError errors in each non-ASCII path. Restoring the exact head gives another 21/21 passes. This is actual Windows execution, complementing the earlier Linux CP1252 model. Exact archived tools source, raw logs, script and FIFO receipt are attached; no production behavior, SDK/game/GPU or performance claim.Evidence: https://github.com/JJenkx/AnyPS5/releases/download/linux-validation-20261023/owned-current-windows-20261030-repeat2.tar.gz, SHA256 e423c97373758bf59239e02c138b74f35f1a45488793bc92878a4c836f94a584. AI-assisted: yes (Codex).
AI-assisted: yes (Codex).
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)