+ PyInstaller: create CArchive (PKG) payloads, validated by PyInstaller's own reader and writer - #407
Merged
Merged
Conversation
Hawkynt
force-pushed
the
feat/pyinstaller-onefile-writer
branch
from
September 30, 2026 13:11
aae7e5c to
10a65df
Compare
Hawkynt
changed the title
Hawkynt
force-pushed
the
feat/pyinstaller-onefile-writer
branch
from
September 30, 2026 15:48
74bf21f to
3ad4b51
Compare
PyInstaller CArchive handling could inspect and extract onefile payloads but could not create an archive. The generic archive descriptor had no writer, even though CArchive defines a documented TOC and cookie for standalone PKG payloads. Add a clean-room CArchive writer from the official layout documentation. Support zlib and stored entries, compression effort tiers, entry type codes, and Python cookie fields; explicitly report metadata that the generic input API cannot preserve. Tests cover compressed and stored round trips and reject unsafe paths. The writer emits a bare PKG payload; a runnable onefile executable still requires a matching bootloader.
The Python library name occupies a fixed-width cookie field, so bytes after a short name must be NUL padding. Stack-allocated bytes are not a format guarantee. Clear the field before encoding the name.
…uld not be reopened Symptom: every Convert_*__to__PyInstaller case failed with "Cannot list format: Unknown": the writer's .pkg output carried no extension anyone claims and no leading magic. Root cause: PyInstaller detection only scanned PE (and extension-less ELF/Mach-O) overlays; a bare CArchive, whose only signature is the cookie at its end, fell through. Fix: when neither extension nor magic decides, a file whose last 88 bytes are the MEI cookie, whose package length is the file length and whose TOC ends where the cookie starts is detected as PyInstaller.
Symptom: PyInstaller 6.21's CArchiveReader rejected the default output
("Python shared library name not set in the archive!") and failed a
compressed empty file with "incomplete or truncated stream"; PyInstaller's
CArchiveWriter, given the same stored entries, produced different bytes.
Root cause: the cookie's library name defaulted to empty; .NET's ZLibStream
emits zero bytes for empty input; TOC entries were not padded to the 16-byte
multiple the bootloader needs on strict-alignment targets; nested names
always used '/', while PyInstaller stores '\' for the Windows bootloader;
and the writer truncated its output stream and used absolute positions, so
a PKG could not follow a bootloader stub in the same stream.
Fix: the library name defaults to python<version>.dll and must be under 64
ASCII bytes; an empty compressed entry is the canonical 8-byte zlib stream;
TOC entries are NUL-padded to 16 bytes; a TargetOs option picks the
separator; offsets are relative to where the archive starts and nothing
before it is touched. TypeCode is limited to x (data) and b (binary), since
the other codes tell the bootloader to unmarshal code, apply an option or
open a nested archive.
Verification: PyInstallerCArchiveOracleTests runs PyInstaller's own
CArchiveReader over our zlib and stored PKGs (every entry extracts to the
same SHA-256) and compares a stored PKG byte for byte with CArchiveWriter's
output for the same entries; it is ignored where no Python with PyInstaller
is on the PATH. Sourcing rung 3 (PyInstaller's CArchive documentation) with
PyInstaller as a black-box oracle; no PyInstaller code is used.
Hawkynt
force-pushed
the
feat/pyinstaller-onefile-writer
branch
from
September 30, 2026 19:46
8f36ef7 to
1c160a2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
x/btype codes, Python version and library name in the cookie, 16-byte-aligned TOC entries,TargetOsseparator, offsets relative to the archive start (so a PKG can follow a bootloader stub).Verification
PyInstallerCArchiveOracleTests(runs where Python with PyInstaller is on PATH; verified locally with PyInstaller 6.21): PyInstaller'sCArchiveReaderextracts every entry we write, and a stored PKG is byte-identical toCArchiveWriteroutput for the same entries. PyInstaller and conversion-matrix tests green locally.