Skip to content

TeX Live 2026 + more - #9

Open
adam-weber wants to merge 6 commits into
mainfrom
upgrade/texlive-2026
Open

TeX Live 2026 + more#9
adam-weber wants to merge 6 commits into
mainfrom
upgrade/texlive-2026

Conversation

@adam-weber

@adam-weber adam-weber commented Mar 14, 2026

Copy link
Copy Markdown
Member

Upgrades the engine to TeX Live 2026, and bundles three workstreams that grew alongside it:

  1. on-demand package normalization for build-required packages
  2. a normalized-artifact (prebuilt) cache
  3. data-driven font-file resolution w/ memory-management improvements

TeX Live 2026 upgrade

  • Bump the busytex submodule to the TL2026 build (WASM engines + TLPDB).
  • Regenerated bundle metadata: packages/bundles/file-manifest.json, bundles.json, file-to-package.json.
  • Updated package.json (description → TeX Live 2026).

Package normalization (build-required packages)

A provider chain (packages/providers/) that turns fetched packages into ready-to-use runtime files:

  • prebuilt / tds-zip (no build), ins-docstrip / dtx (run docstrip via a headless busytex engine).
  • build-engine.ts + busytex-engine.ts: TeX build step on the same WASM the compiler uses; lazy-imported
    so the provider core stays browser-safe.
  • Declarative overrides.json registry; structured unbuildable errors instead of crashes.
  • Wired into processZipData (ctan-core.ts) so both the proxy and dev server pick it up.
  • Fixes the fflate-under-Bun unzip crash (unzipSync).

Normalized-artifact (prebuilt) cache

  • prebuilt-cache.ts: serve a pre-normalized artifact for build-required packages (e.g. acrotex/insdljs)
    instead of building in-request — the only way these work on the (FS-less) Cloudflare Worker.
  • scripts/prebuild-packages.js: offline prebuild → packages/bundles/prebuilt/<pkg>.json (+ _index.json manifest);
    deterministic, atomic writes.
  • Both proxies read it from disk; the Worker reads from R2 (gated by the manifest, loaded per isolate like the font index).
  • Artifacts/manifest are generated, not committed (gitignored; shipped to R2/releases).

Font-file resolution

  • font-index.ts builds a stem→package index from the TLPDB (committed font-file-to-package.json),
    so TeX font requests by file stem (phvr8t, …) resolve to their package on every runtime.
  • TDS-routing fix in processExtractedFiles (anchored matchers so fonts land where kpathsea searches).

CTAN proxy / fetch improvements

  • Shared ctanDownloadUrls() in ctan-core.ts — one source of truth for CTAN URL conventions
    (used by ctan-proxy.ts, serve-local.ts, and the prebuild).
  • Resolve fontspec fonts by name; fix CTAN file vs. package routing.
  • serve-local.ts: configurable port (SIGLUM_DEV_PORT).

Memory management & storage

  • src/worker.js, src/storage.js, src/compiler.js: memory-cleanup improvements and format-cache handling,
    including capture & replay of preamble side-effect files so cached .fmt reuse doesn't drop runtime-written files.
  • Lazy bundle loading + decompress-lz4.ts.

Tests

  • New suite: tests/unit, tests/integration, tests/worker, tests/browser, shared tests/setup mocks/fixtures, vitest configs.
  • Normalization: tests/unit/providers.test.ts, tests/integration/normalize-corpus.test.ts,
    tests/unit/prebuilt-cache.test.ts, and scripts/providers-bun-smoke.ts (Bun parity).
  • All green locally: 395 vitest tests, tsc clean, bun-parity OK.

Docs

  • docs/self-hosting.md: how to run the engine on any host (proxy contract, reference servers, where artifacts come from).

@adam-weber

Copy link
Copy Markdown
Member Author

Closes #10

@adam-weber adam-weber changed the title Upgrade to TeX Live 2026 with memory management improvements and test… TeX Live 2026 + more May 26, 2026
adam-weber and others added 2 commits May 27, 2026 17:55
- Bump busytex submodule to merged main (c2c008e: TL2026 + build hardening)
- Regenerate bundles.json / file-manifest.json; re-merge cm-super (422 fonts)
- bundle-cm-super.ts: pretty-print the merged manifest (was minified)
- storage.js: bump BUNDLE_CACHE_VERSION 6 / MANIFEST_CACHE_VERSION 7
- docs: TL2026 + required bundle-cm-super.ts step; building.md corrections
- gitignore bundle-deps.json / package-map.json (regenerable intermediates)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
List the bundle (acrotex), not sub-packages; sub-package requests resolve to the
bundle artifact via proxy/worker parent-resolution.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant