Skip to content

Parse native TUX3 allocation trees and journal metadata - #362

Merged
Hawkynt merged 8 commits into
mainfrom
feature/tux3-allocation-log-parser
Sep 19, 2026
Merged

Hawkynt merged 8 commits into
mainfrom
feature/tux3-allocation-log-parser

Conversation

@Hawkynt

@Hawkynt Hawkynt commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • parse native TUX3 internal bnodes, inode leaves, inode attributes, data-tree leaves, and the bitmap inode
  • parse the reverse-linked TUX3 journal into chronological native records
  • apply active allocation-only journal records to the on-disk bitmap to produce an effective allocation map
  • make the filesystem extent map expose in-volume free blocks only when the tree and journal state are provably trustworthy
  • fail closed when the journal is malformed or contains active structural tree mutations that require upstream-style stage-1 replay
  • keep create/modify/defrag/purge unavailable until directory/inode mutation and transactional metadata/log writing are implemented

Design

TUX3 stores the allocation bitmap as special inode 1. This implementation independently follows the native on-disk structures required to find that inode: packed superblock root -> generic bnodes -> ileaf dictionary/attributes -> DATA_BTREE root -> dleaf/direct mapping -> bitmap data. Bitmap bits are interpreted little-endian within each byte, matching the native allocator.

The journal is read from the superblock logchain/logcount reverse chain, validated block-by-block, decoded in chronological order, and split at the latest UNIFY marker. Allocation-only records (BALLOC, BFREE, BFREE_RELOG; with BFREE_ON_UNIFY remaining allocated until unify) are applied to the bitmap. Log blocks are conservatively reserved.

Structural records (LEAF_*/BNODE_*) require the native structural replay phase before stale tree roots can be trusted. If any such record is active after the latest UNIFY, AllocationMapValid is false and layout/Wipe fall back to reserving the entire declared volume. This intentionally prefers false negatives over destructive false-free classification.

Allocated bitmap runs are surfaced as MetadataReserved, not Used, because object ownership and logical file sizes are not yet resolved. Consequently the generic wipe path can zero only explicit Free runs and cannot infer cluster-tip slack inside allocated TUX3 blocks.

Tests

Added original NUnit fixtures that build a compact native metadata graph in memory and cover:

  • two-level generic bnode traversal into an ileaf
  • backward ileaf attribute dictionary offsets and DATA_BTREE decoding
  • dleaf logical-to-physical mapping and sentinel validation
  • little-endian allocation bitmap decoding and coalesced allocation runs
  • allocation-only journal replay changing effective allocation state
  • in-volume Wipe preserving allocated blocks while clearing proven free blocks
  • active structural journal records disabling the allocation map and Wipe
  • latest UNIFY making older structural records inactive
  • malformed/unknown journal records failing closed
  • existing truncated-volume and shift-overflow regressions

References / licensing

Consulted the upstream OGAWAHirofumi/linux-tux3 tree, especially tux3.h, btree.c, ileaf.c, iattr.c/iattr.h, dleaf.c, filemap.c, balloc.c, log.c, replay.c, and super.c.

The upstream tree is GPLv2. No expressive implementation code was copied or translated. The implementation is clean-room C#: it uses only factual/specification-defined field layouts, constants, record sizes, replay semantics, and independently written algorithms/tests. No new dependency is introduced.

Scope after this PR

This moves TUX3 beyond superblock-only maintenance: native free-space layout and Wipe work inside a clean/replayable volume. Full native R/W, purge, and defragmentation are still intentionally not advertised because safe mutation also requires directory/inode traversal, orphan/version handling, structural journal replay, and a transactional writer.

@Hawkynt
Hawkynt force-pushed the feature/tux3-allocation-log-parser branch 7 times, most recently from 7f4a8e9 to 6cdfeab Compare September 19, 2026 03:44
The docs gate compares README.md and REFERENCE.md against the built assembly.
Tux3BlockRun, Tux3JournalRecordType and Tux3JournalRecord raise the public type
count from 852 to 855 and add three reference entries, so both generated files
drifted. Rendered with the package-readme generator; no hand edits.
@Hawkynt
Hawkynt force-pushed the feature/tux3-allocation-log-parser branch from 2bbeda0 to ffc58c1 Compare September 19, 2026 04:48
@Hawkynt
Hawkynt merged commit ce7cf5e into main Sep 19, 2026
5 checks passed
@Hawkynt
Hawkynt deleted the feature/tux3-allocation-log-parser branch September 19, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant