Skip to content

Latest commit

 

History

History
350 lines (288 loc) · 57.5 KB

File metadata and controls

350 lines (288 loc) · 57.5 KB

Hawkynt.FileFormats.FileSystems

NuGet NuGet downloads License CI Target

Pure-managed filesystem handling for .NET, without mounting anything through the host OS. The package claims the WHOLE domain — every filesystem and disk-image container, modern, legacy, virtual-machine, optical, forensic and retro-computing alike — not a selection of it. Where a format is read-only, create-only or a deliberate subset, that is a tracked gap recorded in the support matrix below, which is generated from the descriptors and checked by the test suite.

📦 Installation

dotnet add package Hawkynt.FileFormats.FileSystems

The package bundles every FileSystem.* assembly and the disk-image FileFormat.* assemblies, with Hawkynt.Compression.Core as its one NuGet dependency.

✨ Features

  • Inspect filesystems in-process without libguestfs, loop mounts, kernel drivers or elevated privileges — NTFS from Linux, ext4 from Windows, HFS+ from either.
  • Create fresh images for every format marked WORM or R/W below, with real nested directory trees where the format has them.
  • Edit existing images for every format marked R/W: add, replace, remove, purge. Some formats edit blocks in place, some lay the volume out again; the matrix says which.
  • Defragment, wipe unused space, shrink and re-lay volumes at a chosen geometry on the formats whose descriptors offer it.
  • Open disk-image containers (VHD, VHDX, VMDK, VDI, QCOW2, DMG, EWF, optical and tape images, firmware volumes) and reach the filesystem inside.
  • Every writer is held against the format's own tools where one exists: a real kernel under QEMU, its fsck, or its canonical third-party utility.

🧩 Support matrix

State Meaning
N/A Registered filesystem domain that is not represented by a standalone image/container descriptor; specialized multi-source drivers may expose mounted capabilities separately.
R Open, list and extract only. For network, distributed and encrypted formats this may be signature/metadata detection; deliberately unverified descriptors can instead require explicit format selection and expose an opaque object.
WORM Read plus create a fresh image; no supported edit of an existing image.
R/W Read plus add / replace / remove / purge on an existing image. The edit may update blocks in place or lay the volume out again — the Notes column says when it is the latter.

The six verb columns are the maintenance operations the CLI and the UI gate on, defined once in docs/ARCHIVE-MODEL.md:

  • Defrag lays a volume out again, and says how: moving relocates only the runs that are out of place through a block mover of the format's own, rebuild writes the volume out afresh.
  • Wipe zeroes what no file holds — free space, cluster-tip slack, deleted directory entries — without touching a live byte.
  • Shrink reduces a volume to what it needs, keeping its geometry. It is absent wherever the geometry is fixed: for most read-only filesystems and for optical images there is no smaller canonical size to step to, so nothing could be reclaimed.
  • Layout re-lays a volume at a chosen cluster or block geometry. This needs a creator to write the new volume with, so a format that can only analyse its geometry is marked absent here however thoroughly it reads it.
  • Purge erases every live entry and leaves a valid empty image.
  • Compact is the composite the UI offers as one click: defrag, then layout, then shrink, so it is available wherever any of those three is.

Proof is how the format is held against something outside this repository, and every tool it names links to that tool's own source, so the thing doing the checking can be reached from here. "own reader + struct-parity tests" means nothing outside this repository currently checks the format — either no such tool exists, or none is wired into the suite — so the on-disk structures are checked field by field instead. That cell is the honest gap, and it is deliberately preferred over naming a tool nobody runs: a cell that cites a checker which never executes stops anyone from going looking for one.

The Id, State and verb columns are read off the descriptors by FilesystemReadmeIsCurrentTests, and cross-checked against the registry's own filesystem list by CapabilityDocumentationTests; a row that disagrees with the code fails the build. FilesystemVerbsAreBackedTests then runs each ticked verb, so a cell cannot be earned by declaring an interface that refuses when called.

Disk-image and firmware containers

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
Apple IPSW Ipsw R/W ✅ rebuild own reader + struct-parity tests ZIP-based firmware archive IPSW
BIN/CUE BinCue R/W own reader + struct-parity tests Sector-image editor CUE sheet
CDI Cdi R/W ✅ rebuild own reader + struct-parity tests ISO 9660 edits rebuild; low-level existing-sector rewrite remains available DiscJuggler
Commodore Lynx archive Lynx R/W ✅ rebuild own reader + struct-parity tests C64 Lynx archive Lynx
DMG Dmg R/W ✅ rebuild own reader + struct-parity tests raw-profile edits in place; readable foreign/compressed profiles rebuild to canonical raw UDIF Apple disk images
EnCase EWF (E01) Ewf R/W ✅ rebuild own reader + struct-parity tests EnCase E01 segment chain libewf documentation
Flattened Device Tree Blob Dtb R/W ✅ rebuild own reader + struct-parity tests Flattened Device Tree walked as a pseudo-archive Devicetree specification
Intel HEX IntelHex R/W ✅ rebuild own reader + struct-parity tests Intel HEX and Motorola S-record, normalised to firmware.bin + metadata; the writer takes one payload and the base and start addresses from metadata.ini Intel HEX / S-record
MDF/MDS Mdf R/W ✅ rebuild own reader + struct-parity tests Fixed-track ISO 9660 editor; raw-sector EDC/ECC regenerated Alcohol 120%
Nintendo PartitionFS Pfs0 R/W ✅ rebuild own reader + struct-parity tests Nintendo Switch PartitionFS PFS0 (switchbrew)
NRG Nrg R/W ✅ rebuild own reader + struct-parity tests NRG v1/v2 reader; v2 TAO writer; named edits rebuild the embedded ISO Nero Burning ROM
PSP CSO/ZSO Cso R/W ✅ rebuild own reader + struct-parity tests Compressed ISO; edits re-pack the affected blocks CSO
QCOW2 Qcow2 R/W ✅ rebuild qemu-img check / convert -O raw / reverse qemu-img create Edits go to the inner filesystem QCOW2 specification
T64 T64 R/W ✅ moving own reader + struct-parity tests C64 tape archive T64 (VICE)
TAP Tap R/W ✅ moving own reader + struct-parity tests Sinclair / Commodore tape image TAP (World of Spectrum)
TI-TXT (MSP430) TiTxt R/W ✅ rebuild own reader + struct-parity tests TI-TXT sparse firmware dump; edits and canonical rebuild preserve omitted address holes TI-TXT
U-Boot uImage UImage R/W own reader + struct-parity tests U-Boot legacy image header + payload; the writer takes one payload and the header fields from metadata.ini, and computes both CRCs U-Boot image format
UEFI Firmware Volume UefiFv R/W ✅ rebuild own reader + struct-parity tests Firmware volume / FFS UEFI specification
VDI Vdi R/W ✅ rebuild qemu-img check / convert -O raw / reverse qemu-img create Edits go to the inner filesystem VirtualBox storage
VHD Vhd R/W ✅ rebuild qemu-img check / convert -O raw / reverse qemu-img create Standalone fixed and dynamic; differencing requires a parent chain and is rejected; edits go to the inner filesystem Microsoft VHD
VHDX Vhdx R/W ✅ rebuild qemu-img check / convert -O raw / reverse qemu-img create Edits go to the inner filesystem MS-VHDX
VMDK Vmdk R/W ✅ rebuild qemu-img check / convert -O raw / reverse qemu-img create Edits go to the inner filesystem VMware Virtual Disk API

Microsoft / DOS

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
DoubleSpace CVF DoubleSpace R/W ✅ moving MS-DOS 6.22 DRVSPACE driver under QEMU mounts, reads and writes; real driver CVFs read byte-exact MSDBL6.0 CVF; stored clusters DoubleSpace
DriveSpace 3 CVF DriveSpace3 R/W ✅ moving dmsdos mounts the genuine GenuineDvr3Writer output and reads byte-exact Win95 DVR3 CVF; a Win95 guest oracle and the HiPack codec are still open DriveSpace 3 notes
DriveSpace CVF DriveSpace R/W ✅ moving MS-DOS 6.22 DRVSPACE mounts the DoubleSpace / DriveSpace CVFs; real driver CVFs read byte-exact MSDSP6.0 CVF; stored clusters DriveSpace
exFAT ExFat R/W ✅ moving host-kernel exfat mount r/w + fsck.exfat exFAT specification
FAT Filesystem Image Fat R/W ✅ moving host-kernel vfat mount r/w + fsck.fat + mtools + reverse mkfs.vfat FAT12/16/32 with long filenames; in-place FatModifier FAT specification
FAT+ Filesystem Image (large-file extension) FatPlus R/W ✅ rebuild host-kernel vfat mount + fsck.fat read the written volume FAT+ large-file extension FAT+ draft
FATX (Xbox) Fatx R/W ✅ moving own reader + struct-parity tests Xbox FATX FATX (xboxdevwiki)
HPFS Hpfs R/W ✅ moving own reader + struct-parity tests OS/2 HPFS HPFS layout (OS/2 Museum)
NEC PC-98 DOS Pc98 R/W ✅ moving own reader + struct-parity tests NEC IPL keeps the BPB at 0x80, so the Linux msdos driver rejects it PC-98 disk layout
NTFS Ntfs R/W ✅ moving ntfsls / ntfsinfo / ntfsfix (ntfs-3g) + reverse mkfs.ntfs Writes NTFS 1.2, 3.0 or 3.1 as the NtfsVersion create option selects, and reads all three: the version drives the FILE record header layout (3.1 names the record at 44 with the USA at 48; 3.0 and 1.2 have no record number and put the USA at 42), the metadata set ($Secure+$Extend from 3.0, $Quota and no $Extend before it), the $AttrDef table and the $STANDARD_INFORMATION size (72 from 3.0, 48 on 1.2). No security descriptors are stored at any version. Edits relayout the volume; ntfs3 mounts it but lists an empty root (root $I30 index pending) MS-FSCC
ReFS Refs R/W ✅ moving own reader + struct-parity tests ReFS 3.x; mutation is offline-quiescent only (see limitations). Its geometry analysis is real, but there is no creator to write a re-laid volume with, so Layout stays absent ReFS overview
Stacker CVF Stacker R/W ✅ rebuild dmsdos mounts the genuine GenuineStackerWriter output and reads byte-exact STACVOL; Genuine/Extended flavor-preserving rebuild edits Stacker notes
Transactional FAT (TFAT) TFat R/W ✅ rebuild host-kernel vfat mount + fsck.fat read the written volume Transaction-safe FAT (Windows CE) TFAT

Unix / Linux

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
AdvFS (Tru64 UNIX) AdvFs R/W ✅ moving own reader + struct-parity tests Tru64 UNIX AdvFS
BcacheFS BcacheFs R/W ✅ moving bcachefs fsck -n + show-super; alloc / freespace / backpointer / accounting witness tests True in-place edits for the single-device, regular-extent profile; other profiles are refused for mutation bcachefs
BFS Bfs R/W ✅ moving own reader + struct-parity tests BeFS; edits rebuild the volume Be File System
Btrfs Filesystem Image Btrfs R/W ✅ moving btrfs check --readonly + show-super; alloc / freespace / backpointer / accounting witness tests Edits relayout the volume Btrfs on-disk format
Coherent FS Coherent R/W ✅ moving own reader + struct-parity tests PDP-endian sysv
EFS (SGI Extent File System) Efs R/W ✅ moving host-kernel efs mount reads byte-exact SGI EFS EFS
ext1 Ext1 R/W ✅ moving dumpe2fs accepts as ext2 1992 ext, magic 0xEF51; no mkfs.ext1 exists Extended file system
ext2/3/4 Ext R/W ✅ moving e2fsck -fn / dumpe2fs / debugfs + reverse mke2fs ext2 / ext3 / ext4 ext4 documentation
F2FS F2fs R/W ✅ moving host-kernel f2fs mount r/w + fsck.f2fs F2FS documentation
GFS (Sistina/Red Hat, original) Gfs1 R/W ✅ moving own reader + struct-parity tests; the Linux gfs2 driver rejects GFS1 Sistina GFS GFS2
GFS2 (Global File System 2) Gfs2 R/W ✅ moving fsck.gfs2 -n + debugfs.gfs2; mkfs.gfs2 image read by the package Standalone lock_nolock; stuffed + indirect-tree files; edits rebuild while preserving image-size floor and lock table GFS2
HAMMER (DragonFly BSD) Hammer R/W ✅ moving DragonFly kernel mount r/w under QEMU + hammer show / hammer checkmap Single volume, about 1 GB minimum; edits rebuild the volume HAMMER
HAMMER2 (DragonFly BSD) Hammer2 R/W ✅ moving DragonFly kernel mount r/w under QEMU HAMMER2
HTFS (SCO High Throughput File System) Htfs R/W ✅ moving own reader + struct-parity tests SCO HTFS HTFS
JFS Jfs R/W ✅ moving host-kernel jfs mount + fsck.jfs JFS2 (Linux / AIX) JFS project
JFS1 (OS/2 original IBM JFS) Jfs1 R/W ✅ moving own reader + struct-parity tests; the Linux jfs driver only knows JFS2 OS/2 JFS1 JFS project
Minix FS MinixFs R/W ✅ moving mkfs.minix round-trip MINIX v1 / v2 / v3 superblock families MINIX filesystem
Minix V1 FS MinixV1 R/W ✅ moving fsck.minix clean after in-place edit + host-kernel loop-mount reads the added file byte-exact 14-character names MINIX filesystem
Minix V2 FS MinixV2 R/W ✅ moving fsck.minix clean after in-place edit + host-kernel loop-mount reads the added file byte-exact 30-character names MINIX filesystem
NILFS v1 Nilfs1 R/W ✅ moving own reader + struct-parity tests NILFS v1 NILFS
NILFS2 Nilfs2 R/W ✅ moving host-kernel loop-mount reads the in-place-added file byte-exact; mkfs.nilfs2 superblock re-validated NILFS
OCFS2 (Oracle Cluster Filesystem 2) Ocfs2 R/W ✅ moving fsck.ocfs2 -fn + debugfs.ocfs2; mkfs.ocfs2 image read by the package Edits rebuild the volume OCFS2
QNX4 FS Qnx4 R/W ✅ moving own reader + struct-parity tests qnx4
QNX6 Neutrino FS Qnx6 R/W ✅ moving own reader + struct-parity tests qnx6
Reiser4 Reiser4 WORM ✅ moving own reader + struct-parity tests; reiser4progs not installable here Empty tree from mkfs.reiser4 captures; files live in a payload area the reiser4 driver does not see, so the writer stays WORM Reiser4
ReiserFS ReiserFs R/W ✅ moving reiserfsck, also after mutation ReiserFS 3.6; edits rebuild the tree ReiserFS documentation
TUX2 Tux2 R/W ✅ moving own reader + struct-parity tests Phase-tree filesystem Tux2
TUX3 Tux3 R own reader + struct-parity tests Tux3
UFS Ufs R/W ✅ moving FreeBSD kernel mount r/w + fsck_ffs under QEMU UFS1 / FFS UFS
UNIX System V FS SysV R/W ✅ moving host-kernel sysv mount reads byte-exact System V filesystem sysv
VxFS (Veritas) VxFs R/W ✅ moving Linux freevxfs driver mounts the written volume One fileset, direct extents, flat root; edits rebuild the volume freevxfs
Xenix FS Xenix R/W ✅ moving host-kernel sysv (detect_xenix) mount reads byte-exact sysv
XFS Xfs R/W ✅ moving host-kernel xfs mount r/w + xfs_repair -n XFS v5; edits rebuild the volume XFS documentation
ZFS Zfs R/W ✅ moving zdb -l label path; own reader + struct-parity tests OpenZFS v28 single-vdev profile; edits rebuild the pool OpenZFS documentation

Apple

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
APFS Apfs R/W ✅ moving own reader + struct-parity tests Single container / volume, unencrypted; edits rebuild the tree Apple File System Reference
Apple DOS 3.3 AppleDos R/W ✅ moving own reader + struct-parity tests DOS 3.3, catalog at T17 S15 Apple DOS
Apple IIgs GS/OS (2IMG) GsOs R/W ✅ rebuild own reader + struct-parity tests 2IMG wrapper delegating to the inner ProDOS / HFS / DOS 3.3 reader 2IMG
Apple UCSD Pascal ApplePascal R/W ✅ moving own reader + struct-parity tests UCSD Pascal volume UCSD p-System
HFS (Classic) Hfs R/W ✅ moving hmount / hls (hfsutils) list the written files Classic HFS; edits rebuild the catalog Inside Macintosh: Files
HFS+ HfsPlus R/W ✅ moving fsck.hfsplus -fn + hfsutils + reverse mkfs.hfsplus Edits rebuild the catalog TN1150
MFS (Macintosh File System) Mfs R/W ✅ moving own reader + struct-parity tests Macintosh File System (1984) MFS
ProDOS ProDos R/W ✅ moving own reader + struct-parity tests ProDOS technical reference

Optical

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
ISO 9660 Iso R/W ✅ moving host-kernel iso9660 mount + isoinfo / xorriso ISO 9660 + Joliet; edits rebuild the image ECMA-119
UDF Udf R/W ✅ moving host-kernel udf mount + udfinfo / mkudffs round-trip Edits rebuild the image OSTA UDF

Compressed, embedded and flash

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
CramFS CramFs R/W ✅ moving mkfs.cramfs image read byte-exact Edits rebuild the image cramfs
EROFS Erofs R/W ✅ moving mkfs.erofs image read byte-exact Compact inodes, FLAT_PLAIN; edits rebuild the image EROFS
JFFS2 Jffs2 R/W ✅ moving mkfs.jffs2 image read byte-exact Log-structured JFFS2
LittleFS LittleFs R/W ✅ moving own reader + struct-parity tests littlefs
ROMFS RomFs R/W ✅ moving host-kernel romfs mount + 7-Zip read the written image Edits rebuild the image romfs
SmartFS SmartFs R/W ✅ moving own reader + struct-parity tests NuttX; flat root; edits rebuild the volume NuttX SmartFS
SquashFS SquashFs R/W ✅ moving host-kernel squashfs mount + unsquashfs Edits rebuild the image SquashFS
UBIFS Ubifs R/W ✅ moving own reader + struct-parity tests Log-structured; edits rebuild the image UBIFS
YAFFS2 Yaffs2 R/W ✅ moving own reader + struct-parity tests YAFFS

Amiga, Atari, Acorn and other home computers

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
Acorn ADFS Adfs R/W ✅ moving own reader + struct-parity tests; the adfs module is absent from the QEMU guest Acorn ADFS ADFS
ADF Adf R/W ✅ moving Linux affs driver reads the written volume OFS / FFS ADF (Clévy)
Amiga Professional FS AmigaPfs R/W ✅ moving own reader + struct-parity tests PFS3
Amiga SFS Sfs R/W ✅ moving own reader + struct-parity tests; every block self-checksums Flat root; edits rebuild the volume AROS SFS
ATR (Atari 8-bit) Atari8 R/W ✅ moving own reader + struct-parity tests ATR, Atari DOS 2 VTOC Atari DOS
BBC DFS Bbc R/W ✅ moving own reader + struct-parity tests BBC DFS DFS
CP/M 2.2 (8" SSSD) Cpm R/W ✅ moving cpmtools cpmls / cpmcp read the written IBM-3740 image CP/M 2.2, 8-inch SSSD CP/M format notes
CPC DSK CpcDsk R/W ✅ rebuild own reader + struct-parity tests; cpmtools / libdsk expect a different EDSK geometry MV - CPCEMU DSK DSK format
Cromemco RDOS Cromemco R/W ✅ moving own reader + struct-parity tests RDOS Cromemco
D64 D64 R/W ✅ moving own reader + struct-parity tests VICE disk images
D71 D71 R/W ✅ moving own reader + struct-parity tests VICE disk images
D81 D81 R/W ✅ moving own reader + struct-parity tests VICE disk images
DragonFS DragonFs R/W ✅ moving own reader + struct-parity tests DragonDOS DragonDOS
G64 (Commodore GCR) G64 R/W ✅ rebuild own reader + struct-parity tests Raw GCR tracks; the writer GCR-encodes a 1541 image G64 (VICE)
GEMDOS (Atari ST) Gemdos R/W ✅ rebuild own reader + struct-parity tests Atari ST FAT variant GEMDOS
MFS-1 (Acorn Master File System v1) Mfs1 R/W ✅ moving own reader + struct-parity tests Acorn MFS-1; geometry-preserving catalog rebuild for edits BBC Master
MSA (Magic Shadow Archiver) Msa R/W ✅ rebuild own reader + struct-parity tests Magic Shadow Archive MSA (Atari ST)
NIB (Commodore nibble dump) Nib R/W ✅ rebuild own reader + struct-parity tests Raw nibble tracks NIB
PlayStation Memory Card Ps1MemoryCard R/W ✅ rebuild own reader + struct-parity tests Deletion marks the directory frame; add / replace re-pack the blocks PS1 memory card
SCL (ZX Spectrum) ZxScl R/W ✅ moving own reader + struct-parity tests SCL, SINCLAIR magic + checksum SCL
Sharp X68000 Human68k Human68k R/W ✅ moving own reader + struct-parity tests Sharp X68000 Human68k
TI-99/4A DSR Ti99 R/W ✅ moving own reader + struct-parity tests TI-99/4A disk format
TR-DOS TrDos R/W ✅ moving own reader + struct-parity tests ZX Spectrum TRD TR-DOS
TRSDOS / LDOS Trsdos R/W ✅ moving own reader + struct-parity tests TRSDOS
VDFS Vdfs R/W ✅ moving own reader + struct-parity tests Gothic-engine VDFS VDFS

Minicomputer and workstation

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
DEC RT-11 (RX01) Rt11 R/W ✅ moving own reader + struct-parity tests DEC RT-11 RT-11 documentation
HP LIF (Logical Interchange Format) Lif R/W ✅ moving own reader + struct-parity tests HP LIF, 256-byte sectors LIF
Microware OS-9 RBF Os9Rbf R/W ✅ moving own reader + struct-parity tests Microware OS-9 RBF OS-9
ODS-1 (VAX/VMS Files-11 L1) Ods1 R/W ✅ moving own reader + struct-parity tests Files-11 ODS-1 (RSX-11) Files-11
OpenVMS Files-11 OpenVms R/W ✅ moving own reader + struct-parity tests Files-11 ODS-2 / ODS-5 Files-11

Network, distributed and proprietary

Format Id State Compact Defrag Wipe Shrink Layout Purge Proof Notes Reference
BeeGFS BeeGfs N/A V3/V6 decoder + RAID0 stripe arithmetic against documented layout, over real ext/XFS backing images and their native user.fhgfs xattrs — no BeeGFS-produced target set serves as an oracle Read-only mounting of offline metadata/storage target sets through FilesystemStreamSet; V3 dentries + V6 inline/separate regular-file inodes; non-mirrored non-sparse RAID0. Nothing is created or modified: BeeGFS-level mutation would be a transaction across every target plus management state BeeGFS
CephFS / RADOS pool export CephFs R/W detection of the on-disk signature RADOS objects only Ceph
Dell EMC Isilon OneFS OneFs R Dell architecture/diagnostics; raw serialization not public Isilon OneFS OneFS
eCryptfs Ecryptfs R/W ecryptfs-utils passphrase vector + Linux kernel layout Passphrase lower-file AES-128/192/256 read/create; private-key auth and xattr-only metadata are unsupported eCryptfs
GlusterFS brick GlusterFs R detection of the on-disk signature Brick metadata only GlusterFS
IBM Storage Scale / GPFS Gpfs R detection of the on-disk signature IBM Spectrum Scale GPFS
JuiceFS JuiceFs R canonical juicefs dump JSON/protobuf framing + struct-parity tests Portable metadata backups only; JSON shrink strips insignificant whitespace; file payloads remain in object storage JuiceFS metadata backup
Lustre Lustre R own reader + struct-parity tests ldiskfs MDT/OST backing store; conservative offline free-block wipe + trailing-block shrink; namespace edits remain R/O Lustre
MooseFS MooseFs R detection of the on-disk signature Chunk-server objects only MooseFS
NetApp WAFL Wafl R detection of the on-disk signature NetApp WAFL WAFL
NSS (Novell Storage Services) Nss WORM ✅ moving own reader + struct-parity tests, against vectors built to the OES-derived on-disk notes Native read covers validated nested names and ordinary contiguous extents; the writer still emits a private container, not a mountable NSS pool NSS on-disk notes
NWFS (Novell NetWare 386 Traditional Filesystem) Nwfs R/W ✅ rebuild detection of the on-disk signature NetWare 286 / 3.x NWFS
NWFS386 (Novell Traditional NetWare filesystem) Nwfs386 R/W ✅ rebuild own reader + struct-parity tests Plain single-segment DOS namespace; rebuild edits preserve block size and volume name; compression/suballocation/alternate namespaces are read-only Traditional NetWare partitions
OrangeFS / PVFS2 DBPF OrangeFs R/W ✅ rebuild own reader + struct-parity tests One DBPF storage object, not a namespace; the opaque payload is what is edited OrangeFS
SGI CXFS (Cluster XFS) Cxfs R/W ✅ rebuild own reader + xfs_repair -n CXFS is XFS on disk; authoring targets the pre-CRC XFS v4 (crc=0) profile with dir2 short-form directories, and edits rebuild the volume CXFS
Tahoe-LAFS share / capability namespace TahoeLafs R/W ✅ rebuild own reader + struct-parity tests Opaque share payload; mutable storage-container gaps can be wiped, packed and shrunk without changing share or lease bytes Tahoe-LAFS
TFS (unverified historical BBN Trans-FS label) Tfs R none — explicit format selection only No normative on-disk specification, reference implementation or genuine sample was located; the historical TFS\x01 magic and .tfs extension are unsourced and no longer detect, so the image is exposed opaquely and only when the caller names the format; see the format notes BBN

🚀 Quick start

Walk a filesystem image

using FileSystem.Fat;

var image = File.ReadAllBytes("disk.img");
using var fs = new FatReader(image);
foreach (var entry in fs.ListRecursive())
  Console.WriteLine($"{entry.Path} {entry.Size,10} {entry.Modified:O}");

Open a virtual disk and inspect its inner filesystem

using FileFormat.Vhd;
using FileSystem.Ntfs;

using var stream = File.OpenRead("system.vhd");
var inner = new VhdReader().OpenContents(stream);
foreach (var partition in inner.Partitions) {
  if (partition.Type != "NTFS")
    continue;
  var ntfs = new NtfsReader(partition.Open());
  foreach (var path in ntfs.WalkPaths())
    Console.WriteLine(path);
}

Edit an image through the registry

using Compression.Registry;

Compression.Lib.FormatRegistration.EnsureInitialized();
var ops = (IArchiveModifiable)FormatRegistry.GetArchiveOps("Ext")!;
using var image = File.Open("root.ext4", FileMode.Open, FileAccess.ReadWrite);
ops.Add(image, [new ArchiveInputInfo("", "etc/motd", false, "hello\n"u8.ToArray())]);
ops.Remove(image, ["var/log/old.log"]);

📚 API reference

Every public and protected member of all 868 types, generated from the built assembly and its XML documentation, is in REFERENCE.md.

🧪 How the proofs run

The Proof column is produced by these suites, all in Compression.Tests. External tools that are absent make the corresponding test skip rather than fail, so the matrix records what has been shown, not what a particular runner had installed. A tool too old to read the format at all counts as absent for this purpose and skips with the version it reported — it never examined the image, so it has not judged it. A tool that does read the image and then objects has judged it, and that is a failure, never a skip.

Suite What it shows
Support/ThirdPartyFsCheck Mounts the written image with the host kernel's own driver, or extracts it with 7-Zip, and compares every payload. A driver that is present and refuses the volume fails the test; only a driver that is absent lets it skip.
Support/QemuRunner (BSD guests) A headless FreeBSD guest runs fsck_ffs over the written UFS volume; a DragonFly guest runs hammer over the HAMMER and HAMMER2 volumes. There is no Linux guest — Linux drivers are reached through the host kernel, not through QEMU.
KernelMount/InPlaceRwKernelMountTests On a Linux host whose kernel carries the module, the host loop-mounts the package's image directly and reads the in-place-added file (MinixV1, MinixV2, NILFS2). Skips without sudo, losetup or the module.
ExternalConformance*, ExternalFsInteropTests (Category("ExternalFsInterop")) e2fsck, xfs_repair, btrfs check, fsck.f2fs, fsck.jfs, reiserfsck, fsck.minix, fsck.hfsplus, hfsutils, ntfs-3g, mkudffs, unsquashfs, mtools, qemu-img accept what the package writes, and the package reads what they write.
DOS-era drivers under QEMU / dmsdos MS-DOS 6.22 DRVSPACE mounts the DoubleSpace / DriveSpace CVFs; the dmsdos driver mounts the DriveSpace 3 and Stacker volumes.
Struct-parity unit tests For formats with no reachable external tool, each on-disk structure the writer emits is compared field by field against the specification and read back by the package's own reader.
Disk-image containers follow the same rule with qemu-img: forward check, raw round-trip via convert -O raw, and reverse — an image qemu-img create made is opened by the package reader. A forensic-style path builds an inner filesystem with known files, wraps it in a container, validates the container externally, then walks it back and compares bytes.

🧯 Filesystem-aware recovery

FilesystemCarver in Compression.Analysis scans raw images for known superblock signatures at canonical offsets, asks the matching reader to validate each candidate, and can then extract readable entries — useful when a partition table is lost but an inner superblock survives.

using var fs = File.OpenRead("sdcard.img");
var hits = FilesystemCarver.CarveStream(fs);
foreach (var c in hits) {
  var result = FilesystemExtractor.ExtractCarved(fs, c, $"out/{c.FormatId}_0x{c.ByteOffset:X}");
  Console.WriteLine($"{c.FormatId}: {result.FilesExtracted} files, {result.FilesFailed} failed");
}
cwb recover sdcard.img
cwb recover raw.img --mode filesystems --out out/
cwb recover raw.img --mode files --format Jpeg,Png

🔌 Dependencies

Dependency Role
Hawkynt.Compression.Core Compression, checksums, bit I/O, partition helpers and the shared registry primitives
Host filesystem drivers / libguestfs Not required at runtime.
External fsck / mkfs / qemu tools Optional, used by the test suite only

⚠️ Limitations

  • This is an image-manipulation toolkit, not a kernel filesystem driver. R/W does not imply concurrent mount semantics, crash-consistent journaling under arbitrary interruption, or a drop-in replacement for the OS driver.
  • Where Notes says a format's edits rebuild the volume, an add or remove costs the whole image rather than the bytes that changed, and the block placement of untouched files may move.
  • bcachefs mutation is profile-gated: the in-place writer owns single-device regular pointer extents as emitted by this package. Bucket generations are carried through a mutation: emptying a bucket advances its generation, and the alloc key, the bucket_gens slot, the extent pointer and its backpointer are written from one source and checked against each other before the commit is published. Foreign volumes with extra live b-trees, non-canonical bucket_gens or alloc_v4 keys, inline / reflink / compressed extent keys, or unsupported inode and dirent types are refused for mutation rather than rewritten speculatively; read support is broader. How the accounting, backpointer and LRU keys were established is written up in BCACHEFS-ACCOUNTING.md.
  • ReFS edits are offline-quiescent: the image must not be mounted, and RefsMutationMode.NativeCow stays fail-closed. The reader walks ReFS 3.x containers, MSB+ trees, allocators, Block Refcount rows and MLog framing; the offline editor relocates data and metadata pages, replaces and removes regular files and empty directories through CoW B+ replacement and alternate-checkpoint publication. Block Refcount row creation and whole-file clone semantics are implemented and tested: RefsOfflineBlockCloner.CloneWholeFile repoints an existing destination stream at an existing, equal-sized, cluster-aligned source stream, materialises the shared-cluster reference counts — including the first clone of a slot the sparse table does not yet track — and publishes Object Table root #0 and Block Refcount root #6 through one alternate checkpoint, reclaiming the destination's former allocation only once that checkpoint is durable. Writing to a clone afterwards reallocates rather than overwriting the shared clusters. No public verb reaches this primitive: it is a library entry point, not something cwb can be asked to do, and the profile is deliberately narrower than FSCTL_DUPLICATE_EXTENTS_TO_FILE — resident, sparse, integrity, zero-length, already-shared and partial-range clones are all refused. Still open before a mounted driver could use the same core: redo-payload codecs and replay for every opcode, every Schema Table key rule, the allocation-zone policies of the three allocators, container create / delete / move, partial-range and EOF-extending clones, hard links, sparse and integrity-stream mutation beyond 4 KiB clusters, snapshots, ADS, security descriptors, reparse points, USN, rename, truncate, a format path, locking and a Windows chkdsk corpus.
  • Reiser4 remains WORM: the current writer captures the empty native tree but stores added files in a payload area the reiser4 driver does not see. It must not be promoted until the native tree writer owns those files.
  • GFS2 R/W is the standalone lock_nolock profile with regular files in the root directory; small files are stuffed and larger files use the indirect tree. Existing-image edits rebuild the volume while preserving its size floor and lock-table value. ExHash/nested-directory writing, journal replay, cluster-lock-manager state and extended attributes remain out of scope.
  • NSS native read support is intentionally narrow and quiescent: the reader reconstructs validated DirH/LEAF namespace relationships and ordinary contiguous file extents using the OES-derived clean-room profile in NSS-ON-DISK.md, while retaining anchor diagnostics for unsupported images. Native allocation trees, journals, trustee metadata, ZLBH history, compressed/sparse/fragmented data profiles and writable NSS pools remain out of scope; the existing writer still emits a private Workbench container rather than a pool NetWare/OES would mount.
  • Network and distributed formats generally carry no self-contained image. eCryptfs is the encrypted exception handled here: a lower file is structurally self-contained, and the passphrase profile is supported for AES-128/192/256; private-key authentication and xattr-only metadata remain unsupported.
  • Disk-image container support and inner-filesystem support are separate capabilities; a container marked R/W edits the filesystem inside it through that filesystem's own descriptor.

❤️ Support

If this project saves you time or money, consider supporting its development:

GitHub Sponsors PayPal

📜 License

Licensed under LGPL-3.0-or-later — see the repository LICENSE.