Asset System

The Original Asset System

How the retail Darkspore client (Darkspore.exe) loads its assets, reverse-engineered from the binary. AssetData.Parser is a faithful C# mirror of this system, so understanding the original is the best way to understand the parser. The companion pages Binary Format and Parser Architecture cover the wire format and the C# implementation respectively.

Source: a Ghidra decompilation of the Asset* namespaces in Darkspore.exe — 337 functions across 10 namespaces. Of those, only 17 are infrastructure; the other 320 are auto-generated reflection code, two stubs per format.

The core idea

The client ships a single global asset runtime backed by:

• one generic recursive parser (AssetParser::DeserializeObject), and
• one global catalog (AssetCatalog, a singleton).

Every asset format — Noun, Level, Markerset, ability, and ~140 others — is a pair of auto-generated reflection stubs (AssetType::Foo + AssetData::Foo) that, at startup, calls AssetTypeRegistry::Register to describe its fields. No format gets its own loader. The parser dispatches on a small, fixed set of FNV-1a-hashed wire-shape sentinels.

The practical consequence — for both the client and this port — is that adding a format is data, not code: you describe its field table once and the generic parser handles the rest.

The namespaces

Namespace	Funcs	Role	Mirrored in C# by
AssetCatalog	47	Master registry / database singleton (g_GlobalManager).	(server-side concern; the parser returns trees)
AssetTypeRegistry	4	Linked list of every registered type; binary-search index.	TypeRegistry
AssetType	140	Per-format reflection stubs (one per format).	AssetCatalog subclasses under Catalog/
AssetData	137	Twin stubs that emit the field-descriptor table.	the Build() override of each stub
AssetLoader	4	Path build + file read + deserialize entry point.	AssetService / DbpfReader
AssetParser	1	DeserializeObject — the generic recursive deserializer.	AssetParser.Parse
AssetObject	1	CreateInstance — allocator + live-list insertion.	(not modeled — C# returns POCO trees)
AssetCache	1	FindCachedAsset — hash-keyed live-object lookup.	(not modeled — re-parses on demand)
AssetDestructor	1	DestroyObjectRecursive — symmetric tear-down.	n/a (GC)

Hashing

Everything is keyed by FNV-1a 32-bit, case-insensitive (the client lowercases input before hashing):

uint FNV1a_Hash(byte* data, int length, uint seed /* = 0x811c9dc5 */) {
    for (byte* p = data; p < data + length; ++p)
        seed = seed * 0x1000193 ^ *p;
    return seed;
}

Type names, field names, and asset names all run through this. Wire-shape sentinels are themselves just FNV-1a hashes of canonical type names ("array", "Nullable", "char*", …) — they collide with no real struct hash by construction. The full sentinel table is on Binary Format.

Boot: registering every type

flowchart TD
    A[C runtime startup<br/>static initializers] --> B["AssetData::Foo + AssetType::Foo<br/>per format"]
    B --> C[AssetTypeRegistry::Register<br/>name, hash, fields, count, size]
    C --> D[push onto g_AssetTypeRegistryHead]
    D -->|all types registered| E[InitializeAll]
    E --> F[Pass 1 — IndexType<br/>build field hash buckets, sum flattened size]
    F --> G[Pass 2 — BuildTypeMetadata<br/>recursive FNV structural fingerprint]
    G --> H[AssetCatalog::Create + Initialize]
    H --> I[LoadCatalogFile<br/>catalog_LANG.bin]
    I --> J[ProcessCatalogItem per entry]

Loading

Each registered type is a fixed-layout descriptor record carrying its name, type hash, an array of field descriptors, the instance size, and a recursive structural fingerprint (BuildTypeMetadata) — a hash that flips whenever any field name, type, offset, order, or nested layout changes. The client uses it as a schema-version / disk-cache-invalidation key. C# encodes the same field tables via the catalog DSL (see Adding a Format).

Loading a single asset

flowchart TD
    A[GetAsset name] --> B[FindCachedAsset<br/>FNV1a name -> bucket]
    B -->|hit| Z[return cached object]
    B -->|miss| C[CreateInstance]
    C --> D[strip extension, FNV1a remainder = nameHash]
    D --> E[FindTypeByHash extHash]
    E -->|none| X[unknown format]
    E -->|ok| J[LoadBinaryFile]
    J -->|"mode 0"| K[fopen + fread raw bytes]
    J -->|"mode 1"| L[stream from DBPF package]
    K & L --> M[LoadAndParseAsset]
    M --> O[DeserializeObject<br/>buffer, type, base]
    O --> Z2[status = loaded]

Loading

The client picks one of two byte sources based on a global flag:

• mode 0 — direct fopen/fread of a loose file on disk (e.g. _Generated/foo.bin).
• mode 1 — stream the bytes out of a DBPF package (AssetData_Binary.package).

AssetData.Parser covers both: DbpfReader handles the DBPF/DBBF wrapper (including RefPack/EA-QFS decompression), and AssetParser handles the per-entry payload. See Binary Format for the DBPF and catalog-file layouts.

Parsing: recursive descent

DeserializeObject walks the field descriptors of a type. For each field it calls FindTypeByHash(field.typeHash):

• registered struct -> recurse into it on the same byte stream;
• sentinel hash -> handle that wire shape inline (string in the blob, nullable indicator, array of N elements, …);
• value-type hash -> copy the fixed number of raw bytes.

flowchart TD
    A[DeserializeObject] --> B[for each field]
    B --> D[FindTypeByHash field.typeHash]
    D -->|registered struct| E[recurse]
    D -->|Asset| F[resolve name string -> live object]
    D -->|Nullable| G[indicator; if set, recurse child in blob]
    D -->|Key| H[indicator; if set, read key string]
    D -->|CharPtr / Char| J[read string from blob]
    D -->|LocalizedAssetString| I[two indicators -> up to two strings]
    D -->|Array| K[count x stride; dispatch per element type]
    E & F & G & H & I & J & K --> L{more fields?}
    L -->|yes| B
    L -->|no| M[done]

Loading

A key performance detail the port preserves: strings, blobs, and sub-structs are pointers into the original buffer, not copies. The client's destructor (DestroyObjectRecursive) relies on this — it only frees pointers that fall outside the asset buffer. C# sidesteps all of this with the GC, but the parse logic is otherwise 1:1.

Asset vs Key — a distinction that matters

Two sentinels both reference other assets, but with different timing:

• Asset — "resolve now": the parser immediately loads (or instantiates) the referenced object and stores a live pointer.
• Key — "store the hash, resolve later": the field holds a deferred reference string, resolved on demand.

This is why the client tolerates an O(n²)-ish cross-reference walk at warm-up, while a lazy consumer can treat everything as a Key and resolve on access.

Why mirror the client instead of hand-writing loaders?

An alternative server port (dalkon's C++ NounDatabase) hand-rolls eight per-format loaders. That name is a historical misnomer — Noun is just 1 of 143 formats; the database was named after the first one tackled, then seven more were bolted on. AssetData.Parser instead mirrors the client's design: one generic parser, formats as data. The client itself runs exactly one parser (DeserializeObject, ~183 lines). Replicating per-format loaders would invent complexity the original never carried.

See also

• Binary Format — the full sentinel/value-type tables, array dispatch, type-descriptor layout, DBPF and catalog-file formats.
• Parser Architecture — how this maps onto the C# classes.
• Catalog Reference — every format currently described, generated from the schema.