Feat/transcode ac3 dts - #43
Merged
Merged
Conversation
VuIO has no audio decoder for AC-3, E-AC-3 or DTS, so a TV without those licences plays the picture and no sound. Symphonia identifies those tracks (CODEC_ID_AC3 / CODEC_ID_EAC3 / CODEC_ID_DCA) and demuxes them fine — it just has no decoder. These four crates are that missing piece. Vendored rather than depended on. oxideav-dts is published to crates.io only as a yanked 0.0.1, so no registry dependency can ship DTS at all; the rest of the family moves fast enough that a floating version would change what a release decodes without anyone choosing it. The copies are verbatim — same crate names, same layout, no patches — so a diff against upstream stays meaningful and a refresh is a re-run of the script rather than a merge. Only three mechanical manifest changes: sibling deps become path deps, publishing is off (we do not own these names), and lints are allowed, because normalising 6 MB of foreign code to our clippy settings would destroy the diffability that is the point of vendoring verbatim. Upstream tests/ and benches/ are dropped, except the two fixtures the inline src/ tests include_bytes! — 17 KB total, and worth it: `cargo test -p oxideav-ac3` still checks the decoder we ship against a real bitstream. All 2330 upstream tests pass against the vendored copies. Costs no new external crates: thiserror, serde_json and bytemuck were already in the tree. Workspace members so the path deps resolve, but not default members, so their test and bench targets stay off the release path. Verified: cargo test -p oxideav-core -p oxideav-ac3 -p oxideav-dts -p oxideav-aac --lib → 869 + 459 + 280 + 722 passed, 0 failed.
The decode half of the feature: identify the three codecs, index an elementary stream's frames, and drive the vendored decoders to interleaved S16. `TranscodeCodec` is static identification and compiles in every build, so a server without a decoder still knows what an AC-3 track is and can say so honestly. Only the decode path is behind `transcode-ac3` / `transcode-dts` — the same split `mediainfo` draws between knowing what a provider is and being able to call one. `is_decodable()` is what lets the DIDL writer ask "can this build handle this?" without carrying a pile of #[cfg] at the call site. The index reads headers only and streams the file through a fixed buffer: each frame declares its own length, so the walk hops frame to frame without decoding, and the AC-3 track of a two-hour film is around 170 MB. It exists to make the transcoded resource *seekable* — decoded PCM is constant-bitrate, so an exact sample count is an exact Content-Length and a byte offset divides back into a frame. Leading tags and damaged frames resync rather than failing the file. AC-3 and E-AC-3 share a syncword and are told apart by bsid, which both syntaxes place at bit 40; base AC-3 reaches it through crc1+fscod+frmsizecod and Annex E through strmtyp+substreamid+frmsiz+fscod+numblkscod+acmod+lfeon. One probe serves both, so a stream may even switch mid-file. DTS frames carry raw NBLKS and run (NBLKS + 1) x 32 samples. Channel count is measured from a decoded frame rather than predicted from acmod: what matters downstream is what the decoder actually emits after its downmix. That downmix is the decoder's own, because AC-3 carries the §7.8 coefficients in the bitstream — summing channels ourselves would discard the mix the encoder intended and clip besides. A frame that fails to decode yields silence of the length the index promised. The Content-Length is already committed by then, so a decode error has to cost a tick rather than a truncated download. Verified: 16 tests, including both fixtures decoding to exactly the byte count their index predicted and to non-silent audio. Builds clean with no features, with `transcode` alone, and with each decoder alone.
Adds GET/HEAD /media/{id}/transcode/audio.wav, the [transcode] config section,
and the elementary extensions (.ac3/.eac3/.ec3/.dts) that were not in the
supported-media table at all — VuIO could neither play nor even list those
files before, which is why they were left out.
A separate handler from serve_media rather than a mode of it, because almost
every assumption differs: there is no file to open, the length comes from a
plan rather than from stat, the DLNA headers must declare a conversion
(DLNA.ORG_CI=1, hardcoded to 0 on the passthrough path), and the work is
CPU-bound and therefore rationed.
What it keeps is the contract a renderer cannot recover from if it is broken:
an exact Content-Length, Accept-Ranges, and real 206s. Decoded PCM is
constant-bitrate, so a byte offset divides cleanly back into a sample and a
seek is a genuine seek rather than a restart. That is what the frame index
buys, and why AudioPlan resolves total samples, sample rate and channel count
before a byte is sent — a renderer asks for the size and the bytes on separate
connections and will not tolerate the two disagreeing.
Seeking decodes one frame of pre-roll and discards it. AC-3 and DTS frames
overlap by half a window, so the sample at a seek point is reconstructed from
state the previous frame carried; without the pre-roll every seek ticks.
Plans are cached per (id, size, mtime): a renderer issues HEAD, then GET, then
ranges as someone scrubs, and building a plan re-reads the track's headers.
Keying on size and mtime means a file replaced in place is re-indexed rather
than served offsets it no longer has.
max_concurrent refuses rather than queues. A renderer told to wait looks like a
file that will not open, and the streams already playing would lose CPU to it.
A HEAD releases its slot immediately so a renderer that probes before playing
cannot starve one that is playing. Decoding runs on a blocking thread behind a
bounded channel, so a slow reader slows the decoder instead of pulling a film
into memory.
Verified: 8 integration tests against the real router, including a range being
byte-identical to that slice of the full decode, HEAD and GET agreeing on
length, and the fixture decoding to non-silent audio. 498 unit tests pass.
An AC-3/E-AC-3/DTS item now carries two resources in its DIDL: the file as
stored, and the decoded alternative at /media/{id}/transcode/audio.wav. Both
are offered and the renderer picks.
That is the design, not a compromise. There is no reliable table of which
television model licensed which codec — brands dropped DTS at different times,
and the same model differs by region — so any guess we made would be wrong for
someone, and wrong in the direction of silence. Offering both and letting
protocolInfo matching decide is what the standard is for. The decoded resource
carries a different MIME precisely so that matching can tell them apart, and
DLNA.ORG_CI=1 to say the bytes were produced rather than stored; every other
<res> this server writes says CI=0.
`prefer` handles the renderers that take the first resource without checking.
It defaults to "original", which cannot make anything worse: a set that can
already play AC-3 sees exactly the DIDL it saw before. Someone whose TV still
plays silently flips it.
The alternative is only advertised when this build can actually decode the
codec — is_decodable() is a compile-time constant asked at runtime — because
advertising a resource we cannot produce turns a silent film into a broken one.
Both DIDL writers emit it: the indexed path and the older fallback. They were
already near-duplicates, and an item must not lose its alternative depending on
which one happened to serve it. The browse cache is keyed by client profile
already, so per-profile output cannot cross-contaminate.
Verified: 12 integration tests, including two <res> elements for an AC-3 item
browsed as a Samsung set, the ordering flipping with `prefer`, and exactly one
<res> with the feature off.
`audio_format = "aac"` now serves /media/{id}/transcode/audio.aac, so the
config key offers two real choices rather than one working value and one 404.
The trade is explicit in what the resource claims. AAC output size is not known
until it has been produced, so there is no honest Content-Length, and the
response therefore carries no Accept-Ranges and DLNA.ORG_OP=00: no seeking,
rather than advertising ranges it would then refuse. LPCM stays the default
precisely because it keeps those.
ADTS rather than a container — every frame carries its own header, so the
stream is self-describing from any point and needs no muxer, no seek table and
no rewrite at the end. The decoder already emits interleaved S16, which is what
the encoder takes, so this is a pipe rather than a conversion.
Bitrate is left at the encoder's 64 kbps per channel, the conventional AAC-LC
operating point and about a tenth of the same audio as LPCM. Deliberately not a
config key: it is a knob whose wrong setting is audible and which nothing here
benefits from tuning.
Also moves the enabled check to the live config rather than the startup
snapshot, so the admin UI's Live impact for [transcode] is honest.
Verified: 14 integration tests including ADTS framing on the wire, the absent
length and range headers, and the DIDL advertising whichever resource the
config selected.
Adds [transcode] to docs/configuration.md, the decoder rows to the vuio-core feature table, the two new routes to CLAUDE.md, and a CLAUDE.md rule for crates/vendor mirroring the existing one for the vuio-web bundle. The feature table's numbers were stale — 217/148 against a tree that now resolves 248/142 — so every row is re-measured rather than leaving fresh numbers beside old ones. Two rows were missing entirely (mediainfo, web-ui). The zeroes now carry their explanation: dashboard and mcp shed compiled code rather than dependencies, and metadata shares symphonia with casting so dropping it alone sheds nothing. The decoders cost 4 crates, not a subtree: thiserror, serde_json and bytemuck were already in the tree, so what they add is about 6 MB of compiled decoder. CI's feature-matrix loop gains the four new features and the mediainfo it had been missing since that feature landed. It also gains a leg that builds and tests with the decoders dropped — these ship on by default, which makes their absence the path that rots unnoticed, so proving a feature builds is no longer the only thing that job needs to prove. The Docker image ships with the decoders, since someone whose television plays a film silently will not rebuild an image to fix it. VUIO_CARGO_FLAGS is there for an operator who knows their renderers and wants them out.
Two bugs a live server found that the unit tests could not. The scanner never saw these files. Adding the MIME types was not enough: the extension allowlist is a separate list with four independent copies — the platform default, the Docker env default, the generated config template and the documented example. Every integration test injects MediaFile rows straight into the database, so all of them passed against a scanner that indexed nothing. Seeking decoded the primed frame twice. Opening a decoder necessarily decodes a frame, and the loop then fed that same frame in again, pushing its samples through the IMDCT overlap buffer a second time — so every frame after a seek landed somewhere a sequential decode never goes. It cancels out at frame 1, where the pre-roll is frame 0 and a whole-file decode primes on frame 0 too, which is exactly where the existing range test seeked. `prime` now hands back the frame it already decoded instead of it being decoded again. The range tests now assert the real contract rather than a stronger one that happened to hold: a ranged response is the same audio as that slice of the whole to within one LSB, not byte-for-byte. A seek reaches the decoder's floating-point state by a different route, so a sample on a rounding boundary can quantise to the adjacent integer — 1/32768, about -90 dBFS, and the same tolerance the decoder's own conformance suite uses. The comparison respects sample phase, because a range may begin mid-sample and pairing bytes from the wrong phase reports a 1-LSB difference as 256. The new test seeks into frames 2, 5 and 8, where the pre-roll is real. Verified against a running server: 206s with exact lengths and <= 1 LSB for a mid-file range, a header-straddling range and a range to the last byte; 416 past the end; both <res> elements present in a Samsung browse; the AAC variant at 3244 bytes against 98348 of LPCM.
VuIO re-encodes DTS soundtracks to AC-3 5.1 @ 640 kbps for the video.ts route, on a single blocking thread that also carries demux, decode and TS packetisation, and a television seeking through a film pays that cost twenty times over in ranged probes. Measured against ffmpeg on an M1 the encoder ran at 35.6x realtime where ffmpeg manages 188x — 5.3x slower. This takes it to 109x. Three changes, in descending order of what the profiler actually blamed rather than what looked expensive: crc: 25% of encode time. The LFSR ran one bit at a time, and crc1 — which sits at the front of the region it protects — was solved by Gaussian elimination over GF(2), costing seventeen passes over the 5/8 region plus sixteen heap allocations per frame. Now a compile-time byte table, and crc1 by the closed form X = R·(x^-1)^(8·len-16), one multiply against a factor computed once per encoder. Bit-exact: the golden digests are unchanged. mdct: the forward transform was a dense 256x256 matrix-vector product, 65,536 multiply-accumulates against 256 KiB of cosine table streamed through cache per block per channel, 36 times a frame — despite the file calling itself "fast SIMD-accelerated". Now a half-length complex FFT with tabled twiddles, checked against the direct-form oracle. bit allocation: the rate-control tuner evaluates ~250 candidate snroffst values per frame and re-ran the entire psychoacoustic chain for each, though snroffst only enters at the final table lookup. Split into a MaskCurve computed once per channel-block and a cheap lookup, with a test asserting the two agree across every snroffst the search can reach. The crate moves out of crates/vendor: it is no longer a verbatim copy of github.com/OxideAV/oxideav-ac3, so keeping it there would break the diffability that vendoring is for. LICENSE and attribution are retained.
…env switches The rate-control search only ever wanted a bit count, but it materialised 10.5 KiB of bap arrays per candidate and walked them back to add up the bits. A block's mantissa cost turns out to depend only on how many bap values of each grouped class it holds — a group opens on every third (or second) value of its class regardless of what sits between them — so the count can be accumulated as the baps are derived and the arrays never built. A test checks that against the sequential accounting it replaces. Separately, the AC3_* debug switches were read with std::env::var where they were used, which put a global lock, a scan of environ and sometimes a String allocation inside per-block and per-candidate loops: ~190 lookups a frame. They are process-lifetime constants; they now load once. 5.1 @ 640 kbps: 109x -> 117x realtime. Both changes are bit-exact.
Everything a syncframe needs after transient detection depends only on that frame: the §8.2.2 detector carries biquad state across frames and the delay line carries 256 samples of left context, but the transform, coupling, exponents, rate control and packing do not. So emit_syncframe splits in two — analyse_frame, which runs in order and is a pair of biquads per block, and encode_analysed, which takes &self and is where all the work is — and a batch of eight frames fans out across rayon. Eight frames is 256 ms of buffering, well inside the 0.4 s the TS muxer already batches before it emits anything, so it costs a seek nothing it was not paying. flush() now pads to a frame boundary rather than assuming one short frame is all that remains. 5.1 @ 640 kbps on an M1: 117x -> 267x realtime, against ffmpeg's 189x. Output is bit-identical to the single-threaded encoder — packets are collected in order and no frame reads another's state. Also clears the clippy errors this crate carried from upstream: the simd.rs dispatchers put their scalar fallback after an early return, so on aarch64 it was unreachable rather than cfg'd out. The gate now passes where it did not before.
Transient detection is the one part of encoding that has to run in frame order — the §8.2.2 biquads carry state across frames — so after the rayon split it sits on the serial path and caps what the parallel half can reach. It was reaching through self.biquad_state[stage] inside the sample loop, paying a bounds check and four stores per sample per stage for a filter whose arithmetic is nine operations. Both stages' history now lives in locals and is written back once. The expression order is unchanged, so the filter output is bit-identical. The peak hierarchy was scanning the same 256 samples three times over. Max is associative, so the four level-3 segment peaks are all the scanning needed and the level-2 and level-1 peaks fall out of those; the silence branch was recomputing two it already had. Also replaces 2f32.powi(exp) on the per-coefficient quantiser path with a 25-entry table. Exponents are 0..=24 by construction and powers of two are exact in binary floating point, so this is the same values looked up rather than an approximation — but powi with a runtime exponent lowers to a loop of squarings, several thousand times a frame. 5.1 @ 640 kbps: 267x -> 275x realtime (121x on one core). Bit-exact.
The soundtrack format is now config-driven and defaults to AC-3, so the surfaces that still described LPCM had to catch up, and one of them was wrong rather than merely stale. An audio item whose own codec is AC-3 or E-AC-3 was being offered an AC-3 alternative. The decoded resource exists for a renderer that may hold no Dolby licence, so handing it the same codec re-encoded offers it nothing it had not already refused. Such an item now falls back to LPCM, which is what this path produced before and what every renderer decodes; `aac` and `lpcm` cannot collide this way and are unaffected. `item_audio_is_dolby` answers the question next to `item_needs_transcode`, keeping the XML writers clear of feature-gated codec knowledge. The transport-stream length promise needed no change — it is derived from the output codecs' own rates, so it already followed a soundtrack up to AC-3 5.1 as it had followed one down to stereo AAC. The tests asserting otherwise were written when stereo was the only outcome, and now check both formats: five DTS soundtracks reach the renderer at half the rate they left at rather than an eighth, and the promise says so. Also: `audio_format = "ac3"` in both config templates and the admin enum, the docs' account of what the setting decides for an item versus a film, and a checked-division lint in the AC-3 range handler. Verified end to end: a DTS 5.1 film reaches video.ts as AC-3 5.1 at 640 kbps with no environment variable set, a 2.0 source stays 2.0 at 192, and a byte range into audio.ac3 returns the whole encode's own bytes from the frame after the seek — the one frame the MDCT overlap cannot reproduce is the frame the viewer dragged to.
Both regressed in f8b616c, and neither showed up until now: CI last ran on this branch before the codec fork landed, so the first run after it went red on code nobody had touched. CLAUDE.md says vuio-codec-ac3 is lint-exempt — 13.7k inherited lines that are their own cleanup chore, deliberately kept off the performance work. The `[lints]` table it points at is the right place to say so but cannot be the only one: CI runs `cargo clippy -p vuio-core … -- -D clippy::all`, cargo hands those trailing flags to every workspace member unit it builds rather than only the named ones, and a command-line `-D` outranks the manifest. An attribute outranks both, so the exemption is now also stated at the crate root, where it actually holds. Without it every clippy release breaks the build here; 1.98 found eight. The MSRV came along with the same commit's publish preparation — version 0.0.45 to 0.0.1, a new licence, a new author list — which raised `rust-version` from 1.80 to 1.97 while every other crate in the workspace promises 1.95. Nothing in the codec needs it: it checks clean on 1.95, so the promise is restored rather than the other crates dragged up to meet it. Two failures on this run are neither: libxaac-sys' cmake passes /O2 and /RTC1 together on MSVC, and the aarch64 and armv7 Linux runners failed to fetch the toolchain action and the soft-float headers.
libxaac-sys arrived on this branch in place of oxideav-aac, and its bundled cmake build has never compiled on two of the targets CI covers. Both failures are in that build, not in anything calling it. On MSVC the library's own cmake asks for /O2 unconditionally. A Visual Studio generator is multi-config, so that reaches the Debug configuration too, and MSVC refuses /O2 beside the /RTC1 Debug adds — error D8016, which took out both Windows test jobs and Windows clippy. /O2 is now asked for on every configuration except Debug. Debug was also the wrong configuration to be building at all: it links the debug CRT while rustc links the release one whatever the cargo profile says, so a dev build was quietly mixing the two. A non-release profile now builds RelWithDebInfo, which keeps the symbols with neither problem and matches the opt-level 3 the workspace already asks of this package in dev. Single-config generators ignore --config, so nothing outside Windows changes. On armv7 the library builds 32-bit arm as -mfloat-abi=softfp. That leaves __ARM_PCS_VFP undefined, so glibc's gnu/stubs.h reaches for gnu/stubs-soft.h — which an arm-linux-gnueabihf sysroot does not ship, and the build stopped at the first #include. The ABI now follows the Rust target triple, defaulting to the previous softfp for a genuinely soft-float toolchain. The generator expression was checked against a multi-config generator rather than guessed at: Debug resolves to /W3 alone, every other configuration to /W3 /O2.
The last thing keeping armv7 red. libxaac's 61 hand-written armv7 kernels are written in unified (UAL) syntax — the condition code goes after the S flag and after an LDM/STM addressing mode, so SUBSNE and LDMFDLE rather than the divided syntax's SUBNES and LDMLEFD. GNU as still defaults to divided and rejects those outright. Upstream builds with clang, whose integrated assembler defaults to unified, which is why the files never said which dialect they are. Now they do. Six lines needed rewriting as well: `SUB r3, r1, LSL #3` is the pre-UAL shorthand for `SUB r3, r3, r1, LSL #3`, and unified syntax wants the destination named. The two forms assemble to the same e0433181. Verified against the assembler CI actually uses rather than reasoned about — arm-linux-gnueabihf-as 2.47, with the cortex-a8/neon/hard-float flags the library's cmake passes: - before, 5 of 61 files fail to assemble; after, 0 of 61 do - of the 56 that did assemble before, all 56 disassemble byte-identically afterwards, so the directive changed no instruction that was already being accepted, and the six rewritten SUBs encode as they did That check is also why this touches all 61 rather than the one file the build log named: grepping for the ambiguous forms found three files, and the assembler then found two more.
The last armv7 failure, one layer up from the assembly. bindings.rs is
committed bindgen output, and bindgen records the layout it measured: nine
blocks of size, alignment and offset assertions written as subtractions
against literals. Every struct they describe holds pointers, so on a 32-bit
target the struct is smaller than the literal, the subtraction underflows, and
it is a compile error rather than a failed assertion — six of them on armv7.
Only the assertions are 64-bit facts. The structs need no help: they are
#[repr(C)] over real pointer types with no explicit padding, so rustc lays
them out for whatever pointer width the target has. So the assertions are
gated on target_pointer_width = "64" and nothing else changes.
Checked against a real 32-bit target rather than reasoned about, by compiling
bindings.rs standalone — it is self-contained:
- i686-unknown-linux-gnu: 7 E0080 errors before, none after
- the 64-bit host still compiles it with the assertions active
- corrupting one of those assertions is still caught on 64-bit, so the gate
disabled the check only where it was never true
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.