Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions API.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ This document is the practical API reference for integrating `heic_decoder` in a

- Decode HEIF/HEIC/AVIF to RGBA (`u8` or `u16` storage).
- Decode HEVC-coded HEIC/HEIF straight to RGB8 when alpha is not needed.
- Decode HEVC-coded HEIC within a memory budget to a size-capped, display-oriented sRGB RGB8 image (`decode_bounded`).
- Decode from bytes, `Read`, `BufRead`, and file paths.
- Optional `image` crate integration through hook registration.
- Keep default decode output aligned with primary item transforms (`clap`/`irot`/`imir`) and expose explicit EXIF orientation helpers.
Expand Down Expand Up @@ -417,6 +418,38 @@ fn decode(path: &Path) -> Result<(), heic_decoder::DecodeError> {
}
```

## Bounded decoding

`decode_bounded(BoundedInput, BoundedDecodeOptions)` decodes a HEIC still image
to display-oriented sRGB RGB8 within a memory budget. It returns
`Result<BoundedRgbImage, BoundedDecodeError>`. The output is reduced with area
averaging so that neither side exceeds `max_side` (1 through 6000, default
6000), and `original_dimensions` is the display size before reduction.
`max_memory_bytes` (at least 64 KiB, default 128 MiB) caps the decoder-owned
memory that depends on the input. `decoder-tracing` disables bounded decoding.

The primary item may be a grid of uniform HEVC tiles or a single HEVC image
(`hvc1`/`hev1`). Every coded item must be opaque 8-bit 4:2:0 HEVC with one IDR
slice; WPP is supported. A single image is decoded as a grid with one tile.
`clap`/`irot`/`imir` are applied, and Exif orientation applies when no
effective container orientation is present. RGB matrix/TRC ICC profiles and
SDR nclx colour with BT.709, BT.2020 or Display P3 primaries are converted to
sRGB. HDR transfer functions, grayscale or
lookup-table ICC profiles, unprofiled wide-gamut HEVC colour, alpha, and
unknown required auxiliaries return errors. Known depth, semantic matte, and
gain-map auxiliaries are ignored.

Before any pixel decoding, preflight reads every coded item's headers and
reserves its reconstruction workspace, and admission then reserves the output
together with the largest tile workspace. Inputs that do not fit return
`MemoryBudgetExceeded` or `LimitExceeded` without decoding pixels. Grid tiles
decode one at a time, or concurrently with `parallel-grid` when the budget
allows. A single image's full-resolution reconstruction must fit the budget at
once, so a large non-grid image needs a larger budget than a grid of the same
size. Grid tile payloads are limited to 16 MiB; a single image's payload is
bounded by the budget and the 512 MiB input limit. Admission is conservative,
and allocator overhead and process RSS are not included.

## Experimental incremental bounded decoding

Enable the `incremental-experiment` Cargo feature to use
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Pure Rust HEIF/HEIC/AVIF decoder with first-class `image` crate integration.
- `path_extension_is_heif`
- `DecodedRgbaImage::apply_exif_orientation`
- `image_integration::apply_exif_orientation_dynamic`
- Memory-budgeted HEIC decoding with `decode_bounded`: grid and single-image HEVC primaries decode to display-oriented sRGB RGB8 with a capped maximum side, and inputs that would exceed the budget are refused before pixel decoding.
- Optional guardrails for bounded production use:
- max input bytes
- max decoded pixel count
Expand Down
16 changes: 9 additions & 7 deletions src/bounded.rs
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,8 @@ pub fn decode_bounded(
let (pool, _pool_reservation) = {
let threads = std::thread::available_parallelism()
.map_or(1, usize::from)
.min(8);
.min(8)
.min(grid.tiles.len());
let mut workers = threads;
while workers > 1
&& max_job
Expand Down Expand Up @@ -285,13 +286,14 @@ pub fn decode_bounded(
}

fn job_bytes(tile: &grid::Tile<'_>, region: resample::Region, layout: resample::Layout) -> usize {
let contributions = if layout.is_unscaled() {
0
} else {
resample::Contribution::workspace_bytes(region)
};
tile.workspace_bytes
+ tile.payload_len
+ if layout.is_unscaled() {
0
} else {
resample::Contribution::workspace_bytes(region)
}
.saturating_add(tile.payload_len)
.saturating_add(contributions)
}

fn render_tile(
Expand Down
17 changes: 4 additions & 13 deletions src/bounded/container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -273,15 +273,6 @@ fn unique<'a>(slot: &mut Option<&'a [u8]>, data: &'a [u8]) -> Result<()> {

impl<'a> Index<'a> {
pub(super) fn parse(metadata: &'a [u8], source_len: u64, budget: &Budget) -> Result<Self> {
Self::parse_primary(metadata, source_len, budget, false)
}

pub(super) fn parse_primary(
metadata: &'a [u8],
source_len: u64,
budget: &Budget,
direct: bool,
) -> Result<Self> {
let mut reader = Reader::new(metadata);

if reader.full_box()? != (0, 0) {
Expand Down Expand Up @@ -361,10 +352,10 @@ impl<'a> Index<'a> {
let primary = items
.binary_search_by_key(&primary_id, |item| item.id)
.map_err(|_| Error::Malformed("primary item missing"))?;
if items[primary].kind != *b"grid"
&& !(direct && matches!(&items[primary].kind, b"hvc1" | b"hev1"))
{
return Err(Error::Unsupported("primary item must be a HEIC grid"));
if !matches!(&items[primary].kind, b"grid" | b"hvc1" | b"hev1") {
return Err(Error::Unsupported(
"primary item must be a HEIC grid or HEVC image",
));
}
let idat = idat.unwrap_or_default();
Self::locations(
Expand Down
142 changes: 86 additions & 56 deletions src/bounded/grid.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,6 @@ pub(super) struct Properties<'a> {

impl<'a> Properties<'a> {
pub(super) fn read(index: &Index<'a>, item: usize, tile: bool) -> Result<Self> {
Self::read_with_compatibility(index, item, tile, false)
}

#[cfg(feature = "incremental-experiment")]
pub(super) fn read_incremental(index: &Index<'a>, item: usize, tile: bool) -> Result<Self> {
Self::read_with_compatibility(index, item, tile, true)
}

fn read_with_compatibility(
index: &Index<'a>,
item: usize,
tile: bool,
legacy_pixi: bool,
) -> Result<Self> {
let mut result = Self {
config: None,
dimensions: None,
Expand Down Expand Up @@ -92,10 +78,8 @@ impl<'a> Properties<'a> {
return Err(Error::Unsupported("pixi version or flags"));
}
let count = r.uint(1)? as usize;
if (count != 3 && !(legacy_pixi && count == 1))
|| r.take(count)?.iter().any(|&n| n != 8)
{
return Err(Error::Unsupported("8-bit RGB channels required"));
if !matches!(count, 1 | 3) || r.take(count)?.iter().any(|&n| n != 8) {
return Err(Error::Unsupported("8-bit channels required"));
}
r.finish()?;
}
Expand Down Expand Up @@ -195,12 +179,87 @@ pub(super) struct Grid<'a> {
pub(super) exif: Option<u8>,
}

impl<'a> Tile<'a> {
fn preflight(
index: &Index<'a>,
source: &mut Source<'_>,
budget: &Budget,
item: usize,
properties: &Properties<'a>,
max_payload: usize,
) -> Result<Self> {
let config = properties.config.ok_or(Error::Malformed("missing hvcC"))?;
let payload_len = index.items[item]
.location
.ok_or(Error::Malformed("missing coded item location"))?
.length;
if payload_len == 0 || payload_len > max_payload {
return Err(Error::LimitExceeded("coded item payload bytes"));
}
let mut payload = budget.zeroed(payload_len, "preflight payload")?;
index.read_item(item, source, &mut payload)?;
let (geometry, reconstruction_bytes) = {
let _headers = budget.reserve(
payload_len.saturating_mul(5).saturating_add(1024 * 1024),
"HEVC header parsing",
)?;
let prepared = codec::prepare(config, &payload)?;
(prepared.geometry, prepared.reconstruction_bytes())
};
if properties.dimensions != Some((geometry.width, geometry.height)) {
return Err(Error::Malformed("ispe differs from SPS"));
}
let workspace_bytes = (reconstruction_bytes + payload_len as u64 * 3 + 1024 * 1024)
.try_into()
.map_err(|_| Error::LimitExceeded("codec workspace"))?;
let _admission = budget.reserve(workspace_bytes, "tile reconstruction")?;
Ok(Self {
item,
config,
geometry,
color: properties.color.clone(),
payload_len,
workspace_bytes,
})
}
}

impl<'a> Grid<'a> {
pub(super) fn preflight(
index: &Index<'a>,
source: &mut Source<'_>,
budget: &Budget,
) -> Result<Self> {
if index.items[index.primary].kind == *b"grid" {
Self::tiled(index, source, budget)
} else {
Self::single(index, source, budget)
}
}

fn single(index: &Index<'a>, source: &mut Source<'_>, budget: &Budget) -> Result<Self> {
let properties = Properties::read(index, index.primary, false)?;
let mut tiles = budget.buffer::<Tile<'a>>(1, "tile index")?;
tiles.push(Tile::preflight(
index,
source,
budget,
index.primary,
&properties,
usize::MAX,
)?)?;
let exif = validate_references(index, source, &tiles, budget)?;
Ok(Self {
width: tiles[0].geometry.width,
height: tiles[0].geometry.height,
columns: 1,
tiles,
properties,
exif,
})
}

fn tiled(index: &Index<'a>, source: &mut Source<'_>, budget: &Budget) -> Result<Self> {
let primary = index.primary;
let primary_id = index.items[primary].id;
let location = index.items[primary]
Expand Down Expand Up @@ -255,56 +314,27 @@ impl<'a> Grid<'a> {
return Err(Error::Unsupported("grid requires direct HEVC tiles"));
}
let props = Properties::read(index, item, true)?;
let config = props.config.ok_or(Error::Malformed("missing tile hvcC"))?;
let payload_len = index.items[item]
.location
.ok_or(Error::Malformed("missing tile location"))?
.length;
if payload_len == 0 || payload_len > MAX_ITEM_BYTES {
return Err(Error::LimitExceeded("tile payload bytes"));
}
let mut payload = budget.zeroed(payload_len, "preflight payload")?;
index.read_item(item, source, &mut payload)?;
let (geometry, reconstruction_bytes) = {
let _headers =
budget.reserve(payload_len * 5 + 1024 * 1024, "HEVC header parsing")?;
let prepared = codec::prepare(config, &payload)?;
(prepared.geometry, prepared.reconstruction_bytes())
};
if props.dimensions != Some((geometry.width, geometry.height)) {
return Err(Error::Malformed("tile ispe differs from SPS"));
}
let tile = Tile::preflight(index, source, budget, item, &props, MAX_ITEM_BYTES)?;
if tiles.first().is_some_and(|first| {
(
geometry.width,
geometry.height,
geometry.full_range,
geometry.matrix,
geometry.primaries,
geometry.transfer,
tile.geometry.width,
tile.geometry.height,
tile.geometry.full_range,
tile.geometry.matrix,
tile.geometry.primaries,
tile.geometry.transfer,
) != (
first.geometry.width,
first.geometry.height,
first.geometry.full_range,
first.geometry.matrix,
first.geometry.primaries,
first.geometry.transfer,
) || props.color != first.color
) || tile.color != first.color
}) {
return Err(Error::Unsupported("nonuniform grid tiles"));
}
let workspace_bytes = (reconstruction_bytes + payload_len as u64 * 3 + 1024 * 1024)
.try_into()
.map_err(|_| Error::LimitExceeded("codec workspace"))?;
let _admission = budget.reserve(workspace_bytes, "tile reconstruction")?;
tiles.push(Tile {
item,
config,
geometry,
color: props.color,
payload_len,
workspace_bytes,
})?;
tiles.push(tile)?;
}
let tile = &tiles[0];
if u64::from(tile.geometry.width) * u64::from(columns) < u64::from(width)
Expand Down
6 changes: 3 additions & 3 deletions src/bounded/incremental.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ impl<R: Read> Read for Rbsp<R> {

fn plan<'a>(index: &Index<'a>, source: &mut Source<'_>, budget: &Budget) -> Result<Grid<'a>> {
let primary = index.primary;
let properties = Properties::read_incremental(index, primary, false)?;
let properties = Properties::read(index, primary, false)?;
let (width, height) = properties
.dimensions
.ok_or(Error::Malformed("primary dimensions"))?;
Expand Down Expand Up @@ -203,7 +203,7 @@ fn plan<'a>(index: &Index<'a>, source: &mut Source<'_>, budget: &Budget) -> Resu
if !matches!(&index.items[item].kind, b"hvc1" | b"hev1") {
return Err(Error::Unsupported("direct HEVC item required"));
}
let props = Properties::read_incremental(index, item, is_grid)?;
let props = Properties::read(index, item, is_grid)?;
let config = props.config.ok_or(Error::Malformed("missing hvcC"))?;
let _headers = budget.reserve(2 * 1024 * 1024, "header parsing")?;
let slice = slice(index, source, item, config)?;
Expand Down Expand Up @@ -261,7 +261,7 @@ pub fn decode(
let _runtime = budget.reserve(65536, "runtime")?;
let mut source = Source::new(input)?;
let metadata = source.metadata(&budget)?;
let index = Index::parse_primary(&metadata, source.len(), &budget, true)?;
let index = Index::parse(&metadata, source.len(), &budget)?;
let grid = plan(&index, &mut source, &budget)?;
let layout = Layout::new(&grid, options.max_side)?;
for (i, transform) in grid.properties.transforms.iter().flatten().enumerate() {
Expand Down
Loading
Loading