mosaic-temporal

Bit-exact GPU SSIM in PyTorch — PSNR ~300 dB parity with skimage. Hungarian + RAFT optical flow for temporally coherent video mosaic.

Related: mosaic-temporal-gpu — high-speed NVDEC/NVENC variant that drops CPU fallbacks for maximum GPU throughput.

What it does

Reconstructs a video from thousands of small image tiles ("photo mosaic"), with temporal coherence: same regions across frames map to the same tiles when visually plausible. Unlike per-frame mosaic generators (which flicker), this library:

• Solves a Hungarian assignment between tiles and grid cells per frame
• Uses RAFT optical flow to propagate tile choices across consecutive frames
• Computes a bit-exact GPU SSIM (~300 dB PSNR parity with skimage) for tile re-ranking, so GPU and CPU produce md5-identical output

Verified on Kaggle T4: 240/240 frames bit-exact match between CPU and GPU paths.

Install

pip install mosaic-temporal              # CPU-only (numpy/scipy/cv2/skimage)
pip install "mosaic-temporal[gpu]"       # +torch, torchvision, numba (RAFT, GPU SSIM)
pip install "mosaic-temporal[all]"       # +kornia for GPU postprocessing

Quickstart

from mosaic_temporal import D1Config, run_pipeline

config = D1Config.from_preset("vivid", tile_size=64)
run_pipeline(
    frames_dir="frames/",
    tile_pool="tiles/",
    output="mosaic.mp4",
    config=config,
)

CLI:

mosaic-temporal --frames frames/ --tiles tiles/ --output mosaic.mp4 --preset vivid

Demo

See examples/synthetic_24f.py for a 24-frame procedural smoke test (no external assets, runs in <60s on CPU).

For a Big Buck Bunny demo with Fruits-360 tiles, see docs/DEMO.md.

Why bit-exact?

Traditional GPU SSIM uses kornia's box convolution, which rounds differently from skimage and breaks reproducibility (PSNR ~80 dB only). mosaic-temporal implements skimage's exact uniform-7 convolution in torch.conv2d (fp64 path), so CPU and GPU SSIM scores match to PSNR > 250 dB on uint8 inputs.

This means: the same input video + tile pool produces md5-identical mp4 output regardless of CPU or GPU. CI parity tests verify this on every commit.

Performance

Configuration	Resolution	Per-frame time	Speedup
skimage SSIM (CPU)	1080p	29.8s	1.0× (baseline)
mosaic-temporal[gpu] (Kaggle T4)	1080p	21.9s	1.36×

See BENCHMARKS.md for full reproducible numbers.

Architecture

• core — MosaicEngine + run_pipeline, the Hungarian + RAFT entrypoint
• torch_exact — bit-exact GPU SSIM (PSNR ~300 dB parity)
• solvers/ — SolverBackend Protocol + scipy / torch_lap / auto backends
• flow/ — FlowBackend Protocol + RAFT (extras gpu) and Identity (CI) backends
• ssim/ — SSIMBackend Protocol + skimage and torch_exact backends
• gpu/ — optional GPU paths (assemble / postprocess / fast_swap)
• legacy — naive per-frame baseline for parity comparison

See DESIGN.md for the full design rationale, including why kornia SSIM was rejected (F7 NOGO: bit-exact incompatibility).

Status

v0.1.0 — beta. API may evolve before v1.0.0. Bit-exact contract is stable.

Verification (sigstore)

Releases from v_next_ (released after 2026-05-16) include a sigstore keyless signature bundle (.sigstore per artifact) attached to the GitHub Release.

Verify a PyPI install

pip download <pkg-name>==<version> --no-deps -d ./verify
python -m sigstore verify github \
    --cert-identity 'https://github.com/hinanohart/mosaic-temporal/.github/workflows/release.yml@refs/tags/v<version>' \
    --cert-oidc-issuer 'https://token.actions.githubusercontent.com' \
    ./verify/*.whl ./verify/*.tar.gz

The corresponding .sigstore bundles can be downloaded from the GitHub Release page.

Historic releases (pre-2026-05-16)

Earlier releases were published without sigstore bundles. Re-installing those versions provides no cryptographic provenance — pin to a current release if assurance matters.

License

MIT. See LICENSE. Demo asset credits in CREDITS.md; third-party attribution is consolidated there.

Acknowledgements

• RAFT optical flow: Princeton (BSD-3, via torchvision)
• Hungarian assignment: scipy linear_sum_assignment
• bit-exact SSIM: skimage structural_similarity (BSD-3)

Citation

@software{mosaic_temporal_2026,
  author = {hinanohart},
  title = {mosaic-temporal: Bit-exact GPU SSIM + Hungarian + RAFT for video mosaic},
  year = {2026},
  url = {https://github.com/hinanohart/mosaic-temporal},
}