image-processing-lifecycle.md

Image Processing Lifecycle

Overview

The image processing pipeline converts uploaded source files into deep-zoom (DZI) tile sets. It supports SVS, pyramidal TIFF, and standard image formats.

Pipeline stages

Upload (API pod)
  │  File validated → streamed to disk → SourceImage(status="pending")
  ▼
Enqueue
  │  arq/Redis job, or BackgroundTasks fallback in local execution mode
  ▼
Process (dedicated worker, or in-process only in local execution mode)
  │  pending → processing (5%) → tiles (10-78%) → thumbnail (80-85%)
  │  → saving record (90%) → completed (100%)
  ▼
Serve
     DZI tiles at /api/tiles/<source_id>/image.dzi
     Thumbnail at /api/tiles/<source_id>/thumbnail.jpeg

Upload actor attribution

Single uploads, image replacement, and bulk-import child sources persist the acting user as SourceImage.uploaded_by (FK to users.id, SET NULL). BulkImportJob.requested_by records the user who started the bulk import. These columns are not exposed in API response schemas.

When processing completes, app.processing.process_source_image resolves the uploader from SourceImage.uploaded_by and enriches the image.upload.processed structured log with user.id, user.role, and event.synthetic derived from is_synthetic_user(user). If the uploader is unknown or has been deleted, user.id and user.role are omitted and event.synthetic is false. Role and synthetic metadata are never snapshotted into SourceImage itself.

Filename normalization

Client-supplied filenames are normalized once at ingestion by sanitize_upload_filename() (backend/app/filenames.py) before they are persisted as SourceImage.original_filename or AdminTask.original_filename. It applies to single uploads, image replacement, bulk-import members (including ZIP entries), and filesystem-import archives, so the API, logs, span attributes, and processing error messages all show the same value.

Normalization takes the basename (dropping / and \ components), removes control characters and newlines, collapses whitespace runs to single spaces, NFC-normalizes unicode, truncates to the 500-character column limit, and falls back to unnamed when nothing usable remains. Spaces and non-ASCII characters are preserved, and the value is stored as plain text — markup is not HTML-escaped at ingestion; renderers escape.

The on-disk copy is named from a UUID plus a bounded suffix (storage_extension()); a client suffix longer than 32 bytes falls back to .bin, so an over-long display name cannot produce an invalid path component.

Staged writes and orphan prevention

Single uploads and image replacements stream the request body to a .staging-<uuid>.<ext> sibling of the final stored path (backend/app/upload_staging.py), then atomically rename it into place just before the SourceImage row commits. Because staging names are never persisted, a request cancelled or killed mid-stream can only leave a staging artifact — never a final-path file that looks like a committed upload (#1248).

The rename deliberately precedes the commit so a committed row always has its file. Failures before the commit attempt (metadata bumps, queue work staged in the same transaction) are handled by cleanup_unowned_final(), which deletes the final-path file only after a fresh-session query proves no committed SourceImage.stored_path owns it. Once commit() has been attempted the outcome is ambiguous — the server can still commit after the client sees an error — and a fresh ownership query could race that in-flight transaction, so the file is retained unconditionally. An orphaned file is recoverable; a wrongly deleted owned file is not. Queue rejections follow the same rule: once the bookkeeping row is committed with status="failed", that row owns the file, so it stays on disk and consistent with the record.

The shared reconciliation sweep (run_reconciliation_sweep) removes .staging-* artifacts whose mtime is older than four hours (STAGING_MAX_AGE_SECONDS — deliberately above the 7200-second ingress upload timeout so a stalled but still-connected request can never outlive the bound); an in-flight upload refreshes its staging file's mtime on every chunk write, so only dead requests age out. The same sweep also reports aged final-path files that no SourceImage row owns (upload.unowned_final_detected) for operator reconciliation — it never deletes them automatically.

The deployment also sets TMPDIR=<source_images_dir>/.staging/ on the API container, so python-multipart's SpooledTemporaryFile upload spool and every other tempfile consumer land on the source-images PVC instead of the pod's 1 GiB ephemeral-storage budget (#1365). The directory is created at startup (next to the tiles_dir bootstrap in app/main.py); the sweep provisions it too and removes files inside it once their mtime ages past STAGING_MAX_AGE_SECONDS. On a shared RWX mount a sibling pod's in-flight spool can be listed, so the age bound is what keeps the sweep from deleting a live request's file — everything inside .staging/ is a transient artifact never referenced by a committed row.

Bulk-import zip extraction limits

Bulk import (POST /admin/bulk-import/, admin/instructor only) expands uploaded .zip archives into source_images_dir before any SourceImage rows exist. Extraction is bounded by a single _ZipExtractBudget (backend/app/routers/bulk_import.py) shared by every archive in the request, so a crafted archive — or the same payload split across several archives — cannot exhaust the data volume:

• each eligible entry is pre-screened from its central-directory metadata — declared file_size over BULK_IMPORT_MAX_ENTRY_BYTES (2 GiB), running total over BULK_IMPORT_MAX_TOTAL_BYTES (20 GiB), or a file_size / compress_size ratio above BULK_IMPORT_MAX_COMPRESSION_RATIO (100:1) rejects the archive before the entry is opened;
• entries are copied with a bounded read loop that re-checks the per-entry and cumulative caps against the bytes actually decompressed, so a forged header gains nothing;
• more than BULK_IMPORT_MAX_ENTRIES (2000) image entries rejects the archive;
• a single archive with more than BULK_IMPORT_MAX_ARCHIVE_ENTRIES (10,000) total entries — images and non-images alike — is rejected from the end-of-central-directory record (with ZIP64 fallback) before ZipFile allocates the ZipInfo list, and re-checked on the parsed total after open, so an archive flooded with tiny non-image entries cannot charge unbounded parse cost against the API (BULK_IMPORT_MAX_ENTRIES only counts eligible image entries);
• free space on source_images_dir must stay above BULK_IMPORT_MIN_FREE_BYTES (1 GiB), checked before each entry and every 512 MiB written (the same pattern as the filesystem-import staging check).

The raw request body is also bounded: the ingress accepts unlimited bodies, so UploadBodyLimitMiddleware (pure ASGI) watches the streamed body on the multipart upload paths and answers 413 the moment a cap is crossed, before python-multipart's temp-file spool can absorb it. It tracks two counters: bytes since the last multipart --boundary delimiter — a per-part cap matching the endpoint's per-file limit (BULK_IMPORT_MAX_UPLOAD_BYTES / UPLOAD_MAX_BYTES, plus 1 MiB of framing slack), so a batch of individually valid files is never rejected for its combined size — and total body bytes against a whole-request ceiling that bounds the spool (BULK_IMPORT_MAX_REQUEST_BYTES, 80 GiB by default, on bulk import; the per-part cap plus slack on the single-file routes). The endpoint read loops re-apply the per-part cap to each uploaded part (zip archives are consumed in place from the spooled upload file — no second archive copy — and plain images stream to source_images_dir), and write_upload_to_staging enforces UPLOAD_MAX_BYTES for single uploads and replacements; exceeding a cap returns 413 and removes the partial file.

The limits are validated at import time (_validate_zip_limits): non-positive values, or a non-finite / sub-1 compression ratio, fail startup rather than silently disabling a guard. Limit violations return HTTP 413 (free-space exhaustion returns 507) with a message naming the archive/entry (basenames truncated to 40 characters so the reason fits the frontend's 200-character display cutoff) and the limit, and every file already extracted for that request is unlinked before the response is sent. The frontend surfaces the 413 detail verbatim in the upload modal (userMessage() in frontend/src/api.ts). Defaults are listed in backend/README.md.

Zip parts never get a second archive copy: the endpoint rewinds the already-spooled UploadFile.file and hands it straight to zipfile (#1365), so an archive larger than pod-local ephemeral storage — up to BULK_IMPORT_MAX_UPLOAD_BYTES — imports under the default chart profile. Combined with TMPDIR pointing at the PVC, a cancelled or crashed import leaves nothing on pod-local storage at all; a dead request's spool file ages out under the sweep described above.

Central-directory parsing and the bounded extraction loop run on a worker thread via asyncio.to_thread (_extract_zip_image_entries), so a large archive does not stall the API worker's event loop; the budget and the 400/413/507 error mapping are unchanged. A request cancelled mid-extraction cannot interrupt the thread, so the endpoint keeps waiting on the worker's done-event for it to finish and then unlinks every path the worker staged — plus everything staged earlier in the request — rather than leaving unowned files on the data volume.

Status transitions

Status	Progress	Description
pending	0%	SourceImage created, awaiting processing
processing	5%	Worker picked up the job
(tiles)	10-78%	pyvips.dzsave() running (progress via eval signal)
(thumb)	80-85%	Thumbnail generation
(saving)	90%	Creating/updating Image record in DB
completed	100%	Image record created, tiles on disk
failed	—	Error; error_message set on SourceImage

Failure messages

_processing_failure_message() builds the persisted error_message. ENOSPC failures (including libvips write errors that carry the strerror text) become "Insufficient storage — the tiles volume is full"; every other failure keeps the underlying exception text after Tile generation failed: / Image replacement failed:, collapsed to one line, with absolute paths reduced to their basename, the generated storage name swapped for the uploaded filename, and the detail truncated to 300 characters. The frontend polls the source image and shows that message in the processing snackbar; for bulk imports the per-file entries of BulkImportJob.errors are listed instead.

Progress values in the 10-78% range come from pyvips eval signal callbacks mapped via ProgressTracker. The async _flush_tracker_progress() coroutine writes tracker state to the database every 1.5 seconds without blocking tile generation. Polls whose tracker values are unchanged still touch SourceImage.updated_at, and the flusher stays alive through the provenance checksum pass, so a live processing row never ages past the stale-source-image cutoff during one flat-progress stretch (a huge single image, a pyvips build without progress signals, or a slow multi-GB source hash).

Worker configuration

Setting	Value	Rationale
WORKER_MAX_JOBS	4 (configurable)	Concurrent processing slots per worker pod; also bounds in-process bulk-import concurrency in local mode
job_timeout	7200s	2 hours — large filesystem archives need headroom

Task types registered on the worker:

• process_source_image_task — new upload
• replace_image_task — image replacement
• bulk_import_task — legacy compatibility wrapper for already-queued bulk-import coordinators; new bulk imports use API-hosted coordination
• rebuild_tile_pump_task — fills a durable tile-rebuild job's persisted, PostgreSQL-derived execution window
• rebuild_tile_item — reserves and processes one durable source-image item

Durable rebuild children use their configured arq timeout as an execution boundary. Task cancellation sets libvips' kill flag, waits for the generation thread to stop, removes the temporary tile tree, and then releases the worker slot. Operator-requested durable-job cancellation does not interrupt an active libvips call: the child continues heartbeating through generation and observes the persisted cancelling state before promotion. Lease heartbeats stop with the child; an uncommitted item is recovered after its lease expires. If heartbeat renewal loses the claim or raises an error, the child cancels active preparation, waits for libvips and temporary tree cleanup, and exits without committing the rebuild or finalizing that attempt.

Task execution modes and Redis fallback

TASK_EXECUTION_MODE=local preserves the development behavior: a Redis reachability or submission failure returns a fallback result and the caller may use FastAPI's BackgroundTasks. TASK_EXECUTION_MODE=required never runs image processing in the API process. A queue failure returns HTTP 503 with Retry-After: 30; rows created before submission are marked failed and staged files are removed.

The dedicated worker uses arq's built-in arq:queue:health-check key, refreshed independently of job slots every 30 seconds. Kubernetes liveness should run arq app.worker.WorkerSettings --check; arq returns a non-zero exit code when the key is missing or expired. The queue health endpoint reports Redis reachability separately from worker liveness, and is intentionally not part of the readiness probe.

Trace context propagation

The API pod serialises W3C trace context into arq job arguments via opentelemetry.propagate.inject(carrier). The worker extracts it so the full upload → enqueue → worker → tile-gen → DB-write pipeline appears as a single distributed trace.

Tile generation

# processing.py → generate_tiles()
# DZI parameters live in tile_provenance.py so they also feed the settings hash.
image = pyvips.Image.new_from_file(source_path, access="sequential")
image.dzsave(
    output,
    tile_size=DZI_TILE_SIZE,    # 254
    overlap=DZI_OVERLAP,        # 1
    suffix=DZI_TILE_SUFFIX,     # ".jpeg[Q=85]"
)

• access="sequential" — memory-efficient streaming; the file is read once without random access.
• tile_size=254, overlap=1 — standard DeepZoom parameters. These (and the JPEG suffix) are defined as constants in app/tile_provenance.py, so the recorded tile_settings_hash always reflects the parameters actually used — see tile-cache-provenance.md.
• JPEG quality 85 — balance between file size and visual fidelity.
• Runs via asyncio.to_thread() so the event loop is not blocked.

Thumbnail

thumb = pyvips.Image.thumbnail(source_path, 256, height=256, crop="centre")

A fresh file handle is needed because the sequential stream was consumed by dzsave. Center-cropping ensures card previews show a recognisable portion regardless of aspect ratio.

Tile count estimation

_estimate_tile_count(width, height) sums tiles across all pyramid levels (halving dimensions each level) for progress logging and span attributes.

Pyramidal image detection

detect_pyramid_info(source_path) inspects a source file for pre-existing pyramidal structure and microscopy metadata.

Loader	Detection method	Metadata extracted
openslideload	openslide.level-count > 1	mpp_x/y, objective_power (incl. Aperio)
tiffload	SubIFD-based or multi-page pyramid	Resolution from TIFF xres/yres

Derived fields:

• measurement_scale = 1.0 / mpp_x (pixels per µm)
• measurement_unit = "um" (always, when MPP is available)

TIFF resolution conversion: libvips stores resolution in pixels/mm, so mpp_x = 1000.0 / xres. Values outside 0.01–100 µm/px are discarded as unreasonable for microscopy.

After a successful upload or replacement commits, processing calls scan_after_processing, which queues a one-image measurement scan (or runs it inline when the queue is unavailable). An embedded scale is recorded as an approved embedded finding; other images get a pending RBC proposal. See measurement-review.md.

Image replacement

process_replace_image(source_image_id, target_image_id):

1. Generates new tiles + thumbnail from the replacement source file
2. Updates the existing Image record: tile_sources, thumb, width, height, file_size, version (bumped)
3. Clears canvas_annotations and locked_overlays from metadata (coordinates reference old image geometry)
4. Re-derives pyramid metadata from the new file
5. Removes old tile directory from disk after the DB commit succeeds

In required task-execution mode, a queue rejection marks the replacement source as failed and retains its file (the committed failed row owns the path — see the staged-writes section), but applies no metadata or version changes to the target image.

See image-metadata-and-versioning.md for the full metadata preservation/clearing rules.

File paths

Env var	Default	Contents
SOURCE_IMAGES_DIR	/data/source_images	Uploaded raw images
TILES_DIR	/data/tiles	Generated DZI tiles + thumbs

Tiles are served by the authorized FastAPI route GET /api/tiles/{source_image_id}/{path}, which validates an image-scoped tile_token query parameter (issued on tile_sources/thumb URLs in API responses — see tile-delivery-boundary.md). In Kubernetes the nginx tiles sidecar serves the PVC directly and authorizes requests via auth_request against GET /api/tiles-auth.

Tile-cache provenance

On successful tile generation (both new uploads and replacements) the SourceImage records provenance so tile currentness can be evaluated later without filesystem inspection:

• source_checksum — SHA-256 of the source file (best-effort; never blocks completion).
• tile_settings_hash — fingerprint of the DZI settings + pipeline version.
• tiles_generated_at — generation timestamp.

The effective tile_cache_status (current / missing / stale / failed) is computed from these fields plus the current pipeline settings. The DZI parameters live in app/tile_provenance.py and feed both dzsave and the settings hash. See tile-cache-provenance.md for the staleness rules and API surface.

Bulk-import coordinators distinguish a lost child job from a stopped worker. New bulk-import requests run their lightweight coordinator via the API BackgroundTasks path so the coordinator never occupies an arq worker slot while it waits for child image jobs. Coordination is durable (#1173): the import snapshots one JobItem per staged file under a job_type="bulk_import" supervisor Job, the coordinator holds a PostgreSQL lease on that row, and every API replica runs a periodic sweep that re-claims expired leases — so an API restart, crash, rolling deploy, or OOM kill resumes the import instead of abandoning it, without duplicating SourceImage rows or leaking staged files. BULK_IMPORT_COORDINATOR_LEASE_SECONDS (120), BULK_IMPORT_COORDINATOR_HEARTBEAT_SECONDS (30), BULK_IMPORT_ITEM_LEASE_SECONDS (300), BULK_IMPORT_MAX_COORDINATOR_CLAIMS (10), BULK_IMPORT_MAX_ITEM_ATTEMPTS (3), and BULK_IMPORT_COORDINATOR_SWEEP_SECONDS (30) tune the resume machinery. Child image processing still prefers the arq worker queue, and TASK_EXECUTION_MODE=required still rejects child queue submission failures rather than processing images in the API process. Coordinators register liveness in Redis. The bulk-import endpoint returns HTTP 409 (A bulk import is already in progress) while another pending or processing import has a live coordinator registration, or while a pending or processing row is still within the short coordinator-registration window. When Redis positively confirms no coordinator registration is live, an older row is treated as abandoned regardless of how stale it is. When Redis is unreadable (disabled, unreachable, or erroring), the guard no longer fails open on recency alone: the short coordinator-registration window only proves a coordinator hasn't written to its row recently, not that it has stopped, so an unreadable check instead falls back to the same conservative, Redis-independent staleness threshold reconcile_stale_bulk_import_jobs already uses in the reconciliation sweep (BULK_IMPORT_STALE_SECONDS, defaulting to the worker job timeout) before treating the row as abandoned; anything still within that window blocks the new import with 409 instead. The Redis/staleness pre-check above is a check-then-act race under concurrent requests: two requests can both pass it before either commits its new BulkImportJob row. A partial unique index on bulk_import_jobs (idx_bulk_import_jobs_single_active, added in migration 0024_bulk_import_single_job) makes "at most one pending/processing row" an actual database guarantee. Because the pre-check's own stale-coordinator recovery path intentionally admits an old pending/processing row once its coordinator is judged dead (whether confirmed dead via Redis, or judged abandoned via the staleness threshold above when Redis is unreadable), bulk_import_images finalizes that abandoned row to a terminal state in the same transaction as the new job it is about to insert — otherwise the new unique index would keep rejecting every replacement until the next reconcile_stale_bulk_import_jobs sweep. On the insert itself, bulk_import_images catches IntegrityError, checks that it is specifically idx_bulk_import_jobs_single_active (an unrelated integrity failure, such as the target category being deleted concurrently, propagates unchanged rather than masquerading as a conflict), removes the files it had already staged to disk, and returns the same 409 the pre-check produces. Once the new job's insert has committed, bulk_import_images also runs the same orphaned-file cleanup reconcile_stale_bulk_import_jobs performs during its sweep, but only for finalized rows whose coordinator was positively confirmed dead via Redis — a row admitted through the staleness-threshold fallback above is finalized but its files are left alone, since an unreadable liveness check is not proof the coordinator has actually stopped. This cleanup is best-effort: a failure there is logged and swallowed rather than blocking the already-committed job from being scheduled. When a queued child observes an absent worker heartbeat for the configured wall-clock window, it writes arq's abort latch before marking the SourceImage failed. The latch narrows the race window; the processor's terminal-row guard is what guarantees a late child cannot overwrite a recorded failure. If Redis cannot accept the latch, the row remains pending and the coordinator keeps waiting. The pending wait ceiling stays below the coordinator's job_timeout as an evidence-gated last resort, so it can record a child whose status has remained unknown without failing healthy queued work. The processing backstop is above the child's job_timeout because its clock starts when processing is first observed. Worker-hosted coordinators normally rely on the 900-second no-progress stale check because arq's coordinator timeout is reached first; new API-hosted coordination retains the processing backstop. Both paths use the latch before failing the row. This guarantee relies on the pinned arq version's past-dated abort-marker behavior and is covered by test_abort_latch_survives_pruning_and_is_consumed_before_job_start in backend/tests/test_worker.py.

Every source-image processor also refuses to start when its row is already terminal, regardless of whether reconciliation, a bulk-import detector, or another failure path recorded that terminal state. This prevents late queued jobs from overwriting a recorded failure; arq retries remain valid because retryable rows stay in processing rather than becoming terminal.

Stale SourceImage reconciliation

reconcile_stale_source_images(), along with the other reconciliation steps described below, runs from a shared sweep (run_reconciliation_sweep() in app/reconciliation.py). In TASK_EXECUTION_MODE=local (no dedicated worker pod) the sweep runs once on backend (API pod) startup, since the API process is the only one running and must self-heal on boot. In TASK_EXECUTION_MODE=required a dedicated arq worker pod is guaranteed, so the sweep instead runs as a periodic arq cron job (WorkerSettings.cron_jobs, every 15 minutes, plus once at worker boot) — this catches staleness accumulated between deployments/restarts, not just at boot, and keeps the reconciliation work off the API pod's startup critical section. reconcile_stale_source_images() marks SourceImages as failed when they exceed status-specific cutoffs. processing rows use the 900-second no-progress cutoff because they have started work. pending rows are never failed by elapsed time alone. Once they exceed the job_timeout // 2 observation bound (1 hour by default), they are failed only when Redis is reachable, a worker heartbeat is present, and the arq queue is empty. A non-empty or unhealthy queue leaves pending rows untouched so a valid queued job can still self-heal. This handles genuinely lost jobs without failing healthy queued work.

reconcile_stale_bulk_import_jobs() also runs as part of that same sweep. It marks a pending or processing bulk-import row as terminal only when its updated_at is older than the configured stale threshold and it has no live coordinator registration: incomplete rows become failed, while rows whose completed count equals their total become completed. A live registration keeps a long-running import untouched. When Redis is unavailable, the sweep uses the conservative stale timestamp bound alone rather than leaving abandoned rows permanently in processing; the default bound is the coordinator job timeout. API-hosted local fallbacks register in the general coordinator liveness set and therefore use the same conservative abandoned-row path. Existing per-file errors and failed_count accounting is preserved. This finalises coordinator rows left behind by a cancelled or killed arq job; the coordinator uses max_tries=1 because retrying its non-idempotent batch would duplicate SourceImage rows.

The same sweep reconciles default-off durable tile rebuild jobs independently. It reclaims expired JobItem leases, clears their execution reservation, and requests a new pump for each active rebuild. Rebuild reconciliation failure is logged without preventing the existing source-image, bulk-import, and AdminTask recovery steps. A separate worker cron cadence supplies a more frequent backstop for missed rebuild pump triggers.

Because the coordinator now lives entirely in the API process (BackgroundTasks, not arq), its in-memory file_entries manifest does not survive an API restart — an interrupted import is not resumed, only finalized to a terminal state as described above. To avoid leaking staged files, bulk_import_images persists the same manifest as BulkImportJob.file_manifest before scheduling the coordinator. When reconcile_stale_bulk_import_jobs() finalizes a stale job, it also removes any manifest-listed staged file that never became a SourceImage row (via _cleanup_orphaned_bulk_import_files), so an interrupted import doesn't hold disk space indefinitely. This does not resume in-flight processing; that would require the durable job/queue model tracked for a future iteration (see the Job/JobItem foundation work).

Surfacing failures after a reload

SourceImage.error_message is persisted, so failures survive a reload — but the frontend's useProcessingJobs state only holds jobs created during the current page session. rehydrateFailedJobs() closes that gap: on the first admin/instructor entry it calls GET /api/source-images/?status=failed&limit=20 (status and limit are optional query params on the existing admin/instructor-only list endpoint, both applied in SQL) and restores each row as a terminal failed job carrying the persisted sanitized message. Restored jobs never start a poller and never count against MAX_PROCESSING_JOBS; rows older than REHYDRATED_FAILURE_MAX_AGE_MS (7 days, measured from created_at) are ignored, and a failed fetch clears the once-per-session guard so the next entry retries.

Failure snackbars no longer auto-hide — an operator must dismiss them. Dismissed source-image ids are persisted per user scope in localStorage (hrivpref:dismissed-failed-uploads:user:<scope>, capped at 200 ids) so a dismissed failure does not return on the next reload. Five or more image failures collapse into a single N uploads failed. snackbar to avoid a pile-up.

Only server-confirmed failures (serverFailed, set when the backend reports the SourceImage as failed and on rehydration) participate in dismissal persistence and in that collapse. Client-side failures — an aborted upload, or status tracking lost to an expired session while the server keeps working — keep their own snackbar and are never remembered, so they cannot suppress the genuine failure the backend may record later.

Because a collapsed snackbar cannot show every filename, the Failed uploads dialog (frontend/src/components/FailedUploadsDialog.tsx) lists every failed source image with its filename, failure time, and persisted reason. It is reachable from the collapsed snackbar's Details action and from the Manage page header, so it remains available after every snackbar has been dismissed; rows can be dismissed individually or all at once.

Related code

• Upload router: backend/app/routers/upload.py
• Processing pipeline: backend/app/processing.py
• Worker config: backend/app/worker.py
• Image validation: backend/app/image_validation.py

Related tests

• backend/tests/test_processing.py — tile generation, pyramid detection, replacement
• backend/tests/test_worker.py — enqueue, fallback, trace propagation
• backend/tests/test_router_upload.py — upload validation, file streaming

Related skills

• Testing Image Processing — hands-on testing guide