test(qa): MLX image-gen QA lane — t2i matrix + adversarial VLM scoring (stacked on #311)

[ziyu4huang]

What

Fills a QA lane for the mlx_image provider (shipped in #311): a small deterministic t2i matrix + adversarial VLM scoring. Dual purpose — (1) the local-generation slot in QA_PLAN.md, and (2) a regression gate for mlx_image: a score collapse or generation failure here catches a provider/env regression before it reaches a user.

tests/qa/test_02_mlx_image_gen.py + a QA_PLAN.md table/run-order row.

⚠️ Stacked on #311 — merge #311 first

This imports from tools.graphics.mlx_image import MLXImage, which lands in #311. Until #311 merges, this PR's diff includes #311's commits (that's why it looks large). Once #311 merges to main, GitHub recomputes the diff against main and it collapses to just the two files this PR adds:

tests/qa/QA_PLAN.md                |  2 ++
tests/qa/test_02_mlx_image_gen.py  | 264 ++++++++++++++++++++++++++++++++

Please review/merge #311 first; this one becomes a clean 2-file PR after.

What it does

For each row in a fixed-seed matrix (portrait / still_life / hands — each stresses a known weak spot):

1. MLXImage().execute({prompt, width, height, seed, output_path}) — generates via the local MLX provider ($0, Apple-Silicon-native), shells out to mlx-movie-director run.py.
2. run.py caption <img> --style score --samples 3 — scores it with Qwen3-VL via LM Studio. --samples 3 writes the per-dimension median of 3 VLM passes, which is necessary because the local Gemma-4 VLM is image-dependent and can return an empty response on a single pass.
3. Writes tests/qa/output/qa_image_gen_receipt.json (per-image scores + flags + gen metadata) and leaves the PNGs for human inspection per the QA_PLAN protocol.

Exit code is non-zero only on a generation or scoring failure (provider broke / caption broke). Low scores are flagged ⚠ in the table — the scorer is deliberately adversarial ("FIND flaws, not praise"), so soft thresholds flag rather than fail; QA is human-inspected.

Not CI pytest

Needs MLX_MOVIE_DIRECTOR_DIR + the mlx venv + staged models + Apple Silicon + LM Studio on localhost:1234. All execution is under if __name__ == "__main__", so pytest collection imports the module cleanly and runs 0 tests — it won't break CI. Run manually:

MLX_MOVIE_DIRECTOR_DIR=/path/to/video_generation \
    .venv/bin/python tests/qa/test_02_mlx_image_gen.py

Baseline receipt (mlx-zimage, --samples 3)

tag	overall	detail	sharpness	composition	prompt	artifacts	flag
portrait	6	6	8	8	9	5	⚠artifacts
still_life	9	8	9	9	10	9	—
hands	6	5	8	8	9	4	⚠artifacts

The flags are expected known base-model traits, not regressions: plasticky/too-smooth skin (a documented zimage base characteristic) and finger fusion (a known AI weak spot). The still_life scores 9 across the board. This is the honest baseline — a future change that improves skin texture shows up as artifacts rising on portrait/hands; a regression that breaks generation shows up as a FAILURE exit.

🤖 Generated with Claude Code

[calesthio]

Thank you for adding this QA lane. The idea is useful, but I?m requesting changes because this PR is explicitly stacked on #311, and #311 currently has requested changes / failing portability tests. As-is, this PR?s diff includes the entire MLX provider family, so it is not safe to approve independently.

Verification I ran on this branch:

python -m py_compile tests/qa/test_02_mlx_image_gen.py
# passed

python -m pytest tests/qa/test_02_mlx_image_gen.py -q
# no tests ran

git diff --check origin/main...HEAD
# clean

The ?no tests ran? result matches the PR description that this is a manual QA script, not a CI pytest suite, so that part is not by itself a blocker. The blocker is dependency hygiene: please rebase this after #311 is merge-ready/merged so the review diff collapses to the intended QA files only. Once the branch contains just tests/qa/QA_PLAN.md and tests/qa/test_02_mlx_image_gen.py, this can be reviewed on its own merits.

I did not find an actionable security issue in the reviewed QA diff, but I?m not approving a stacked PR while the base provider PR is still blocked.