fix: clean up PDF tmpdir and validate --image_mode in PDF mode - #26
fix: clean up PDF tmpdir and validate --image_mode in PDF mode#26iamadhitya1 wants to merge 2 commits into
Conversation
infer.py had two bugs in the PDF processing path: 1. pdf_to_images() created a tempdir via mkdtemp() but never removed it. Long batch runs or crashes left /tmp/pdf_ocr_*/ directories behind indefinitely. Fixed by returning the tmpdir from pdf_to_images() and having run() delete it in a try/finally block. 2. --image_mode gundam was silently accepted with --pdf even though the README specifies that PDF/multi-page input requires --image_mode base. The mismatch produces worse OCR output with no warning. Fixed by raising ValueError in build_jobs() when the combination is detected. Adds tests/test_infer.py with 8 unit tests covering both fixes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
PDF clean |
rajpratham1
left a comment
There was a problem hiding this comment.
This is a well-scoped bug fix that addresses two practical issues together: temporary directories created during PDF processing are now cleaned up reliably, and invalid --image_mode values are rejected early for PDF inputs. I also like that the implementation includes comprehensive regression tests covering success, failure, validation, and cleanup paths.
|
Good fix, and the On the tmpdir cleanup approachReturning a
def pdf_to_images(pdf_path: str, dpi: int = 300):
# Docstring note: caller must keep the returned TemporaryDirectory alive for
# the duration of inference; discarding it releases the rendered PNGs immediately.
On the
|
Two improvements based on review by kushdab on PR baidu#26: 1. Added docstring to pdf_to_images() clarifying that the caller is responsible for deleting the returned tmp_dir after use. 2. Moved --image_mode validation out of build_jobs() and into parse_args() using parser.error(), so invalid CLI combinations exit with a proper argparse error message instead of raising an internal ValueError. Updated tests: replaced ValueError-based build_jobs tests with parse_args-level SystemExit tests that exercise the parser directly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the detailed review @kushdab. Two clarifications and the fixes are now pushed: On the GC concern: our implementation uses On Updated the tests accordingly — validation is now tested at the |
Summary
Fixes two bugs in the PDF processing path of
infer.py, both reported in issue #17.Bug 1 — tmpdir leak (
pdf_to_images, line 54)pdf_to_images()created a temp directory withmkdtemp(prefix="pdf_ocr_")but never removed it. After a normal run or a crash, all rendered page PNGs were left behind in/tmp/pdf_ocr_*/indefinitely. Long batch runs or repeated re-runs accumulate these silently.Fix:
pdf_to_images()now returns(image_paths, tmp_dir)instead of justimage_paths.run()stores the path and deletes it in atry/finallyblock so cleanup happens whether inference succeeds or fails.Bug 2 —
--image_mode gundamsilently accepted in PDF mode (build_jobs, line 241)The README (lines 87, 93) states that PDF/multi-page input must use
--image_mode base. With--image_mode gundamand a PDF, the script cropped every page into 640px tiles and fed them to the model with no warning — producing worse output than the documented setting.Fix:
build_jobs()raisesValueErrorimmediately when--pdfis combined with anyimage_modeother thanbase.Changes
infer.py: addimport shutil;pdf_to_imagesreturnstuple[list[str], str];build_jobsreturnstuple[list, str | None]and validatesimage_mode;rununpacks the tuple and cleans up intry/finallytests/test_infer.py: 8 unit tests covering both fixes (mock-based, no GPU or model required)Test plan
python -m pytest tests/test_infer.py -v→ 8 passedflake8 infer.py tests/test_infer.py --max-line-length=120→ no errors