Flatten to standard src-layout at root#10
Merged
Merged
Conversation
The repo IS pulpie now, so adopt a standard src-layout at root instead of
nesting under pulpie/:
- pulpie/{src,tests,assets} -> {src,tests,assets}
- pulpie/{pyproject.toml,MANIFEST.in,uv.lock,BENCHMARKS.md} -> root
- pulpie/README.md becomes the single root README (drops the thin top README)
Reference updates: CI workflows drop 'working-directory: pulpie' and fix the
cache-dependency-glob; README banner raw URLs use /main/assets/; conftest
fixture path drops one '..'; AGENTS.md + SOURCE.md test paths updated.
Verified: build from root (wheel/sdist correct, assets/tests excluded, twine
passes) and the exact CI install+test sequence (65 parity tests pass).
There was a problem hiding this comment.
Pull request overview
This PR flattens the repository into a standard Python src/ layout at the repo root and updates packaging/CI/docs accordingly, while also bringing in (and testing against) a vendored MinerU-HTML oracle to ensure simplify/reconstruct parity.
Changes:
- Move project files to root + adopt
src/layout with updatedpyproject.toml, workflows, and README asset links. - Add/relocate core extraction implementation (
simplify,reconstruct, chunking, extractor, multi-GPU pipeline) undersrc/pulpie/. - Add parity tests against a vendored MinerU-HTML oracle (via a shim to avoid importing heavyweight inference deps).
Reviewed changes
Copilot reviewed 7 out of 47 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_simplify_parity.py | Adds simplify/map_html/reconstruct parity + roundtrip smoke tests against oracle. |
| tests/gpu/AGENTS.md | Updates test invocation paths for new layout. |
| tests/conftest.py | Adjusts fixture corpus path for flattened repo; installs oracle shim. |
| tests/_oracle/SOURCE.md | Updates documentation references to new test paths. |
| tests/_oracle/MinerU-HTML/mineru_html/utils.py | Vendored oracle utility helpers. |
| tests/_oracle/MinerU-HTML/mineru_html/process/simplify_html.py | Vendored oracle simplify implementation (golden reference). |
| tests/_oracle/MinerU-HTML/mineru_html/process/parse_result.py | Vendored oracle parse-result logic. |
| tests/_oracle/MinerU-HTML/mineru_html/process/map_to_main.py | Vendored oracle reconstruct/extract-main logic. |
| tests/_oracle/MinerU-HTML/mineru_html/process/html_utils.py | Vendored oracle HTML parsing/serialization helpers. |
| tests/_oracle/MinerU-HTML/mineru_html/process/convert2content.py | Vendored oracle conversion hook. |
| tests/_oracle/MinerU-HTML/mineru_html/process/build_prompt.py | Vendored oracle prompt builders. |
| tests/_oracle/MinerU-HTML/mineru_html/process/init.py | Exposes vendored oracle process API. |
| tests/_oracle/MinerU-HTML/mineru_html/inference/vllm_backend.py | Vendored oracle vLLM backend (not used by tests via shim). |
| tests/_oracle/MinerU-HTML/mineru_html/inference/transformers_backend.py | Vendored oracle transformers backend (not used by tests via shim). |
| tests/_oracle/MinerU-HTML/mineru_html/inference/openai_backend.py | Vendored oracle OpenAI backend (not used by tests via shim). |
| tests/_oracle/MinerU-HTML/mineru_html/inference/factory.py | Vendored oracle backend factory. |
| tests/_oracle/MinerU-HTML/mineru_html/inference/base_backend.py | Vendored oracle inference base types/utilities. |
| tests/_oracle/MinerU-HTML/mineru_html/inference/init.py | Vendored oracle inference package exports. |
| tests/_oracle/MinerU-HTML/mineru_html/implementations/vllm_api.py | Vendored oracle implementation wrapper. |
| tests/_oracle/MinerU-HTML/mineru_html/implementations/transformers_api.py | Vendored oracle implementation wrapper. |
| tests/_oracle/MinerU-HTML/mineru_html/implementations/openai_api.py | Vendored oracle implementation wrapper. |
| tests/_oracle/MinerU-HTML/mineru_html/implementations/init.py | Vendored oracle implementations exports. |
| tests/_oracle/MinerU-HTML/mineru_html/exceptions.py | Vendored oracle exception types. |
| tests/_oracle/MinerU-HTML/mineru_html/constants.py | Vendored oracle constants/enums. |
| tests/_oracle/MinerU-HTML/mineru_html/base.py | Vendored oracle pydantic models + base types. |
| tests/_oracle/MinerU-HTML/mineru_html/api.py | Vendored oracle end-to-end API (not used by tests via shim). |
| tests/_oracle/MinerU-HTML/mineru_html/init.py | Vendored oracle top-level exports. |
| src/pulpie/simplify.py | Implements MinerU-compatible HTML simplification + id mapping. |
| src/pulpie/reconstruct.py | Implements MinerU-compatible main-content reconstruction. |
| src/pulpie/py.typed | Marks package as typed. |
| src/pulpie/pipeline.py | Adds overlapped multi-GPU pipeline implementation. |
| src/pulpie/model_utils.py | Adds shared model loading + label conversion utilities. |
| src/pulpie/extractor.py | Adds user-facing Extractor API (single-device). |
| src/pulpie/chunker.py | Adds block extraction + chunk packing utilities. |
| src/pulpie/_html_utils.py | Adds low-level HTML parse/serialize helpers. |
| src/pulpie/init.py | Exposes public API and sets package version. |
| README.md | Replaces thin top-level README with full project README; fixes asset URLs. |
| pyproject.toml | Adds root packaging metadata for src/ layout + deps/extras/tooling. |
| pulpie/README.md | Removes nested-package README after flattening. |
| MANIFEST.in | Excludes tests from sdists (and other manifest rules). |
| BENCHMARKS.md | Adds benchmark documentation at repo root. |
| assets/banner.svg | Adds banner asset at root. |
| assets/banner-dark.png | Adds/relocates banner image asset at root. |
| .github/workflows/test.yml | Removes working-directory, updates cache dependency glob + test command. |
| .github/workflows/publish.yml | Removes working-directory so build/publish run from repo root. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The repo is pulpie itself, so drop the nested
pulpie/wrapper and adopt a standard Python src-layout at root.Moves:
pulpie/{src,tests,assets}→{src,tests,assets}pulpie/{pyproject.toml,MANIFEST.in,uv.lock,BENCHMARKS.md}→ rootpulpie/README.md→ root README (drops the thin top-level one)Reference updates:
working-directory: pulpie, fixedcache-dependency-glob/main/assets/...tests/conftest.pyfixture path (one fewer..)tests/gpu/AGENTS.md,tests/_oracle/SOURCE.mdtest-path refsVerified locally:
uv buildfrom root produces correct wheel/sdist (assets/tests excluded, twine passes); the exact CI install+test sequence runs 65/65 parity tests green.