Shape boolean transformation

[timschmidt]

This PR implements axis aligned bounding box index accelerated Shape level booleans, transformations, a transformation builder for reducing unnecessary recomputation of the axis aligned bounding box index, tests, and a Shape boolean demo.

[TimTheBig]

This seems like the better chose for offset in csgrs

[timschmidt]

Still some unresolved issues in the results of the MultiPolyLine booleans. Should be easy enough to see in the GUI demo. Maybe you want to take a stab at fixing them up? Much work is done already. If we can get the MultiPolyLine booleans working properly, the https://github.com/timschmidt/csgrs/tree/cavalier_contours branch uses cavalier_contours in place of geo for all 2D ops.

[timschmidt]

(I jumped the gun, still another bug hiding in here, working at it)

[timschmidt]

This branch hardens and extends the shape boolean layer, especially for multi-polyline shapes with holes, nested islands/lakes, open linework, and interactive UI drag cases.

Key changes:

• Added shape-level boolean support and fixes for OR, AND, NOT, and XOR across signed CCW material loops and CW hole loops.
• Fixed clipping/assembly failures where open polylines or closed loops leaked into filled material.
• Added support for deeper nested island/lake structures.
• Improved robustness around degenerate and adversarial geometry: shared boundaries, tangencies, near-coincident loops, large coordinates, slivers,
  arc-heavy cases, and UI vertex-drag scenarios.
• Added ShapeView and BorrowedIndexedPolyline for borrowed shape inputs, with borrowed fast paths for common boolean cases.
• Added a benchmark harness for owned vs borrowed shape construction and boolean operation costs.
• Expanded docs and docs.rs readiness across public APIs, FFI structs/constants, UI helpers, and shape boolean rationale.
• Hid polyline::internal from generated docs while preserving diagnostic visibility.
• Added extensive regression, adversarial, property, differential, and fuzz-oriented tests for shape booleans.
• Imported/adapted useful regression-style coverage inspired by geo/overlay-style boolean edge cases.

Validation performed:

• cargo fmt --all --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test --workspace
• env RUSTDOCFLAGS="-D warnings -D missing_docs" cargo doc --workspace --all-features --no-deps
• cargo bench -p cavalier_contours --bench shape_borrowed_api
• git diff --check

[jbuckmccready]

Hey, looks like you've been cooking. I gave it a first pass skim, I haven't played with it yet, here's some initial feedback.

• It would help if you could provide:
  • Your approach to building this up at a high level and your workflow with LLMs (sources used, general prompting/iteration, review process, testing, etc.)
  • Detailed summary of algorithm and how it works, deals with different inputs, and handles edge cases (ideally citing back to code)
  • Your recommended approach for reviewing the changes (where to start, what is critical/most important, etc.)
  • Anything not included in this PR that is useful contextually, this could be features/changes you left out that you think should be made, or ideas you had that you decided to skip
• I see there is bibliography with some sources, how were these sources used:
  • Are they direct approaches in them that were implemented, or just related and helpful to solving the problem?
  • How much were you diving in vs. just utilized as a source for LLM implementation and/or fixes?
• Fuzzing was added that looks reasonable at a glance, but does add a lot of code. What was the decision process for adding it, were there bugs caught by using it, and if so what bugs and how were they fixed? Should these maybe be added for other parts of the project?
• For changes that normalize xy values to be near the origin: I know why this is beneficial for the floating point ops, but similarly to the fuzzing changes, how did this come about, and should this be added to other parts of the project?
• There are added constants for fuzzy float comparisons, how were these constant values decided? They also do not match other parts of the project which allows configuring the values as part of the arguments.

The reason for all this high level/non-code specific requests for information is I don't want to waste time crawling through details and (mis)interpret what you've done, or what your intent was.

I know what the algorithm intends to do, but there is a lot of details in the algorithm. And I understand the high level reason for most of the changes (fuzzing, testing, bibliography, etc.), but I don't have the context for the "why" which would allow me to review.

[timschmidt]

All good. It's a big PR with a lot of code. I can pare it down if desired. I'm doing a lot of test-driven porting of csgrs and csgrs-related algorithms to hyperreals at the moment and figured I'd throw codex at the old shape boolean transformation code to see if we could fix it. Our approach: - Reuse the existing Polyline boolean as the clipping primitive. I did not try to implement a new sweep-line overlay from scratch. The shape layer classifies loops, routes to lower-level polyline booleans, and then reassembles results into signed material/hole loops. - Define the shape model around signed loop bins. In this branch, Shape is represented as CCW loops for positive material and CW loops for holes, with per-loop segment AABB indexes and a top-level loop AABB index. ShapeView/BorrowedIndexedPolyline provide borrowed versions for common read-only cases without cloning vertex storage. - Make the public Shape::boolean function mostly a dispatcher. The public method handles open linework, canonicalizes standalone CW loops, applies empty/same-shape identities, chooses special paths for common one-shell or holeless cases, and falls back to cell/pairwise assembly for harder cases. - Harden by invariants rather than only examples. The deterministic tests assert loop orientation, finite coordinates, fresh indexes, no duplicate loops, and sampled set-membership semantics. The fuzz harnesses assert the same shape validity and sampled semantics across compact generated cases. I used codex as a pair-programming and review aid, not as the source of truth. The useful prompts were around “find missing edge cases from this algorithm,” “explain this failing trace,” “suggest regression cases for hole/island nesting,” “summarize this clipping-paper concept,” and “review this helper for stale-index or orientation mistakes.” I reviewed changes and am responsible for any junk left in there. I've checked builds, manually verified artifacts, and tested for working behavior. The bibliography should be read as design background, not as “I directly implemented Martinez/Vatti/Greiner-Hormann line by line. The source use was often conceptual: regularized polygon-set semantics, signed boundary reconstruction, and awareness that degenerate cases like shared edges, tangencies, hole-boundary intersections, and vertex-only containment probes are dangerous. ## Algorithm ## At the representation level, a Shape is a set of indexed loops: CCW loops add material and CW loops subtract holes. ShapeView mirrors that representation with borrowed polyline references and cloned/built spatial indexes, so common boolean cases can avoid cloning all vertex storage until necessary. For open polylines, the algorithm separates area from linework. Open plines do not affect the filled boolean result. The closed portions are booleaned first, then open segments are clipped against the relevant operand area depending on the operation, and finally clipped again against the filled result so internal construction lines do not remain visible inside material. For closed-loop booleans, the lower-level primitive is pline_boolean. That helper clones the input plines, computes their combined bounding-box center, translates both near the origin, calls boolean_opt with a shape-specific PlineBooleanOptions, and translates the result back. This came from precision hardening. The code path is intentionally local: pline_boolean computes a combined bounding-box center, translates cloned inputs near the origin, runs the lower-level boolean, then translates result plines back. That avoids changing the caller’s shape while improving numerical behavior in the operation that is most sensitive to coordinate magnitude. For orientation normalization, lower-level results are converted back into the shape convention: positive results are normalized CCW, subtractive results are normalized CW, and PlineInversionView is used where CW holes need to be presented as positive-area contours to the lower-level polyline boolean. The public evidence for why this mattered is the added large-coordinate stress coverage via the fuzzer. I would not blindly add origin normalization to every method, but it may make sense for more of them since the library is built around hardware floats. For union, there are fast paths and then more careful hole handling. Holeless shapes can merge CCW loops and remove loops whose area is already covered. Single-shell union preserves lower-level negative loops as hole regions, rejects suspicious tiny/inverted union results with a degenerate fallback, subtracts material from hole regions, intersects holes where empty space survives in both operands, clips holes back to final material, and drops holes that are outside the final material. The merge code is deliberately opportunistic; during transient UI-drag states it catches lower-level panics and keeps loops separate rather than making the whole shape unusable for the next frame. For difference and intersection, the code decomposes shapes into “material cells,” meaning a shell plus its immediate holes. Difference subtracts cutter cells from current cells, largest cutters first. Intersection is implemented by intersecting material cells, with special handling when both cells have holes so it can compute the outside region and rebuild nested signed loops correctly. For XOR, the main identity is A xor B == (A \ B) union (B \ A). The code computes both differences, preserves holes that still represent area not covered by the opposite difference, cancels coincident signed pairs, and rebuilds signed loops. There is also a direct lower-level XOR path for some single-shell arc-heavy cases where composing two differences and a union could lose a valid piece. For edge cases, the implementation specifically calls out empty operands, identical operands, same loop geometry with different loop order/start vertex, shared edges, tangencies, hole-boundary intersections, near-zero slivers, deeper nesting, large coordinates, open linework, and UI vertex-drag cases. The tests and fuzz targets are aimed at the same areas: adversarial corpora include deep island/lake nesting, dense hole grids, arc rings, sawtooth intersections, near-coincident shared edges, and the UI demo scene; the singularity fuzz target focuses on point/edge contacts, epsilon-width overlaps, hole tangencies, collinear notches, and large-coordinate precision loss That's to the best of my understanding. If there is a mistake, it is my own. And I'm happy to correct it. The fuzzing was added, primarily by codex as directed by me, to help surface, isolate, and fix degeneracies and failures in the clipping and stitching stages of the algorithm(s). The current constants are private shape-boolean defaults: SHAPE_BOOLEAN_POS_EQUAL_EPS = 1e-8, SHAPE_BOOLEAN_PLINE_POS_EQUAL_EPS = 1e-7, and SHAPE_BOOLEAN_AREA_EPS = 1e-9. The shape helper passes 1e-7 into PlineBooleanOptions while using the area epsilon to filter near-zero slivers during shape assembly. These were empirically derived from the fuzzing, not from some mathematical foundation. Would collapsed_area_eps be a better choice? Or I could do a ShapeBooleanOptions: pub struct ShapeBooleanOptions<T = f64> { pub pos_equal_eps: T, pub pline_pos_equal_eps: T, pub area_eps: T, pub normalize_origin: bool, } ## Recommended review approach ## I would start with cavalier_contours/src/shape_algorithms/mod.rs, but not by reading every helper linearly. First review the invariants and public surface: Shape, IndexedPolyline, ShapeView, from_plines, from_signed_plines, as_view, transform/index rebuild methods, and the constants. The most important invariant is that loop orientation and indexes must stay consistent after every construction or mutation. Next review Shape::boolean as a router. Check whether the identities and special cases are semantically right: empty shapes, identical shapes, holeless union, one-shell-with-holes paths, direct/special XOR paths, and the fallback pairwise signed-loop assembly. Then review the high-risk helpers by behavior: pline_boolean for normalization/tolerance choices, open-pline clipping, merge_ccw_plines, remove_ccw_plines_covered_by_others, material-cell construction, difference/intersection, and signed-pair cancellation. I would focus less on whether every helper is “elegant” and more on whether the result preserves the shape invariants and set semantics. After that, use cavalier_contours/tests/test_shape_boolean.rs as the executable spec. The tests document the intended invariants: closed loops, orientation, no repeated positions, finite coordinates, child/top-level spatial indexes matching loop bounds, no duplicate loops, and sampled set-op membership. Finally, review fuzz as support code rather than the main algorithm. If I can help in any way, please let me know. I did this to try to give back a little, since cavalier_contours inspired https://github.com/timschmidt/hypercurve which credits cavalier_contours twice in its references. The code is not 1:1 but a lot of the inspiration still shows through. Big differences in how the lines are stored semantically, and in depending on https://github.com/timschmidt/hyperlimit for exact geometric predicates.

…

-- Timothy Schmidt (517) 292-4030 ***@***.***

[timschmidt]

I should also say that it's possible there's a better way to do this. I am decidedly a non-expert. I read all the references I cite in these projects. And I understand enough. But I often find myself in the same shoes as you. Mystified as to how the solution was arrived at. This is where the approach I took led me. But if there's a better way, I'm all ears.