Add a COLMAP export format to polyform (closes #9) - #17
Open
kunjikakgp wants to merge 4 commits into
Open
Conversation
Adds a COLMAP text-model exporter (cameras.txt/images.txt/points3D.txt), addressing PolyCam#9.
Wires COLMAPConvertor into the CLI entry point (adds --shared_camera flag).
18 tests covering quaternion math, ARKit->COLMAP axis conversion, and end-to-end convertor output, using synthetic capture-folder fixtures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9 ("How to get the raw data into Colmap format to use 3DGaussian?"). Also relevant to #11 (LiDAR Point Cloud Alignment) and #13 (Alignment between camera poses and generated mesh) -- same root need, though this PR focuses specifically on #9.
Summary
Adds a COLMAPConvertor (alongside the existing InstantNGPConvertor) that exports a Polycam capture as a standard COLMAP text model (cameras.txt, images.txt, points3D.txt). This is the format expected by the reference 3D Gaussian Splatting implementation and by Nerfstudio's COLMAP data parser, enabling direct use with COLMAP-based tooling without re-running COLMAP's feature matching and SfM pipeline, since Polycam already provides globally optimized camera poses..
Usage, matching the existing CLI pattern:
python3 -m polyform.convert --format colmap
or, for a single shared camera instead of one-per-image:
python3 -m polyform.convert --format colmap --shared_camera
What's included
Design decisions
Axis-convention conversion: Polycam's raw transform_rows (rotate=False) is a camera-to-world matrix in ARKit/OpenGL camera axes (+X right, +Y up, +Z out of the screen). COLMAP/OpenCV expect +X right, +Y down, +Z into the scene, stored as world-to-camera rather than camera-to-world. The conversion flips the local Y/Z axes of the camera-to-world rotation, then transposes to get world-to-camera. I deliberately did not reuse the existing rotate=True branch in Camera.init (the one InstantNGPConvertor uses) -- that's a different axis permutation tuned for instant-ngp/nerfstudio's convention, not COLMAP's, and mixing the two would silently produce wrong poses.
Rotation matrix to quaternion: implemented by hand (Shepperd's method) rather than adding scipy as a new dependency, since polyform currently only depends on numpy, fire and Pillow. A naive sqrt(trace)/2 formula is numerically unstable close to 180-degree rotations, which do happen in real scan trajectories (e.g. looping around an object).
Per-image cameras by default: InstantNGPConvertor already notes that Polycam's per-frame intrinsics vary by a pixel or two frame to frame. I followed that precedent -- every image gets its own COLMAP camera by default, with an opt-in shared_camera=True (--shared_camera on the CLI) for workflows that assume one shared camera.
points3D.txt is intentionally empty: Polycam's raw export doesn't include COLMAP-style per-image 2D/3D correspondences (that's normally the output of the feature-matching + triangulation step this convertor lets people skip). An empty points3D.txt is valid input for tools that only need camera poses -- 3D Gaussian Splatting falls back to random point initialization when it's empty. Documented explicitly in the README.
Possible follow-ups (not in this PR)
Testing
Ran via PYTHONPATH=. python3 -m pytest tests/ -v -- 18 passed. Also ran flake8 (--extend-ignore=F403,F405,E305 to match the star-import and spacing conventions already used in instant_ngp.py) with no findings on the new files.
Coverage: rotation-matrix/quaternion round-tripping (including a rejection check for non-rotation-matrix input), the ARKit->COLMAP axis conversion in isolation, and end-to-end convertor behavior (correct files written, per-image vs shared_camera, unit-norm quaternions, empty points3D.txt, optimized-vs-raw pose handling with cropped intrinsics, empty-folder no-crash, default output path). Also manually verified end-to-end via the CLI against a small synthetic capture.