Skip to content

Add a COLMAP export format to polyform (closes #9) - #17

Open
kunjikakgp wants to merge 4 commits into
PolyCam:mainfrom
kunjikakgp:add-colmap-convertor
Open

Add a COLMAP export format to polyform (closes #9)#17
kunjikakgp wants to merge 4 commits into
PolyCam:mainfrom
kunjikakgp:add-colmap-convertor

Conversation

@kunjikakgp

Copy link
Copy Markdown

Closes #9 ("How to get the raw data into Colmap format to use 3DGaussian?"). Also relevant to #11 (LiDAR Point Cloud Alignment) and #13 (Alignment between camera poses and generated mesh) -- same root need, though this PR focuses specifically on #9.

Summary

Adds a COLMAPConvertor (alongside the existing InstantNGPConvertor) that exports a Polycam capture as a standard COLMAP text model (cameras.txt, images.txt, points3D.txt). This is the format expected by the reference 3D Gaussian Splatting implementation and by Nerfstudio's COLMAP data parser, enabling direct use with COLMAP-based tooling without re-running COLMAP's feature matching and SfM pipeline, since Polycam already provides globally optimized camera poses..

Usage, matching the existing CLI pattern:

python3 -m polyform.convert --format colmap

or, for a single shared camera instead of one-per-image:

python3 -m polyform.convert --format colmap --shared_camera

What's included

  • polyform/convertors/colmap.py -- the convertor itself, plus a standalone arkit_pose_to_colmap() helper that converts Polycam's raw ARKit camera-to-world transform into COLMAP's world-to-camera quaternion + translation.
  • polyform/convert.py -- wires --format colmap into the existing CLI entry point.
  • README.md -- new "COLMAP" section (mirrors the existing "Instant-ngp" section's structure and tone).
  • tests/ (new) -- 18 pytest tests covering the coordinate-conversion math and the end-to-end convertor, using synthetic capture folders since the repo doesn't ship any sample data.

Design decisions

Axis-convention conversion: Polycam's raw transform_rows (rotate=False) is a camera-to-world matrix in ARKit/OpenGL camera axes (+X right, +Y up, +Z out of the screen). COLMAP/OpenCV expect +X right, +Y down, +Z into the scene, stored as world-to-camera rather than camera-to-world. The conversion flips the local Y/Z axes of the camera-to-world rotation, then transposes to get world-to-camera. I deliberately did not reuse the existing rotate=True branch in Camera.init (the one InstantNGPConvertor uses) -- that's a different axis permutation tuned for instant-ngp/nerfstudio's convention, not COLMAP's, and mixing the two would silently produce wrong poses.

Rotation matrix to quaternion: implemented by hand (Shepperd's method) rather than adding scipy as a new dependency, since polyform currently only depends on numpy, fire and Pillow. A naive sqrt(trace)/2 formula is numerically unstable close to 180-degree rotations, which do happen in real scan trajectories (e.g. looping around an object).

Per-image cameras by default: InstantNGPConvertor already notes that Polycam's per-frame intrinsics vary by a pixel or two frame to frame. I followed that precedent -- every image gets its own COLMAP camera by default, with an opt-in shared_camera=True (--shared_camera on the CLI) for workflows that assume one shared camera.

points3D.txt is intentionally empty: Polycam's raw export doesn't include COLMAP-style per-image 2D/3D correspondences (that's normally the output of the feature-matching + triangulation step this convertor lets people skip). An empty points3D.txt is valid input for tools that only need camera poses -- 3D Gaussian Splatting falls back to random point initialization when it's empty. Documented explicitly in the README.

Possible follow-ups (not in this PR)

  • Seed points3D.txt from the depth maps Polycam already exports (keyframes/depth/*.png), by back-projecting a subsample of pixels into a sparse point cloud. Didn't attempt this here since I don't have a real Polycam capture to confirm the exact depth encoding (units, 16-bit scale factor) against, and didn't want to ship unverified back-projection code.
  • Binary model export (colmap model_converter can already do this conversion, so kept out of scope here).

Testing

Ran via PYTHONPATH=. python3 -m pytest tests/ -v -- 18 passed. Also ran flake8 (--extend-ignore=F403,F405,E305 to match the star-import and spacing conventions already used in instant_ngp.py) with no findings on the new files.

Coverage: rotation-matrix/quaternion round-tripping (including a rejection check for non-rotation-matrix input), the ARKit->COLMAP axis conversion in isolation, and end-to-end convertor behavior (correct files written, per-image vs shared_camera, unit-norm quaternions, empty points3D.txt, optimized-vs-raw pose handling with cropped intrinsics, empty-folder no-crash, default output path). Also manually verified end-to-end via the CLI against a small synthetic capture.

Adds a COLMAP text-model exporter (cameras.txt/images.txt/points3D.txt), addressing PolyCam#9.
Wires COLMAPConvertor into the CLI entry point (adds --shared_camera flag).
18 tests covering quaternion math, ARKit->COLMAP axis conversion, and end-to-end convertor output, using synthetic capture-folder fixtures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How to get the raw data into Colmap format to use 3DGaussian?

1 participant