Optional static vs dynamic splat region separation for deformation mode speedup

[Cedric-Perauer]

This PR adds

a python script that samples timesteps, computes the splat translation over time and stores the movement for each splat in a mask.bin file (see Issue #13)

More Details

• modifications within the app to optionally activate the mask static mode which masks static splats from the deformation network, leading to over 5 times faster inference speed
• an fps counter is visualised in the top right to
• a slider with a save button is introduced that can be used to store the default threshold for each scene and tweak runtime/quality trade off
• due to the fact that some scenes are only captured from a single view which leads to increased movement of splats even in static regions, the mode is currently an optional setting but is especially recommended for multi-view captures where static parts of the scene are more stable

Visualisations

Scenes with Speedup

Works very well for scenes with multi-view static camera capture

Macbook M5 Pro : ~15 -> ~100 FPS

fabien_masking.mp4

iPhone 17 Air : ~3 -> ~15 FPS

sear_phone_mask.mov

Macbook M5 Pro : ~20 -> ~35 FPS

basin_masking.mov

Example scene with no achievable speedup

Here we show the split-cookie scene which experiences heavy camera movement as it is not a static multi-view recording, in this case no static splat masking can be done without loosing significant quality in the 4D rendering. Therefore choosing the ideal setting in masking mode converges to the full unmasked rendering for these types of scenes, we therefore recommend keeping it off in that case (which is the default setting).
Further improvements could consider decoupling the camera and scene movement to adjust for this.

cookie_masking_failure.mov

[Cedric-Perauer]

@yunjinli the example_data folders with included mask files are now uploaded on google drive here