Skip to content

Incorrect ray count calculation due to floating-point precision #2

@X1a0ming

Description

@X1a0ming

Description

There is an inconsistency in how the number of rays is calculated in RayCaster.cpp due to floating-point precision issues. When dividing the size by the resolution, the implicit casting to integer causes truncation errors for specific values (e.g., 1.2), resulting in missing rays.

Reproduction Steps

Set resolution = 0.1.

Initialize RayCaster with size = (1.2, 1.6).

Observe the calculated ray count.

Expected Behavior For a size of 1.2 and resolution of 0.1: Calculation: (1.2 / 0.1) + 1 should equal 13. Expected Ray Count: 17x13

Actual Behavior The calculated v_ray_num is 12. Actual Ray Count: 17x12

However, if size is set to (1.3, 1.6), the calculation works as expected (17x14), highlighting the inconsistency

(this->size[1] / resolution) + 1

In floating-point arithmetic, 1.2 / 0.1 may result in 11.99999.... When this is implicitly cast to an int, it is truncated to 11 instead of 12. Adding 1 results in 12, missing the expected count by 1.

Suggested Fix Use std::round to ensure the division result is rounded to the nearest integer before casting, or use a small epsilon.

Fix

in ray_caster_plugin.cc

int nray = ((int)(size[0] / resolution[0]) + 1) *
               ((int)(size[1] / resolution[0]) + 1);

that is

int nray = (static_cast<int>(std::round(size[0] / resolution[0])) + 1) *
               (static_cast<int>(std::round(size[1] / resolution[0])) + 1);

Apply similar fix in RayCaster.cpp

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions