Skip to content

Reduce precision in outputs to eliminate floating point errors#18

Merged
shawest merged 2 commits intoshawest:masterfrom
mpiercy827:fix-integration-test-precision
Feb 13, 2026
Merged

Reduce precision in outputs to eliminate floating point errors#18
shawest merged 2 commits intoshawest:masterfrom
mpiercy827:fix-integration-test-precision

Conversation

@mpiercy827
Copy link
Contributor

This commit reduces the flakiness of the integration tests introduced in the previous commits by reducing the precision of NeucBOT's outputs. Currently, the outputs are formatted in scientific notation with 12+ digits after the decimal, which is well beyond the precision of these calculations.

Even small changes to the NeucBOT code can create small differences in the outputs in these last decimal places due to errors produced by floating point aritmetic limitations, resulting in flaky integration tests that make it hard to tell if a regression is introduced. This commit solves this problem by making the following changes:

  • Installing the numpy package, which will be used in subsequent work
  • Rounds neucbot.py output values using numpy.format_float_scientific
  • Modifies tests/integration_tests.sh to produce better error messages

This commit reduces the flakiness of the integration tests introduced in
the previous commits by reducing the precision of NeucBOT's outputs.
Currently, the outputs are formatted in scientific notation with 12+
digits after the decimal, which is well beyond the precision of these
calculations.

Even small changes to the NeucBOT code can create small differences in
the outputs in these last decimal places due to errors produced by
floating point aritmetic limitations, resulting in flaky integration
tests that make it hard to tell if a regression is introduced. This
commit solves this problem by making the following changes:

- Installing the numpy package, which will be used in subsequent work
- Rounds neucbot.py output values using numpy.format_float_scientific
- Modifies tests/integration_tests.sh to produce better error messages
@mpiercy827 mpiercy827 marked this pull request as ready for review February 13, 2026 06:38
@shawest shawest merged commit d6d5eb0 into shawest:master Feb 13, 2026
1 check passed
@mpiercy827 mpiercy827 deleted the fix-integration-test-precision branch February 28, 2026 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants