Skip to content

Add one out forecast-implied inference in network inference bundle#772

Merged
guilherme-brandao merged 8 commits intodevfrom
guilherme/engn-3553-calcnetworkinferences-calc-and-populate
Mar 27, 2025
Merged

Add one out forecast-implied inference in network inference bundle#772
guilherme-brandao merged 8 commits intodevfrom
guilherme/engn-3553-calcnetworkinferences-calc-and-populate

Conversation

@guilherme-brandao
Copy link
Contributor

@guilherme-brandao guilherme-brandao commented Mar 18, 2025

Purpose of Changes and their Description

This PR adds a new feature to calculate forecast-implied inferences when each inferer is removed from the network. These values are critical for evaluating the influence of individual inferers on forecasters' implied predictions.

  • Added GetOneOutInfererForecastImpliedInferences function to calculate what each forecaster's implied inference would be when each inferer is removed
  • Updated CalcNetworkInferences to include these values in the network inference bundle
  • Used existing OneOutInfererForecasterValues type to maintain consistent structure with other one-out calculations

Performance Related Changes:

  • The GetNetworkInferences function employs an in-memory cache that temporarily stores results of expensive mathematical operations (in this case, just Exp) to avoid redundant calculations during network inference processing; the cache is enabled at the function's start, utilized during calculations, and then intentionally disabled and cleared upon completion.
  • Other minor optimizations.

Link(s) to Ticket(s) or Issue(s) resolved by this PR

Are these changes tested and documented?

  • If tested, please describe how. If not, why tests are not needed.
    ✅ Added new unit tests
  • If documented, please describe where. If not, describe why docs are not needed. -- Added Data Submission section to ./README.md.
  • Added to Unreleased section of CHANGELOG.md?

Copy link
Contributor

@xmariachi xmariachi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contains a triple-nested for.
I am concerned about the time this will take when executing in an endBlocker.
Could we time-measure this by having a full max inferers and max forecasters test?
If this has an impact we want to know ... it could add some non-negligible time to the block
cc @spooktheducks @amimart

@amimart
Copy link
Contributor

amimart commented Mar 19, 2025

it could add some non-negligible time to the block

It seems to be used on queries only so it should not impact block production

Copy link
Contributor

@amimart amimart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've let some time/alloc optimization remarks to see if we can improve it

Copy link
Contributor

@amimart amimart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments that may be great to discuss

@guilherme-brandao guilherme-brandao force-pushed the guilherme/engn-3553-calcnetworkinferences-calc-and-populate branch from cd50837 to 42a2a38 Compare March 26, 2025 15:05
Copy link
Contributor

@amimart amimart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@guilherme-brandao guilherme-brandao merged commit 4d563f0 into dev Mar 27, 2025
9 checks passed
@guilherme-brandao guilherme-brandao deleted the guilherme/engn-3553-calcnetworkinferences-calc-and-populate branch March 27, 2025 10:06
@xmariachi
Copy link
Contributor

it could add some non-negligible time to the block

It seems to be used on queries only so it should not impact block production

I think this was left unreplied, but just for completeness: this is internalized into endblocker in another PR

@xmariachi xmariachi mentioned this pull request Mar 27, 2025
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants