Skip to content

New stdnorm calc + weights onchain#747

Merged
xmariachi merged 24 commits intodevfrom
diego/engn-3462-fixing-normalization-change-outlier-problem
Feb 21, 2025
Merged

New stdnorm calc + weights onchain#747
xmariachi merged 24 commits intodevfrom
diego/engn-3462-fixing-normalization-change-outlier-problem

Conversation

@xmariachi
Copy link
Contributor

@xmariachi xmariachi commented Feb 14, 2025

Purpose of Changes and their Description

This PR implements the rework on weights that fixes the latest outlier issues observed from experience and provides a cutoff on low weights for the stddev.

  • Adds stores for latestStdnorm and latestWeights
  • regrets returned on calculation
  • At CloseReputerNonce, calculate stdnorm based on old weights, store it and recalculate weights.
  • stdnorm used as normalization for all weights calculation cases.
  • refactor and additional tests on weights.
  • emits events on new stdnorm set, and on each new actor weight
  • autocli query addition of latest stdnorm, latest inferer weight and latest forecaster weight
  • adds v0.9.0 migration

Link(s) to Ticket(s) or Issue(s) resolved by this PR

Are these changes tested and documented?

  • If tested, please describe how. If not, why tests are not needed.
    Tested with allora-simulator. The expectations from research are met.
  • If documented, please describe where. If not, describe why docs are not needed.
    Internal implementation details, no need to post to the user.
  • Added to Unreleased section of CHANGELOG.md?

Still Left Todo

  • OPTIONAL: Optimization on weights from other actors taken from the latest ones. Implement a function that fills these weights from the store.
    • instead of calling "CalcWeightsGivenWorkers", take them from the store - however they will be different in the one-out cases because the elements differ.

@xmariachi xmariachi requested a review from kpeluso February 14, 2025 20:55
@github-actions
Copy link

github-actions bot commented Feb 14, 2025

The latest Buf updates on your PR. Results from workflow Buf Linter / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed⏩ skippedFeb 21, 2025, 4:09 AM

@xmariachi xmariachi changed the base branch from dev to release-v0.9.0 February 17, 2025 19:08
@xmariachi xmariachi changed the base branch from release-v0.9.0 to dev February 18, 2025 22:46
@xmariachi xmariachi force-pushed the diego/engn-3462-fixing-normalization-change-outlier-problem branch from 94c489d to 44f4640 Compare February 18, 2025 23:01
@xmariachi xmariachi marked this pull request as ready for review February 18, 2025 23:02
@xmariachi xmariachi changed the title WIP - stdnorm and weights onchain New stdnorm calc + weights onchain Feb 18, 2025
Copy link
Contributor

@amimart amimart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job thx!
I've made some remarks, the main point is to introduce a new migration for the next software upgrade to v0.9.0.

Copy link
Contributor

@kpeluso kpeluso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments + Upgrade Flow then gtg

xmariachi and others added 6 commits February 20, 2025 12:57
more efficient allocation of arrays

Co-authored-by: Arnaud Mimart <33665250+amimart@users.noreply.github.com>
Signed-off-by: Diego C <xmariachi@gmail.com>
@xmariachi xmariachi force-pushed the diego/engn-3462-fixing-normalization-change-outlier-problem branch from 1ce5007 to 93d82b9 Compare February 20, 2025 15:16
kpeluso
kpeluso previously approved these changes Feb 20, 2025
kpeluso
kpeluso previously approved these changes Feb 20, 2025
Copy link
Contributor

@amimart amimart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@fernandofcampos fernandofcampos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great job, especially with the test coverage.

@xmariachi xmariachi merged commit 4bde6e3 into dev Feb 21, 2025
10 checks passed
@xmariachi xmariachi deleted the diego/engn-3462-fixing-normalization-change-outlier-problem branch February 21, 2025 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants