Skip to content
This repository was archived by the owner on Oct 4, 2026. It is now read-only.

Preserve warm-start baselines across boosting rounds - #11

Merged
mmikhasenko merged 1 commit into
mainfrom
codex/fix-warm-start
Oct 4, 2026
Merged

mmikhasenko merged 1 commit into
mainfrom
codex/fix-warm-start

Conversation

@mmikhasenko

Copy link
Copy Markdown
Member

Problem and change

Training uses warm_start in round one but drops it from subsequent rounds. With a nonzero baseline, later trees therefore optimize derivatives at the wrong score.

Include the baseline alongside accumulated tree predictions on every round, in both full-data and row-sampled training. For sampled rows, select the corresponding baseline entries. The fitted model continues to return only the learned correction; the docstring now explains how to add the external baseline for full predictions.

Add regression tests with row-varying baselines: an independent weighted logistic Newton oracle and least-squares equivalence to fitting residual targets, covering 1/2/4 rounds, full and 50% sampled data, observation weights, regularization, and preservation of the input baseline.

Addresses the warm-start finding in #8. Merge this fix before the retirement documentation in #10.

Validation

  • New logistic tests reproduced failures at rounds two and four before the fix.
  • Full suite: 128 assertions pass on Julia 1.11.5.
  • Against the saved outputs from pinned EvoTrees commit 0d83f261e0a32f506ecf1cc144ff2a939c67d1ae, the original baseline-0.7 comparison now has maximum probability differences of 4.21e-8 after one round and 5.40e-8 after two rounds (previously 0.03527 after two).
  • Patch whitespace checked with CRLF-aware git diff --check.

@mmikhasenko
mmikhasenko merged commit e18e0a0 into main Oct 4, 2026
1 check failed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant