Skip to content
This repository was archived by the owner on Oct 4, 2026. It is now read-only.

Fix reversed LossFunctions arguments in regression training - #9

Merged
mmikhasenko merged 1 commit into
mainfrom
codex/fix-loss-argument-order
Oct 4, 2026
Merged

mmikhasenko merged 1 commit into
mainfrom
codex/fix-loss-argument-order

Conversation

@mmikhasenko

Copy link
Copy Markdown
Member

Problem and change

L2DistLoss() training passes the target as the prediction to LossFunctions, reversing its gradient: a single unregularized update on constant target 3 predicts −3 instead of 3. Other losses with prediction-dependent curvature also receive incorrect Hessians.

Pass (prediction, target) to the generic LossFunctions deriv and deriv2 adapters. Keep specialized adapters for the existing target-first LogitLogLoss methods, preserving the built-in logistic behavior and public derivative API.

Add coverage for constant-target and multi-round regression, weighted regularized leaf values, Poisson gradients/Hessians, and logistic compatibility, plus an unreleased changelog entry.

Addresses the derivative-order finding in #8. The separate warm-start, child-threshold, and weight-validation findings are outside this PR.

Validation

  • Reproduced six derivative/Hessian failures with the new tests before the fix.
  • julia --startup-file=no --project=. test/runtests.jl: all 98 assertions pass on Julia 1.11.5 with LossFunctions 1.1.0, including the existing logistic/XGBoost-reference checks.
  • Fixed native L2DistLoss() predictions match all eight saved EvoTrees cases (1/10 rounds, uniform/nonuniform weights, L2 penalties 0/2), with maximum absolute difference 1.7953e-7 against EvoTrees commit 0d83f261e0a32f506ecf1cc144ff2a939c67d1ae.
  • The comparison data and methodology are linked in issue #8. No custom MSE workaround was used for the fixed-native comparison.

@mmikhasenko
mmikhasenko merged commit bbdb260 into main Oct 4, 2026
1 check passed
@mmikhasenko
mmikhasenko deleted the codex/fix-loss-argument-order branch October 4, 2026 11:31

Copy link
Copy Markdown
Member Author

Maintainer decision following the EvoTrees comparison in #8: merge this correctness fix, then archive this JLBoost fork and use EvoTrees for future work.

The comparison showed matching controlled logistic calculations, substantially faster EvoTrees training in the measured workloads, and broader functionality. JLBoost's exact split search, explicit tree objects, and convenient per-tree reweighting remain useful reference material, but do not justify maintaining a separate general-purpose boosting library for our needs.

This PR fixes the LossFunctions argument-order bug before archival. The separate warm-start, child-threshold, and weight-validation findings documented in #8 remain unresolved; merging this PR should not be read as resolving those issues. The comparison report, reproduction scripts, and raw results linked from #8 will remain available as a reference.

The maintainer will perform the merge and repository archival.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant