Repository navigation
Fix reversed LossFunctions arguments in regression training - #9
Conversation
|
Maintainer decision following the EvoTrees comparison in #8: merge this correctness fix, then archive this JLBoost fork and use EvoTrees for future work. The comparison showed matching controlled logistic calculations, substantially faster EvoTrees training in the measured workloads, and broader functionality. JLBoost's exact split search, explicit tree objects, and convenient per-tree reweighting remain useful reference material, but do not justify maintaining a separate general-purpose boosting library for our needs. This PR fixes the LossFunctions argument-order bug before archival. The separate warm-start, child-threshold, and weight-validation findings documented in #8 remain unresolved; merging this PR should not be read as resolving those issues. The comparison report, reproduction scripts, and raw results linked from #8 will remain available as a reference. The maintainer will perform the merge and repository archival. |
Problem and change
L2DistLoss()training passes the target as the prediction to LossFunctions, reversing its gradient: a single unregularized update on constant target 3 predicts −3 instead of 3. Other losses with prediction-dependent curvature also receive incorrect Hessians.Pass
(prediction, target)to the generic LossFunctionsderivandderiv2adapters. Keep specialized adapters for the existing target-firstLogitLogLossmethods, preserving the built-in logistic behavior and public derivative API.Add coverage for constant-target and multi-round regression, weighted regularized leaf values, Poisson gradients/Hessians, and logistic compatibility, plus an unreleased changelog entry.
Addresses the derivative-order finding in #8. The separate warm-start, child-threshold, and weight-validation findings are outside this PR.
Validation
julia --startup-file=no --project=. test/runtests.jl: all 98 assertions pass on Julia 1.11.5 with LossFunctions 1.1.0, including the existing logistic/XGBoost-reference checks.L2DistLoss()predictions match all eight saved EvoTrees cases (1/10 rounds, uniform/nonuniform weights, L2 penalties 0/2), with maximum absolute difference 1.7953e-7 against EvoTrees commit0d83f261e0a32f506ecf1cc144ff2a939c67d1ae.