You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 4, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Oct 4, 2026. It is now read-only.
EvoTrees comparison: results and reproducible scripts
Compared JLBoost at 06d9924 with the requested EvoTrees commit 0d83f261 (version 0.18.8), on 2026-09-29.
EvoTrees uses gradient/Newton boosting, not classical AdaBoost. The controlled logistic calculations match, including positive observation weights. The packages are not interchangeable under their defaults, and the comparison exposed correctness issues in this JLBoost fork.
Maximum probability difference below 3.4e-8 on controlled discrete-feature data
Additional stump and matched gamma cases
Differences below 3.9e-8
Positive integer weights vs duplicated observations
Agree within numerical tolerance in both packages
All positive weights scaled by 10, with penalties/threshold effects disabled
Predictions unchanged within numerical tolerance
Native L2DistLoss() regression
Does not match: reversed gradient arguments in JLBoost
Diagnostic custom MSE with correctly ordered arguments
Eight controlled configurations agree within 1.8e-7
Nonzero warm start
One round agrees; after two rounds the maximum residual-probability difference is 0.03527
On continuous features, predictions are not pointwise identical: EvoTrees uses histogram boundaries, while JLBoost searches exact distinct-value splits. Held-out logistic quality was very similar in the tested datasets.
Correctness findings
Regression derivative argument order.g_h.jl calls deriv(loss, target, prediction), while LossFunctions 1.1.0 expects deriv(loss, output, target). With a constant feature, target 3, one round, eta 1, and no penalty, L2DistLoss() predicts −3; EvoTrees predicts 3. JLBoost's own target-first LogitLogLoss is unaffected. The custom MSE in the bundle is a diagnostic, not a library fix.
Warm-start baseline is dropped after the first round.jlboost-fit.jl evaluates later rounds using only prior-tree predictions, omitting the original baseline.
Child threshold only checks the left child.find_best_split.jl checks the selected split's left Hessian, without checking the right child or choosing the best eligible alternative.
Weight validation is incomplete. JLBoost accepts negative weights; all-zero weights produced nonfinite predictions. EvoTrees rejects any nonpositive weight at this commit. JLBoost accepts some zero weights, but this does not universally guarantee equivalence to deleting those rows.
All 63 existing JLBoost test assertions pass, so these cases reveal coverage gaps. Library source was left unchanged.
Timing and quality
Apple M2, Julia 1.11.5, one Julia compute thread, CPU only. Eight continuous features, positive nonuniform weights, 20 rounds, path depth three, no sampling. Median of three warmed repetitions, packages run sequentially. Fit includes initialization/binning; input-table construction is excluded. EvoTrees uses 64 bins and JLBoost exact splits.
Logistic training rows
JLBoost fit
EvoTrees fit
EvoTrees speedup
JLBoost held-out log loss
EvoTrees held-out log loss
1,000
31.7 ms
5.22 ms
6.1×
0.569373
0.569025
10,000
238 ms
14.5 ms
16.4×
0.557754
0.557357
50,000
1.276 s
60.3 ms
21.2×
0.555512
0.555266
For the 50k-trained model, prediction on 2,000 held-out rows took 0.358 ms for JLBoost vs 0.740 ms for EvoTrees. This small-batch result should not be extrapolated to large-batch inference. Logistic allocation volume per fit was about 2.33 GB vs 11.4 MB, respectively; these are cumulative allocations, not peak memory.
Native JLBoost regression RMSE was 8.4–8.7 because of the derivative bug. The diagnostic custom MSE gave RMSE 0.747–0.759, compared with EvoTrees' 0.751–0.764. Native regression timings therefore do not represent equal-quality work. Full timings and raw samples are in the bundle. These are synthetic, single-seed CPU measurements, not GPU/thread-scaling or production-data benchmarks.
Functionality, API, and documentation
Area
JLBoost fork
EvoTrees at the pinned commit
Objectives
Binary logistic; generic LossFunctions adapter currently affected by the bug above
Regression objectives, binary/multiclass classification, multiple targets and distributional objectives
Features
Numeric/Boolean; missing and categorical features unsupported per README
Numeric/Boolean and categorical features; missing features require preprocessing
Tree growth
Depth-wise or best-first max_leaves
Binary depth-wise or oblivious/symmetric trees
Training controls
Row/column sampling; explicit tree scaling
Sampling, early stopping, monotonic constraints, bagging, explicit seed
Integration/compute
Tables API; serial implementation; old MLJ wrapper archived
Tables/matrix APIs, in-package MLJ, threaded CPU and GPU extensions
Observation weights
weights=w; multiplies gradients and Hessians
weight_name or w_train; same gradient/Hessian scaling
Inspection
Node objects, gain/coverage/frequency, per-tree eta wrapper
Array-based trees, gain importance, plotting, TreeSHAP
Documentation
Concise README and Markdown tutorials; hosted Quarto site planned
Broader stable/dev documentation and tutorials, but some examples have stale keywords
Capabilities outside the shared logistic/MSE paths were source-reviewed, not all runtime-tested. EvoTrees documentation issues include README fnames instead of feature_names, overview examples using rng instead of seed, and assigning dtrain.y .= ... before creating the column.
Required parameter translation: JLBoost max_depth=d corresponds to EvoTrees max_depth=d+1; JLBoost lambda=λ corresponds to EvoTrees L2=λ, lambda=0; JLBoost gamma=γ corresponds to EvoTrees gamma=γ/2 for the shared objectives. JLBoost returns raw logistic margins; EvoTrees returns probabilities. Neutral initialization must be requested in EvoTrees, whose default bias is computed from the unweighted targets. Child thresholds have different units: Hessian mass vs observation-weight mass.
Dependency compatibility: these versions do not resolve together in one clean environment: SortingLab 0.3 requires CategoricalArrays 0.10, while EvoTrees requires CategoricalArrays 1. The bundle runs them in separate processes and environments without overriding compatibility.
Activity
mmikhasenko commentedon Sep 29, 2026
EvoTrees comparison: results and reproducible scripts
Compared JLBoost at
06d9924with the requested EvoTrees commit0d83f261(version 0.18.8), on 2026-09-29.EvoTrees uses gradient/Newton boosting, not classical AdaBoost. The controlled logistic calculations match, including positive observation weights. The packages are not interchangeable under their defaults, and the comparison exposed correctness issues in this JLBoost fork.
Scripts, report, and raw results
Downloadable reproduction bundle on GitHub Gist — use Download ZIP on the Gist page. It contains:
compare.jl: controlled numerical checks and warmed training/prediction benchmarks.diagnostics.jl: independent weighted-stump oracle and minimal regression reproduction.summarize.py: validates agreement/invariants and generates the measurement tables.README.md: setup and reproduction commands, with pinned source revisions.COMPARISON.md: full functionality, weights, API, documentation, and correctness review.measurements.md, raw result TOMLs, and dependency-manifest snapshots.Do the results match?
L2DistLoss()regressionOn continuous features, predictions are not pointwise identical: EvoTrees uses histogram boundaries, while JLBoost searches exact distinct-value splits. Held-out logistic quality was very similar in the tested datasets.
Correctness findings
g_h.jlcallsderiv(loss, target, prediction), while LossFunctions 1.1.0 expectsderiv(loss, output, target). With a constant feature, target 3, one round, eta 1, and no penalty,L2DistLoss()predicts −3; EvoTrees predicts 3. JLBoost's own target-firstLogitLogLossis unaffected. The custom MSE in the bundle is a diagnostic, not a library fix.jlboost-fit.jlevaluates later rounds using only prior-tree predictions, omitting the original baseline.find_best_split.jlchecks the selected split's left Hessian, without checking the right child or choosing the best eligible alternative.All 63 existing JLBoost test assertions pass, so these cases reveal coverage gaps. Library source was left unchanged.
Timing and quality
Apple M2, Julia 1.11.5, one Julia compute thread, CPU only. Eight continuous features, positive nonuniform weights, 20 rounds, path depth three, no sampling. Median of three warmed repetitions, packages run sequentially. Fit includes initialization/binning; input-table construction is excluded. EvoTrees uses 64 bins and JLBoost exact splits.
For the 50k-trained model, prediction on 2,000 held-out rows took 0.358 ms for JLBoost vs 0.740 ms for EvoTrees. This small-batch result should not be extrapolated to large-batch inference. Logistic allocation volume per fit was about 2.33 GB vs 11.4 MB, respectively; these are cumulative allocations, not peak memory.
Native JLBoost regression RMSE was 8.4–8.7 because of the derivative bug. The diagnostic custom MSE gave RMSE 0.747–0.759, compared with EvoTrees' 0.751–0.764. Native regression timings therefore do not represent equal-quality work. Full timings and raw samples are in the bundle. These are synthetic, single-seed CPU measurements, not GPU/thread-scaling or production-data benchmarks.
Functionality, API, and documentation
max_leavesweights=w; multiplies gradients and Hessiansweight_nameorw_train; same gradient/Hessian scalingetawrapperCapabilities outside the shared logistic/MSE paths were source-reviewed, not all runtime-tested. EvoTrees documentation issues include README
fnamesinstead offeature_names, overview examples usingrnginstead ofseed, and assigningdtrain.y .= ...before creating the column.Required parameter translation: JLBoost
max_depth=dcorresponds to EvoTreesmax_depth=d+1; JLBoostlambda=λcorresponds to EvoTreesL2=λ, lambda=0; JLBoostgamma=γcorresponds to EvoTreesgamma=γ/2for the shared objectives. JLBoost returns raw logistic margins; EvoTrees returns probabilities. Neutral initialization must be requested in EvoTrees, whose default bias is computed from the unweighted targets. Child thresholds have different units: Hessian mass vs observation-weight mass.Dependency compatibility: these versions do not resolve together in one clean environment: SortingLab 0.3 requires CategoricalArrays 0.10, while EvoTrees requires CategoricalArrays 1. The bundle runs them in separate processes and environments without overriding compatibility.