The methods for symbolic regression (SR) have come a long way since the days of Koza-style genetic programming (GP).
Our goal with this project is to keep a living benchmark of modern symbolic regression, in the context of state-of-the-art ML methods.
Currently these are the challenges, as we see it:
- Lack of cross-pollination between the GP community and the ML community (different conferences, journals, societies etc)
- Lack of strong benchmarks in SR literature (small problems, toy datasets, weak comparator methods)
- Lack of a unified framework for SR, or GP
We are addressing the lack of pollination by making these comparisons open source, reproduceable and public, and hoping to share them widely with the entire ML research community. We are trying to address the lack of strong benchmarks by providing open source benchmarking of many SR methods on large sets of problems, with strong baselines for comparison. To handle the lack of a unified framework, we've specified minimal requirements for contributing a method to this benchmark: a scikit-learn compatible API.
The current edition of the benchmark (SRBench 2025, reported in our call for action paper) evaluates 25 symbolic regression methods under a unified experimental setup: every method runs from a docker container, with hyperparameter tuning and 30 independent runs per dataset, on 24 datasets from PMLB plus a set of first-principles regression problems. This roster includes the 14 methods from the original SRBench together with the methods staged since then.
Methods currently benchmarked:
| Method | ||
|---|---|---|
| AFP - paper | AFP_fe | AFP_ehc - paper |
| Bingo - paper | Brush - paper | BSR - paper |
| E2E - paper | EPLEX - paper | EQL - paper |
| FEAT - paper | FFX - paper | Genetic Engine - paper |
| GPGomea - paper | GPlearn - paper | GPZGD - paper |
| ITEA - paper | NeSymRes - paper | Operon - paper |
| Ps-Tree - paper | PySR - paper | Qlattice - paper |
| Rils-rols - paper | TIR - paper | TPSR - paper |
| uDSR - paper |
The full experiment code and results for this edition live on the srbench_2025 branch (raw results in results/).
If you are choosing baselines for a new symbolic regression paper, please use this roster and these results rather than the 2021 tables below.
Black-box regression (median and 95% confidence interval across datasets):
First-principles problems (accuracy-size Pareto fronts; the star marks the known first-principles expression evaluated on the same data):
The original published edition of the benchmark (NeurIPS 2021) consists of 14 symbolic regression methods, 7 other ML methods, and 252 datasets from PMLB, including real-world and synthetic datasets from processes with and without ground-truth models.
The results pages on the project website currently correspond to this edition; its raw results are the feather files in results/ on this branch.
Methods benchmarked in v2.0:
-
Age-Fitness Pareto Optimization (Schmidt and Lipson 2009) paper , code
-
Age-Fitness Pareto Optimization with Co-evolved Fitness Predictors (Schmidt and Lipson 2009) paper , code
-
Deep Symbolic Regression (Petersen et al. 2020) paper , code
-
Feature Engineering Automation Tool (La Cava et al. 2017) paper , code
-
epsilon-Lexicase Selection (La Cava et al. 2016) paper , code
-
GP-based Gene-pool Optimal Mixing Evolutionary Algorithm (Virgolin et al. 2017) paper , code
-
gplearn (Stephens) code
-
Interaction-Transformation Evolutionary Algorithm (de Franca and Aldeia, 2020) paper , code
-
Semantic Backpropagation GP (Virgolin et al. 2019) paper , code
We are actively updating and expanding this benchmark. Want to add your method? See our Contribution Guide.
We made available all of our experiments' results as feather files: the 2021 (v2.0) results inside /results/ on this branch, and the 2025 results inside /results/ on the srbench_2025 branch.
Check out CONTRIBUTING.md file to see how to set up your algorithm. This guide will detail all the requirements in order to submit a pull request with a compatible interface with SRBench.
The analyze file is the main entry point as it will parse the flags and create specific python commands to run each experiment independently. Some examples on how to invoke the experiments are available at the docs/user_guide.md.
Note on Git LFS: This repository uses Git Large File Storage (LFS) for storing large dataset files. GitHub provides sufficient free bandwidth and storage per month for GitHub Free accounts; additional bandwidth requires a paid plan. If you have a GitHub Student account, you may see larger capacities. If you don't need the actual results files, you can clone without LFS using:
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/cavalab/srbench.gitA detailed guide on how to reproduce the experiments by yourself is provided in docs/user_guide.md.
Once you get all the results, you nee to collate the results using the collate scripts in ./postprocessing/scripts
The current (2025) edition of the benchmark, including all
Imai Aldeia, G. S., Zhang, H., Bomarito, G., Cranmer, M., Fonseca, A., Burlacu, B., La Cava, W., and de França, F. 2025. Call for Action: towards the next generation of symbolic regression benchmark. Proceedings of the Genetic and Evolutionary Computation Conference Companion
v2.0 of the benchmark was reported in our Neurips 2021 paper:
La Cava, W., Orzechowski, P., Burlacu, B., de França, F. O., Virgolin, M., Jin, Y., Kommenda, M., & Moore, J. H. (2021). Contemporary Symbolic Regression Methods and their Relative Performance. Neurips Track on Datasets and Benchmarks.
v1.0 was reported in our GECCO 2018 paper:
Orzechowski, P., La Cava, W., & Moore, J. H. (2018). Where are we now? A large benchmark study of recent symbolic regression methods. GECCO 2018.
William La Cava (@lacava), william dot lacava at childrens dot harvard dot edu