Is this Benchmark Suite abandoned?

[fredericosantos]

Hello, I have seen very little activity on this repository.

Is anyone interested in passing on the torch?

Some improvements that would be nice to see/be discussed:

1. A bring your own optimizer solution for grid_time. Currently it uses GridSearchCV, which is terrible for some datasets that have 6 sample sizes.
2. Clear instructions on time limits: Is the 6h limit per seed? Per dataset?
3. A refactoring of which datasets enter the benchmark. As an example, the 'true' R^2 of many of the datasets is lower than 1; which means finding the 'human true' formula is not obvious.
4. A discussion on using fixed seeds;

[lacava]

Our last push to master was 12 days ago and there are multiple active PRs. So, to answer your title question, no.

1. We did this in 2022 https://cavalab.org/srbench/competition-2022/
2. Depends which version of the experiment you're referring to.
3. That's why we have "black box" and "ground truth" experiments, and experiment with noise.
4. Feel free.

Is anyone interested in passing on the torch?

The torch is already shared. You're welcome to contribute. See https://cavalab.org/srbench/contributing/#ground-rules; read 3 carefully.