Refactor metrics #18
Description
There are some issues with the current metrics implementation.
Some thoughts:
- single estimate metrics / dataset metrics
- dataset metrics are typically defined by a single-estimate metric and aggregation function
- some metrics are composed of other metrics
- for some metrics bigger is better, for other smaller is better
- support swiping single-estimate metric parameters (like a threshold) and automatic plotting
- symmetry handling varies per-metric
Currently most of this is supported for the standard metrics, but it's quite messy.