What feature would you like to see?
Many tasks have multiple objectives which may be complementary, or a trade-off. How can we allow metrics to return such subscores, and have optimizers optimize them?
This will require thinking about:
- What is the right interface for the user-defined metric to return multiple scores
- How can the user specify how different sub-scores should be aggregated? Arithmetic/Geometric mean, weighted avg, etc.
- How should scores across instances be aggregated?
Would you like to contribute?
Additional Context
Relevant discussion: https://discord.com/channels/1161519468141355160/1161519469319946286/1407399169517420718
Upstream GEPA issue: gepa-ai/gepa#2