-
Notifications
You must be signed in to change notification settings - Fork 6
Introduce _benchmarks_ #27
Copy link
Copy link
Open
Labels
featureA new feature to the codebase (this correlates with MINOR in Semantic Versioning).A new feature to the codebase (this correlates with MINOR in Semantic Versioning).
Description
Metadata
Metadata
Assignees
Labels
featureA new feature to the codebase (this correlates with MINOR in Semantic Versioning).A new feature to the codebase (this correlates with MINOR in Semantic Versioning).
Projects
StatusShow more project fields
P2
Benchmarks aim to measure the performance of an agent for a specific problem.
They are pre-set experiments, with a pre-defined list environments to run on, and pre-set evaluation metrics.
What changes, and also the input to a benchmark, is the
Agent.