Skip to content

docs: add evals documentation, update commands reference and configuration guide#572

Merged
tejaskash merged 1 commit intoaws:mainfrom
notgitika:docs/eval-documentation
Mar 19, 2026
Merged

docs: add evals documentation, update commands reference and configuration guide#572
tejaskash merged 1 commit intoaws:mainfrom
notgitika:docs/eval-documentation

Conversation

@notgitika
Copy link
Contributor

@notgitika notgitika commented Mar 19, 2026

Summary

  • Add docs/evals.md — comprehensive guide covering evaluators, on-demand evals, online eval configs, rating scales, placeholders, sampling rate, builtin evaluators, standalone (ARN) mode, and common patterns (CI/CD quality gates)
  • Update docs/commands.md — add command reference for add evaluator, add online-eval, run evals, evals history, pause/resume online-eval, logs evals
  • Update docs/configuration.md — add evaluator and online eval config schema documentation, update project fields table and sample agentcore.json
  • Update README.md — add Evaluations to capabilities, commands table, documentation links, and project structure description

Test plan

  • Verify all doc links resolve correctly
  • Verify CLI examples match actual command flags
  • Verify schema examples match current Zod schemas

@notgitika notgitika requested a review from a team March 19, 2026 15:56
@github-actions github-actions bot added the size/l PR size: L label Mar 19, 2026
@notgitika notgitika changed the title docs: add evaluation feature documentation docs: add evals documentation, update commands reference and configuration guide Mar 19, 2026
Copy link
Contributor

@tejaskash tejaskash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tejaskash tejaskash merged commit df58b41 into aws:main Mar 19, 2026
19 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l PR size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants