Skip to content

Conversation

JoelSpeed
Copy link
Contributor

This adds an initial AGENTS.md configuration for how to API review via an AI agent such as claude.

It also implements a /api-review command that can be used locally to review PRs for anyone who has claude installed.

I hope we can get folks using this to self help, but my long term goal is to integrate this into coderabbit or some other review tool that can post the comments directly on the PR.

As an example of the output, see #2488 (comment)

Currently highlights of its instructions:

  • Explaining valid values
  • Ensuring markers are documented in the godoc
  • Identifying cross field validations that aren't enforced

Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

Hello @JoelSpeed! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

@openshift-ci openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 19, 2025
Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign joelspeed for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

@everettraven everettraven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some general questions for learning purposes.

Nothing worth blocking this on IMO, especially if you are finding it useful.

**Explanation:** [Why this change is needed]


I'll run a comprehensive API review for the OpenShift API changes in the specified GitHub PR.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like something an LLM would spit out when you ask for a review - is it necessary to include this in the instructions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this whole section looks like it is something an LLM spit out as an execution plan. Should something like this be hand-rolled with explicit instructions on how to conduct the review and important considerations?

I guess my curiosity here is if we made an explicit guidelines type document that humans could follow, an LLM should be able to follow along relatively easily and we can potentially enforce more nuanced guardrails.

Not worth blocking this on - more so asking questions for myself as I've not worked with LLMs in this capacity before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an artefact of how these documents are built. Using Claude locally, I've given it text based prompts, and it has converted that into instructions that it can read. So 95% of this document is it translating my instructions and feedback into rules that it can later apply.

Copy link
Contributor

@everettraven everettraven Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. That is interesting. So this file is automatically updated with more detailed instructions by Claude as the AGENTS.md file is updated?

If you update this file by hand to "improve it", what happens?



I'll run a comprehensive API review for the OpenShift API changes in the specified GitHub PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to create a separate H1 heading to explain to the LLM that this is the steps for how to actually conduct the review?

### Testing
```bash
make test-unit # Run unit tests
make integration # Run integration tests (in tests/ directory)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we teach it how to run the integration tests with more focused arguments?

That way running the integration tests don't take longer than necessary when reviewing a change that only impacts a subset of the APIs/tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a good idea, I'll have a go at that

Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

@JoelSpeed: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants