GitHub Annotation Limits ( 10 Errors, 10 Warnings)

[aniaq]

The GitHub annotations system enforces a hard limit of 10 errors/warnings per check run, which can create challenges when running comprehensive checks with a higher number of warnings. This limitation can hinder effective debugging and reviewing for larger projects.

When running the plugin-check-action, only the first 10 errors/ warnings are displayed as annotations. This might lead to missing other important errors/warnings unless users manually check the logs or artifacts for detailed output.

Possible workarounds:

1. Combine multiple errors/warnings into a single annotation message to stay within the 10 error /warning limit while still providing meaningful feedback.
2. Produce an artifact (e.g., HTML or JSON file) containing a full report of all errors / warnings. Users can download this file for in-depth analysis.

Would it make sense to implement one of these workarounds in this action?

Reference: Extremely low annotation limit Thank you for considering this improvement!

[swissspidy]

Thanks for opening this.

When introducing Plugin Check in larger projects it's probably best to start with the plugin and not the GitHub Action, because of the much faster feedback loop. Later on you’ll automatically have fewer annotations. So I'd consider this a bit of an edge case.

As for potential solutions, I think leaving a PR comment is probably best, see #1

Grouping unrelated messages in an unrelated file sounds weird from a UX perspective. An artifact might be a nice addition, but is also easy to miss. If you need a table with all errors, the plugin is the quicker choice.

[aniaq]

We are encountering challenges due to a high number of false positives in our project, making it difficult to review all warnings within a single GitHub Action run. In addition to the time required for multiple workflow runs, the annotation limits are consuming GitHub resources.

Instead of increasing these limits, it would be highly beneficial if the file paths and line numbers were displayed directly in the logs. This would allow us to quickly identify and address issues without relying solely on GitHub annotations.

Would it be possible to modify the output format in the action logs to include file paths and line numbers?

[swissspidy]

Thanks, I'll think about your suggestion.

In the meantime, as I said before:

When introducing Plugin Check in larger projects it's probably best to start with the plugin and not the GitHub Action, because of the much faster feedback loop.

So install the plugin and run the wp plugin check command yourself if you'd like to quickly see all errors across all files.

Then you can fix errors and identify false positives to ignore in following runs.