Skip to content

ci: log workflow results and write GitHub job summaries - #988

Open
Saibamen wants to merge 1 commit into
boykopovar:mainfrom
Saibamen:ci/workflow-job-summary
Open

Saibamen wants to merge 1 commit into
boykopovar:mainfrom
Saibamen:ci/workflow-job-summary

Conversation

@Saibamen

@Saibamen Saibamen commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make maintenance and progress GitHub Actions easier to audit: log what each run did and mirror key lines in the Job summary tab, without extra API calls.

  • PR overlap (pr-overlap.yml, tools/pr_overlap.py): print open-PR scan totals; log comment create/update/delete/unchanged/clean; job summary sync table.
  • Progress comment (progress-comment.yml): log when a Progress report run is stale (PR head moved since workflow_run.head_sha); skip PATCH when the existing bot comment body is unchanged (same pattern as PR overlap); log unchanged comment.
  • Progress report (progress-report.yml): job summary when compare is empty (No library/shader progress changes in this diff).
  • Progress / Pages (progress.yml): tee progress.py output with pipefail; copy libraries / shaders lines into job summary.

Test plan

  • Open a PR and confirm Progress report job summary shows either the diff table or the empty-diff message.
  • After a successful Progress report run, confirm Progress comment logs unchanged comment or skipped stale run (not full overlap-style sync counters).
  • Run PR overlap via workflow_dispatch and confirm log lines + job summary counts.
  • Dispatch Progress on the branch and confirm job summary lists library/shader percentages (fork Pages deploy may fail; grep step is the evidence).

What

CI observability: workflow logs and $GITHUB_STEP_SUMMARY for PR overlap, progress report, progress comment, and Pages progress publish.

Behaviour (intentional): Progress comment no longer PATCHes GitHub when the composed body matches the existing github-actions[bot] comment (reduces API churn; mirrors PR overlap unchanged handling).

Report files keep explicit UTF-8 encoding from merged #2134 (write_text(..., encoding="utf-8")).

Tested

Head: 4973a9c64865299846e3ebd50d708b3ffd2f0943 (rebased onto upstream main).

Workflow Run Evidence
PR overlap https://github.com/Saibamen/AnyPS5/actions/runs/38177930919 pr_overlap: repo=... open_prs=...; pr_overlap sync: created=0 ... clean=1; Job summary table
Progress https://github.com/Saibamen/AnyPS5/actions/runs/38177933167 libraries / shaders lines + Job summary; job fails on fork deploy-pages (expected)
Progress report https://github.com/Saibamen/AnyPS5/actions/runs/38177925075 Log + Job summary: No library/shader progress changes in this diff.
Progress comment https://github.com/Saibamen/AnyPS5/actions/runs/38177996876 progress comment sync: unchanged comment on PR #3 (fork main temporarily FF'd to run this workflow; restored)

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • N/A - no runtime code paths
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • N/A - no third-party code
  • N/A - not game behaviour
  • Depends on: none
  • AI-assisted: yes

@github-actions github-actions Bot added the ci .github label Oct 7, 2026
@Sanjaysrinivas

Sanjaysrinivas commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

I reproduced the empty-report failure on current main (cea1ef13c35044356348bc6221d45f3a80f7df09): the unmatched glob becomes issues/*/comments, then jq fails. The same failure occurred in this fork workflow run.

I tested the PR-overlap shell step at your exact head 09ef50d98941587402e180c2a891b4e349cbae2f on Ubuntu 24.04 under WSL2, using real Bash and jq 1.7 with mocked report generation and GitHub CLI calls:

  • No reports: main fails; this PR succeeds with zero GitHub calls.
  • An empty clean report: succeeds with one comment lookup.
  • A new overlap report: succeeds and selects comment creation.
  • An empty report with an existing stale comment: succeeds and selects deletion.
  • An unchanged overlap report: succeeds without updating the comment.

All five scenarios behaved as expected on this PR. The four scenarios with report files also passed on main, providing controls for the reproduction. No live comment creation, editing or deletion was performed during the checks.

This validates the empty-glob fix and these PR-overlap shell paths. The other workflows and end-to-end GitHub summary presentation were not tested.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Points to address:

  • Mixed topics. shopt -s nullglob and || true in pr-overlap.yml (lines 29-30, 38) change behaviour: they fix the empty-report failure Sanjaysrinivas reproduced. That is a bug fix, not observability, and the description doesn't mention it. Please move it to its own PR (it can land quickly) and keep this one logging-only.
  • No evidence it runs. "Not tested" plus no checks on the branch. Please dispatch PR overlap and Progress on your fork and link the runs showing the log lines and Job summary. The only local check so far (Sanjaysrinivas's comment) covers the pr-overlap step, with mocks. In progress.yml, the grep -E '^libraries |^shaders ' does match what tools/progress.py prints (lines 337-338), but python3 tools/progress.py out | tee runs without pipefail (the default run shell is bash -e), so a failing progress.py is only caught by accident when that grep finds nothing. Please add set -o pipefail.
  • Size vs. value. progress-comment.yml goes from 13 to 56 lines in that step to count six states that are only printed. Please cut it to the stale-run skip line and the unchanged-comment skip, which are the only ones that carry information.
  • Overlap. #1229 and #1325 also touch tools/pr_overlap.py; rebase after they land or confirm there is no conflict.

Workflow behaviour changes outside CONTRIBUTING are boykopovar's call, so the above evidence will help him decide.

@Saibamen

Saibamen commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10

MarcZX10 commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

I reviewed the overlap with Marc's #2134. The changes address separate behavior:

They can coexist and do not depend on each other. Since both touch the report writer, preserve encoding="utf-8" when the two diffs are combined (for this branch's extracted helper, that means write_text(body, encoding="utf-8")).

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@Saibamen
Saibamen force-pushed the ci/workflow-job-summary branch 2 times, most recently from 3dd7b70 to 6b3000d Compare October 11, 2026 13:22
@Saibamen

Copy link
Copy Markdown
Contributor Author

Rebased

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 11, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase. Remaining points:

  • Description vs diff. The summary says Progress comment gets "the same sync counters as overlap", but progress-comment.yml:30-46 only echoes skipped stale run / unchanged comment. There are no created/updated counters and no job summary. Either add them or fix the description.
  • Behaviour change. progress-comment.yml:36-46 now skips the PATCH when the body is unchanged. That is a behaviour change, not logging, and the description doesn't list it as one. Please state it under What (it is fine to keep, pr-overlap already does the same).
  • No evidence for two of the four changes. The linked runs are PR overlap and Progress, on head 1ef9dc62 (the Pages deploy step fails on the fork, as expected). Nothing shows the progress-report.yml:34-41 empty-diff summary or the progress-comment.yml log lines running. Please link a run of each (a PR on your fork is enough) and re-run Progress on the current head, since the pipefail + grep in progress.yml:25-35 fails the publish step if a line is missing.
  • Per the earlier comment by @MarcZX100, keep encoding="utf-8" in the report writer when #2134 lands.

@Saibamen
Saibamen force-pushed the ci/workflow-job-summary branch from 6b3000d to 4973a9c Compare October 11, 2026 21:49
@Saibamen

Copy link
Copy Markdown
Contributor Author

@adriadam10 Addressed the Oct 11 review on head 4973a9c (rebased onto current upstream main).

Description: Progress comment no longer claims overlap-style sync counters; documented the intentional unchanged-body PATCH skip under What.

Evidence (fork, same HEAD):

#2134: write_text(..., encoding="utf-8") still present in tools/pr_overlap.py after rebase.

Please take another look when you have a moment.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 8970 tools/pr_overlap.py
PR 9539 tools/pr_overlap.py
PR 9666 tools/pr_overlap.py

Surface what maintenance and progress workflows did without extra API calls.

- PR overlap: scan summary from pr_overlap.py; comment sync counters and job summary
- Progress comment: stale-run skip and unchanged-comment skip logging
- Progress report: job summary when compare is empty
- Progress (Pages): pipefail on progress.py; copy libraries/shaders lines into job summary

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci .github needs rebase Conflicts with main, rebase needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants