Skip to content

Add a zizmor gate for the Actions setup, and clear what it found - #9

Merged
alex-clickhouse merged 1 commit into
mainfrom
ci/zizmor-actions-audit
Aug 19, 2026
Merged

Add a zizmor gate for the Actions setup, and clear what it found#9
alex-clickhouse merged 1 commit into
mainfrom
ci/zizmor-actions-audit

Conversation

@alex-clickhouse

@alex-clickhouse alex-clickhouse commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Wires zizmor into CI as a gate on workflow changes, and fixes everything it reported. Takes this repo from 7 findings to 0.

Every workflow here is reusable, so a mistake in one becomes a mistake in each caller — and these workflows run an AI agent over untrusted PR content with an app token. That puts them in the trust boundary rather than in build glue.

The companion PR is ClickHouse/integrations-ai-playground#422, which does the same for the central pipeline. This repo was already in much better shape: persist-credentials: false was set, the relay ran on permissions: {}, and the permission grants were commented.

What changed

App tokens are scoped. The relay token makes exactly one gh issue create call and the benchmark token dispatches exactly one workflow, but each was minted with every permission the app is installed with. They now declare permission-issues: write and permission-actions: write + permission-contents: read. claude-docs-drift.yml already scoped its own — this brings the other two in line.

Everything is pinned to a commit hash, with .github/dependabot.yml proposing the bumps weekly, grouped, with a 7-day cooldown — auto-bumping a hash means trusting whatever was just published, and a bad release is usually caught within days. Each hash is the commit the floating tag already resolved to, verified per action, so nothing changes behaviour.

Important

One exception: claude-code-action was pinned at v1.0.111 (2026-05-01) and moves to v1.0.195 (2026-08-18) — 178 commits, 117 files. Same major, so no intended breaking changes, but it is a real jump on the action that triages PRs across the client repos. If you would rather land it separately, revert that one line to fefa07e9c665b7320f08c3b525980457f22f58aa # v1.0.111 and Dependabot will offer it next week.

The example callers are audited too

Client repos copy these, so a mistake in one propagates to seven repos. Auditing them turned up two things in caller-claude-pr-triage.yml:

  • It had no permissions: block. A called workflow can only narrow what its caller grants, never widen it. Any repo copying the example into a org/repo whose default token is read-only would have had triage silently fail to apply labels or post its comment. It now declares the three permissions claude-pr-triage.yml actually needs. This is a functional fix for callers, not a lint fix. (caller-claude-docs-drift.yml already had one.)
  • It used secrets: inherit for a cross-repo call. The reusable workflow declares exactly one secret, so the example now names ANTHROPIC_API_KEY and nothing else in a caller's secret store is reachable. The header comment that argued why inherit was acceptable is replaced by the narrower thing it now does.

The @main refs in the example callers stay unpinned on purpose — that is the point of publishing reusable workflows here, and pinning callers to a commit would mean a bump in every client repo for every fix. Recorded in .github/zizmor.yml with that reason.

The gate

.github/workflows/zizmor.yml runs on every pull request, and on pushes to main that touch a workflow, an action, or either config, plus workflow_dispatch. It fails on any finding, so .github/zizmor.yml is the complete list of accepted risks with a reason each — an exception becomes a reviewable diff instead of a warning nobody reads.

It uses pipx run zizmor==1.29.0: pipx ships on the GitHub-hosted runners, so there is no setup action to trust, and the version is pinned so a new zizmor release cannot break unrelated PRs. It reads examples/*.yml by glob so new examples are covered without anyone remembering.

The pin policy here is stricter than the playground's, which allows a tag for anthropics/*. That is deliberate and noted in the config: this repo is a dependency of every client repo.

It is unfiltered on pull_request on purpose: a run that paths skips reports no status, so a filtered job can never be a required check — every PR leaving the workflows alone would wait on a status that never arrives. At ~10s it is cheaper to run every time. That means it is safe to add to branch protection once you have seen it pass. One caveat: if the online audits cannot reach the API, zizmor warns and continues, so the advisory check can degrade quietly.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds zizmor-based CI auditing and hardens reusable workflows against supply-chain and permission risks.

Changes:

  • Adds a zizmor gate with documented exceptions.
  • Pins Actions and configures Dependabot updates.
  • Narrows app-token permissions and caller secret exposure.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
examples/caller-claude-pr-triage.yml Adds explicit permissions and secret forwarding.
.github/zizmor.yml Documents accepted unpinned caller references.
.github/workflows/zizmor.yml Adds the Actions security audit gate.
.github/workflows/cross-repo-bug-relay.yml Pins the token action and limits issue permissions.
.github/workflows/clickhouse-benchmark.yml Pins and scopes the benchmark dispatch token.
.github/workflows/claude-pr-triage.yml Pins Actions and updates secret guidance.
.github/workflows/claude-docs-drift.yml Pins the app-token action.
.github/dependabot.yml Adds grouped, delayed Action updates.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/zizmor.yml Outdated
Every workflow here is reusable, so a mistake in one becomes a mistake in
each caller. These workflows also run an AI agent over untrusted pull
request content with an app token. zizmor reported 7 findings. This commit
clears all of them and adds a CI gate that fails on any new one.

* Give the app tokens explicit `permission-*` inputs. The relay token made
  one `gh issue create` call, and the benchmark token dispatched one
  workflow, but each carried every permission the app is installed with.
  claude-docs-drift.yml already scoped its token.
* Pin every action to a commit hash, and add .github/dependabot.yml to
  propose the bumps every week. It waits 7 days before it offers a
  release, because a bad release is usually found in that time. Each hash
  is the commit that the floating tag already resolved to, so the
  behaviour stays the same. The exception is claude-code-action, which was
  held at v1.0.111 from May and now moves to v1.0.195.
* Pass ANTHROPIC_API_KEY explicitly in the pr-triage example caller, in
  place of `secrets: inherit`. The reusable workflow declares that one
  secret, so a caller does not need to expose the rest of its store to a
  workflow in another repository.
* Add a `permissions:` block to the pr-triage example caller. A called
  workflow can only narrow the permissions of its caller. Without the
  block, a repo that copies the example runs the job with its default
  token, and triage cannot apply labels or write its comment if that
  default is read-only. The docs-drift example already had a block.

The gate reads the example callers as well, because client repos copy
them. It fails on any finding. .github/zizmor.yml holds the accepted
risks, with the reason for each one.

Co-Authored-By: Claude <noreply@anthropic.com>
@alex-clickhouse
alex-clickhouse force-pushed the ci/zizmor-actions-audit branch from 2e2a623 to a1ef9c7 Compare August 19, 2026 10:09
@alex-clickhouse
alex-clickhouse merged commit 6934351 into main Aug 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants