Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

releng: Update references to sigs.k8s.io/promo-tools #2752

Merged
merged 2 commits into from
Sep 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions artifacts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,10 +117,10 @@ To promote a set of artifacts, follow these steps:
schedule (currently every 4 hours)
7. Published artifacts will appear artifacts.k8s.io

[generate-manifest]: https://sigs.k8s.io/k8s-container-image-promoter/cmd/kpromo#generating-a-file-promotion-manifest
[generate-manifest]: https://sigs.k8s.io/promo-tools/cmd/kpromo#generating-a-file-promotion-manifest
[google-groups]: /groups/README.md
[infra.yaml]: /infra/gcp/infra.yaml
[kpromo]: https://sigs.k8s.io/k8s-container-image-promoter/cmd/kpromo
[kpromo]: https://sigs.k8s.io/promo-tools/cmd/kpromo
[periodic]: https://prow.k8s.io/job-history/gs/kubernetes-jenkins/logs/ci-k8sio-file-promo
[postsubmit]: https://prow.k8s.io/job-history/gs/kubernetes-jenkins/logs/post-k8sio-file-promo
[presubmit]: https://prow.k8s.io/job-history/gs/kubernetes-jenkins/pr-logs/directory/pull-k8sio-file-promo
Expand Down
18 changes: 10 additions & 8 deletions infra/gcp/bash/backup_tools/backup_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,14 @@ set -o xtrace
export GCRANE_CHECKOUT_DIR="${GOPATH}/src/github.com/google/go-containerregistry"
GCRANE="${GCRANE_CHECKOUT_DIR}/cmd/gcrane/gcrane"

export CIP_CHECKOUT_DIR="${GOPATH}/src/sigs.k8s.io/k8s-container-image-promoter"
CIP_SNAPSHOT_CMD="${CIP_CHECKOUT_DIR}/cip -minimal-snapshot -output-format=CSV -snapshot"
# CIP_REF is the commit SHA to use for building the cip binary (used only for
export PROMO_TOOLS_CHECKOUT_DIR="${GOPATH}/src/sigs.k8s.io/promo-tools"
SNAPSHOT_CMD="${PROMO_TOOLS_CHECKOUT_DIR}/cip -minimal-snapshot -output-format=CSV -snapshot"
# PROMO_TOOLS_REF is the commit SHA to use for building the cip binary (used only for
# testing; not used by the actual prod backup job).
#
# TODO(releng): Update me
# Known-good commit from 2020-04-01
CIP_REF="feb5dc08b2cbfa2c779c4c5d397dad40e669bc84"
PROMO_TOOLS_REF="feb5dc08b2cbfa2c779c4c5d397dad40e669bc84"

SCRIPT_ROOT="$(dirname "$(readlink -f "$0")")"
# shellcheck disable=SC1090
Expand All @@ -47,9 +49,9 @@ build_cip()
{
local cip_path

git clone https://github.com/kubernetes-sigs/k8s-container-image-promoter "${CIP_CHECKOUT_DIR}"
pushd "${CIP_CHECKOUT_DIR}"
git reset --hard "${CIP_REF}"
git clone https://github.com/kubernetes-sigs/promo-tools "${PROMO_TOOLS_CHECKOUT_DIR}"
pushd "${PROMO_TOOLS_CHECKOUT_DIR}"
git reset --hard "${PROMO_TOOLS_REF}"
make
# Leave a symlink to the cip binary that was built from the above command.
cip_path=$(find "$(bazel info bazel-bin)" -type f -name cip)
Expand Down Expand Up @@ -107,7 +109,7 @@ verify_repo()

diff -u \
<(cat "${TEST_IMAGES_FILE}") \
<(${CIP_SNAPSHOT_CMD} "${repo}")
<(${SNAPSHOT_CMD} "${repo}")
}

# Backup GCRs for prod.
Expand Down
6 changes: 2 additions & 4 deletions infra/gcp/bash/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,8 @@ function empower_group_as_viewer() {
ensure_project_role_binding "${project}" "group:${group}" "roles/viewer"
}

# Grant roles for running cip-auditor E2E test
# (https://github.com/kubernetes-sigs/k8s-container-image-promoter/tree/master/test-e2e#cip-auditor-cip-auditor-e2ego).
# Grant roles for running cip-auditor E2E tests
# (https://sigs.k8s.io/promo-tools/test-e2e)

# $1: The GCP project
# $2: The service account
Expand Down Expand Up @@ -407,8 +407,6 @@ function empower_file_promoter() {

ensure_service_account "${project}" "${FILE_PROMOTER_SVCACCT}" "artifact promoter"

# TODO(kpromo): Determine if admin access is required here.
# ref: https://github.com/kubernetes-sigs/k8s-container-image-promoter/issues/413
empower_svcacct_to_write_gcs_bucket "${acct}" "${bucket}"
}

Expand Down
99 changes: 58 additions & 41 deletions k8s.gcr.io/Vanity-Domain-Flip.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,49 +38,48 @@ To get new images into the old `gcr.io/google-containers`, a Googler must approv
change in Google's private repository.

On the other hand, the new `{asia,eu,us}.gcr.io/k8s-artifacts-prod` is integrated with a
publicly-visible GitHub repository, named [k8s.io][k8sio]. The [promoter][CIP]
publicly-visible GitHub repository, named [k8s.io][k8sio]. The [promoter][cip]
watches this repository for changes and promotes images. In addition, a system
of setting up staging repos, and promoting from them into
`{asia,eu,us}.gcr.io/k8s-artifacts-prod` has been [created][staging-subproject] so that owners of
subprojects in the community can take control of how their images are released.

## The Promoter (cip)

The [Container Image Promoter][CIP] (henceforth "the promoter") is the OSS
rewrite of the [promoter used internally within Google][internal-promoter]. It
The [Container Image Promoter][cip] (henceforth "the promoter") is the OSS
rewrite of the [promoter used internally within Google][google-promoter]. It
works by reading in a set of promoter manifests (YAMLs) that describe the
desired state of a Docker registry's image contents, and proceeds to copy in any
missing images. Currently the toplevel `k8s.gcr.io` directory at the [k8s.io
Github repo][k8sio] defines such a set of promoter manifests.
missing images. Currently, the `k8s.gcr.io` directory of [this repo][k8sio] defines such a set of promoter manifests.

## Prow Integration

The act of invoking the promoter as a postsubmit against the k8s.io repo is done
by [Prow][prow], as the `post-k8sio-cip` Prow job. There are other Prow jobs
by [Prow][prow], as the `post-k8sio-image-promo` Prow job. There are other Prow jobs
that integrate with the promoter, and the ones relevant to this doc are outlined
in the list below:

- [`pull-k8sio-cip`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-k8sio-cip))
Dry run version of `post-k8sio-cip`. It is run as a presubmit
- [`pull-k8sio-image-promo`][pull-k8sio-image-promo] ([logs][pull-k8sio-image-promo-logs])
Dry run version of `post-k8sio-image-promo`. It is run as a presubmit
check to any PR against [k8s.io Github repo][k8sio]. In particular, it
catches things like tag moves (which are disallowed). Unlike
`post-k8sio-cip`, it does not run in the trusted cluster, because it does
`post-k8sio-image-promo`, it does not run in the trusted cluster, because it does
not need to use prod credentials (in fact, it doesn't use any creds).
- [`post-k8sio-cip`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/post-k8sio-cip))
- [`post-k8sio-image-promo`][post-k8sio-image-promo] ([logs][post-k8sio-image-promo-logs])
Postsubmit job against k8s.io repo holding promoter manifests. The promoter
manifests here are those that promote from the various staging subproject
repos to `{asia,eu,us}.gcr.io/k8s-artifacts-prod/<subproject>/<image>`. It uses the
`k8s-infra-gcr-promoter@k8s-artifacts-prod.iam.gserviceaccount.com` service
account to write to `{asia,eu,us}.gcr.io/k8s-artifacts-prod`. For all
intents and purposes, **this is the gatekeeper for new images going into
`k8s-artifacts-prod`**.
- [`ci-k8sio-cip`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/ci-k8sio-cip))
Like `post-k8sio-cip`, but runs periodically. This is to ensure
- [`ci-k8sio-image-promo`][ci-k8sio-image-promo] ([logs][ci-k8sio-image-promo-logs])
Like `post-k8sio-image-promo`, but runs periodically. This is to ensure
that even if images are accidentally deleted from
`{asia,eu,us}.gcr.io/k8s-artifacts-prod`, they are automatically copied back. It also
acts as a kind of sanity check, to ensure that the promoter can run at all.
- [`pull-cip-e2e`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-e2e))
Runs an [E2E][CIP-e2e] [test][CIP-e2e-promotion] for changes to the promoter source code. This
- [`pull-cip-e2e`][pull-cip-e2e] ([logs][pull-cip-e2e-logs])
Runs an [E2E][cip-e2e] [test][cip-e2e-promotion] for changes to the promoter source code. This
test checks that the promoter can promote images (its main purpose). It uses
the `[email protected]`
service account to use the `k8s-cip-test-prod` GCP project resources for its
Expand All @@ -89,10 +88,10 @@ in the list below:
In addition there are some jobs that act solely as a sanity check on the
promoter's own codebase:

- [`pull-cip-unit-tests`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-unit-tests))
- [`pull-cip-unit-tests`][pull-cip-unit-tests] ([logs][pull-cip-unit-tests-logs])
This runs unit tests for the promoter codebase, and are part of
the PR presubmit checks.
- [`pull-cip-lint`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-lint))
- [`pull-cip-lint`][pull-cip-lint] ([logs][pull-cip-lint-logs])
This runs [golangci-lint][golangci-lint] for the promoter
codebase (which is primarily written in Go).

Expand All @@ -102,13 +101,13 @@ In order for a user to push to `k8s-artifacts-prod`, they must:

1. Ensure that they have a [subproject staging repo][staging-subproject] (e.g.,
`gcr.io/k8s-staging-foo` for the `foo` subproject).
2. Add the promotion metadata in the [manifests subdirectory](/k8s.gcr.io/manifests) in the k8s.io repo.
2. Add the promotion metadata in the [manifests subdirectory][k8sio-manifests] in the k8s.io repo.

### Security Restrictions

- **Write-once**: Images promoted to production will NOT be deleted, unless under extreme,
emergency circumstances that require human supervision (see "Breakglass"
section below).
emergency circumstances that require human supervision (see
["Breakglass"](#breakglass) section below).
- **Immutable tags**: New images added to the promoter manifests cannot use an
existing tag for the same image. In other words, tags (once created for an
image) cannot be deleted.
Expand Down Expand Up @@ -138,11 +137,11 @@ The backup GCR locations are:

## Prow Integration

- [`ci-k8sio-backup`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/ci-k8sio-backup))
- [`ci-k8sio-backup`][ci-k8sio-backup] ([logs][ci-k8sio-backup-logs])
Runs a backup of all GCR images in
`{asia,eu,us}.gcr.io/k8s-artifacts-prod` to
`{asia,eu,us}.gcr.io/k8s-artifacts-prod-bak/...`.
- [`pull-k8sio-backup`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-k8sio-backup))
- [`pull-k8sio-backup`][pull-k8sio-backup] ([logs][pull-k8sio-backup-logs])
Checks that changes to the [backup scripts][k8sio-backup] are
valid. Like the `pull-cip-e2e` and `pull-cip-auditor-e2e` jobs, this job
uses GCP resources to check that the backup scripts work as intended in
Expand Down Expand Up @@ -175,7 +174,7 @@ The step-by-step process is:
1. An image is created (new tag), deleted, etc on the `k8s-artifacts-prod` GCR.
2. Cloud Pub/Sub message with the stateful change contents is sent over HTTPS to
the `cip-auditor` service in Cloud Run.
3. `cip-auditor` clones a fresh copy of [promoter manifests][k8sio] at https://github.com/kubernetes/k8s.io.
3. `cip-auditor` clones a fresh copy of [promoter manifests][k8sio] at https://git.k8s.io/k8s.io.
4. `cip-auditor` checks the Pub/Sub message contents against the promoter manifests.
5. If the message agrees with the promoter manifests, nothing happens.
Otherwise, a call is made to the Stackdriver Error Reporting API with a stack
Expand All @@ -194,14 +193,14 @@ The configuration for deploying the prod Cloud Run instance is [here](/infra/gcp

## Prow Integration

- [`pull-cip-auditor-e2e`](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) ([logs](https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-auditor-e2e))
Like `pull-cip-e2e`, but runs E2E [tests][CIP-e2e-auditor] for the auditing
- [`pull-cip-auditor-e2e`][pull-cip-auditor-e2e] ([logs][pull-cip-auditor-e2e-logs])
Like `pull-cip-e2e`, but runs E2E [tests][cip-e2e-auditor] for the auditing
mechanism built into the promoter. While the actual auditing mechanism (known
as "cip-auditor") runs in production in the `k8s-artifacts-prod` project, the
E2E tests here run in the test-only project named `k8s-gcr-audit-test-prod`
which is dedicated solely to this purpose. The auditor code lives
[here][CIP-auditor-code], but the E2E tests for it live
[here][CIP-e2e-auditor]. The E2E test use the
[here][cip-auditor-code], but the E2E tests for it live
[here][cip-e2e-auditor]. The E2E test use the
`k8s-infra-gcr-promoter@k8s-gcr-audit-test-prod.iam.gserviceaccount.com` GCP
project resources for creating/deleting Cloud Run services in
`k8s-gcr-audit-test-prod`, as well as clearing Pub/Sub messages and
Expand Down Expand Up @@ -238,24 +237,42 @@ The auditing mechanism uses 3 service accounts:
## Admin

The `[email protected]` googlegroup manages the
auditor service. Its members are listed [here](../groups/groups.yaml).
auditor service. Its members are listed [here][groups].

# Glossary

- GCR: Google Container Registry
- GCS: Google Cloud Storage

[CIP]:https://github.com/kubernetes-sigs/k8s-container-image-promoter
[internal-promoter]:http://go/registry-promoter
[k8sio]:https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io
[k8sio-manifests]:https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io/manifests
[k8sio-backup]:https://github.com/kubernetes/k8s.io/tree/main/infra/gcp/backup_tools
[staging-subproject]:https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io#staging-repos
[prow]:https://github.com/kubernetes/test-infra/tree/master/prow
[golangci-lint]:https://github.com/golangci/golangci-lint
[groups]:https://github.com/kubernetes/k8s.io/blob/main/groups/groups.yaml
[CIP-e2e]:https://github.com/kubernetes-sigs/k8s-container-image-promoter/tree/master/test-e2e
[CIP-e2e-promotion]:https://github.com/kubernetes-sigs/k8s-container-image-promoter/tree/master/test-e2e/cip
[CIP-e2e-auditor]:https://github.com/kubernetes-sigs/k8s-container-image-promoter/tree/master/test-e2e/cip-auditor
[CIP-auditor-code]:https://github.com/kubernetes-sigs/k8s-container-image-promoter/blob/master/lib/audit/auditor.go
[ci-k8sio-backup-code]:https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml
[ci-k8sio-backup]: https://git.k8s.io/test-infra/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml
[ci-k8sio-backup-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/ci-k8sio-backup
[ci-k8sio-backup-code]: https://git.k8s.io/test-infra/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml
[ci-k8sio-image-promo]: https://git.k8s.io/test-infra/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml
[ci-k8sio-image-promo-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/ci-k8sio-image-promo
[cip]: https://sigs.k8s.io/promo-tools
[cip-e2e]: https://sigs.k8s.io/promo-tools/test-e2e
[cip-e2e-promotion]: https://sigs.k8s.io/promo-tools/test-e2e/cip
[cip-e2e-auditor]: https://sigs.k8s.io/promo-tools/test-e2e/cip-auditor
[cip-auditor-code]: https://sigs.k8s.io/promo-tools/legacy/audit
[golangci-lint]: https://github.com/golangci/golangci-lint
[groups]: /groups
[google-promoter]: http://go/registry-promoter
[k8sio]: https://git.k8s.io/k8s.io/k8s.gcr.io
[k8sio-backup]: /infra/gcp/bash/backup_tools
[k8sio-manifests]: /k8s.gcr.io/manifests
[post-k8sio-image-promo]: https://git.k8s.io/test-infra/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml
[post-k8sio-image-promo-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/post-k8sio-image-promo
[prow]: https://git.k8s.io/test-infra/prow
[pull-cip-auditor-e2e]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-cip-auditor-e2e-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-auditor-e2e
[pull-cip-e2e]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-cip-e2e-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-e2e
[pull-cip-lint]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-cip-lint-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-lint
[pull-cip-unit-tests]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-cip-unit-tests-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-cip-unit-tests
[pull-k8sio-backup]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-k8sio-backup-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-k8sio-backup
[pull-k8sio-image-promo]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml
[pull-k8sio-image-promo-logs]: https://prow.k8s.io/job-history/kubernetes-jenkins/logs/pull-k8sio-image-promo
[staging-subproject]: /k8s.gcr.io/README.md#staging-repos
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Flip][VDF].

**As such, these images are effectively FROZEN. New changes to them WILL BE
REJECTED, and any new promotions MUST happen through individual sub-project
staging repos, as detailed in the policy [here][official-policy].**
staging repos, as detailed [here][staging-repos].**

[VDF]: https://github.com/kubernetes/k8s.io/blob/main/k8s.gcr.io/Vanity-Domain-Flip.md
[official-policy]: https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io#creating-staging-repos
[staging-repos]: /k8s.gcr.io/README.md#staging-repos
[VDF]: /k8s.gcr.io/Vanity-Domain-Flip.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,8 @@
# It was generated automatically by running "cip-mm". To
# reproduce, run:
#
# $ cd
# $ git clone https://github.com/kubernetes-sigs/k8s-container-image-promoter go/src/sigs.k8s.io/k8s-container-image-promoter
# $ cd go/src/sigs.k8s.io/k8s-container-image-promoter
# $ make install
# $ cip-mm -base_dir PATH/TO/k8s.gcr.io -staging_repo gcr.io/google-containers
# $ go install sigs.k8s.io/promo-tools/cmd/cip-mm
# $ $(go env GOPATH)/bin/cip-mm --base_dir k8s.gcr.io/archived --staging_repo gcr.io/google-containers

- name: addon-builder
dmap:
Expand Down