From 272930c820ae0c2005c95a663fa326d0286201bf Mon Sep 17 00:00:00 2001 From: Stephen Augustus Date: Wed, 15 Sep 2021 17:28:37 -0400 Subject: [PATCH] Update references to default development branch ...and other link fixes. Signed-off-by: Stephen Augustus --- README.md | 60 +++++++++++++++++++++++++------------------- checks_interface.md | 58 +++++++++++++++++++++--------------------- cmd/gh2gcs/README.md | 9 ++++--- go.mod | 8 +++--- go.sum | 15 +++++++---- 5 files changed, 83 insertions(+), 67 deletions(-) diff --git a/README.md b/README.md index a667a2e9..0dd927ad 100644 --- a/README.md +++ b/README.md @@ -310,42 +310,46 @@ submitting a new PR. As the promoter uses a combination of network API calls and shell-instantiated processes, we have to fake them for the unit tests. To make this happen, these -mechanisms all use a `stream.Producer` [interface](lib/stream/types.go). The -real-world code uses either the [http](lib/stream/http.go) or -[subprocess](lib/stream/subprocess.go) implementations of this interface to +mechanisms all use a `stream.Producer` [interface](/legacy/stream/types.go). The +real-world code uses either the [http](/legacy/stream/http.go) or +[subprocess](/legacy/stream/subprocess.go) implementations of this interface to create streams of data (JSON or not) which we can interpret and use. -For tests, the [fake](lib/stream/fake.go) implementation is used instead, which +For tests, the [fake](/legacy/stream/fake.go) implementation is used instead, which predefines how that stream will behave, for the purposes of each unit test. A good example of this is the [`TestReadRegistries` -test](lib/dockerregistry/inventory_test.go). +test](/legacy/dockerregistry/inventory_test.go). ### Automated builds The `gcr.io/k8s-staging-artifact-promoter` GCR is a staging repo for Docker -image build artifacts from this project. Every update to the `master` branch in -this Github repo results in a new set of 2 images in the staging GCR repo: +image build artifacts from this project. Every update to the default +development branch of this Github repo results in three images being built in +the staging GCR repo: 1. `gcr.io/k8s-staging-artifact-promoter/cip` 1. `gcr.io/k8s-staging-artifact-promoter/cip-auditor` +1. `gcr.io/k8s-staging-artifact-promoter/kpromo` These images get built and pushed up there by GCB using the [build file here][cloudbuild.yaml]. There are also production versions of these images here: 1. `{asia,eu,us}.gcr.io/k8s-artifacts-prod/artifact-promoter/cip` 1. `{asia,eu,us}.gcr.io/k8s-artifacts-prod/artifact-promoter/cip-auditor` +1. `{asia,eu,us}.gcr.io/k8s-artifacts-prod/artifact-promoter/kpromo` The images from the staging GCR end up in `k8s-artifacts-prod` using the promoter image running in -[Prow](https://github.com/kubernetes/test-infra/tree/master/prow). "Using the -promoter" here means creating a PR in the [k8sio-manifests-dir][k8s.io Github -repo] to promote versions from staging to production, such as in [this -PR](https://github.com/kubernetes/k8s.io/pull/704). +[Prow](https://github.com/kubernetes/test-infra/prow). "Using the +promoter" here means creating a PR in the [k8s.io Github repo][k8sio-manifests-dir] +to promote versions from staging to production, such as in +[this PR](https://github.com/kubernetes/k8s.io/pull/704). #### Connection with Prow -There are a number of Prow jobs that consume the production Docker images of -`cip` or `cip-auditor`. These jobs are defined [cip-prow-integration][here]. +There are a number of Prow jobs that consume the production container images +of `cip`, `cip-auditor`, or `kpromo`. These jobs are defined +[cip-prow-integration][here]. The important thing to note is that ultimately the jobs there are downstream consumers of the production `cip` and `cip-auditor` images discussed above. So @@ -353,23 +357,26 @@ if there is a breaking change where the Docker images don't work any more for these Prow jobs, the sequence of events required to fix those Prow jobs are: 1. fix the bug in this codebase -2. generate new `cip` and `cip-auditor` images in `gcr.io/k8s-staging-artifact-promoter` (automated) +2. generate new `cip` and `cip-auditor` images in + `gcr.io/k8s-staging-artifact-promoter` (automated) 3. promote images into production 4. update Prow jobs to use the new images from Step 3 Step 1 is done in this Github repo. Step 3 is done in [the k8s.io Github -repo](https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io). Step 4 is -done in the [test-infra Github repo](https://github.com/kubernetes/test-infra). +repo][k/k8s.io]. + +Step 4 is done in the [test-infra Github repo][k/test-infra]. ## Versioning -We follow [Semver](https://semver.org/) for versioning. For each new release, +We follow [SemVer](https://semver.org/) for versioning. For each new release, create a new release on GitHub with: - Update VERSION file to bump the semver version (e.g., `1.0.0`) -- Create a new commit for the 1-liner change above with this command with `git commit -m "cip 1.0.0"` -- Create an annotated tag at this point with `git tag -a "v1.0.0" -m "cip 1.0.0"` -- Push this version to the `master` branch (requires write access) +- Create a new commit for the 1-liner change above with this command with + `git commit --signoff -m "v1.0.0: Release commit"` +- Create a signed tag at this point with `git tag -s -m "v1.0.0" "v1.0.0"` +- Push this version to the default development branch (requires write access) ### Default versioning @@ -377,14 +384,9 @@ The Docker images that are produced by this repo are automatically tagged in the following format: `YYYYMMDD-`. As such, there is no need to bump the VERSION file often as the Docker images will always get a unique identifier. -[docker]:https://docs.docker.com/get-docker -[golang]:https://golang.org/doc/install -[k8sio-manifests-dir]:https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io -[cip-prow-integration]:https://github.com/kubernetes/k8s.io/blob/main/k8s.gcr.io/Vanity-Domain-Flip.md#prow-integration - ## Checks Interface -Read more [here](https://github.com/kubernetes-sigs/promo-tools/blob/master/checks_interface.md). +Read more [here](/checks_interface.md). The addition of the checks interface to the Container Image Promoter is meant to make it easy to add checks against pull requests affecting the promoter @@ -399,5 +401,11 @@ The vulnerability dashboard (`vulndash`) has moved to [`kubernetes/release`][k/r Read more [here][vulndash-readme]. +[cip-prow-integration]: https://git.k8s.io/k8s.io/k8s.gcr.io/Vanity-Domain-Flip.md#prow-integration +[docker]: https://docs.docker.com/get-docker +[golang]: https://golang.org/doc/install +[k/k8s.io]: https://git.k8s.io/k8s.io [k/release]: https://git.k8s.io/release +[k/test-infra]: https://git.k8s.io/test-infra +[k8sio-manifests-dir]: https://git.k8s.io/k8s.io/k8s.gcr.io [vulndash-readme]: https://git.k8s.io/release/docs/vuln-dashboard.md diff --git a/checks_interface.md b/checks_interface.md index 15ceb0db..7451ed1f 100644 --- a/checks_interface.md +++ b/checks_interface.md @@ -4,12 +4,12 @@ to make it easy to add checks against pull requests affecting the promoter manifests. The interface allows engineers to add checks without worrying about any pre-existing checks and test their own checks individually, while also giving freedom as to what conditionals or tags might be necessary for the -check to occur. Aditionally, using an interface means easy expandability of +check to occur. Additionally, using an interface means easy expandability of check requirements in the future. ## Interface Explanation The `PreCheck` interface is implemented like so in the -[types.go](https://github.com/kubernetes-sigs/promo-tools/blob/master/lib/dockerregistry/types.go) +[types.go](/legacy/dockerregistry/types.go) file. The `Run` function is the method used in order to actually execute the check that implements this interface. @@ -33,15 +33,13 @@ func (sc *SyncContext) RunChecks( ``` #### Integration With PROW -The Container Image Promoter has several Prow jobs that run whenever a pull -request attempts to modify the promoter manifests. The -[*pull-k8s-cip*](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) -and the -[*pull-k8s-cip-vuln*](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) -Prow jobs call the `RunChecks` function and actually run their respective checks. -New Prow jobs can be [added](https://github.com/kubernetes/test-infra/blob/master/config/jobs/README.md#adding-or-updating-jobs) -to run an individual check in the future if that check requires it's own separate -job for some reason. +The Container Image Promoter has several Prow jobs that run whenever a pull +request attempts to modify the promoter manifests. The +[*pull-k8sio-cip*][k8sio-presubmits] and the +[*pull-k8sio-cip-vuln*][k8sio-presubmits] Prow jobs call the `RunChecks` +function and actually run their respective checks. New Prow jobs can be +[added][add-prow-job] to run an individual check in the future if that check +requires a separate job. ### How To Add A Check In order to add a check, all you need to do is create a check type that @@ -52,13 +50,12 @@ type foo struct {} ... func (f *foo) Run() error ``` -Then add that check type you've created to the input list of PreChecks for -the RunChecks method, which is called in the -[cip.go](https://github.com/kubernetes-sigs/promo-tools/blob/master/cip.go) -file. + +Then add that check type you've created to the input list of PreChecks for +the `RunChecks` method [here](/legacy/dockerregistry/inventory.go). Note that the `Run` method of the precheck interface does not accept any -paramaters, so any information that you need for your check should be passed +parameters, so any information that you need for your check should be passed into the check type as a field. For example, if you are running a check over promotion edges, then you can set up your check like so: @@ -81,11 +78,11 @@ func (f * foo) Run() error { Images that have been promoted are pushed to production; and once pushed to production, they should never be removed. The `ImageRemovalCheck` checks if any images are removed in the pull request by comparing the state of the -promoter manifests in the pull request's branch to the master branch. Two sets -of Promotion Edges are generated (one for both the master branch and pull -request) and then compared to make sure that every destination image (defined -by its image tag and digest) found in the master branch is found in the pull -request. +promoter manifests in the pull request's branch to the default development +branch. Two sets of Promotion Edges are generated (one for both the default +development branch and pull request) and then compared to make sure that every +destination image (defined by its image tag and digest) found in the default +development branch is found in the pull request. This method for detecting removed images should ensure that pull requests are only rejected if an image is completely removed from production, while still @@ -112,21 +109,21 @@ all images for any vulnerabilities they might already have before promoting them. A vulnerability check also serves as a method for surfacing all vulnerabilities regardless if they have a fix to the user. To emphasize this point, the vulnerability check has been implemented in it's own separate Prow -job -[*pull-k8s-cip-vuln*](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml) +job [*pull-k8sio-cip-vuln*][k8sio-presubmits] so that the check's logs (which will detail all the vulnerabilities that exist in the new images to be promoted) won't get mixed in with the logs from the promoter's other checks. -The vulnerability check makes use of the Container Analysis API in order to -1. scan all new staging images for vulnerabilities whenever they are added to -an image staging project -2. get vulnerability information when we are -checking a the images to be promoted from a PR +The vulnerability check makes use of the Container Analysis API in order to: + +1. scan all new staging images for vulnerabilities whenever they are added to + an image staging project +2. get vulnerability information when we are checking the images to be promoted + from a PR To make use of this API, key pieces of infrastructure must be put in place, such as enabling the Container Analysis API on all image staging projects -and authenticating the Prow job (pull-k8s-cip-vuln) with a Google service +and authenticating the Prow job (pull-k8sio-cip-vuln) with a Google service account that is authorized to access the vulnerability data for each staging project. @@ -134,3 +131,6 @@ The vulnerability check will reject a pull request if it finds any vulnerabilities that are both beyond the severity threshold (defined by the *-vuln-severity-threshold*) and have a known fix; otherwise the check will accept the PR. + +[add-prow-job]: https://git.k8s.io/test-infra/config/jobs/README.md#adding-or-updating-jobs +[k8sio-presubmits]: https://git.k8s.io/test-infra/config/jobs/kubernetes/wg-k8s-infra/releng/artifact-promotion-presubmits.yaml diff --git a/cmd/gh2gcs/README.md b/cmd/gh2gcs/README.md index 5c2a7183..c7b1179b 100644 --- a/cmd/gh2gcs/README.md +++ b/cmd/gh2gcs/README.md @@ -12,8 +12,8 @@ Google Cloud has [documentation on installing and configuring the Google Cloud S The simplest way to install the `gh2gcs` CLI is via `go get`: -``` -$ go get k8s.io/release/cmd/gh2gcs +```console +go get k8s.io/release/cmd/gh2gcs ``` This will install `gh2gcs` to `$(go env GOPATH)/bin/gh2gcs`. @@ -46,4 +46,7 @@ The following GCS buckets are managed by SIG Release: - k8s-artifacts-cni - contains [CNI plugins](https://github.com/containernetworking/plugins) artifacts - k8s-artifacts-cri-tools - contains [CRI tools](https://github.com/kubernetes-sigs/cri-tools) artifacts (`crictl` and `critest`) -The artifacts are pushed to GCS by [Release Managers](https://github.com/kubernetes/sig-release/blob/master/release-managers.md). The pushing is done manually by running the appropriate `gh2gcs` command. It's recommended for Release Managers to watch the appropriate repositories for new releases. +The artifacts are pushed to GCS by +[Release Managers](https://k8s.io/releases/release-managers/). The pushing is +done manually by running the appropriate `gh2gcs` command. It's recommended for +Release Managers to watch the appropriate repositories for new releases. diff --git a/go.mod b/go.mod index 47137d51..db0296ff 100644 --- a/go.mod +++ b/go.mod @@ -25,8 +25,8 @@ require ( golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e // indirect golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 - google.golang.org/api v0.56.0 - google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71 + google.golang.org/api v0.57.0 + google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83 gopkg.in/src-d/go-git.v4 v4.13.1 gopkg.in/yaml.v2 v2.4.0 k8s.io/utils v0.0.0-20210305010621-2afb4311ab10 @@ -36,7 +36,7 @@ require ( ) require ( - cloud.google.com/go v0.93.3 // indirect + cloud.google.com/go v0.94.1 // indirect github.com/Microsoft/go-winio v0.5.0 // indirect github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7 // indirect github.com/acomagu/bufpipe v1.0.3 // indirect @@ -69,7 +69,7 @@ require ( golang.org/x/mod v0.4.2 // indirect golang.org/x/net v0.0.0-20210525063256-abc453219eb5 // indirect golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect - golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf // indirect + golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365 // indirect golang.org/x/text v0.3.6 // indirect golang.org/x/tools v0.1.5 // indirect google.golang.org/appengine v1.6.7 // indirect diff --git a/go.sum b/go.sum index e5c743ea..e328d531 100644 --- a/go.sum +++ b/go.sum @@ -26,8 +26,9 @@ cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aD cloud.google.com/go v0.92.1/go.mod h1:cMc7asehN84LBi1JGTHo4n8wuaGuNAZ7lR7b1YNJBrE= cloud.google.com/go v0.92.2/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= cloud.google.com/go v0.92.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= -cloud.google.com/go v0.93.3 h1:wPBktZFzYBcCZVARvwVKqH1uEj+aLXofJEtrb4oOsio= cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= +cloud.google.com/go v0.94.1 h1:DwuSvDZ1pTYGbXo8yOJevCTr3BoBlE+OVkHAKiYQUXc= +cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= @@ -1041,8 +1042,9 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf h1:2ucpDCmfkl8Bd/FsLtiD653Wf96cW37s+iGx93zsu4k= golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365 h1:6wSTsvPddg9gc/mVEEyk9oOAoxn+bT4Z9q1zx+4RwA4= +golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -1154,8 +1156,9 @@ google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtuk google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw= google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU= google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k= -google.golang.org/api v0.56.0 h1:08F9XVYTLOGeSQb3xI9C0gXMuQanhdGed0cWFhDozbI= -google.golang.org/api v0.56.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.57.0 h1:4t9zuDlHLcIx0ZEhmXEeFVCRsiOgpgn2QOH9N0MNjPI= +google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= @@ -1222,8 +1225,10 @@ google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKr google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w= google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= google.golang.org/genproto v0.0.0-20210825212027-de86158e7fda/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71 h1:z+ErRPu0+KS02Td3fOAgdX+lnPDh/VyaABEJPD4JRQs= google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83 h1:3V2dxSZpz4zozWWUq36vUxXEKnSYitEH2LdsAx+RUmg= +google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=