Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: CI/CD Pipelines #259

Merged
merged 318 commits into from
Feb 8, 2024
Merged

refactor: CI/CD Pipelines #259

merged 318 commits into from
Feb 8, 2024

Conversation

Johannes-Schneider
Copy link
Contributor

Context

SAP/cloud-sdk-java-backlog#318

Continuation of #235

This PR refactors our GitHub actions to, first and foremost, avoid the massive code duplication.
Additionally, it also refactors the release flow, which now provides a lot more automation and should be easier to use in general.

How CI Works

The CI portion of our pipelines has been changed in the following ways:

  1. There is now one general purpose CI workflow (continuous-integration.yaml).
  2. This workflow is triggered whenever a PR (targeting our main branch) receives an update.
  3. Additionally, this workflow is also triggered from within other workflows (i.e. from our new main-build.yaml, our prepare-release.yaml, and our deploy-snapshot.yaml). This is how we are getting rid of the code duplication.
How CD Works

The CD portion (i.e. performing releases) has also been changed.
Here is the new flow:

  1. The release facilitator triggers the Prepare Release (prepare-release.yaml) workflow. There are sensitive default parameters in place, so no manual input is required.
  2. This workflow will do the following things:
    1. Bump the version of our module to the version to be released (e.g. 5.3.0)
    2. Run the new continuous-integration.yaml workflow on the release commit. This includes code checks, such as Black Duck, CodeQL, etc.
    3. Create a new release tag (rel/<VERSION>) and also create a new draft release in our GitHub repo
    4. Create a PR for our updated release notes
    5. Create a PR for our updated java docs
    6. Create a PR for our updated code, which also contains further instructions for the release facilitator
    7. If anything along the way fails, everything will be cleaned up automatically by the workflow - no manual interaction needed.
  3. If the Prepare Release workflow succeeds, the release facilitator will find a new PR with detailed instructions about what to do in our Code Repo (i.e. SAP/cloud-sdk-java). These TODOs include the following:
    1. Review the release commit (there is a link in the PR body)
    2. Review the draft release (there is a link in the PR body)
    3. Review and approve the Release Notes PR (there is a link in the PR body)
    4. Review and approve the Java Docs PR (there is a link in the PR body)
    5. Review and approve the Code Updates (i.e. version bump and resetting the release-notes.md)
  4. As soon as the code PR has been approved, the Perform Release workflow will be triggered automatically.
  5. This workflow checks whether all PRs can be merged. If that is not the case, the workflow fails.
    1. In case of any failure, the Perform Release workflow can also be triggered manually now.
  6. The Workflow will download the release artifacts and push them to Maven Central - this is still a staging repo, which needs to be released manually by the release facilitator.
  7. The Workflow will also merge all of the PRs

@@ -19,6 +18,7 @@ env:
jobs:
prerequisites:
name: "Prerequisites"
if: ${{ github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request_review' && github.event.review.state == 'approved' && startsWith(github.event.pull_request.head.ref, 'RELEASE-')) }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussion

We are still having a pull_request_review trigger here. So even though the job would be skipped for any PR that is not coming from a RELEASE-* branch, it would still appear within the status checks of every other PR.

So maybe we should get rid of the automatic trigger altogether?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would agree, one extra button press isn't the world, better not to have our PR checks (even more) cluttered.. no strong opinion though ofc...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the automatic trigger and added another checkbox to the release PR body (i.e. Trigger the Perform Release Workflow).

Copy link
Member

@MatKuhr MatKuhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly LGTM, one minor question on the snapshot deployment

didn't look at the release notes python yet

description: "The username to use when authenticating with the repository."
required: true
repository-password:
description: "The password to use when authenticating with the repository."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not too sure if this is safe to do. Maybe input parameters are logged in plaintext somewhere? Maybe just pass the name of the secret instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, it seems to be a "best practice" to pass around secrets like this
(see the token parameter of the checkout action for example).

But I'd still agree: As this is our own action, we don't need to pass around the actual secrets. Using the secret names should be sufficient 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, passing the secret names (instead of their actual values), doesn't work as it would require some sort of "nested GitHub macro extension" like so:

- run: something
  env:
    GH_TOKEN: ${{ secrets.${{ inputs.secret-name}} }}

This doesn't seem to be supported by GH actions.

@Johannes-Schneider Johannes-Schneider dismissed CharlesDuboisSAP’s stale review February 7, 2024 13:18

comments have been addressed.

@MatKuhr MatKuhr merged commit a7765b3 into SAP:main Feb 8, 2024
13 of 14 checks passed
This was referenced Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please merge Request to merge a pull request please review Request to review a pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants