Skip to content

OVERVIEW.md

Latest commit

 

History

History
82 lines (52 loc) · 7.12 KB

OVERVIEW.md

File metadata and controls

82 lines (52 loc) · 7.12 KB

CICD-Tools

Technical Overview

CICD-Tools provides four consumable resources that together form the basis for a complete CI solution:

  1. Customized pre-commit hooks for end-user projects.
  2. A custom Docker container which supplies the required binary tools for the pre-commit hooks.
  3. Remotely consumable GitHub "Jobs" that are actively maintained.
  4. A custom packaging system that securely delivers upgradable Toolboxes full of scripts for the workflows.

End-user projects consume these components to create their own custom CI solution, all while shifting the maintenance of the CI itself to CICD-Tools.

1. Pre-Commit Hooks

The pre-commit hooks are designed to be consumed by end-user projects directly: integration requires creating a .pre-commit-config.yaml file.

These hooks form the basic building blocks of your codebase's quality controls and are used in two ways:

  1. To perform standard pre-commit checks when committing code locally to end-user projects.
  2. As a way of keeping the CI and the local dev experience congruent, the pre-commit hooks that run at various stages are also run by the CI. (See the pre-commit documentation for more detail on stages.)

However, this leads to a problem where the tools used to perform these quality controls now need to be available…

2. Docker Containers

The CICD-Tools container provides vetted binaries that are integrated with the pre-commit hooks.

This allows a single container definition to securely provide most third party software. Where necessary, other trusted containers are leveraged to create a complete solution. Together these containers provide a way of leveraging third party tools without polluting your codebase with extra dependencies.

Finally, the containers ensure the same tools are used to check the codebase locally, and in the CI.

This should help in the elimination of environment drift (where the codebase or test tooling performs differently in local, test, or production environments).

3. GitHub "Jobs"

There is a large collection of remotely consumable GitHub Jobs that provide various CI/CD services. These Jobs are reusable GitHub workflows that are actively maintained as part of the CICD-Tools project.

  • It's important to stop for a moment and note that ONLY the files named "job" are intended to be consumed by end-user projects.
  • The "workflow" files are used to provide CI/CD to CICD-Tools itself.

The Job Naming Convention

Each Job is named in accordance with this convention:

  • job-step-project type-function

The step is an integer between 0 and 100 which tries to give you an idea of how far along the software release process it should be used:

  • Low digit steps are generally used early on as setup steps, leading the way to high digit steps that facilitate release or deployment.
  • Low digit steps may also be low complexity Jobs that can be used at any stage in the CI process.

The project type describes the type of project this workflow supports:

  • generic Jobs could be used by any sort of end-user project.
  • container Jobs are designed specifically for workflows that are generating or testing containers.
  • cookecutter Jobs are designed to be consumed by cookiecutter template projects.
  • mac_maker Jobs are designed to be used with the mac_maker binary tool.
  • poetry Jobs make use of the Python library poetry and as such require the presence of a pyproject.toml file. However, the projects themselves may not necessarily be Python projects.

The function indicates the specific deliverable or service the Job provides.

Using CICD-Tools "Jobs"

End-user projects contain workflows that in turn call the CICD-Tools Jobs, in a manner similar to this example.

It's important to look at the Job files themselves before incorporating them into your project- the Job files define a clear API and state their scripting requirements.

4. The Packaging System

The CICD-Tools Jobs in turn consume Toolboxes:

The Advantages of Decoupling

The Packaging System allows the API of each Job to be decoupled from its actual implementation:

  • This allows multiple Jobs to reuse the same Toolbox components.
  • This also allows the Toolboxes to be developed independently of the Jobs.
  • Finally, it's possible to customize your project's action-00-toolbox GitHub Action to use your own manifest. In this way you can create your own Toolbox tarballs and customize the CI even further; all while adhering to the same API.