Skip to content

enh(ci): fixes after #389 review - #412

Open
sdepassio wants to merge 4 commits into
developfrom
CTOR-2525-fix-ci-updates
Open

sdepassio wants to merge 4 commits into
developfrom
CTOR-2525-fix-ci-updates

Conversation

@sdepassio

Copy link
Copy Markdown
Contributor

Description

Fixes for CI after #389 review.

Type of change

  • Patch fixing an issue (non-breaking change)
  • New functionality (non-breaking change)
  • Breaking change (patch or feature) that might cause side effects breaking part of the Software

Target serie

  • 24.04.x
  • 24.10.x
  • 25.10.x
  • master

How this pull request can be tested ?

Please describe the procedure to verify that the goal of the PR is matched. Provide clear instructions so that it can be correctly tested.

Any relevant details of the configuration to perform the test should be added.

Checklist

Community contributors & Centreon team

  • I have followed the coding style guidelines provided by Centreon
  • I have commented my code, especially new classes, functions or any legacy code modified. (docblock)
  • I have commented my code, especially hard-to-understand areas of the PR.
  • I have rebased my development branch on the base branch (master, maintenance).

…ith the delivered sc)

don't build/test everything when only the "stream-connectors-ci" workflow is updated (it's only a dispatcher)
@sdepassio
sdepassio force-pushed the CTOR-2525-fix-ci-updates branch from 635ffea to 73b37f5 Compare October 7, 2026 09:34
@sdepassio
sdepassio requested a review from tuntoja October 7, 2026 09:34
@sdepassio
sdepassio marked this pull request as ready for review October 7, 2026 09:34
@sdepassio
sdepassio requested a review from a team as a code owner October 7, 2026 09:35
@sdepassio
sdepassio requested review from kduret and pkriko and removed request for a team October 7, 2026 09:35

@tuntoja tuntoja left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. The orchestrator split, the single ci-status check and delivering the lib only after the connectors are tested against it all look good to me.

Three points, two with a suggestion you can apply directly:

  1. The lib is not delivered when only its version changes. After merge, unstable would get connectors requiring lib >= 3.9.0 while the lib there stays 3.8.1.
  2. Dependency fixes of the connectors set in stream-connectors.yml are no longer delivered.
  3. The connectors are delivered before the lib they depend on. On a release branch, testing can end up with connectors that cannot be installed.

Note for #409 / #410 (MON-205723), which will be rebased on this: the release tag trigger will move to stream-connectors-ci.yml, and .version.stream-connectors-lib will replace the version literal in point 1.

Comment on lines +61 to +63
lib_code: &lib_code
- modules/centreon-stream-connectors-lib/**
- packaging/connectors-lib/**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The library is not delivered when only its version changes.

The lib version is set in stream-connectors-lib.yml (version: "3.9.0"), which is only in the ci group. As a result, deliver_lib stays false when the version is bumped.

What happens when this PR is merged on develop:

  • stream-connectors-lib.yml changed, so the lib 3.9.0 is built and tested, but not delivered.
  • packaging/connectors/** changed, so all the connectors are delivered to unstable, and they now require centreon-stream-connectors-lib >= 3.9.0.
  • Unstable ends up with connectors that cannot be installed, because the lib there is still 3.8.1. The run stays green, because the connectors are tested against the lib built in the same run.

Suggested fix: deliver the lib when its workflow changes. A CI-only change then re-delivers the same version to unstable, which is allowed there. #410 will later replace this line with a .version.stream-connectors-lib file.

Suggested change
lib_code: &lib_code
- modules/centreon-stream-connectors-lib/**
- packaging/connectors-lib/**
lib_code: &lib_code
- modules/centreon-stream-connectors-lib/**
- packaging/connectors-lib/**
# the library version is set in its workflow: a version bump must deliver the library
- .github/workflows/stream-connectors-lib.yml

Comment on lines +45 to 47
# changes of the CI files are handled by stream-connectors-ci.yml (build_all_connectors input)
packaging:
- modified: .github/workflows/stream-connectors.yml
- added|modified: packaging/connectors/**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependency fixes of the connectors are no longer delivered.

This workflow is not only CI: it holds packaging data of the connectors. The "Add specific dependencies" step sets the rpm/deb dependencies of each connector (for example the libcurl4t64 fix for trixie), and the "Replace package and connector name variables" step sets the package names.

With this line removed, a fix in those steps is built and tested (through build_all_connectors) but never delivered: users only get it when a connector file also changes.

Suggested fix: keep the workflow in the packaging filter, so that a change here delivers every connector, as before #389. A later improvement would be to move the dependency map into packaging/connectors/, and then drop this line.

Suggested change
# changes of the CI files are handled by stream-connectors-ci.yml (build_all_connectors input)
packaging:
- modified: .github/workflows/stream-connectors.yml
- added|modified: packaging/connectors/**
# the dependencies of the stream connectors packages are set in this workflow
packaging:
- modified: .github/workflows/stream-connectors.yml
- added|modified: packaging/connectors/**

Comment on lines +102 to +105
# the library is delivered only once the stream connectors have also been tested against it
stream-connectors-lib-delivery:
needs: [detect-changes, stream-connectors-lib, stream-connectors]
if: ${{ !cancelled() && needs.detect-changes.outputs.deliver_lib == 'true' && needs.stream-connectors-lib.result == 'success' && needs.stream-connectors.result != 'failure' }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connectors are delivered before the library they depend on.

The connectors are delivered inside the stream-connectors job (its deliver-* jobs), while the lib is delivered afterwards by stream-connectors-lib-delivery. When both change in the same run (as in this PR: lib 3.9.0, connectors requiring >= 3.9.0):

  • the repositories hold connectors whose lib isn't published yet, at least for a while;
  • if the lib delivery fails, they stay that way: unstable, or testing on a release branch, which QA installs from, contains connectors that cannot be installed.

Suggested fix: deliver the connectors after the lib, the same way this PR already splits the lib delivery.

  1. In stream-connectors.yml, move deliver-rpm, deliver-deb and deliver-pulp to a new stream-connectors-delivery.yml, and expose the list of connectors to deliver:
on:
  workflow_call:
    inputs:
      # ... existing inputs
    outputs:
      delivered_connectors:
        description: "JSON list of the stream connectors built in this run that must be delivered"
        value: ${{ jobs.detect-changes.outputs.delivered_connectors }}
  1. stream-connectors-delivery.yml holds the three jobs unchanged, except that the matrix reads the input. The cache keys built by the package job still match within the run, as for the lib:
name: stream-connectors-delivery

# triggered by stream-connectors-ci.yml, once the library (if any) has been delivered
on:
  workflow_call:
    inputs:
      connectors:
        description: "JSON list of the stream connectors to deliver"
        type: string
        required: true

jobs:
  get-environment:
    uses: ./.github/workflows/get-environment.yml

  deliver-rpm:
    needs: [get-environment]
    # ...
    strategy:
      matrix:
        distrib: [el8, el9, el10]
        connector_path: ${{ fromJson(inputs.connectors) }}
  # deliver-deb / deliver-pulp: same change
  1. Call it in this workflow, after the lib delivery, and only if the lib delivery didn't fail:
  # the stream connectors are delivered after the library they depend on
  stream-connectors-delivery:
    needs: [stream-connectors, stream-connectors-lib-delivery]
    if: ${{ !cancelled() && needs.stream-connectors.result == 'success' && needs.stream-connectors.outputs.delivered_connectors != '[]' && needs.stream-connectors-lib-delivery.result != 'failure' }}
    uses: ./.github/workflows/stream-connectors-delivery.yml
    with:
      connectors: ${{ needs.stream-connectors.outputs.delivered_connectors }}
    secrets: inherit
    permissions:
      contents: read
      id-token: write
      pull-requests: read

Also add stream-connectors-delivery to the ci filter and to the needs of ci-status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants