Make semaphore pipeline run less frequently, reduce machine sizes, avoid clearing cache #362

milindl · 2025-09-02T04:27:55Z

List of changes for making pipeline more efficient:

remove cache clear - should allow node dependencies to be cached from now on as they're cached automatically
reduction in size of agent whenever possible
switch out Linux: Build, test, lint, docs to use arm64 instead of amd64. This will be the most frequently run block
conditional running: Kafka and SR client blocks are run only when there's an applicable change. There's only one block which runs unconditionally on all PRs and that is "Linux: Build, test, lint, docs"
many blocks are now run on PRs, but not run on the master branch anymore - there's no point in repeating things. We don't merge non-passing PRs anyway.

…oid clearing cache

Copilot

Pull Request Overview

This PR optimizes the Semaphore CI pipeline by making it run less frequently, reducing machine resource usage, and removing cache clearing operations. The changes are aimed at improving efficiency and reducing costs while maintaining test coverage.

Adds conditional execution rules to prevent unnecessary pipeline runs on version tags
Downgrades machine types from size-2 to size-1 for resource optimization
Removes cache clearing step to improve build performance

Comments suppressed due to low confidence (1)

.semaphore/semaphore.yml:1

The same conditional logic is repeated multiple times. Consider defining this as a YAML anchor or variable to reduce duplication and make maintenance easier.

# This file is managed by ServiceBot plugin - Semaphore. The content in this file is created using a common

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-02T04:29:10Z

.semaphore/semaphore.yml


 blocks:
  - name: "Linux amd64 (musl): Build and test"
+    run:
+      when: "change_in('['/package.json', '/deps', '/lib', '/src', '/test']') AND tag !~ '^v[0-9]\\.'"


The change_in condition uses an array syntax with single quotes inside double quotes, which may cause parsing issues. Consider using proper YAML array syntax: change_in(['/package.json', '/deps', '/lib', '/src', '/test'])

.semaphore/semaphore.yml

Copilot · 2025-09-02T04:29:11Z

.semaphore/semaphore.yml

@@ -48,6 +49,8 @@ blocks:
            - docker run -v "$(pwd):/v" node:18-alpine /v/.semaphore/build-docker-alpine.sh

  - name: "Linux arm64 (musl): Build and test"
+    run:
+      when: "change_in('['/package.json', '/deps', '/lib', '/src', '/test']') AND tag !~ '^v[0-9]\\.'"


The same conditional logic is repeated multiple times. Consider defining this as a YAML anchor or variable to reduce duplication and make maintenance easier.

Copilot · 2025-09-02T04:29:11Z

.semaphore/semaphore.yml

-  - name: "Linux arm64: Build and test"
+  - name: "Linux amd64: Build and test"
+    run:
+      when: "change_in('['/package.json', '/deps', '/lib', '/src', '/test']') AND tag !~ '^v[0-9]\\.'"


The same conditional logic is repeated multiple times. Consider defining this as a YAML anchor or variable to reduce duplication and make maintenance easier.

Copilot · 2025-09-02T04:29:11Z

.semaphore/semaphore.yml

@@ -75,6 +80,8 @@ blocks:
            - make test

  - name: 'macOS arm64/m1: Build and test'
+    run:
+      when: "change_in('['/package.json', '/deps', '/lib', '/src', '/test']') AND tag !~ '^v[0-9]\\.'"


The same conditional logic is repeated multiple times. Consider defining this as a YAML anchor or variable to reduce duplication and make maintenance easier.

Copilot · 2025-09-02T04:29:11Z

.semaphore/semaphore.yml

@@ -146,6 +157,8 @@ blocks:
            - make integtest

  - name: "Linux amd64: Performance"
+    run:
+      when: "change_in('['/package.json', '/deps', '/lib', '/src', '/ci/tests']') AND tag !~ '^v[0-9]\\.'"


The same conditional logic is repeated multiple times. Consider defining this as a YAML anchor or variable to reduce duplication and make maintenance easier.

… not needed

… PR anyway

emasab · 2025-09-11T10:37:15Z

.semaphore/semaphore.yml

@@ -48,6 +49,8 @@ blocks:
            - docker run -v "$(pwd):/v" node:18-alpine /v/.semaphore/build-docker-alpine.sh

  - name: "Linux arm64 (musl): Build and test"
+    run:
+      when: "change_in(['/package.json', '/lib', '/src', '/test']) AND tag !~ '^v[0-9]\\.' AND branch != 'master'"


We should run tests on master commits as it's possible squash and merge succeeds and there are no conflicts but code isn't rebased before that.

sonarqube-confluent · 2025-09-17T09:55:19Z

Analysis Details

0 Issues

0 Bugs
0 Vulnerabilities
0 Code Smells

Coverage and Duplications

No coverage information (70.80% Estimated after merge)
No duplication information (2.00% Estimated after merge)

Project ID: confluent-kafka-javascript

View in SonarQube

Make semaphore pipeline run less frequently, reduce machine sizes, av…

3c687eb

…oid clearing cache

Copilot AI review requested due to automatic review settings September 2, 2025 04:27

milindl requested review from a team as code owners September 2, 2025 04:27

Copilot AI reviewed Sep 2, 2025

View reviewed changes

milindl added 4 commits September 2, 2025 04:34

Fix change_in

3528b03

Add pipeline ignore temporarily to change in

3c1c48a

Test changes for change_in pipeline

ce4657c

Test changes for change_in pipeline - 2

d39013b