Add dockers to Fast and Basic Builds #1298

rbanka1 · 2025-05-06T15:26:57Z

Added UMF docker

// workflow executed on fork:
https://github.com/rbanka1/unified-memory-framework/actions/runs/15134054789
https://github.com/rbanka1/unified-memory-framework/actions/runs/15134054814

I added a new Docker image for Ubuntu 24.04 and updated the others.
I also created two new workflows that check for changes in the Docker files.
If any changes are detected, the relevant Docker image will be rebuilt and pushed (unless it's a pull request).
The rebuilt images will be available after the merge.

KFilipek · 2025-05-07T13:17:07Z

@lukaszstolarczuk
We have https://github.com/bb-umf account on GitHub.

KFilipek · 2025-05-07T13:28:58Z

.github/docker/ubuntu-20.04.Dockerfile


 # Dependencies for tests (optional)
 ARG TEST_DEPS="\
-	libnuma-dev"
+	libnuma-dev \
+	libtbb-dev\


Scalable pool relies on TBB library (src/pool/pool_scalable.c), so this is an UMF dependency.

The same as for libnuma - as long as you don't use that pool it's not required. Hence, I'd leave as test deps.

KFilipek · 2025-05-07T13:30:33Z

.github/docker/ubuntu-20.04.Dockerfile


 # Dependencies for tests (optional)
 ARG TEST_DEPS="\
-	libnuma-dev"
+	libnuma-dev \


It's requirement for OS memory provider, according to README.md

As mentioned in the readme, it's for testing :)

While I agree with you that for runtime this is a requirement, it is not the hard requirement. As long as you don't use that provider it's not really required, hence we only mark this as "for testing" (or, in the user's case - for the application that uses the provider).

I'd personally leave libnuma as test deps.

KFilipek · 2025-05-07T13:35:11Z

.github/workflows/reusable_basic.yml


 env:
  BUILD_DIR : "${{github.workspace}}/build"
-  INSTL_DIR : "${{github.workspace}}/install-dir"
+  INSTL_DIR : "${{github.workspace}}/../install-dir"


I believe going outside the workspace is not the best idea.

…-framework into newDocker

lukaszstolarczuk · 2025-05-14T11:01:59Z

.github/docker/ubuntu-20.04.Dockerfile

@@ -1,10 +1,10 @@
-# Copyright (C) 2024 Intel Corporation
+# Copyright (C) 2024-2025 Intel Corporation


pls order commits and add a proper commit title/message; while we're adding one new docker, the main change is something else 😉

pls also update the title of the PR, and if ready, please un-draft

lukaszstolarczuk · 2025-05-14T11:03:19Z

.github/docker/ubuntu-22.04.Dockerfile

@@ -20,21 +20,24 @@ ENV DEBIAN_FRONTEND noninteractive
 ARG BASE_DEPS="\
 	build-essential \
 	cmake \
-	git"
+	git \
+	libtool"


we don't install hwloc from source on this image, libtool seems unneeded here...?

no, it is required in Build UMF

https://github.com/rbanka1/unified-memory-framework/actions/runs/15064086737/job/42344995284#step:9:72

och well, this build uses the old container, I suppose 😉nonetheless, libtool is not required if you have hwloc installed

It seems that libtool isnt needed but it is required in build umf so I left it

lukaszstolarczuk · 2025-05-14T11:04:37Z

.github/docker/ubuntu-20.04.Dockerfile

-	libtbb-dev"
+# Hwloc installation dependencies
+ARG HWLOC_DEPS="\
+	dos2unix \


do we need this?

no, removed

lukaszstolarczuk · 2025-05-14T11:05:16Z

.github/docker/ubuntu-20.04.Dockerfile

+
+# Copy hwloc
+# libhwloc-dev is required - installed via script because hwloc version is to old on this OS
+COPY .github/scripts/install_hwloc.sh /opt/umf/install_hwloc.sh


pls move this COPY step after all ARGs

lukaszstolarczuk · 2025-05-14T11:05:33Z

.github/docker/ubuntu-20.04.Dockerfile

+	libtool"
+
+# Copy hwloc
+# libhwloc-dev is required - installed via script because hwloc version is to old on this OS


is too old

misspelling :D

lukaszstolarczuk · 2025-05-14T11:14:46Z

.github/workflows/reusable_basic.yml

-      if: matrix.compiler.cxx == 'g++-7'
-      run: sudo apt-get install -y ${{matrix.compiler.cxx}}
-
-    - name: Get UMF version


why this step is removed?

lukaszstolarczuk · 2025-05-14T11:14:53Z

.github/workflows/reusable_basic.yml

@@ -192,7 +166,6 @@ jobs:
        -DUMF_DISABLE_HWLOC=${{matrix.disable_hwloc}}
        -DUMF_LINK_HWLOC_STATICALLY=${{matrix.link_hwloc_statically}}
        ${{ matrix.build_type == 'Debug' && matrix.compiler.c == 'gcc' && '-DUMF_USE_COVERAGE=ON' || '' }}
-        ${{ matrix.llvm_linker || '' }}


why this is removed?

lukaszstolarczuk · 2025-05-14T11:15:29Z

.github/workflows/reusable_basic.yml

@@ -205,11 +178,14 @@ jobs:
        ${{ matrix.compiler.cxx == 'icpx' && '. /opt/intel/oneapi/setvars.sh' || true }}
        LD_LIBRARY_PATH="${{env.BUILD_DIR}}/lib/:${LD_LIBRARY_PATH}" ctest --output-on-failure

+    - name: Set OS name
+      run: echo "OS_VER=ubuntu-${{ matrix.ubuntu_ver }}" >> $GITHUB_ENV


we set OS and OS_VER already within docker image

lukaszstolarczuk · 2025-05-14T11:19:16Z

.github/workflows/reusable_basic.yml

    - name: Check coverage
      if:  ${{ matrix.build_type == 'Debug' && matrix.compiler.c == 'gcc' }}
      working-directory: ${{env.BUILD_DIR}}
      run: |
-        export COVERAGE_FILE_NAME=${{env.COVERAGE_NAME}}-${{matrix.os}}-shared-${{matrix.shared_library}}-no_hwloc-${{matrix.disable_hwloc}}
+        export COVERAGE_FILE_NAME=${{env.COVERAGE_NAME}}-${{env.OS_VER}}-shared-${{matrix.shared_library}}-no_hwloc-${{matrix.disable_hwloc}}


can't we still use matrix.os here?

yes but now I have 'ubuntu_ver' variable in matrix. If it is important I can change the name

either add ubuntu string here, or (preferably) bring back "os" var which contains full name of the os (as we will probably also add some fedora or other distros later on)

I added full name

lukaszstolarczuk · 2025-05-14T11:22:10Z

.github/workflows/reusable_basic.yml

    - name: Install oneAPI basekit
      if: matrix.compiler.cxx == 'icpx'
      run: |
+        sudo apt-get update


pls note sudo is probably not passwordless on the docker, so e.g. use syntax like here: https://github.com/pmem/pmemstream/blob/master/utils/docker/prepare-for-build.sh#L20

because I ran docker as root it is not necessary to use sudo, so removed it.

according to sec. rules, we should not use root as the default user, that's why I believe sudo should be used if needed for a specific step

done, now I ran docker as a test_user

rbanka1 added 2 commits May 5, 2025 15:58

add dockers

b668842

fix

c2d35fc

KFilipek requested changes May 8, 2025

View reviewed changes

rbanka1 added 2 commits May 8, 2025 11:22

test

ef8733e

Merge branch 'main' into newDocker

fd055d3

lukaszstolarczuk mentioned this pull request May 8, 2025

Docker workflow #1105

Closed

10 tasks

rbanka1 added 5 commits May 13, 2025 09:51

separated detect changes - test

5bc7506

Merge branch 'newDocker' of https://github.com/rbanka1/unified-memory…

4b44fe4

…-framework into newDocker

Merge branch 'main' into newDocker

b442229

separated detect changes - test

e14f45e

Merge branch 'newDocker' of https://github.com/rbanka1/unified-memory…

ae65dbf

…-framework into newDocker

rbanka1 force-pushed the newDocker branch 2 times, most recently from 6039c67 to b1d2ec2 Compare May 13, 2025 11:58

lukaszstolarczuk reviewed May 14, 2025

View reviewed changes

rbanka1 changed the title ~~New docker UMF~~ Add dockers to Fast and Basic Builds May 15, 2025

rbanka1 force-pushed the newDocker branch 4 times, most recently from 11980b4 to 3552ecc Compare May 20, 2025 09:33

corrections

aefd38a

rbanka1 force-pushed the newDocker branch from 3552ecc to aefd38a Compare May 20, 2025 10:43

		@@ -1,10 +1,10 @@
		# Copyright (C) 2024 Intel Corporation
		# Copyright (C) 2024-2025 Intel Corporation

Add dockers to Fast and Basic Builds #1298

Are you sure you want to change the base?

Add dockers to Fast and Basic Builds #1298

Conversation

rbanka1 commented May 6, 2025 • edited Loading

KFilipek commented May 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbanka1 commented May 6, 2025 •

edited

Loading

KFilipek commented May 7, 2025 •

edited

Loading