Skip to content

Commit 14392b3

Browse files
Brian Woodssstabellini
Brian Woods
authored and
sstabellini
committed
automation: add automation/
Add support for gitlab CI. Add the automation/ directory with only arm64v8 support for now. These were taken from the Xen project and modified for runX. Only Alpine Linux is used since Debian was the only other arm64v8 distro and there were compiler issues since Debian unstable (what Xen used) is a moving target. Signed-off-by: Brian Woods <[email protected]> Reviewed-by: Stefano Stabellini <[email protected]>
1 parent 9250af6 commit 14392b3

18 files changed

+986
-0
lines changed

‎.gitlab-ci.yml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
stages:
2+
- build
3+
- test
4+
5+
include:
6+
- 'automation/gitlab-ci/build.yaml'
7+
- 'automation/gitlab-ci/test.yaml'

‎automation/build/Makefile

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
2+
# the base of where these containers will appear
3+
REGISTRY := registry.gitlab.com/lf-edge/runx
4+
CONTAINERS = $(subst .dockerfile,,$(wildcard */*.dockerfile))
5+
6+
help:
7+
@echo "Builds containers for building Xen based on different distros"
8+
@echo "To build one run 'make DISTRO/VERSION'. Available containers:"
9+
@$(foreach file,$(sort $(CONTAINERS)),echo ${file};)
10+
@echo "To push container builds, set the env var PUSH"
11+
12+
%: %.dockerfile ## Builds containers
13+
docker build -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
14+
@if [ ! -z $${PUSH+x} ]; then \
15+
docker push $(REGISTRY)/$(@D):$(@F); \
16+
fi
17+
18+
.PHONY: all
19+
all: $(CONTAINERS)

‎automation/build/README.md

+96
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
Docker Containers
2+
=================
3+
4+
These Docker containers should make it possible to build runX in
5+
any of the available environments on any system that supports
6+
running Docker. They are organized by distro and tagged with
7+
the version of that distro. They are available from the GitLab
8+
Container Registry under the Xen project at the [registry] and
9+
can be pulled with Docker from the following path:
10+
11+
```
12+
docker pull registry.gitlab.com/lf-edge/runx/DISTRO:VERSION
13+
```
14+
15+
To see the list of available containers run `make` in this
16+
directory. You will have to replace the `/` with a `:` to use
17+
them.
18+
19+
Building runX
20+
-------------
21+
22+
From the top level of the source tree it should be possible to
23+
run the following:
24+
25+
```
26+
./automation/scripts/containerize ./build.sh
27+
```
28+
29+
Which will cause the top level `./build.sh` to execute within the
30+
efault container, which is currently defined as Alpine 4.12. Any
31+
arguments specified to the script will be executed within the container
32+
from the default shell.
33+
34+
There are several environment variables which the containerize script
35+
understands.
36+
37+
- DOCKED_CMD: Whether to use docker or podman for running the containers.
38+
podman can be used as a regular user (rootless podman), but for that
39+
to work, /etc/subuid and /etc/subgid needs to containe the proper
40+
entries, for such user.
41+
docker is the default, for running with podman, do:
42+
43+
```
44+
DOCKER_CMD=podman ./automation/scripts/containerize ./build.sh
45+
```
46+
47+
- CONTAINER: This overrides the container to use. For Alpine 3.12
48+
aarch64, use:
49+
50+
```
51+
CONTAINER=alpine ./automation/scripts/containerize ./build.sh
52+
```
53+
54+
- CONTAINER_PATH: This overrides the path that will be available under the
55+
`/build` directory in the container, which is the default path.
56+
57+
```
58+
CONTAINER_PATH=/some/other/path ./automation/scripts/containerize ls
59+
```
60+
61+
- CONTAINER_ARGS: Allows you to pass extra arguments to Docker
62+
when starting the container.
63+
64+
- CONTAINER_UID0: This specifies whether root is used inside the container.
65+
66+
- CONTAINER_NO_PULL: If set to 1, the script will not pull from docker hub.
67+
This is useful when testing container locally.
68+
69+
- XEN_CONFIG_EXPERT: If this is defined in your shell it will be
70+
automatically passed through to the container.
71+
72+
If your docker host has Linux kernel > 4.11, and you want to use containers
73+
that run old glibc (for example, CentOS 6 or SLES11SP4), you may need to add
74+
75+
```
76+
vsyscall=emulate
77+
```
78+
79+
to the host kernel command line. That enables a legacy interface that is used
80+
by old glibc.
81+
82+
83+
Building a container
84+
--------------------
85+
86+
There is a makefile to make this process easier. You should be
87+
able to run `make DISTRO/VERSION` to have Docker build the container
88+
for you. If you define the `PUSH` environment variable when running the
89+
former `make` command, it will push the container to the [registry] if
90+
you have access to do so and have your Docker logged into the registry.
91+
92+
To login you must run `docker login registry.gitlab.com`. For more
93+
information see the [registry help].
94+
95+
[registry]: https://gitlab.com/runx/runx/container_registry
96+
[registry help]: https://gitlab.com/help/user/project/container_registry

‎automation/build/alpine/3.12-arm64v8.dockerfile

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
FROM arm64v8/alpine:3.12
2+
LABEL maintainer.name="The runX Project" \
3+
maintainer.email="[email protected]"
4+
5+
ENV USER root
6+
7+
RUN mkdir /build
8+
WORKDIR /build
9+
10+
# build depends
11+
RUN \
12+
# apk
13+
apk update && \
14+
\
15+
# for building runx
16+
apk add make && \
17+
apk add bash && \
18+
apk add bison && \
19+
apk add cpio && \
20+
apk add findutils && \
21+
apk add flex && \
22+
apk add gcc && \
23+
apk add glib-dev && \
24+
apk add gzip && \
25+
apk add openssl-dev && \
26+
apk add musl-dev && \
27+
apk add ncurses-dev && \
28+
apk add patch && \
29+
apk add perl && \
30+
apk add tar && \
31+
apk add wget && \
32+
\
33+
# for running qemu
34+
apk add elfutils && \
35+
apk add pixman && \
36+
apk add python3 && \
37+
apk add xz && \
38+
apk add zlib && \
39+
\
40+
# cleanup
41+
rm -rf /tmp/* && \
42+
rm -f /var/cache/apk/*

‎automation/gitlab-ci/build-each-commit.sh

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
# For a newly pushed branch the BEFORE_SHA will be all 0s
4+
if [[ ${BASE} == 0000000000000000000000000000000000000000 ]]; then
5+
echo "Newly pushed branch, skipped"
6+
exit 0
7+
fi
8+
9+
git merge-base --is-ancestor ${BASE} ${TIP}
10+
if [[ $? -ne 0 ]]; then
11+
echo "${TIP} is not a descendent of ${BASE}, skipped"
12+
exit 0
13+
fi
14+
15+
echo "Building ${BASE}..${TIP}"
16+
17+
NON_SYMBOLIC_REF=1 ./automation/scripts/build-test.sh ${BASE} ${TIP} \
18+
bash -c "git clean -ffdx && ./automation/scripts/build"

‎automation/gitlab-ci/build.yaml

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
.build-tmpl: &build
2+
stage: build
3+
image: registry.gitlab.com/lf-edge/runx/${CONTAINER}
4+
script:
5+
- ./automation/scripts/build 2>&1 | tee build.log
6+
artifacts:
7+
paths:
8+
- binaries/
9+
- target/
10+
- '*.log'
11+
when: always
12+
except:
13+
- master
14+
- smoke
15+
- /^coverity-tested\/.*/
16+
- /^stable-.*/
17+
18+
.gcc-tmpl:
19+
variabes: &gcc
20+
CC: gcc
21+
CXX: g++
22+
23+
.arm64-build-tmpl:
24+
<<: *build
25+
variables:
26+
XEN_TARGET_ARCH: arm64
27+
tags:
28+
- arm64
29+
30+
.arm64-build:
31+
extends: .arm64-build-tmpl
32+
variables:
33+
debug: n
34+
35+
.gcc-arm64-build:
36+
extends: .arm64-build
37+
variables:
38+
<<: *gcc
39+
40+
# Arm builds
41+
42+
alpine-3.12-gcc-arm64:
43+
extends: .gcc-arm64-build
44+
variables:
45+
CONTAINER: alpine:3.12-arm64v8
46+
47+
# Arm test artifacts
48+
49+
alpine-3.12-arm64-rootfs-export:
50+
stage: build
51+
image: registry.gitlab.com/lf-edge/runx/tests-artifacts/alpine:3.12-arm64v8
52+
script:
53+
- mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
54+
artifacts:
55+
paths:
56+
- binaries/initrd.tar.gz
57+
tags:
58+
- arm64
59+
60+
kernel-5.10.30-arm64-export:
61+
stage: build
62+
image: registry.gitlab.com/lf-edge/runx/tests-artifacts/kernel:5.10.30-arm64v8
63+
script:
64+
- mkdir binaries && cp /Image binaries/Image
65+
artifacts:
66+
paths:
67+
- binaries/Image
68+
tags:
69+
- arm64
70+
71+
qemu-system-aarch64-5.2.0-arm64-export:
72+
stage: build
73+
image: registry.gitlab.com/lf-edge/runx/tests-artifacts/qemu-system-aarch64:5.2.0-arm64v8
74+
script:
75+
- mkdir binaries && cp /qemu-system-aarch64 binaries/qemu-system-aarch64
76+
artifacts:
77+
paths:
78+
- binaries/qemu-system-aarch64
79+
tags:
80+
- arm64
81+
82+
xen-4.14-arm64-export:
83+
stage: build
84+
image: registry.gitlab.com/lf-edge/runx/tests-artifacts/xen:4.14-arm64v8
85+
script:
86+
- mkdir binaries && cp /xen.tar.gz binaries/xen.tar.gz
87+
artifacts:
88+
paths:
89+
- binaries/xen.tar.gz
90+
tags:
91+
- arm64

‎automation/gitlab-ci/test.yaml

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Test jobs
2+
qemu-alpine-arm64-gcc:
3+
stage: test
4+
image: registry.gitlab.com/lf-edge/runx/${CONTAINER}
5+
variables:
6+
CONTAINER: alpine:3.12-arm64v8
7+
CC: gcc
8+
script:
9+
- BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
10+
- mv ../build-each-commit-gcc.log .
11+
dependencies: []
12+
artifacts:
13+
paths:
14+
- smoke.serial
15+
- '*.log'
16+
when: always
17+
tags:
18+
- arm64
19+
except:
20+
- master
21+
- smoke
22+
- /^coverity-tested\/.*/
23+
- /^stable-.*/
24+
25+
26+
qemu-alpine-arm64-test:
27+
stage: test
28+
image: registry.gitlab.com/lf-edge/runx/${CONTAINER}
29+
variables:
30+
CONTAINER: alpine:3.12-arm64v8
31+
CC: gcc
32+
script:
33+
- ./automation/scripts/qemu-alpine-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
34+
dependencies:
35+
- alpine-3.12-gcc-arm64
36+
- alpine-3.12-arm64-rootfs-export
37+
- kernel-5.10.30-arm64-export
38+
- qemu-system-aarch64-5.2.0-arm64-export
39+
- xen-4.14-arm64-export
40+
artifacts:
41+
paths:
42+
- smoke.serial
43+
- '*.log'
44+
when: always
45+
tags:
46+
- arm64
47+
except:
48+
- master
49+
- smoke
50+
- /^coverity-tested\/.*/
51+
- /^stable-.*/

‎automation/scripts/build

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/bin/bash -ex
2+
3+
./build.sh clean
4+
5+
./build.sh

0 commit comments

Comments
 (0)