Skip to content

Commit f5ce047

Browse files
max-modelsspossann
max-models
and
spossann
authored
Only run the unit tests which touch changed code in the ci (#96)
Redo of #50 **Solves the following issue(s):** Closes #45 **Core changes:** There are now four PR workflows that must pass for a feature to be merged: 1. unit 2. models 3. pure Python 4. with clones Tests are made with and without MPI with `--testmon`. Testmon data is cached for the next workflow run to use. Cache cannot be altered, thus new cache keys are necessary in every run. The key convention is, for unit tests for example: ``` key: testmon-unit-${{ github.event.number }}-${{ github.run_number }} restore-keys: | testmon-unit-${{ github.event.number }}- testmon-unit- ``` The `event.number` is the number of the PR. The `run_number` refers to the workflow run, thus it increases monotonically. The one that is fetched has a run number one less than the current run, and is thus the most recent. --------- Co-authored-by: Stefan Possanner <[email protected]> Co-authored-by: Stefan Possanner <[email protected]>
1 parent d31c648 commit f5ce047

27 files changed

+489
-1700
lines changed

‎.github/actions/install/install-struphy/action.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: "Clone and install struphy"
1+
name: "Install struphy in env"
22

33
inputs:
44
optional-deps:
@@ -12,6 +12,8 @@ runs:
1212
- name: Install struphy
1313
shell: bash
1414
run: |
15+
python3 -m venv env
16+
source env/bin/activate
1517
pip install --upgrade pip
1618
pip install ".[${{ inputs.optional-deps }}]"
1719
pip list

‎.github/actions/tests/unit-mpi/action.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
name: "Run unit tests with MPI"
2+
3+
runs:
4+
using: composite
5+
steps:
6+
- name: Run unit tests with MPI
7+
shell: bash
8+
run: |
9+
struphy test unit --mpi 2

‎.github/actions/tests/unit/action.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ runs:
55
steps:
66
- name: Run unit tests with MPI
77
shell: bash
8+
env:
9+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-${{ matrix.test-type }}
810
run: |
911
struphy compile --status
1012
# run: |

‎.github/workflows/test-PR-models-clones.yml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: PR - domain cloning in Container
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- devel
7+
push:
8+
branches:
9+
- devel
10+
workflow_dispatch:
11+
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
defaults:
17+
run:
18+
shell: bash
19+
20+
permissions:
21+
contents: read
22+
23+
jobs:
24+
model-tests-in-container-with-struphy:
25+
runs-on: ubuntu-latest
26+
27+
container:
28+
image: ghcr.io/struphy-hub/struphy/ubuntu-with-struphy:latest
29+
credentials:
30+
username: spossann
31+
password: ${{ secrets.GHCR_TOKEN }}
32+
33+
steps:
34+
- name: Check for dockerenv file
35+
run: (ls /.dockerenv && echo Found dockerenv) || (echo No dockerenv)
36+
37+
- name: Checkout repo
38+
uses: actions/checkout@v4
39+
40+
- name: Check .testmondata 1
41+
run: |
42+
ls .testmon* || echo "No .testmondata"
43+
44+
- name: Setup cache for testmon
45+
uses: actions/cache@v4
46+
with:
47+
path: |
48+
.testmondata-clones
49+
key: testmon-clones-${{ github.ref_name }}-${{ github.event.number }}-${{ github.run_number }}
50+
restore-keys: |
51+
testmon-clones-${{ github.ref_name }}-${{ github.event.number }}-
52+
testmon-clones-${{ github.ref_name }}
53+
testmon-clones-devel
54+
testmon-clones-
55+
56+
- name: Check .testmondata 2
57+
run: |
58+
ls .testmon* || echo "No .testmondata"
59+
60+
# This step can be removed when the updated ghcr images are pushed
61+
- name: Install sqlite
62+
shell: bash
63+
run: apt install -y sqlite3
64+
65+
- name: Install Struphy in Container
66+
uses: ./.github/actions/install/struphy_in_container
67+
68+
- name: Compile Struphy
69+
run: |
70+
source /struphy_c_/env_c_/bin/activate
71+
struphy compile
72+
73+
- name: Verification tests with clones
74+
shell: bash
75+
env:
76+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-clones
77+
run: |
78+
source /struphy_c_/env_c_/bin/activate
79+
struphy test verification --mpi 4 --nclones 2

‎.github/workflows/test-PR-models.yml‎

Lines changed: 66 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@ on:
44
pull_request:
55
branches:
66
- devel
7+
push:
8+
branches:
9+
- devel
710
workflow_dispatch:
811

912
concurrency:
@@ -20,51 +23,98 @@ permissions:
2023
jobs:
2124
model-tests-in-container-with-struphy:
2225
runs-on: ubuntu-latest
26+
2327
container:
2428
image: ghcr.io/struphy-hub/struphy/ubuntu-with-struphy:latest
2529
credentials:
2630
username: spossann
2731
password: ${{ secrets.GHCR_TOKEN }}
28-
steps:
2932

33+
steps:
3034
- name: Check for dockerenv file
3135
run: (ls /.dockerenv && echo Found dockerenv) || (echo No dockerenv)
3236

3337
- name: Checkout repo
3438
uses: actions/checkout@v4
3539

40+
- name: Check .testmondata 1
41+
run: |
42+
ls .testmon* || echo "No .testmondata"
43+
44+
- name: Setup cache for testmon
45+
uses: actions/cache@v4
46+
with:
47+
path: |
48+
.testmondata-model
49+
.testmondata-model-mpi
50+
key: testmon-model-${{ github.ref_name }}-${{ github.event.number }}-${{ github.run_number }}
51+
restore-keys: |
52+
testmon-model-${{ github.ref_name }}-${{ github.event.number }}-
53+
testmon-model-${{ github.ref_name }}
54+
testmon-model-devel
55+
testmon-model-
56+
57+
- name: Check .testmondata 2
58+
run: |
59+
ls .testmon* || echo "No .testmondata"
60+
61+
# This step can be removed when the updated ghcr images are pushed
62+
- name: Install sqlite
63+
shell: bash
64+
run: apt install -y sqlite3
65+
3666
- name: Install Struphy in Container
3767
uses: ./.github/actions/install/struphy_in_container
3868

3969
- name: Compile Struphy
4070
run: |
41-
which python3
4271
source /struphy_c_/env_c_/bin/activate
43-
which python3
4472
struphy compile
4573
46-
- name: Model tests
74+
- name: LinearMHD test
4775
shell: bash
76+
env:
77+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model
4878
run: |
49-
which python3
5079
source /struphy_c_/env_c_/bin/activate
51-
which python3
52-
struphy compile --status
5380
struphy test LinearMHD
81+
82+
- name: Toy test
83+
shell: bash
84+
env:
85+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model
86+
run: |
87+
source /struphy_c_/env_c_/bin/activate
5488
struphy test toy
89+
90+
- name: Model tests
91+
shell: bash
92+
env:
93+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model
94+
run: |
95+
source /struphy_c_/env_c_/bin/activate
5596
struphy test models
56-
struphy test verification
97+
98+
- name: Verification tests
99+
shell: bash
100+
env:
101+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model
102+
run: |
103+
source /struphy_c_/env_c_/bin/activate
104+
struphy test verification
57105
58106
- name: Model tests with MPI
59107
shell: bash
108+
env:
109+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model-mpi
60110
run: |
61-
which python3
62111
source /struphy_c_/env_c_/bin/activate
63-
which python3
64-
struphy compile --status
65-
struphy test models
66112
struphy test models --mpi 2
67-
struphy test verification --mpi 1
68-
struphy test verification --mpi 4
69-
struphy test verification --mpi 4 --nclones 2
70-
struphy test VlasovAmpereOneSpecies --mpi 2 --nclones 2
113+
114+
- name: Verification tests with 2 MPI
115+
shell: bash
116+
env:
117+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-model-mpi
118+
run: |
119+
source /struphy_c_/env_c_/bin/activate
120+
struphy test verification --mpi 2

‎.github/workflows/test-PR-pure-python.yml‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
name: PR - pure Python tests in Container
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- devel
7+
push:
8+
branches:
9+
- devel
10+
workflow_dispatch:
11+
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
defaults:
17+
run:
18+
shell: bash
19+
20+
permissions:
21+
contents: read
22+
23+
jobs:
24+
model-tests-in-container-with-reqs:
25+
runs-on: ubuntu-latest
26+
27+
container:
28+
image: ghcr.io/struphy-hub/struphy/ubuntu-with-reqs:latest
29+
credentials:
30+
username: spossann
31+
password: ${{ secrets.GHCR_TOKEN }}
32+
33+
steps:
34+
- name: Check for dockerenv file
35+
run: (ls /.dockerenv && echo Found dockerenv) || (echo No dockerenv)
36+
37+
- name: Checkout repo
38+
uses: actions/checkout@v4
39+
40+
- name: Check .testmondata 1
41+
run: |
42+
ls .testmon* || echo "No .testmondata"
43+
44+
- name: Setup cache for testmon
45+
uses: actions/cache@v4
46+
with:
47+
path: |
48+
.testmondata-pure-python
49+
.testmondata-pure-python-mpi
50+
key: testmon-pure-python-${{ github.ref_name }}-${{ github.event.number }}-${{ github.run_number }}
51+
restore-keys: |
52+
testmon-pure-python-${{ github.ref_name }}-${{ github.event.number }}-
53+
testmon-pure-python-${{ github.ref_name }}
54+
testmon-pure-python-devel
55+
testmon-pure-python-
56+
57+
- name: Check .testmondata 2
58+
run: |
59+
ls .testmon* || echo "No .testmondata"
60+
61+
# This step can be removed when the updated ghcr images are pushed
62+
- name: Install sqlite
63+
shell: bash
64+
run: apt install -y sqlite3
65+
66+
- name: Install Struphy
67+
uses: ./.github/actions/install/install-struphy
68+
69+
- name: LinearMHD test
70+
shell: bash
71+
env:
72+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python
73+
run: |
74+
source env/bin/activate
75+
struphy compile --status
76+
struphy test LinearMHD
77+
78+
- name: Vlasov test
79+
shell: bash
80+
env:
81+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python
82+
run: |
83+
source env/bin/activate
84+
struphy test Vlasov
85+
86+
- name: GuidingCenter test
87+
shell: bash
88+
env:
89+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python
90+
run: |
91+
source env/bin/activate
92+
struphy test GuidingCenter
93+
94+
- name: VlasovAmpere test
95+
shell: bash
96+
env:
97+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python
98+
run: |
99+
source env/bin/activate
100+
struphy test VlasovAmpereOneSpecies
101+
102+
- name: EulerSPH test
103+
shell: bash
104+
env:
105+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python
106+
run: |
107+
source env/bin/activate
108+
struphy test EulerSPH
109+
110+
- name: Vlasov test MPI
111+
shell: bash
112+
env:
113+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python-mpi
114+
run: |
115+
source env/bin/activate
116+
struphy test Vlasov --mpi 2
117+
118+
- name: GuidingCenter test MPI
119+
shell: bash
120+
env:
121+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python-mpi
122+
run: |
123+
source env/bin/activate
124+
struphy test GuidingCenter --mpi 2
125+
126+
- name: VlasovAmpere test MPI
127+
shell: bash
128+
env:
129+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python-mpi
130+
run: |
131+
source env/bin/activate
132+
struphy test VlasovAmpereOneSpecies --mpi 2
133+
134+
- name: EulerSPH test MPI
135+
shell: bash
136+
env:
137+
TESTMON_DATAFILE: ${{ github.workspace }}/.testmondata-pure-python-mpi
138+
run: |
139+
source env/bin/activate
140+
struphy test EulerSPH --mpi 2

0 commit comments

Comments
 (0)