DAOS-17344 build: Add dependencies as submodules #16162

jolivier23 · 2025-03-27T05:33:43Z

As a step to building all RPMs in DAOS repo, Add dependencies as submodules. Eventually, we may want these to be snapshots.

Tested that it builds locally

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

Add CopyRetriever as alternative to GitRepoRetriever. Use it for submodule option at first. Add support for internal, static library Add patch support to rpm source rpm See PR#16134 for example Signed-off-by: Jeff Olivier <[email protected]>

github-actions · 2025-03-27T05:34:03Z

Ticket title is 'Better support for internal components'
Status is 'Open'
https://daosio.atlassian.net/browse/DAOS-17344

As a step to building all RPMs in DAOS repo, Add dependencies as submodules. Eventually, we may want these to be snapshots. Tested that it builds locally Signed-off-by: Jeff Olivier <[email protected]>

Signed-off-by: Jeff Olivier <[email protected]>

daosbuild1 · 2025-04-02T17:38:09Z

Test stage Python Bandit check completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16162/15/execution/node/135/log

daosbuild1 · 2025-04-02T22:36:47Z

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16162/15/execution/node/274/log

Add backoffs to retries of pool and container rsvc operations. Move tse_task_reinit calls to the end of each completion callback, in order to reduce the chance of new data races between a task and its reinitialized instance. Signed-off-by: Li Wei <[email protected]>

Change-Id: I0817d2aaae3bf6f52d71661d6f7ec271f59c55c7 Signed-off-by: Jeff Olivier <[email protected]>

Signed-off-by: Jeff Olivier <[email protected]>

daosbuild1 · 2025-04-15T17:16:14Z

Test stage Functional on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-16162/41/testReport/

Signed-off-by: Jeff Olivier <[email protected]>

daosbuild1 · 2025-04-16T07:14:25Z

Test stage Test RPMs on EL 8.6 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-16162/42/display/redirect

knard38

Overall LGTM

Jenkinsfile

knard38 · 2025-04-16T08:54:13Z

deps/prepare_submodule.sh

+  field_name=$2
+
+  # shellcheck disable=SC1090,SC2283
+  source <(grep = <(grep -A10 "\[${field_name}\]" "${DAOS_ROOT}/utils/build.config"))


To not depend on the number of line:

Suggested change

source <(grep = <(grep -A10 "\[${field_name}\]" "${DAOS_ROOT}/utils/build.config"))

source <(awk -v RS= "/\[${field_name}\]/" "${DAOS_ROOT}/utils/build.config" | tail -n+2)

daosbuild1 · 2025-04-16T13:59:29Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-16162/42/display/redirect

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 · 2025-04-17T23:29:02Z

No real changes, had to resolve merge conflict on daos_build.py

daosbuild1 · 2025-04-18T22:36:47Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16162/45/execution/node/1471/log

This reverts commit 9ec84fc.

This reverts commit 9ec84fc. Allow-unstable-test: true Priority: 2 Cancel-prev-build: false Signed-off-by: Tomasz Gromadzki <[email protected]>

This reverts commit 4a367f6.

Primary purpose is to bring in title patch. But in order to do that easily, there are a few other build patches pulled in. Most of them are build only but one does update libfuse to fused. Google has been using this exclusively for a few months. DAOS-15136 client: Switch to using libfused (#14077) DAOS-17117 build: build dependencies as git submodules (#15909) DAOS-11866 build: Add --build-deps=fetch option (#14125) DAOS-623 build: Minor build fix (#15749) DAOS-623 build: Update developer package for fused (#15746) DAOS-17344 build: Enhance SCons options (#16157) Co-authored-by: Ashley Pittman <[email protected]> Signed-off-by: Nicholas Murphy <[email protected]> Signed-off-by: Jeff Olivier <[email protected]>

DAOS-17344 build: Enhance SCons options

54937f3

Add CopyRetriever as alternative to GitRepoRetriever. Use it for submodule option at first. Add support for internal, static library Add patch support to rpm source rpm See PR#16134 for example Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 force-pushed the jvolivie/sub_deps branch 3 times, most recently from 3476c13 to 405d208 Compare March 27, 2025 05:51

DAOS-17344 build: Add dependencies as submodules

3f3141b

As a step to building all RPMs in DAOS repo, Add dependencies as submodules. Eventually, we may want these to be snapshots. Tested that it builds locally Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 force-pushed the jvolivie/sub_deps branch from 405d208 to 3f3141b Compare March 27, 2025 05:57

jolivier23 added 7 commits March 27, 2025 00:02

fix pylint issue

f40a625

Signed-off-by: Jeff Olivier <[email protected]>

Merge branch 'jvolivie/fix_copy' into jvolivie/sub_deps

70bad8f

Fix scripts

98fb972

Signed-off-by: Jeff Olivier <[email protected]>

Don't copy patches already in right place

d65c5a8

Signed-off-by: Jeff Olivier <[email protected]>

Add argobots patches and fix script

d5b9c54

Signed-off-by: Jeff Olivier <[email protected]>

Fix isort linting issue

93fdf2a

Signed-off-by: Jeff Olivier <[email protected]>

Get the submodules

ef33152

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 force-pushed the jvolivie/sub_deps branch from c06eee6 to ef33152 Compare March 27, 2025 16:59

jolivier23 added 3 commits March 27, 2025 14:02

Fix spdk issue

b92318d

Signed-off-by: Jeff Olivier <[email protected]>

Merge branch 'jvolivie/fix_copy' into jvolivie/sub_deps

8ae8f96

Extra commit after merge

45100d1

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 mentioned this pull request Mar 27, 2025

DAOS-17344 build: Enhance SCons options #16157

Merged

6 tasks

jolivier23 added 3 commits March 28, 2025 12:04

Fix archive script

a74e861

Signed-off-by: Jeff Olivier <[email protected]>

Fix archive script

a05ae66

Signed-off-by: Jeff Olivier <[email protected]>

Merge branch 'jvolivie/fix_copy' into jvolivie/sub_deps

fce0ae4

liw and others added 6 commits April 3, 2025 12:27

Fix build issue

457582b

Change-Id: I0817d2aaae3bf6f52d71661d6f7ec271f59c55c7 Signed-off-by: Jeff Olivier <[email protected]>

Merge remote-tracking branch 'comm/master' into jvolivie/fix_copy

dc364fe

Extra commit

8b13f7a

Signed-off-by: Jeff Olivier <[email protected]>

Merge branch 'jvolivie/fix_copy' into jvolivie/sub_deps

f73565f

Empty commit

b492a19

Signed-off-by: Jeff Olivier <[email protected]>

daltonbohning previously approved these changes Apr 15, 2025

View reviewed changes

Do recursive update in daos_build test

c70b58a

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 dismissed daltonbohning’s stale review via c70b58a April 16, 2025 04:04

jolivier23 added 2 commits April 15, 2025 22:04

Merge remote-tracking branch 'comm/master' into jvolivie/sub_deps

d843e47

Empty commit

bb14766

Signed-off-by: Jeff Olivier <[email protected]>

knard38 previously approved these changes Apr 16, 2025

View reviewed changes

Review suggestions

5b90e18

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 dismissed knard38’s stale review via 5b90e18 April 16, 2025 17:33

jolivier23 requested review from daltonbohning and knard38 April 16, 2025 18:56

daltonbohning previously approved these changes Apr 16, 2025

View reviewed changes

knard38 previously approved these changes Apr 17, 2025

View reviewed changes

Merge remote-tracking branch 'comm/master' into jvolivie/sub_deps

295f814

Signed-off-by: Jeff Olivier <[email protected]>

jolivier23 dismissed stale reviews from knard38 and daltonbohning via 295f814 April 17, 2025 23:24

jolivier23 requested review from daltonbohning and knard38 April 17, 2025 23:25

daltonbohning approved these changes Apr 17, 2025

View reviewed changes

knard38 approved these changes Apr 18, 2025

View reviewed changes

jolivier23 merged commit 9ec84fc into master Apr 19, 2025
60 of 64 checks passed

jolivier23 deleted the jvolivie/sub_deps branch April 19, 2025 17:13

grom72 added a commit that referenced this pull request May 6, 2025

Revert "DAOS-17344 build: Add dependencies as submodules (#16162)"

3ae295c

This reverts commit 9ec84fc.

grom72 added a commit that referenced this pull request May 6, 2025

Revert "DAOS-17344 build: Add dependencies as submodules (#16162)"

4a367f6

This reverts commit 9ec84fc. Allow-unstable-test: true Priority: 2 Cancel-prev-build: false Signed-off-by: Tomasz Gromadzki <[email protected]>

grom72 added a commit that referenced this pull request May 8, 2025

Reapply "DAOS-17344 build: Add dependencies as submodules (#16162)"

e45d68e

This reverts commit 4a367f6.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

DAOS-17344 build: Add dependencies as submodules #16162

DAOS-17344 build: Add dependencies as submodules #16162

Uh oh!

jolivier23 commented Mar 27, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

daosbuild1 commented Apr 2, 2025

Uh oh!

daosbuild1 commented Apr 2, 2025

Uh oh!

daosbuild1 commented Apr 15, 2025

Uh oh!

daosbuild1 commented Apr 16, 2025

Uh oh!

knard38 left a comment

Uh oh!

Uh oh!

knard38 Apr 16, 2025

Uh oh!

jolivier23 Apr 16, 2025

Uh oh!

daosbuild1 commented Apr 16, 2025

Uh oh!

jolivier23 commented Apr 17, 2025

Uh oh!

daosbuild1 commented Apr 18, 2025

Uh oh!

Uh oh!

Uh oh!

	source <(grep = <(grep -A10 "\[${field_name}\]" "${DAOS_ROOT}/utils/build.config"))
	source <(awk -v RS= "/\[${field_name}\]/" "${DAOS_ROOT}/utils/build.config" \| tail -n+2)

DAOS-17344 build: Add dependencies as submodules #16162

DAOS-17344 build: Add dependencies as submodules #16162

Uh oh!

Conversation

jolivier23 commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

daosbuild1 commented Apr 2, 2025

Uh oh!

daosbuild1 commented Apr 2, 2025

Uh oh!

daosbuild1 commented Apr 15, 2025

Uh oh!

daosbuild1 commented Apr 16, 2025

Uh oh!

knard38 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

knard38 Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

jolivier23 Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

daosbuild1 commented Apr 16, 2025

Uh oh!

jolivier23 commented Apr 17, 2025

Uh oh!

daosbuild1 commented Apr 18, 2025

Uh oh!

Uh oh!

Uh oh!

jolivier23 commented Mar 27, 2025 •

edited

Loading