Add support for modular build structure.

[grafikrobot]

This is part of the effort to make the Boost libraries "modular" for build and consumption. See https://lists.boost.org/Archives/boost/2024/01/255704.php and https://github.com/grafikrobot/boost-b2-modular/blob/b2-modular/README.adoc for more information.

This PR depends on the following other PRs being merged to both develop and master branches of the respective repos:

• Add support for modular build structure. boost#854

This PR will be changed to ready for review, i.e. not draft, when the above are merged. Do not merge this one until that time.

[grafikrobot]

Please review and merge this PR at your earliest convenience.

[mborland]

@grafikrobot Do the modular changes come with an increase in memory usage? The only failure in the CI is OOM with UBSAN that I have not seen before.

[raffienficiaud]

I am seeing a "no space left on device" error.

[grafikrobot]

@grafikrobot Do the modular changes come with an increase in memory usage? The only failure in the CI is OOM with UBSAN that I have not seen before.

They don't as they are build system changes only. But if those tests where close to OOM before it's not surprising that it might OOM at any time.

[mborland]

@grafikrobot Do the modular changes come with an increase in memory usage? The only failure in the CI is OOM with UBSAN that I have not seen before.

They don't as they are build system changes only. But if those tests where close to OOM before it's not surprising that it might OOM at any time.

Build system changes could definitely change the memory consumption. Since test has quite a few dependencies are the build sizes all the same? Were build sizes tested before and after modularity to ensure size parity? Is each dependency dragging in copies of all it's dependencies, or is it like how it used to be where there's just one copy of everything?

[grafikrobot]

@grafikrobot Do the modular changes come with an increase in memory usage? The only failure in the CI is OOM with UBSAN that I have not seen before.

They don't as they are build system changes only. But if those tests where close to OOM before it's not surprising that it might OOM at any time.

Build system changes could definitely change the memory consumption. Since test has quite a few dependencies are the build sizes all the same? Were build sizes tested before and after modularity to ensure size parity? Is each dependency dragging in copies of all it's dependencies, or is it like how it used to be where there's just one copy of everything?

I'm still investigating this but one data point since you asked.. It appears the on-disk build sizes with the modular changes is significantly smaller from before (as in for some test builds 14M vs 50M). I'm trying to figure out the memory size. Which is not easy to do given the various programs that running b2 will launch.