"autogen.pl --force" fails in master tarball due to missing hwloc dirs

[jsquyres]

If you run ./autogen.pl --force in a distribution tarball, it fails because there are hwloc directories not present:

opal/mca/hwloc/hwloc2/hwloc/Makefile.am:23: error: required directory opal/mca/hwloc/hwloc2/hwloc/netloc does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:25: error: required directory opal/mca/hwloc/hwloc2/hwloc/utils does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:25: error: required directory opal/mca/hwloc/hwloc2/hwloc/tests does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:25: error: required directory opal/mca/hwloc/hwloc2/hwloc/contrib/systemd does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:25: error: required directory opal/mca/hwloc/hwloc2/hwloc/contrib/hwloc-ps.www does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:25: error: required directory opal/mca/hwloc/hwloc2/hwloc/doc does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:35: error: required directory opal/mca/hwloc/hwloc2/hwloc/contrib/windows does not exist
opal/mca/hwloc/hwloc2/hwloc/Makefile.am:37: error: required directory opal/mca/hwloc/hwloc2/hwloc/netloc does not exist
autoreconf: automake failed with exit status: 1
Command failed: autoreconf -ivf --warnings=all,no-obsolete,no-override -I config

We used to include these directories -- along with dummy Makefile.am files -- before we made hwloc be a git submodule.

make distcheck works on a master git checkout, but something is not translating correctly to the generated tarball such that if you run autogen in it, 💥.

[bgoglin]

autoreconf -f fails the same in hwloc embedded tarball (with additional errors for missing *.in files).
@ggouaillardet looked at fixing this in the past open-mpi/hwloc#235 but it wasn't really needed at that point.
Not sure how to fix this. We could try changing the hwloc dist script to add all those dummy directories and Makefile.am?

[bgoglin]

This should be fixed in hwloc commit bgoglin/hwloc@18caedf (from hwloc v2.1)
It just creates the missing directories that autogen/automake complained about.
It added tests to hwloc CI so that such issues are now detected before reaching OMPI's submodule.
My quick OMPI test looks good but I'd like a double-check.

[jsquyres]

@bgoglin I confirm that bgoglin/hwloc@18caedf fixes the issue. Would it be possible to get this merged to the hwloc v2.1 branch?

[bgoglin]

I'll decide after seeing your votes this morning :)

[bgoglin]

OK, zapproved.