[question] Compiling Perl with static_ext

[vadimkantorov]

I'm trying to compile for WebAssembly biber which is implemented in Perl. The relevant issue: plk/biber#338.

As a guidance of a quite modern static Perl build I found WebPerl. Thank you for writing https://webperl.zero-g.net/building.html and many comments in the code. I don't need the HTML/JS interaction part, but the static build itself

As part of preparation work, I'm trying to build Perl statically with most (and hopefully, all) extensions that biber requires.

Two of these are List::MoreUtils, List::MoreUtils::XS. However, the build fails. If you have some insight on this, could you advise on how to make the static build work?

ROOT=$PWD
URLPERL=https://www.cpan.org/src/5.0/perl-5.30.0.tar.gz

mkdir -p source build/native/perl build/wasm/perl
wget -nc $URLPERL -P source
tar -xf source/$(basename $URLPERL) --strip-components=1 --directory=build/native/perl

CPANMODULES="IO::File List::MoreUtils List::MoreUtils::XS"
for MOD in $CPANMODULES; do
    URL=$(wget -q -O - https://fastapi.metacpan.org/v1/download_url/$MOD | grep download_url | cut -d'"' -f4)
    PKG=${MOD//::/-}
    wget -nc $URL -P source
    mkdir -p build/native/perl/ext/$PKG
    tar -xf source/$(basename $URL) --strip-components=1 --directory build/native/perl/ext/$PKG
    echo $PKG $URL
done
cd build/native/perl
bash ./Configure -sde -Dprefix=$PWD/../prefix -Dlibs="-lm" -Dstatic_ext='IO/File List/MoreUtils List/MoreUtils/XS'
make

Error:

make[1]: Leaving directory '/home/vadimkantorov/buildbiber/build/native/perl/cpan/JSON-PP'
./miniperl -Ilib make_ext.pl ext/List-MoreUtils/pm_to_blib  MAKE="make" LIBPERL_A=libperl.a
Can't locate IO/File.pm in @INC (you may need to install the IO::File module) (@INC contains: . /home/vadimkantorov/buildbiber/build/native/perl/cpan/AutoLoader/lib /home/vadimkantorov/buildbiber/build/native/perl/dist/Carp/lib /home/vadimkantorov/buildbiber/build/native/perl/dist/PathTools /home/vadimkantorov/buildbiber/build/native/perl/dist/PathTools/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/ExtUtils-Install/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/ExtUtils-MakeMaker/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/ExtUtils-Manifest/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/File-Path/lib /home/vadimkantorov/buildbiber/build/native/perl/ext/re /home/vadimkantorov/buildbiber/build/native/perl/dist/Term-ReadLine/lib /home/vadimkantorov/buildbiber/build/native/perl/dist/Exporter/lib /home/vadimkantorov/buildbiber/build/native/perl/ext/File-Find/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/Text-Tabs/lib /home/vadimkantorov/buildbiber/build/native/perl/dist/constant/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/version/lib /home/vadimkantorov/buildbiber/build/native/perl/cpan/Getopt-Long/lib /home/vadimkantorov/buildbiber/build/native/perl/lib) at inc/latest/private.pm line 10.
BEGIN failed--compilation aborted at inc/latest/private.pm line 10.
Compilation failed in require at inc/latest.pm line 5.
Compilation failed in require at Makefile.PL line 19.
BEGIN failed--compilation aborted at Makefile.PL line 19.
Unsuccessful Makefile.PL(ext/List-MoreUtils): code=512 at make_ext.pl line 518.
makefile:581: recipe for target 'ext/List-MoreUtils/pm_to_blib' failed

It seems that these modules have some build-time dependencies. Not sure how to satisfy them. The build order seems not preserved. Attaching full log:
log.txt

Thank you!

[haukex]

Hi, from a very quick look at Perl's make_ext.pl, it looks to me like build order should be mostly preserved. But do I understand correctly you've written your own script to build the extensions instead of modifying WebPerl's emperl_config.sh? Have you tried adding the modules to the build by modifying the variables EMPERL_EXTENSIONS and EMPERL_STATIC_EXT?

[vadimkantorov]

You're correct! For now, I'm using the snippet above for native and static compilation. Once all modules compile well (plk/biber#338 (comment)), then I'll try to have a compact emscriptenized script based on your ideas in WebPerl.

For now it seems that for some reason IO::File is not installed and definitely not installed in the correct order.

For now it seems that the best course of action is:

1. Compile Perl statically with the required static extensions, except the problematic ones
2. Compile each extension required by biber separately and statically (by perl Makefile.pl LINKTYPE=static && make)
3. Rebuild a new Perl binary by linking it to all compiled modules

This is rather tedious :( but well

[haukex]

WebPerl uses this hints file to set static_ext instead of the command-line option you appear to be using. I unfortunately don't have the time to check right now, but perhaps that has something to do with your issue. I'd be curious what your results are from building with build.pl when you get the chance.

[vadimkantorov]

I tried both hints file and command-line options. The result was equivalent, so I think their action is equivalent as well