Skip to content

Conversation

@Kangie
Copy link
Contributor

@Kangie Kangie commented Oct 1, 2024

Hi Team,

While logging #262 I had the thought that it ought to be relatively straightforward to port Dillo to Meson.

I took a bit of time this afternoon and the result is an initial port.

It builds and links without complaint, but it's currently not producing a "working" dillo - I've missed something with DNS that I'll come back to shortly.

I know this wasn't specifically requested however I still think it's worthwhile:

  • It's possible to run a 'meson' build using muon and samurai which are C99 implementations of Meson and Ninja respectively.
  • It's a lot smaller than automake + autoconf (coming in at just over half the lines of "code" to maintain, which could be reduced by adjusting the blind formatting rules I used)
  • It's faster / more efficient
    • sh -c 'meson setup --wipe build ; ninja -C build' 18.87s user 7.74s system 958% cpu 2.777 total
    • sh -c './configure && make -j' 33.55s user 4.55s system 550% cpu 6.917 total
  • It doesn't rely on which ;)

image

image

I'll keep plugging away at this unless you're not interested until we can get it ready for release.

Closes: #262

Cheers,

Matt

@rodarima
Copy link
Member

rodarima commented Oct 1, 2024

Thanks for the PR and sorry for the late reply.

Even if I hate autotools with all my heart, they are capable of producing a unreadable but dependency-free configure script that can run in (almost?) any POSIX-compliant machine, including very old ones.

There are other issues with autotools too, which are mostly the reason I'll probably change the build system in the future, but I don't like the idea of adding python as a build dependency because of meson (or using the WIP muon).

As FLTK has switched recently to cmake, and it is very likely that it has to be supported in every platform that we want to build Dillo on, we can asume that cmake is already available due to indirect build dependencies. So I will be inclined to choose it over meson, which should yield similar speeds when using ninja instead of make.

In any case, I won't take this decision without making a study beforehand of the options, and see which one is more appropriate. Also consider the fact that I have already some knowledge on cmake, and almost none on meson. Regardless, this change will cause a major version bump, so it will not likely happen soon.

Regarding the which problem, it should be a ~5 line patch, so I don't think it is a reason to change the build system (yet). The IWYU patch is also welcome.

@rodarima rodarima marked this pull request as draft October 1, 2024 17:03
@Kangie
Copy link
Contributor Author

Kangie commented Oct 1, 2024

or using the WIP muon.

Having recently done the fvwm3 meson port where Muon was a desired feature, I can say that the recent 0.3.0 release is effectively on-par with mainline Meson (so no hard Python req)

As FLTK has switched recently to cmake, and it is very likely that it has to be supported in every platform that we want to build Dillo on, we can asume that cmake is already available due to indirect build dependencies. So I will be inclined to choose it over meson, which should yield similar speeds when using ninja instead of make.

CMake is "fine" however IMO Meson results in far more maintainable and readable build system code, and typically one obvious way to accomplish any given task rather than the ~5 ways that there are to do it in CMake (with no indication which is more appropriate for a given situation).

Which platforms that we're building C++11 code on don't support C99 and/or Python anyway ;)

With 99% of the porting done you shouldn't need to dig into the guts for quite a while.

I may submit a patch for the which issue at some point; got to do actual work today. I will, however, get the Meson build to the point that the browser is actually functional at some point so that you can compare.

@rodarima
Copy link
Member

rodarima commented Oct 2, 2024

With 99% of the porting done you shouldn't need to dig into the guts for quite a while.

I need to understand very deeply the build system as to debug weird problems in all kinds of platforms I don't have access to, other than a mail/IRC channel with the person. So, totally the opposite.

I may submit a patch for the which issue at some point; got to do actual work today.

Thanks, that would be nice.

@Kangie
Copy link
Contributor Author

Kangie commented Oct 15, 2024

Following up on this, it looks like it's (almost) all working:

image

build/src/dillo
paths: Cannot open file '/home/kangie/.dillo/dillorc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillodillorc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/keysrc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillokeysrc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/domainrc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillodomainrc': No such file or directory
paths: Using internal defaults...
dillo_dns_init: Here we go! (threaded)
TLS library: OpenSSL 3.3.2 3 Sep 2024
Disabling cookies.
paths: Cannot open file '/home/kangie/.dillo/hsts_preload': No such file or directory
paths: Cannot open file '/usr/local/etc/dillohsts_preload': No such file or directory
paths: Using internal defaults...
Nav_open_url: new url='about:splash'
Nav_open_url: new url='https://www.google.com'
Dns_server [0]: www.google.com is 142.250.67.4 2404:6800:4006:811::2004
Connecting to 142.250.67.4:443
www.google.com: TLSv1.3, cipher TLS_AES_256_GCM_SHA384
sha256 256-bit EC: /CN=www.google.com
sha256 2048-bit RSA: /C=US/O=Google Trust Services/CN=WR2
sha256 4096-bit RSA: /C=US/O=Google Trust Services LLC/CN=GTS Root R1
root: /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
>>>> a_Nav_repush <<<<
Nav_open_url: new url='https://www.google.com'
a_Nav_expect_done: repush!

Not sure why the PNG isn't visible but that should be straightforward to fix, and the etcdir just needs a trailing slash.

@Kangie Kangie force-pushed the a-proper-build-system branch from 8ad51db to 524c8cd Compare October 15, 2024 00:20
@Kangie
Copy link
Contributor Author

Kangie commented Oct 15, 2024

OK, making progress... I had HAVE_PNG not ENABLE_PNG.

This is looking good:

image

aths: Cannot open file '/home/kangie/.dillo/dillorc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillo/dillorc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/keysrc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillo/keysrc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/domainrc': No such file or directory
paths: Cannot open file '/usr/local/etc/dillo/domainrc': No such file or directory
paths: Using internal defaults...
dillo_dns_init: Here we go! (threaded)
TLS library: OpenSSL 3.3.2 3 Sep 2024
Disabling cookies.
paths: Cannot open file '/home/kangie/.dillo/hsts_preload': No such file or directory
paths: Cannot open file '/usr/local/etc/dillo/hsts_preload': No such file or directory
paths: Using internal defaults...
Nav_open_url: new url='about:splash'
Nav_open_url: new url='http://www.google.com'
Dns_server [0]: www.google.com is 142.251.221.68 2404:6800:4006:809::2004
Connecting to 142.251.221.68:80
>>>> a_Nav_repush <<<<
Nav_open_url: new url='http://www.google.com'
a_Nav_expect_done: repush!

@Kangie
Copy link
Contributor Author

Kangie commented Oct 15, 2024

Sorry about the spam, but best to keep this commentary separate:

Regardless, this change will cause a major version bump, so it will not likely happen soon.

There's little reason (aside from keeping autotools in the repo...) not to include the meson build files alongside autotools for a transitional period, then you are able to deprecate autotools and remove it at a time of your choosing with the confidence that it's not going to suddenly cause downstream issues.

@Kangie Kangie force-pushed the a-proper-build-system branch 22 times, most recently from 62e57c9 to 219fd24 Compare October 15, 2024 04:03
@Kangie Kangie force-pushed the a-proper-build-system branch 8 times, most recently from 618c06b to 94a8c81 Compare October 20, 2024 12:43
@Kangie Kangie force-pushed the a-proper-build-system branch from 94a8c81 to 4579efc Compare October 26, 2024 10:58
@Kangie Kangie force-pushed the a-proper-build-system branch from 4579efc to 421db66 Compare November 12, 2024 06:16
@Kangie Kangie mentioned this pull request Jan 30, 2025
It's been a mandatory part of POSIX since 2001 an although
it's an ISO C Extension, cstdint is ISO for C++

Signed-off-by: Matt Jolly <[email protected]>
@Kangie Kangie force-pushed the a-proper-build-system branch from 421db66 to 4008e70 Compare January 30, 2025 02:26
@Kangie
Copy link
Contributor Author

Kangie commented Jan 30, 2025

Rebased on current master, tidied up commit history and squashed meson commits down for comparison with #333

Builds fine, runs fine. Tests are running successfully but timing out - I can dig into that if further development is welcome. CI updated to use matrices to test as many options as possible, but not tested post merge; I expect failures here.

I suspect the remaining issues to be tidied up are to do meson not successfully interpreting the test driver exiting, as my test logs show the diff coming back as 0 = 0.

==================================== 9/44 ====================================
test:         b-div
start time:   02:13:40
duration:     30.00s
result:       exit status 0
command:      MESON_TEST_ITERATION=1 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 MALLOC_PERTURB_=126 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 DILLOBIN=/data/development/temp/dillo/build/src/dillo /data/development/temp/dillo/test/html/driver.sh /data/development/temp/dillo/test/html/render/b-div.html
----------------------------------- stdout -----------------------------------
paths: Cannot open file '/home/kangie/.dillo/dillorc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/dillorc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/keysrc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/keysrc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/domainrc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/domainrc': No such file or directory
paths: Using internal defaults...
dillo_dns_init: Here we go! (threaded)
TLS library: OpenSSL 3.3.2 3 Sep 2024
Disabling cookies.
paths: Cannot open file '/home/kangie/.dillo/hsts_preload': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/hsts_preload': No such file or directory
paths: Using internal defaults...
Nav_open_url: new url='file:/data/development/temp/dillo/test/html/render/b-div.html'
** ERROR **: [Dpi_read_comm_keys] No such file or directory
Dpi_blocking_start_dpid: try 1
paths: Cannot open file '/home/kangie/.dillo/dillorc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/dillorc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/keysrc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/keysrc': No such file or directory
paths: Using internal defaults...
paths: Cannot open file '/home/kangie/.dillo/domainrc': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/domainrc': No such file or directory
paths: Using internal defaults...
dillo_dns_init: Here we go! (threaded)
TLS library: OpenSSL 3.3.2 3 Sep 2024
Disabling cookies.
paths: Cannot open file '/home/kangie/.dillo/hsts_preload': No such file or directory
paths: Cannot open file '/tmp/dillo/etc/dillo/hsts_preload': No such file or directory
paths: Using internal defaults...
Nav_open_url: new url='file:/data/development/temp/dillo/test/html/render/b-div.ref.html'
OK
----------------------------------- stderr -----------------------------------
+ DILLOBIN=/data/development/temp/dillo/build/src/dillo
+ '[' '!' -e /data/development/temp/dillo/build/src/dillo ']'
+ magick_bin=convert
+ command -v magick
+ magick_bin=magick
+ test_file /data/development/temp/dillo/test/html/render/b-div.html
+ html_file=/data/development/temp/dillo/test/html/render/b-div.html
+ '[' '!' -e /data/development/temp/dillo/test/html/render/b-div.html ']'
+ ref_file=/data/development/temp/dillo/test/html/render/b-div.ref.html
+ '[' '!' -e /data/development/temp/dillo/test/html/render/b-div.ref.html ']'
++ basename /data/development/temp/dillo/test/html/render/b-div.html
+ test_name=b-div.html
+ wdir=b-div.html_wdir
+ rm -rf b-div.html_wdir
+ mkdir -p b-div.html_wdir
+ mkfifo b-div.html_wdir/display.fifo
+ exec
+ xorgpid=999693
+ trap 'kill 999693' EXIT
+ read dispnum
+ Xvfb -screen 5 1024x768x24 -displayfd 6
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
+ export DISPLAY=:2
+ DISPLAY=:2
+ render_page /data/development/temp/dillo/test/html/render/b-div.html b-div.html_wdir/html.png
+ htmlfile=/data/development/temp/dillo/test/html/render/b-div.html
+ outpic=b-div.html_wdir/html.png
+ dillopid=999706
+ sleep 1
+ /data/development/temp/dillo/build/src/dillo -f /data/development/temp/dillo/test/html/render/b-div.html
++ xwininfo -all -root
++ awk '/Dillo:/ {print $1}'
+ winid=0x200009
+ '[' -z 0x200009 ']'
+ xwd -id 0x200009 -silent
+ magick xwd:- png:b-div.html_wdir/html.png
+ kill 999706
+ render_page /data/development/temp/dillo/test/html/render/b-div.ref.html b-div.html_wdir/ref.png
+ htmlfile=/data/development/temp/dillo/test/html/render/b-div.ref.html
+ outpic=b-div.html_wdir/ref.png
+ dillopid=999771
+ sleep 1
+ /data/development/temp/dillo/build/src/dillo -f /data/development/temp/dillo/test/html/render/b-div.ref.html
++ xwininfo -all -root
++ awk '/Dillo:/ {print $1}'
+ winid=0x200009
+ '[' -z 0x200009 ']'
+ xwd -id 0x200009 -silent
+ magick xwd:- png:b-div.html_wdir/ref.png
+ kill 999771
++ compare -metric AE b-div.html_wdir/html.png b-div.html_wdir/ref.png b-div.html_wdir/diff.png
+ diffcount=0
+ '[' 0 = 0 ']'
+ echo OK
+ ret=0
+ exec
+ rm b-div.html_wdir/display.fifo
+ '[' -z '' ']'
+ rm -rf b-div.html_wdir
+ return 0
+ exit 0
+ kill 999693
==============================================================================

@Kangie
Copy link
Contributor Author

Kangie commented Jan 30, 2025

It's probably worth, at the very least, cherry-picking the IWYU fixes.

@Kangie Kangie force-pushed the a-proper-build-system branch from 4008e70 to fb092b6 Compare January 30, 2025 02:31
This is a first pass at porting autoconf/automake to Meson.

Test driver modified to remove the trap on EXIT in favour of a function
call after setting the exit code; this was causing meson test to hang
on HTML tests and recording them as failed after the timeout.

Signed-off-by: Matt Jolly <[email protected]>
Use Matrix configurations to make a more concise config that
builds both the Autotools and Meson build paths on supported
platforms, and where possible uses Clang and GCC

Signed-off-by: Matt Jolly <[email protected]>
@Kangie Kangie force-pushed the a-proper-build-system branch from fb092b6 to 7bbddad Compare January 30, 2025 04:26
@Kangie
Copy link
Contributor Author

Kangie commented Jan 30, 2025

Tracked down hanging tests to the exit trap.

If there's willingness to consider merging this I'm happy to tidy up the CI and look into test failures, but otherwise the port is "done", and is up-to-date with current master.

@Kangie Kangie marked this pull request as ready for review January 30, 2025 04:31
@Sunny-Maxis
Copy link

Comment for Rodrigo:

Do not adopt meson if you plan to support old UNIXes. Muon, the C99 Meson, is unfortunately poorly coded with poor support for aligned access on RISC systems and numerous problems (My main critique being, who peppers a setup with hundreds of asserts in a DEFAULT BUILD?), and boson, the C11 version, is poorly maintained.

Out of all options, autotools is preferable, but CMake is workable. I understand as you are moving to C++11 that GCC moreorless will become the sole compiler, but dillo never properly built on anything else.

I do not want to hurt Kengie's feelings or upset their effort, it's just, Meson is not an acceptable system for this project. CMake has additionally decent Windows support, if MS windows support is a priority.

@eli-schwartz
Copy link

eli-schwartz commented Mar 2, 2025

I hope you don't plan on compiling CMake on systems without a C++ (11) compiler.

@Sunny-Maxis
Copy link

I hope you don't plan on compiling CMake on systems without a C++ (11) compiler.

Building a build tool with GCC is different than building a program with a native compiler.

There's hassles and problems with building GCC for everything on niche platforms. On IRIX, for instance, GCC cannot be effectively debugged even with restored GCC support from Kazuo K. or the SGUG GCC, the gdb system on IRIX can't debug threads, stack traces are funky. dbx expects dwarf of a certain kind. On Itanium and PA-RISC, GCC's C++ performance compared to HPE's is baaad. It's a bit better there, you got several compilers to choose from depending on OS (ICC, aCC, Pro64) but yeah, I think ya get my point.

Regardless, I was offering my input. Dillo is a useful browser and you gotta get with the times, Cmake being a necessity is what it is. Meson and Muon are not acceptable IMHO for projects that target older systems.

@rodarima
Copy link
Member

rodarima commented Mar 2, 2025

Meson and Muon are not acceptable IMHO for projects that target older systems.

I also suspect the same, but I will need to do more research to be sure. This is not a priority now, so I will take some time to address other issues first. I recommend not investing more time in this PR until that is clarified on my side, otherwise you risk wasting your time.

I hope you don't plan on compiling CMake on systems without a C++ (11) compiler.

We try to target 20+ year old stacks, including older gcc with no support for C++11 as we only need variadic macros from C++11 which are available in much older compilers. There is no public announcement about what we support and what not because is not an easy task to even measure (I don't have access to many of the old systems that claim to run Dillo).

I won't change the build system until I have researched all platforms in which Dillo works currently and how it will be affected. And I would rather not maintain two build systems.

If you @Sunny-Maxis want to help improving the support on IRIX that would be nice to have. Feel free to open another issue to track the current status and what we can do to improve it.

@eli-schwartz
Copy link

We try to target 20+ year old stacks, including older gcc with no support for C++11 as we only need variadic macros from C++11 which are available in much older compilers. There is no public announcement about what we support and what not because is not an easy task to even measure (I don't have access to many of the old systems that claim to run Dillo).

CMake, when compiling itself, searches for a C++17 compiler, or a C++14 compiler, or a C++11 compiler, and if it cannot at least find a C++11 compiler it errors out with this code:

  if(NOT CMake_HAVE_CXX_UNIQUE_PTR)
    message(FATAL_ERROR "The C++ compiler does not support C++11 (e.g. std::unique_ptr).")
  endif()

So I don't think you can use cmake if you target 20+ year old stacks.

Meson can in theory run on any system with a c99 compiler, though it may require either:

  • python (the entire programming language -- python 3.10 is written in c99, python 3.11 moved to c11, meson supports python 3.7)
  • or muon (a relatively lean codebase implementing a standalone version of meson in c99)

to be ported to that system, possibly a much easier task than porting a C++ compiler.

According to muon-build/muon#115

Muon would benefit IRIX because Python is terrible on our platform for speed and performance.

which indicates that python does work?

Muon is definitely willing to accept patches, but it was made clear that IRIX in particular cannot be emulated, and uses a different MIPS ABI than other OSes running on MIPS, and @Sunny-Maxis decided it was too painful to debug (interesting assessment) so idk what to tell you. I am not convinced that experience is generally applicable to older systems.

I suppose you could argue that the only option is to continue with autotools...

@rodarima
Copy link
Member

rodarima commented Mar 2, 2025

I suppose you could argue that the only option is to continue with autotools...

For very old platforms it probably is the best tradeoff for now, but it slows down development on relatively modern ones. However the current issues I have with autotools are probably fixable with enough perseverance.

@Sunny-Maxis
Copy link

If you @Sunny-Maxis want to help improving the support on IRIX that would be nice to have. Feel free to open another issue to track the current status and what we can do to improve it.

I will definitely give it a shot when I'm at that point. It's going to be a hot minute more than likely because we are trying to get other stuff out of the way.

So I don't think you can use cmake if you target 20+ year old stacks.

This is a fair assessment. However I can get a version of Cmake running on IRIX (3.8 or so) using a somewhat recent version of GCC without dealing with libuv, and people have gotten a later version of it running using their own tool chains.

That is to say he could simply stick with an older version of Cmake as a minimum required version.

which indicates that python does work?
Python does work but Meson presumably will continue to roll forward with more and more stringent system requirements that spells bad news for such a project where you're trying to maintain support for obsolete platforms.

It's extremely painful to use meson on a system like IRIX. Unless you're cross compiling which I am not able to do with our setup because we are trying to use the native compiler whenever possible, and because my boss is of the opinion that cross compiling is not necessary on this particular system.

Muon is definitely willing to accept patches, but it was made clear that IRIX in particular cannot be emulated, and uses a different MIPS ABI than other OSes running on MIPS, and @Sunny-Maxis decided it was too painful to debug (interesting assessment) so idk what to tell you. I am not convinced that experience is generally applicable to older systems.

This is somewhat out of the scope for this discussion but I will briefly elaborate:

  1. It was crashing no matter which compiler are used even before initialization. Manually disabling the asserts that were used in the code in the range of almost 200 was possible to get it further but it ended up failing well before it actually did anything useful. Asserts are a great tool for debugging but I was repeatedly informed throughout the thread that removing them was not supported / they kept on telling me I was doing it wrong which I didn't appreciate. That was severely condescending.

  2. They couldn't answer basic aligned access questions which I suspected was the ultimate issue because you would once you got past the assert bull end up with bus errors which usually means stack alignment issues.

  3. They kept on insisting it was a problem with the compiler which was complete bull crap. We have compiled dozens of programs without problems using both compilers and it was just not productive use of our time anymore.

  4. For the most part when something requires meson/ninja we just use an older release or in some cases my boss has ported applications to new build systems and given the finger to the person who thought it was an educated idea to lock out older systems on a relatively portable application otherwise.

I suppose you could argue that the only option is to continue with autotools...

This is what I would recommend based on what's being said here. If you're going to Target windows as a platform at all honestly you should just tell people that they should have to fork and track the project separately with Ms build or something. The windows and UNIX universes are just not compatible on several levels.

Regardless it's much easier to port GCC to a platform or reinstate support even, then to sometimes deal with poor upstream support or crap code. Sometimes dealing with projects like Muon, which I'm not saying was intentional on their part, feels like pulling teeth and I don't appreciate the way I was lectured on several occasions and told to just throw printf's everywhere.

Rodrigo you'll be hearing from me when I do have something to show on dillo for IRIX. That'll probably be sometime in the next 3 to 4 months or so. That may not be that far off though that's just a rough estimate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configure uses which which is not POSIX compliant

4 participants