Standalone gz sim executable

[sauk2]

🎉 New feature

Related to gazebosim/gz-tools#7

Summary

This PR introduces a standalone executable for gz sim by following a similar approach to gz model. The new executable is integrated into cmdsim.rb.in, replacing the existing functionality. It is also a step in the direction of #2737, which will eventually allow us to decouple the server from GUI components.

The majority of the behavior remains consistent, including:

• Launching the Server using -s
• Launching the GUI using -g
• Launching the Quickstart menu when no argument is provided

Changes to the launch behaviour

• Launching with -s runs the server in the main thread.
• Launching with -g runs the GUI in the main thread.
• Running without -s or -g launches both the Server and GUI in separate threads. Since a single SIGINT terminates only one of the threads while the other continues running, a force kill mechanism ensures that both threads terminate properly.

Test it

Check for UNIT_gz_TEST and INTEGRATION_log_system.

Checklist

• Signed all commits for DCO
• Added tests
• Added example and/or tutorial
• Updated documentation (as needed)
• Updated migration guide (as needed)
• Consider updating Python bindings (if the library has them)
• codecheck passed (See contributing)
• All tests passed (See test coverage)
• While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

[j-rivero]

Before looking into the code: gz-sim-sim seems a weird name to my eyes :) although it can serve as a transitioning step to other naming or model.

Given the client / server architecture I would expect to find something like: gz-sim-client and gz-sim-server separate standalone applications that can be run @sauk2 what do you think? We would need some separate that does not like to Qt libraries for packaging the server-only and get that into the official docker hub.

[traversaro]

Thanks a lot @sauk2 !

I put together an hack (or let's call it a "proof of concept", to be more fancy) to understand if this new standalone executable could be used to be able to get a working gz sim on Windows and macOS (i.e. fixing #168, #44 and #2393), and it seems that indeed it is working fine, at least on Windows, while macOS is untested. For anyone curious the hack/poc is at https://github.com/traversaro/ign-gazebo/tree/sim-standalone-exe-win .

Once a consensus is reached on this PR, I can try to cleanup the code in https://github.com/traversaro/ign-gazebo/tree/sim-standalone-exe-win to have it in PR shape.

[traversaro]

For anyone curious the hack/poc is at https://github.com/traversaro/ign-gazebo/tree/sim-standalone-exe-win .

For clarity, there I used tinyprocesslib because it is a quick copy&paste from https://github.com/gazebosim/gazebo-classic/blob/b22c6e15e52299865b31093b8feebc9ca19e26e8/gazebo/gazebo_main.cc#L259 . However, probably it make more sense to use gz::utils::Subprocess, see https://gazebosim.org/api/utils/2/classgz_1_1utils_1_1Subprocess.html and gazebosim/gz-utils#127 . By the way, I wonder if it could make sense to use two different process for server and client also on Linux, so that we avoid having different code paths for Linux vs macOS/Windows, or there is some advantage of using a single process for gz sim on Linux?

[j-rivero]

By the way, I wonder if it could make sense to use two different process for server and client also on Linux, so that we avoid having different code paths for Linux vs macOS/Windows, or there is some advantage of using a single process for gz sim on Linux?

That is a good point. Ideally we should go with one code path for all unless we find a good reason or a problem in a particular arch. Maybe wrong but I have always assumed that the ruby code currently spawns two different processes via Process.fork for the GUI and the server so we maintain the two separate processes although we change the way that they are coordinated and orchestratred.

[sauk2]

@j-rivero and @traversaro Thanks a lot for all your feedback! Sorry for the delay in making changes and responding. I've moved this PR back to draft since I believe further discussion is needed before it's ready for merge.

Based on your comments, it made sense to handle the server and GUI as separate processes rather than separate threads. To that end, I've split them into two distinct CLI executables - gz-sim-server and gz-sim-gui, each responsible for its own flags. These are now invoked by a main CLI executable, gz-sim-main, which forwards the appropriate flags to the Server and GUI executables while launching them as separate subprocesses using the gz::utils::Subprocess class. This executable is now available to gz-tools and the cmdsim.rb script. By maintaining a single outward-facing executable, process management is handled at the executable level rather than at the Ruby level, as it has been until now.

Handling the -s and -g flags to launch the server and GUI separately is straightforward, but the real challenge lies in orchestrating the full Server+GUI launch. This scenario can terminate in multiple ways - a SIGINT will cleanly shut down both processes but issues arise when the user manually closes the GUI as this does not signal the Server to shut down. Currently, I’m using the Alive() function to monitor both processes and if the GUI exits due to user input, the server is explicitly terminated as well.

Another issue is that logs are not streamed to the terminal in the current implementation. The Stdout() function in gz::utils::Subprocess only provides output after the process completes, which is not ideal. One potential solution is exposing subprocess_read_stdout within gz::utils::Subprocess to enable real-time output streaming. I’ve put together a mock implementation for testing and I’d appreciate any thoughts on how best to integrate this into the utility class.

Additionally, the server and GUI flags are currently defined within their respective executables. I’m exploring ways to extract these flags so they can be included in the main help message - one idea is to invoke each executable with --help and capture their stdout.

I’d appreciate any suggestions or feedback on these points. Thanks again!

[traversaro]

Thanks a lot for the great work!

Currently, I’m using the Alive() function to monitor both processes and if the GUI exits due to user input, the server is explicitly terminated as well.

We used a similar solution in Gazebo Classic on Windows, and it worked fine.

Another issue is that logs are not streamed to the terminal in the current implementation. The Stdout() function in gz::utils::Subprocess only provides output after the process completes, which is not ideal. One potential solution is exposing subprocess_read_stdout within gz::utils::Subprocess to enable real-time output streaming. I’ve put together a mock implementation for testing and I’d appreciate any thoughts on how best to integrate this into the utility class.

Interesting. I do not have experience with subprocess.h, but with reproc and tiny-process-lib the subprocess output is automatically printed in the terminal that launched the main process. Perhaps there is some way to achieve this, without the need to manually process the stdout and stderr from the child processes?

Additionally, the server and GUI flags are currently defined within their respective executables. I’m exploring ways to extract these flags so they can be included in the main help message - one idea is to invoke each executable with --help and capture their stdout.

Good point. Could it make sense to move the logic of those flags to a simple header-only library, and call it from both the main executable and server/client ones?

[j-rivero]

Another issue is that logs are not streamed to the terminal in the current implementation. The Stdout() function in gz::utils::Subprocess only provides output after the process completes, which is not ideal. One potential solution is exposing subprocess_read_stdout within gz::utils::Subprocess to enable real-time output streaming. I’ve put together a mock implementation for testing and I’d appreciate any thoughts on how best to integrate this into the utility class.

Interesting. I do not have experience with subprocess.h, but with reproc and tiny-process-lib the subprocess output is automatically printed in the terminal that launched the main process. Perhaps there is some way to achieve this, without the need to manually process the stdout and stderr from the child processes?

Sounds like a bug/defect in subprocess that would need to be fixed. I think that we can go with the workaround by now if it is producing the expect result and ticket the bug in gz-utils in order not to go too far with this PR.

Additionally, the server and GUI flags are currently defined within their respective executables. I’m exploring ways to extract these flags so they can be included in the main help message - one idea is to invoke each executable with --help and capture their stdout.

Good point. Could it make sense to move the logic of those flags to a simple header-only library, and call it from both the main executable and server/client ones?

+1.

[sauk2]

@traversaro and @j-rivero, Sorry for the delay in getting back to you, and thanks for the feedback!

I initially tried using gz::utils::Subprocess, but it was challenging to retrieve logs and manage termination properly. Instead, I switched to using the system() function which was much simpler to work with and appears to be cross-platform.

I've made a few changes that seem to work well. Instead of maintaining separate executables for gz-sim-main and gz-sim-server, I’ve merged them into a single executable while gz-sim-gui remains separate. The separation between the server and GUI is still maintained.

Now, the server process runs directly from the main executable and the GUI is launched as a separate process using system(). One of the challenges with having separate server and GUI executables was passing the correct flags to each. The allow_extras() function of CLI11 allows additional arguments, but in many cases, it causes the positional argument to be misrecognized. To resolve this, I now launch the server directly from the main executable while constructing and forwarding the correct arguments to the GUI.

I still need to figure out how to make this work on Windows, so it would be great to get some advice on this!

I’d be happy to get your feedback and do let me know if this approach works for you!