Description
Thank you for taking the time to submit an issue!
Background information
I am testing OpenFOAM on a Power 10 server node with 768 hardware threads. If I run -np 768 (anything over about 256, really), Open MPI crashes due to the operating system being out of file handles. I have increased the number of handles to 64k, and it still runs out. Another MPI code, LAMMPS, runs out at np = 240.
What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)
5.0.2
Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)
OS distribution package
If you are building/installing from a git clone, please copy-n-paste the output from git submodule status.
Please describe the system on which you are running
- Operating system/version: RHEL 9
- Computer hardware: A single IBM Power 10 server node
- Network type: None(?).
Details of the problem
I am running the OpenFOAM motorbike test with various mesh sizes. I expect to be able to run with MPI processes populating all the hardware threads, so -np 768. However, the program crashes with an operating system error reporting insufficient file handles. This happens on other MPI codes when the process count is well over 200.
Note: If you include verbatim output (or a code block), please use a GitHub Markdown code block like below:
shell$ mpirun -n 2 ./hello_world