ch4: use MPIR_pmi_allgather_group in address exchange #7392
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
hzhou
force-pushed
the
2504_pmi_group
branch
5 times, most recently
from
19acb5b to
b4dabef
Compare
|
test:mpich/ch3/most |
hzhou
force-pushed
the
2504_pmi_group
branch
4 times, most recently
from
75eb799 to
7186b4e
Compare
|
test:mpich/pmi |
|
test:mpich/ch3/most |
|
test:mpich/ch4/most |
raffenet
approved these changes
Apr 29, 2025
Comment on lines
+214
to
+222
| /* convert the int array into a comma-separated int list */ | ||
| group_str = MPL_malloc(nprocs * 8, MPL_MEM_OTHER); /* assume each integer fits in 8 chars */ | ||
| char *s = group_str; | ||
| for (int i = 0; i < nprocs; i++) { | ||
| int n = sprintf(s, "%d,", procs[i].rank); | ||
| s += n; | ||
| } | ||
| /* overwrite the last comma */ | ||
| s[-1] = '\0'; |
There was a problem hiding this comment.
this is fairly close to MPL_strjoin but since it starts with an int array its not quite a fit. could extend the join function down the road to cover this use-case. nothing to do right now.
Skip optional attributes when the value is the default. This provide better backward compatibility as we won't generate unrecognized attributes if the *new* optional features are not used.
In order to support new PMI features and also support backward compatibility, we need track server PMI versions from the init response.
PMI_Barrier_group run a barrier within a group of processes specified by an array of processes. Constants such as PMI_GROUP_WORLD, PMI_GROUP_NODE, PMI_GROUP_NODE_ROOTS can be used to represent commonly-used special groups.
Old servers may not support PMI 1.2 and fail at init. Use 1.1 for backward compatibility. Servers that support newer versions should set the version in its response. Then client can gracefully handle new features (e.g. PMI_Barrier_group);
Update the the version check in PMI_Init.
In preparation to support group PMI_Barrier, we need functions to convert from a list of global ranks to local ranks.
In preparation to support group PMI_Barrier, we need a way to map from a list of global ranks to proxies in a pg. The rankmap maps from global rank to node_id, and we need a min_node_id offset to map from node_id to the index in pg->proxy_list.
PMI_Barrier wire command may have an optional "group=..." attribute, which restrict the barrier within the process group specified by the group. The group string include special names such as "WORLD", "NODE", or a comma-separated list of world ranks. If the group attribute is missing, it defaults to "WORLD", which is backward compatible to previous PMI-1 versions (1.0 and 1.1).
Add an example using/testing PMI_Barrier_group.
Add support for procs array (that is not just a PMIX_RANK_WILDCARD) using PMI_Barrier_group wire protocol.
Add MPIR_pmi_allgather_group and MPIR_pmi_barrier_group to support allgather values over a group of processes over PMI. Because the groups may overlap, to control the access window, we will need double barrier -- one between put and get and one at the end. However, our use case is for bootstrapping business card exchange. The business card is a constant for each process thus multiple allgather access windows won't corrupt its value. Thus, we avoid the second barrier if it is provided a "name".
Use MPIR_pmi_allgather_group in MPIDI_{OFI,UCX}_comm_addr_exchange when
it is supported. Fallback to use MPIR_pmi_allgather if it is not supported
and assert that it is only called in comm_world.
Fix wrong type of addrname in ofi/init_addrxchg.c.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
True session requires address exchange within a group of processes. This PR supports it by -
PMI_Barrier_groupto PMI-1.2 APIMPIR_pmi_allgather_groupPMI_Barrier_groupMPIDIU_bc_exchange_node_rootsto avoid duplicate code in each netmod[skip warnings]
Author Checklist
Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
Commits are self-contained and do not do two things at once.
Commit message is of the form:
module: short descriptionCommit message explains what's in the commit.
Whitespace checker. Warnings test. Additional tests via comments.
For non-Argonne authors, check contribution agreement.
If necessary, request an explicit comment from your companies PR approval manager.