From 3adb263b4cd6fff014e4330df8baee93cf4f1a74 Mon Sep 17 00:00:00 2001 From: Jim Garlick Date: Wed, 2 Sep 2026 08:44:34 -0700 Subject: [PATCH 1/4] rfc13: add PMI_Barrier_group() Problem: RFC 13 does not document the PMI 1.2 PMI_Barrier_group() API function or its group constants. Add the function. Assisted-by: Claude:claude-opus-4.8 --- spec_13.rst | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/spec_13.rst b/spec_13.rst index 6995284..0ad72ef 100644 --- a/spec_13.rst +++ b/spec_13.rst @@ -557,6 +557,47 @@ Notes: - The barrier operation MUST be usable as a generic synchronization mechanism, without requiring KVS data to be queued for exchange. +.. function:: int PMI_Barrier_group (const int *group, int count, const char *stringtag) + +This function is a collective call across a subset of the process group, +introduced in PMI subversion 1.2. :var:`group` is an array of :var:`count` +process ranks identifying the subset, or one of the special constants below, +in which case :var:`count` is ignored: + +.. list-table:: + :header-rows: 1 + + * - Constant + - Value + - Meaning + * - PMI_GROUP_WORLD + - (int \*)0 + - all processes in the process group + * - PMI_GROUP_SELF + - (int \*)1 + - the calling process only + * - PMI_GROUP_NODE + - (int \*)2 + - the processes co-located on the same node as the caller + +The PMI library SHALL block until all processes in the group have entered the +call, or an error occurs. Upon successful return, KVS values put by group +members before the call are available to :c:func:`PMI_KVS_Get` within the +group. :var:`stringtag`, which MAY be NULL, distinguishes group barriers that +are outstanding concurrently, for example when issued from multiple threads. + +Errors: + +- PMI_FAIL - barrier failed, or the process manager does not support the + subversion 1.2 group barrier + +Notes: + +- PMI_GROUP_SELF is a local operation that requires no communication with the + process manager. Because a subversion 1.1 process manager fails the call, a + client MAY call PMI_Barrier_group with PMI_GROUP_SELF to probe whether the + group barrier is supported. + .. function:: int PMI_KVS_Create (char kvsname[], int length) .. function:: int PMI_KVS_Destroy (const char kvsname[]); From 95a1ff06fbc57c7fb7c83b14868509614fa02339 Mon Sep 17 00:00:00 2001 From: Jim Garlick Date: Wed, 2 Sep 2026 08:53:58 -0700 Subject: [PATCH 2/4] rfc13: update barrier wire protocol ABNF for 1.2 Problem: the PMI 1.2 wire protocol defines additional barrier attributes to support PMI_barrier_group(), but this is not documented in the RFC 13 ABNF. Add it. --- spec_13.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/spec_13.rst b/spec_13.rst index 0ad72ef..01e469a 100644 --- a/spec_13.rst +++ b/spec_13.rst @@ -838,9 +838,13 @@ Protocol Definition [SP "kvsname=" word] LF - C:barrier = "cmd=barrier_in" LF + C:barrier = "cmd=barrier_in" + [SP "group=" group] ; subversion >= 2 only + LF + S:barrier = "cmd=barrier_out" [SP "rc=" int] + [SP "tag=" int] ; subversion >= 2 only LF C:get = "cmd=get" [SP "kvsname=" word] SP "key=" word LF @@ -889,6 +893,8 @@ Protocol Definition ; macros + group = ( "WORLD" / "NODE" / ranklist ) ":" int + ranklist = uint *("," uint) ; comma-delimited process ranks intlist = int *["," int] ; comma-delimited integers word = 1*(%x21-3C %x3E-7E) ; visible char minus = string = 1*(SP HTAB VCHAR) ; visible char plus tab, space From 878a1245a2dcde46a3c315c7e255175f9622768d Mon Sep 17 00:00:00 2001 From: Jim Garlick Date: Wed, 2 Sep 2026 10:15:50 -0700 Subject: [PATCH 3/4] rfc13: add PMI 1.2 group barrier protocol notes Problem: the ABNF is not sufficient to guide an implementation of PMI 1.2 group barriers. Add some prose, informed by reviewing MPICH 5's PMI_Barrier_group() implementation. --- spec_13.rst | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/spec_13.rst b/spec_13.rst index 01e469a..40dafc5 100644 --- a/spec_13.rst +++ b/spec_13.rst @@ -757,6 +757,29 @@ If a protocol error occurs, the detecting side SHALL immediately close the connection and abort the program. IT SHOULD log the message so that the problem can be tracked down. +Barrier Operation +================= + +The group barrier introduced in PMI 1.2 has the following wire protocol +characteristics: + +- The barrier_in "group" key SHALL be a group designator string with a + decimal integer :tag suffix. + +- The group designator, which identifies the barrier scope, SHALL be one + of "WORLD" (all ranks), "NODE" (this rank), or a comma-delimited list of + ranks. "SELF" SHALL NOT occur on the wire because PMI_GROUP_SELF is fully + handled within the client implementation. + +- The tag suffix SHOULD uniquely identify the barrier instance within the + client. For example, it MAY be implemented as a thread safe, client + resident barrier counter. + +- The server SHALL return the client's integer tag value from barrier_in + in the corresponding barrier_out. This allows multiple threads to perform + concurrent barriers, which implies that a PMI 1.2 client implementation + MUST manage concurrent thread access to PMI_FD. + Spawn Operation =============== From 60962e566e6d7e5d883a715b0da2baa2beee803d Mon Sep 17 00:00:00 2001 From: Jim Garlick Date: Wed, 2 Sep 2026 13:15:28 -0700 Subject: [PATCH 4/4] rfc13: update back compatibility info Problem: The Back Compatibility section does not capture the implications of the updated version handshake description. Update this section. --- spec_13.rst | 15 ++++++++++++--- spell.en.pws | 1 + 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/spec_13.rst b/spec_13.rst index 40dafc5..95f86fb 100644 --- a/spec_13.rst +++ b/spec_13.rst @@ -928,9 +928,18 @@ Protocol Definition Back Compatibility ================== -Earlier versions of the PMI-1 wire protocol did not include the init -operation in which versions are exchanged. Protocol operations that -were culled in PMI 1.1 are not covered here. +PMI subversion 1 is not backwards compatible with version 0. Practically +speaking, subversion 0 no longer exists in the wild and therefore is +not covered by this document. The effective minimum PMI-1 subversion is 1. + +As implied in the Version Handshake section, a client sending a minimum +version of 1.1 may be answered with a server maximum of 1.2 (or beyond). +Clients need not speak subversions greater than their stated minimum, +but servers MUST speak subversions less than or equal to their stated +maximum. + +In practice, clients SHOULD always send subversion 1 and adaptively use +features offered by the server if available. Local Process Group Information ******************************* diff --git a/spell.en.pws b/spell.en.pws index 033db6a..dd65447 100644 --- a/spell.en.pws +++ b/spell.en.pws @@ -532,3 +532,4 @@ proctable Claude validators backfilled +subversions