OpenPMIx

Reference Implementation of the Process Management Interface Exascale (PMIx) standard

View the Project on GitHub

Downloads   Privacy Policy   Security Policy   Publications   Community   Contribute

RFC0018

Title

Extend the Event Notification RFC

Abstract

As use of the event notification system grows, requests for some added capabilities have begun to surface. These include the ability to notify across threads in the same process, and the ability to direct ordering of event notifications when registering event handlers. This RFC addresses those needs without requiring any changes to the involved APIs.

Labels

[ATTRIBUTES][BEHAVIOR]

Action

[APPROVED]

Copyright (c) 2017 Intel, Inc. All rights reserved.

This document is subject to all provisions relating to code contributions to the PMIx community as defined in the community’s LICENSE file. Code Components extracted from this document must include the License text as described in that file.

Description

Several use-cases have identified desirable extensions to the behavior of the event notification system, especially in the registration area. These include:

None of these require modification of existing PMIx APIs, nor addition of new ones. Instead, all can be supported by adding attributes to direct the behavior of the existing event registration/notification functions.

Event registration extensions

There are three proposed changes to event registration.

Registering multiple event handlers against same codes

This change stipulates that the implementation must support multiple event handlers being registered against the same status code(s). PMIx has always had a requirement that a status code could be used in both a single status event handler, and multiple multi-status event handlers. However, we previously required that a single-status event registration could only be done once using that status code, and that a given combination of status codes could only be registered once as a multi-status handler. This RFC removes those restrictions and requires that an implementation support multiple single-status handlers registered against a given code, and multiple handlers registered against the same combination of codes. The precedence attributes described below can be applied to direct ordering across these handlers.

Note that the current standard doesn’t explicitly state that any PMIX_EVENT_HDLR_NAME attribute given at registration be unique. The RFC corrects that lack by making this an explicit requirement. The PMIx implementation is required to return an error whenever this is violated, regardless of any other attributes that may or may not be provided. However, an event registration is not required to provide a handler name – any non-named handlers cannot be subsequently cited when requesting precedence.

Desired ordering

The current PMIx standard does not actually specify a default ordering for event handlers as they are being registered. However, it does include an inherent ordering for invocation. Specifically, PMIx stipulates that handlers be called in the following categorical order:

The intent of this RFC is to add a capability to support arbitrary ordering within these categories – i.e., one can direct that a single status event handler come before another single status handler, but not that a multi-event handler precede a single event handler. There are only two allowed exceptions – a user may request that a specified event handler be executed at the beginning or at the end of the chain, regardless of category.

The primary need here is for attributes indicating desired ordering of the event handler being registered vs other handlers that have already been registered or will subsequently be registered. In both cases, it is necessary that the user provide something to identify each handler so the relative position can later be specified – this is to be done via the existing PMIX_EVENT_HDLR_NAME attribute. In addition, the RFC retains the existing PMIX_EVENT_ORDER_PREPEND attribute that directs PMIx to prepend the handler being registered to the front of the chain for that category of handler, but renames that attribute to PMIX_EVENT_HDLR_PREPEND for consistency.

Since no default ordering was previously specified, users could not be certain of expected behavior when registering handlers. Thus, this RFC removes the ambiguity by requiring implementations to default to prepending event handlers within their respective categories.

Within these constraints, the following new attributes are proposed:

Proc-Local data range

In addition to the above attributes, this RFC proposes a new data range definition:

#define PMIX_RANGE_PROC_LOCAL    7   // restrict range to the local proc

When given in a registration, the attribute indicates that the requestor only wishes this event handler to receive events generated by itself (e.g., by another thread within the same process).

Event notification extensions

Three changes are proposed to the event notification area.

Aggregation of event handler results

The current event notification standard requires that the implementation simply append any results returned by an event handler to the array of results passed in to subsequent handlers. It was left to each invoked event handler to scan the array of prior results and determine what, if any, action to take based on their content.

This RFC proposes to modify this behavior by allowing event handlers to modify the incoming results array. Thus, the implementation will be supplying a consolidated set of results to subsequent event handlers in place of the current aggregated array.

Event handlers can modify the results array in two ways:

Additions to the results array can be provided in the event handler’s notification callback function. Any pmix_info_t structures provided in that array will continue to be appended to the results array prior to invoking the next handler. Note that the status returned by each event handler will be appended to the results array just before addition of any returned structures, as is currently done.

Receiver-level filtering of events

This change requires that the implementation apply any data range directives given during event handler registration to the notification prior to invoking the handler. Prior to this RFC, the delivery of an event to a specific process was governed solely by the data range given by the event generator. This RFC adds a second filter to the delivery procedure by allowing the registration to also specify the range of sources it will accept. Thus, even though an event generator may direct that this event go to a target process, the individual handlers registered by that process can specify which of them want to be invoked to service an event from that source range. Registrations that do not include a range directive will not be filtered.

New attributes

The following event attributes are added to support fault tolerance behavior within the application:

Protoype Implementation

Prototype implementation available in PMIx master repo in Pull Request 344

Author(s)

Ralph H. Castain
Intel, Inc.
Github: rhc54