19.2.262. PMIx_server_IOF_flow_control

PMIx_server_IOF_flow_control — Suspend or resume the processes feeding stdin to this PMIx server.

19.2.262.1. SYNOPSIS

#include <pmix_server.h>

pmix_status_t PMIx_server_IOF_flow_control(const pmix_proc_t *source,
                                           pmix_iof_channel_t channel,
                                           bool xoff,
                                           const pmix_info_t directives[], size_t ndirs,
                                           pmix_op_cbfunc_t cbfunc, void *cbdata);

19.2.262.1.1. Python Syntax

from pmix import *

foo = PMIxServer()
# ... after a successful foo.init() ...
pydirs = []

# stop every process feeding us stdin
rc = foo.iof_flow_control(None, PMIX_FWD_STDIN_CHANNEL, True, pydirs)

# ... and later, let them resume
rc = foo.iof_flow_control(None, PMIX_FWD_STDIN_CHANNEL, False, pydirs)

# or name a single producer
source = {'nspace': "mytool", 'rank': 0}
rc = foo.iof_flow_control(source, PMIX_FWD_STDIN_CHANNEL, True, pydirs)

The Python binding is the blocking form — it passes no callback, and returns the status directly. A source of None is the Python spelling of a NULL source, and means every process feeding this server.

19.2.262.2. INPUT PARAMETERS

source: Pointer to a pmix_proc_t(5) identifying the process whose stdin flow is to be controlled. A NULL value, or a source carrying a wildcard rank and/or namespace, applies the request to every process feeding stdin to this server.
channel: The pmix_iof_channel_t(5) identifying the channel to control. Only PMIX_FWD_STDIN_CHANNEL can be flow-controlled.
xoff: true to suspend the stream, false to resume it.
directives: Array of pmix_info_t(5) structures qualifying the request. A NULL value (with ndirs of zero) is supported when no qualifiers are provided.
ndirs: Number of elements in the directives array.
cbfunc: Callback function of type pmix_op_cbfunc_t invoked once the request has been applied. A NULL value makes the call blocking.
cbdata: Opaque pointer that is passed, unmodified, to cbfunc.

19.2.262.3. DESCRIPTION

A host environment that finds itself falling behind on the stdin it is being handed through its push_stdin upcall has, without this function, only two options: drop bytes, or queue them without bound. PMIx_server_IOF_flow_control provides the third — reach back to whoever is producing the data and stop them at the source.

The library applies the request to both kinds of producer it can reach:

any stdin the library is reading on this process’s own behalf — the read is left un-armed while the stream is suspended; and
every tool that has pushed stdin to this server — each is sent the request, and a tool that is itself a server relays it onward to its own producers. A chain of launchers therefore carries the request all the way back to the process actually holding the input stream.

Nothing is buffered by PMIx on behalf of a suspended stream, and nothing is lost. The bytes that would have been read simply stay in the producer’s input stream, where the operating system applies the back-pressure. An XOFF is therefore never permission to drop data.

A suspension persists until the function is called again with xoff set to false. Every XOFF must eventually be paired with an XON, or the stream stalls for the life of the producer.

PMIx_server_IOF_flow_control supports both blocking and non-blocking operation. When cbfunc is non-NULL the call is non-blocking: it thread-shifts the request into the library’s progress thread, returns immediately, and invokes cbfunc when the request has been applied. When cbfunc is NULL the call is blocking: it does not return until the request has been applied.

The host is required to retain the directives array until the callback is executed — or, in the blocking case, until the function returns.

19.2.262.4. SUSPENDING WITHOUT CALLING THIS FUNCTION

A host that would rather not track its producers can suspend a stream opportunistically instead, by completing a push_stdin upcall with PMIX_ERR_IOF_XOFF in place of PMIX_SUCCESS. That status means “I have taken this data, now stop sending” — the data it accompanies is delivered, no error is reported to the producer, and the producing stream is suspended exactly as it would be by an xoff of true here.

PMIX_ERR_IOF_XOFF is deliberately not a failure: no data has been lost and the stream has not been closed, so a tool whose stdin forwarding is suspended this way is not notified and does not see a PMIX_ERR_IOF_FAILURE event.

Resumption is only ever through this function — there is no status that means “resume”.

19.2.262.5. RETURN VALUE

For the non-blocking form (cbfunc is non-NULL), a return of PMIX_SUCCESS indicates only that the request was accepted for processing; the final status is delivered through cbfunc. For the blocking form (cbfunc is NULL), a return of PMIX_OPERATION_SUCCEEDED indicates that the request was applied successfully — cbfunc is not called. Possible return values include:

PMIX_SUCCESS — (non-blocking form) the request was accepted for processing.
PMIX_OPERATION_SUCCEEDED — (blocking form) the request was applied.
PMIX_ERR_NOT_SUPPORTED — channel did not include PMIX_FWD_STDIN_CHANNEL. stdin is the only stream whose producer this library can reach; output flows the other way, from processes the library does not control.
PMIX_ERR_NOMEM — the library was unable to allocate memory for the request.
PMIX_ERR_NOT_AVAILABLE — the operation cannot be serviced because the library’s progress engine has been stopped.
PMIX_ERR_INIT — the PMIx server library has not been initialized.

Any other negative value indicates an appropriate error condition. PMIx error constants are defined in pmix_common.h.

19.2.262.6. NOTES

Naming a source that is not currently producing stdin is not an error; the request simply reaches nobody. Likewise, an XON for a stream that was never suspended is a no-op.

A producer running a PMIx release that predates flow control is never sent a request — it keeps producing, exactly as it did before this function existed. A host can detect support at build time through the PMIX_CAP_IOF_FLOW_CONTROL capability flag in pmix_version.h.

19.2.262.7. PROGRESS THREAD RESTRICTION

A blocking PMIx call must not be made from within the PMIx progress thread. Any code the library itself invokes runs on that thread: an event handler registered through PMIx_Register_event_handler(3), a callback passed to a non-blocking PMIx API, and — in a server or tool — the completion of a host-module up-call. A blocking call waits for work that the progress thread has to perform, so making one from that thread waits for itself and never returns. The PMIx Standard disallows it, and there is no way for an implementation to service such a request.

Where this call has a blocking form — including the blocking behavior a non-blocking entry point adopts when it is passed a NULL cbfunc — that form detects the situation and returns PMIX_ERR_WOULD_BLOCK immediately, accompanied by a diagnostic naming the call. Nothing is done and no callback is invoked.

PMIX_ERR_WOULD_BLOCK here is not a transient condition to retry: it reports a call that cannot be serviced from where it was made. Reissue it as the non-blocking form with a callback, or from a thread of your own.