19.2.8. PMIx_Get

PMIx_Get, PMIx_Get_nb — Retrieve a value posted by a process against a specified key.

19.2.8.1. SYNOPSIS

#include <pmix.h>

pmix_status_t PMIx_Get(const pmix_proc_t *proc, const char key[],
                       const pmix_info_t info[], size_t ninfo,
                       pmix_value_t **val);

pmix_status_t PMIx_Get_nb(const pmix_proc_t *proc, const char key[],
                          const pmix_info_t info[], size_t ninfo,
                          pmix_value_cbfunc_t cbfunc, void *cbdata);

19.2.8.1.1. Python Syntax

from pmix import *

foo = PMIxClient()
# ... after a successful foo.init() ...
# the proc is a Python ``pmix_proc_t`` dictionary
proc = {'nspace': "testnspace", 'rank': 0}
# the directives is a list of Python ``pmix_info_t`` dictionaries
pydirs = [{'key': PMIX_TIMEOUT,
           'value': 10, 'val_type': PMIX_INT}]
rc, value = foo.get(proc, "mykey", pydirs)

# the non-blocking form returns as soon as the request has been accepted
# and reports the result by executing a callback on the PMIx progress
# thread. The callback is run if and only if the call returned
# PMIX_SUCCESS, and must not itself make a blocking PMIx call.
def valuecb(status, value, cbdata):
    # value is a Python ``pmix_value_t`` dictionary, or None if the key
    # was not found
    print("get completed:", foo.error_string(status), value)
rc = foo.get_nb(proc, "mykey", pydirs, valuecb, "mycbdata")

19.2.8.2. INPUT PARAMETERS

proc: Pointer to a pmix_proc_t identifying the process whose posted data is to be retrieved. A NULL value is used in place of the caller’s own identifier. A rank of PMIX_RANK_WILDCARD retrieves job-level information for the process’s namespace.
key: A NULL-terminated string, no longer than PMIX_MAX_KEYLEN characters, naming the value to retrieve. A NULL key requests all data associated with the identified process.
info: Pointer to an array of pmix_info_t(5) structures conveying directives that qualify the operation (see DIRECTIVES). A NULL value is supported when no directives are desired.
ninfo: Number of elements in the info array.

19.2.8.3. OUTPUT PARAMETERS

val (blocking form): Address of a pmix_value_t pointer. On success it is set as described under DESCRIPTION. When the PMIX_GET_STATIC_VALUES directive is used, the caller must instead provide storage and pass a pointer to it here as an input.

The non-blocking form replaces val with a callback:

cbfunc: Callback function of type pmix_value_cbfunc_t invoked with the retrieved value once it is available.
cbdata: Opaque pointer that is passed, unmodified, to cbfunc.

19.2.8.4. DESCRIPTION

Retrieve information for the specified key as posted by the process identified in proc. PMIx_Get is the blocking form: it does not return until the data is available (or the operation fails or times out). PMIx_Get_nb is the non-blocking form: it returns immediately, and the provided cbfunc is invoked with the final status and value once the data has been retrieved by the local server.

The precedence rules governing where the library looks for the data — the local client cache, the local PMIx server, and the host resource manager — differ for reserved keys (those provided by the host environment, beginning with "pmix") and non-reserved keys (those posted by the application via PMIx_Put(3)). Those rules are governed by the directives described below.

Return of the value (blocking form) is controlled by the directives:

In the absence of any directive, the returned pmix_value_t is a freshly allocated memory object. The caller is responsible for releasing it with PMIX_VALUE_RELEASE when done.
With PMIX_GET_POINTER_VALUES, the function returns a pointer directly into the PMIx library’s key-value store. The caller must not release the returned data.
With PMIX_GET_STATIC_VALUES, the value is returned in caller-provided storage: the caller must pass a pointer to a pmix_value_t it owns in val and destruct it with PMIX_VALUE_DESTRUCT when done. If the implementation cannot return a static value, PMIx_Get returns PMIX_ERR_NOT_SUPPORTED.

As with all non-blocking PMIx APIs, callers of PMIx_Get_nb must keep the proc and info arrays valid until cbfunc is invoked.

19.2.8.5. DIRECTIVES

The following attributes are relevant to this operation. All but PMIX_TIMEOUT are required to be supported by all PMIx libraries; PMIX_TIMEOUT is optional for host environments.

PMIX_OPTIONAL (bool) — look only in the client’s local data store; do not request the data from the PMIx server if it is not found locally.
PMIX_IMMEDIATE (bool) — direct the PMIx server to immediately return an error if the requested data is not in its store, rather than requesting it from the host resource manager.
PMIX_GET_REFRESH_CACHE (bool) — refresh the local cache for the target process from the server before returning, picking up values put and committed since the last refresh. A NULL key refreshes all values for the process; a rank of PMIX_RANK_WILDCARD refreshes job-related information.
PMIX_DATA_SCOPE (pmix_scope_t) — restrict the scope of data to be searched (see PMIx_Put(3)).
PMIX_GET_POINTER_VALUES (bool) — return a pointer to the value in the library’s key-value store instead of a copy; the caller must not free it.
PMIX_GET_STATIC_VALUES (bool) — return the value in storage provided by the caller via the val parameter.
PMIX_SESSION_INFO / PMIX_JOB_INFO / PMIX_APP_INFO / PMIX_NODE_INFO (bool) — direct that the requested key be retrieved from the session-, job-, application-, or node-level information, respectively.
PMIX_TIMEOUT (int) — maximum time, in seconds, for the get to complete before returning an error. Helps avoid “hangs” when the target process never exposes the requested data.

19.2.8.6. RESERVED KEYS

In addition to keys posted by applications via PMIx_Put(3), the host environment provides a range of reserved keys (those beginning with "pmix") that describe the process, its job, and the resources it occupies. These are passed as the key parameter and, unless otherwise noted, are retrieved for the process identified in proc. A rank of PMIX_RANK_WILDCARD retrieves the job-level value for the process’s namespace. The following are commonly available:

Process identity and rank

PMIX_NSPACE (char*) — namespace of the process’s job.
PMIX_JOBID (char*) — job identifier assigned by the scheduler.
PMIX_PROCID (pmix_proc_t*) — process identifier (namespace and rank). Self-referential: see below.
PMIX_RANK (pmix_rank_t) — rank of the process within its job. Self-referential: see below.
PMIX_GLOBAL_RANK (pmix_rank_t) — rank of the process spanning across all jobs in this session.
PMIX_APP_RANK (pmix_rank_t) — rank of the process within its application.
PMIX_LOCAL_RANK (uint16_t) — rank of the process on its node within its job.
PMIX_NODE_RANK (uint16_t) — rank of the process on its node spanning all jobs.
PMIX_PACKAGE_RANK (uint16_t) — rank of the process within its job on the package where it resides.
PMIX_NPROC_OFFSET (pmix_rank_t) — starting global rank of this job.
PMIX_LOCALLDR (pmix_rank_t) — lowest rank on this node within this job.
PMIX_APPLDR (pmix_rank_t) — lowest rank in this application within this job.
PMIX_APPNUM (uint32_t) — application number within the job in which the process is included.
PMIX_SESSION_ID (uint32_t) — session identifier.
PMIX_PROC_PID (pid_t) — operating-system process identifier (pid) of the specified process.

Note

PMIX_RANK and PMIX_PROCID are self-referential — they name the very thing the proc parameter must already supply, so asking for them about a process identified in proc conveys no information and is not supported. Each is instead retrieved by leaving the corresponding part of proc unspecified, which asks “who am I?”:

PMIX_RANK — pass a proc whose namespace is the caller’s own and whose rank is PMIX_RANK_INVALID. The caller’s own rank is returned, as a PMIX_PROC_RANK value.
PMIX_PROCID — pass NULL for proc. The caller’s own process identifier is returned.

Passing a fully specified proc with either key returns PMIX_ERR_NOT_FOUND. Note in particular that PMIX_RANK is not retrievable for another process: it serves as the identifier of the process-realm information supplied by the host (see PMIX_PROC_INFO_ARRAY) rather than as a stored value. A process’s rank is already known to any caller able to name it. The remaining rank keys in this section — PMIX_GLOBAL_RANK, PMIX_APP_RANK, PMIX_LOCAL_RANK, PMIX_NODE_RANK and PMIX_PACKAGE_RANK — are ordinary stored values and are retrieved for the process identified in proc in the usual way.

In the same vein, PMIX_VERSION_NUMERIC reports the version of the library the caller is linked against and ignores proc entirely.

Node and host information

PMIX_HOSTNAME (char*) — name of the host on which the specified process is located.
PMIX_HOSTNAME_ALIASES (char*) — comma-delimited list of names by which the node is known.
PMIX_HOSTNAME_KEEP_FQDN (bool) — indicates that fully-qualified domain-name hostnames are being retained.
PMIX_NODEID (uint32_t) — node identifier where the specified process is located.
PMIX_CLUSTER_ID (char*) — string name of the cluster on which the process is executing.
PMIX_NODE_LIST (char*) — comma-delimited list of nodes running processes for the specified namespace.
PMIX_ALLOCATED_NODELIST (char*) — comma-delimited list of all nodes in this allocation, regardless of whether or not they currently host processes.
PMIX_LOCAL_PEERS (char*) — comma-delimited string of ranks on this node within the specified namespace.
PMIX_LOCAL_PROCS (pmix_data_array_t*) — array of pmix_proc_t(5) structures identifying the processes on the specified node.
PMIX_LOCAL_CPUSETS (char*) — colon-delimited cpusets of the local peers within the specified namespace.

Sizes and counts

PMIX_UNIV_SIZE (uint32_t) — number of slots in this session.
PMIX_JOB_SIZE (uint32_t) — number of processes in this job.
PMIX_JOB_NUM_APPS (uint32_t) — number of applications in this job.
PMIX_APP_SIZE (uint32_t) — number of processes in the application.
PMIX_LOCAL_SIZE (uint32_t) — number of processes in this job on the local node.
PMIX_NODE_SIZE (uint32_t) — number of processes across all jobs on this node.
PMIX_MAX_PROCS (uint32_t) — maximum number of processes for this job.
PMIX_NUM_SLOTS (uint32_t) — number of slots allocated.
PMIX_NUM_NODES (uint32_t) — number of nodes currently hosting processes in the specified realm.
PMIX_NUM_ALLOCATED_NODES (uint32_t) — number of nodes in the specified realm, regardless of whether or not they currently host processes.

Scratch directories

PMIX_TMPDIR (char*) — top-level temporary directory assigned to the session.
PMIX_NSDIR (char*) — sub-directory of the session temporary directory assigned to the namespace.
PMIX_PROCDIR (char*) — sub-directory of the namespace directory assigned to the process.
PMIX_TDIR_RMCLEAN (bool) — the resource manager will clean up the session directories, so the application need not do so.

Process state and launch

PMIX_SPAWNED (bool) — true if this process resulted from a call to PMIx_Spawn(3).
PMIX_PARENT_ID (pmix_proc_t*) — identifier of the process that called PMIx_Spawn(3) to launch this process’s application.
PMIX_EXIT_CODE (int) — exit code returned when the specified process terminated.
PMIX_REINCARNATION (uint32_t) — number of times this process has been instantiated — i.e., a count of the number of times it has been restarted.
PMIX_NODE_OVERSUBSCRIBED (bool) — true if the number of processes from this job on this node exceeds the number of slots allocated to it.
PMIX_CPUSET_BITMAP (pmix_cpuset_t*) — bitmap applied to the process at launch.
PMIX_CREDENTIAL (char*) — security credential assigned to the process.

Process set membership

PMIX_PSET_NAMES (pmix_data_array_t*) — array of the string names of the process sets in which the specified process is a member.

Fabric locality

PMIX_SWITCH_PEERS (pmix_data_array_t*) — the peer ranks that share a switch with the specified process. Returns an array of pmix_info_t, one per local fabric device; each element is itself a PMIX_SWITCH_PEERS entry holding a three-element array that gives the device’s PMIX_FABRIC_DEVICE_ID, the PMIX_FABRIC_SWITCH to which it is connected, and a comma-delimited string of the peer ranks sharing that switch.

19.2.8.7. RETURN VALUE

For the blocking form, PMIX_SUCCESS indicates that the requested data was returned in the manner requested. For the non-blocking form, a return of PMIX_SUCCESS indicates only that the request was accepted for processing; the final status and value are delivered to cbfunc.

PMIX_SUCCESS — the requested data has been returned.
PMIX_ERR_NOT_FOUND — the requested data was not available. This includes a key that was available earlier and has since been deleted — see NOTES.
PMIX_ERR_EXISTS_OUTSIDE_SCOPE — the requested key exists, but was posted in a scope that does not include the requester.
PMIX_ERR_BAD_PARAM — an invalid argument was supplied — for example, an over-length key, or the PMIX_GET_STATIC_VALUES directive with a NULL storage location.
PMIX_ERR_NOT_SUPPORTED — the implementation cannot satisfy a requested directive (e.g., PMIX_GET_STATIC_VALUES).
PMIX_ERR_NOT_AVAILABLE — the operation cannot be serviced because the library’s progress engine has been stopped.
PMIX_ERR_INIT — the PMIx library has not been initialized.

Any other negative value indicates an appropriate error condition. PMIx error constants are defined in pmix_common.h.

19.2.8.8. NOTES

For non-reserved keys, retrieval blocks until the target process has posted the key with PMIx_Put(3), committed it with PMIx_Commit(3), and the data has been circulated — typically via a PMIx_Fence(3) — unless the PMIX_OPTIONAL or PMIX_IMMEDIATE directives are used to bound the search.

Qualified values. When a key was posted together with qualifiers (using the PMIX_QUALIFIED_VALUE key to PMIx_Put(3)), supply the desired qualifiers in the info array to select among the values stored under that key. PMIx_Get returns the primary value of the matching entry; the qualifiers themselves are used only for matching and are not returned.

A key that was there can go away. Published data used only to accumulate, so a key that was once retrievable stayed retrievable. That is no longer true: a process can retract one of its own keys with the PMIX_DEL_* scopes of PMIx_Put(3), and a host environment can retract job-level information it registered with PMIx_server_deregister_resources(3). A call that succeeded for a key can therefore return PMIX_ERR_NOT_FOUND later in the same job, and callers that cannot tolerate that must be prepared for it.

The removal reaches this process as a notification from its local PMIx server, which drops the cached copy the caller was given. For a key belonging to a process on another node it arrives with the collecting PMIx_Fence(3) that carries the removal. The notification is one-way and unacknowledged, so it arrives promptly rather than synchronously: a PMIx_Get that races it can return the old value once more. Once it has landed, a request for the key behaves exactly like one for a key that was never published — for a non-reserved key that means the call blocks until someone publishes it, unless PMIX_OPTIONAL or PMIX_IMMEDIATE bounds the search.

Values from another process are cached, and a later exchange does not refresh them. Once a process’s data has been retrieved, subsequent calls for that process are answered from the local cache without consulting the server. So if the target process posts a new value under a key it had already published, and the data is circulated again, this call keeps returning the value read the first time. That is deliberate — it is what makes a lookup after a full exchange cost nothing — but it means a caller that expects an updated value must ask for one with PMIX_GET_REFRESH_CACHE:

pmix_info_t info;
bool refresh = true;

PMIX_INFO_LOAD(&info, PMIX_GET_REFRESH_CACHE, &refresh, PMIX_BOOL);
rc = PMIx_Get(&proc, "mykey", &info, 1, &val);

Note that a refresh for a non-reserved key brings across everything the target process has published, not only the key named in the call, so one refresh updates the caller’s whole view of that process.

19.2.8.9. PROGRESS THREAD RESTRICTION

A blocking PMIx call must not be made from within the PMIx progress thread. Any code the library itself invokes runs on that thread: an event handler registered through PMIx_Register_event_handler(3), a callback passed to a non-blocking PMIx API, and — in a server or tool — the completion of a host-module up-call. A blocking call waits for work that the progress thread has to perform, so making one from that thread waits for itself and never returns. The PMIx Standard disallows it, and there is no way for an implementation to service such a request.

Where this call has a blocking form — including the blocking behavior a non-blocking entry point adopts when it is passed a NULL cbfunc — that form detects the situation and returns PMIX_ERR_WOULD_BLOCK immediately, accompanied by a diagnostic naming the call. Nothing is done and no callback is invoked.

PMIX_ERR_WOULD_BLOCK here is not a transient condition to retry: it reports a call that cannot be serviced from where it was made. Reissue it as the non-blocking form with a callback, or from a thread of your own.