.. _man5-pmix_server_module_t: pmix_server_module_t ==================== .. include_body `pmix_server_module_t` |mdash| The set of callback functions a host environment provides to the PMIx server library. SYNTAX ------ C Syntax ^^^^^^^^ .. code-block:: c #include typedef struct pmix_server_module_4_0_0_t { /* v1x interfaces */ pmix_server_client_connected_fn_t client_connected; // DEPRECATED pmix_server_client_finalized_fn_t client_finalized; pmix_server_abort_fn_t abort; pmix_server_fencenb_fn_t fence_nb; pmix_server_dmodex_req_fn_t direct_modex; pmix_server_publish_fn_t publish; pmix_server_lookup_fn_t lookup; pmix_server_unpublish_fn_t unpublish; pmix_server_spawn_fn_t spawn; pmix_server_connect_fn_t connect; pmix_server_disconnect_fn_t disconnect; pmix_server_register_events_fn_t register_events; pmix_server_deregister_events_fn_t deregister_events; pmix_server_listener_fn_t listener; /* v2x interfaces */ pmix_server_notify_event_fn_t notify_event; pmix_server_query_fn_t query; pmix_server_tool_connection_fn_t tool_connected; // DEPRECATED pmix_server_log_fn_t log; // DEPRECATED pmix_server_alloc_fn_t allocate; pmix_server_job_control_fn_t job_control; pmix_server_monitor_fn_t monitor; /* v3x interfaces */ pmix_server_get_cred_fn_t get_credential; pmix_server_validate_cred_fn_t validate_credential; pmix_server_iof_fn_t iof_pull; pmix_server_stdin_fn_t push_stdin; /* v4x interfaces */ pmix_server_grp_fn_t group; pmix_server_fabric_fn_t fabric; /* v6x interfaces */ pmix_server_client_connected2_fn_t client_connected2; pmix_server_tool_connection2_fn_t tool_connected2; pmix_server_log2_fn_t log2; /* pending interfaces */ pmix_server_session_control_fn_t session_control; pmix_server_resource_block_fn_t resource_block; } pmix_server_module_t; DESCRIPTION ----------- The `pmix_server_module_t` structure is the primary integration point between the PMIx server library and its host environment |mdash| a resource manager, launcher, or other system-management-stack (SMS) element. It is a table of function pointers, one per PMIx server operation, that the host implements and passes by pointer to :ref:`PMIx_server_init(3) `. PMIx uses a "function-shipping" model: each PMIx client request that requires host action is mirrored by an entry in this structure. When the PMIx server library receives such a request from a local client (or tool), it invokes the corresponding callback in the host's module, handing the host the parameters needed to service the request. This lets a resource manager implement the server role without any knowledge of PMIx internals. **Unimplemented callbacks may be left NULL.** A host is not required to support every operation. Any function the host does not provide is indicated by a NULL pointer in the structure; the library detects the NULL entry and returns ``PMIX_ERR_NOT_SUPPORTED`` to the requesting client for that operation rather than dereferencing the pointer (see, for example, ``pmix_server_publish`` in ``src/server/pmix_server_ops.c``, which returns ``PMIX_ERR_NOT_SUPPORTED`` when ``NULL == pmix_host_server.publish``). This behavior is documented at the top of ``include/pmix_server.h``: "Any functions not supported by the RM can be indicated by a NULL for the function pointer. Client calls to such functions will have a 'not supported' error returned." **Callbacks are asynchronous.** For performance and scalability, the host is required to return from every callback as quickly as possible, performing the actual work asynchronously. Nearly all callbacks are therefore non-blocking: they accept a completion callback function (``cbfunc``) and an opaque ``cbdata`` pointer, and the host is expected to invoke ``cbfunc`` with the result once the operation completes. The type of the completion function varies by operation (for example, ``pmix_op_cbfunc_t`` for simple status returns, ``pmix_modex_cbfunc_t`` for fence/modex data, ``pmix_info_cbfunc_t`` for query, allocate, job-control, monitor, group, fabric, and session operations, and so on). **Return-value contract.** The value a callback returns tells the library how completion will be signaled: * ``PMIX_SUCCESS`` |mdash| the request has been accepted and is being processed asynchronously. The host **will** invoke the supplied ``cbfunc`` later with the final result. * ``PMIX_OPERATION_SUCCEEDED`` |mdash| the operation completed immediately and successfully. The host **must not** invoke ``cbfunc``; the library treats the operation as already finished. * Any other (error) value |mdash| the request failed and the host **will not** invoke ``cbfunc``. The library reports the error to the client directly and releases any state it was holding for the request. **Data ownership.** All data passed into a host callback is owned by the PMIx server library and must not be freed by the host. Conversely, data the host returns through a completion callback is owned by the host, which may release it once the callback returns. ABI STABILITY ------------- The `pmix_server_module_t` structure is consumed by host environments that may initialize it using **positional** (non-designated) initializers. Consequently the order of fields is a binary-compatibility contract and is **frozen**: * Existing fields are never removed and never reordered. * New callbacks are only ever **appended** at the end of the structure. The ``/* v1x interfaces */`` ... ``/* pending interfaces */`` group comments record the release series in which each block of members was added. * When a callback's signature must change, the original field is **retained in place** (marked ``// DEPRECATED``) and a replacement is appended at the end with a numeric suffix on its name. The structure currently carries three deprecation pairs. In each case the library prefers the newer member and falls back to the deprecated one only if the newer member is NULL: * ``client_connected`` |rarrow| ``client_connected2``. The original ``pmix_server_client_connected_fn_t`` takes only ``proc``, ``server_object``, ``cbfunc``, and ``cbdata``. The replacement ``pmix_server_client_connected2_fn_t`` inserts an ``info[]``/``ninfo`` array so additional information can be passed about the connecting client. * ``tool_connected`` |rarrow| ``tool_connected2``. The original ``pmix_server_tool_connection_fn_t`` returns ``void``; the replacement ``pmix_server_tool_connection2_fn_t`` has an otherwise identical signature but returns a ``pmix_status_t`` so the host can synchronously reject a tool connection request. * ``log`` |rarrow| ``log2``. The original ``pmix_server_log_fn_t`` returns ``void``; the replacement ``pmix_server_log2_fn_t`` has an identical parameter list but returns a ``pmix_status_t`` so the host can indicate synchronously whether the log request was accepted. CALLBACK FUNCTIONS ------------------ The members are described below in structure order. client_connected ^^^^^^^^^^^^^^^^^ ``client_connected`` (``pmix_server_client_connected_fn_t``) |mdash| **DEPRECATED**; use ``client_connected2``. Invoked when a registered client connects to the local server. The client is held blocked until the host invokes ``cbfunc``, allowing the library to release it. This original form conveys only the client's ``proc`` and ``server_object``. client_finalized ^^^^^^^^^^^^^^^^^ ``client_finalized`` (``pmix_server_client_finalized_fn_t``) |mdash| Notifies the host that a local client has called `PMIx_Finalize`. The client is held blocked until the host invokes ``cbfunc``. The host should release any per-client state it associated with the process via ``PMIx_server_register_client``. abort ^^^^^ ``abort`` (``pmix_server_abort_fn_t``) |mdash| Services :ref:`PMIx_Abort(3) `. A local client has requested that a set of processes be terminated; the ``procs`` array names the targets, or a NULL array means all processes in the caller's namespace. The requesting client is held blocked until the host invokes ``cbfunc``. The host performs the requested termination and reports the result. fence_nb ^^^^^^^^ ``fence_nb`` (``pmix_server_fencenb_fn_t``) |mdash| Services :ref:`PMIx_Fence(3) ` and its non-blocking form. Called once all local participants of a fence have contributed; the host collectively exchanges the supplied ``data`` blob among all servers hosting participants and returns the aggregated result through a ``pmix_modex_cbfunc_t``. A NULL ``data`` means the local processes had nothing to contribute. Directives in the ``info`` array steer the collective and are optional unless the mandatory flag is set. The ``info`` array may include ``PMIX_LOCAL_COLLECTIVE_STATUS`` (pmix_status_t), by which the PMIx server library reports to the host the status of the local portion of the collective |mdash| for example, an error detected among the local participants. A host that receives a failing status should propagate it into the collective result rather than proceeding with the data exchange. The library may likewise supply ``PMIX_LOCAL_PARTICIPANTS`` (pmix_data_array_t*), an array of ``pmix_proc_t`` identifying the local processes that contributed to the collective, and ``PMIX_SORTED_PROC_ARRAY`` (bool) to indicate that the array of participating processes has already been sorted into canonical order so the host need not sort it again. These attributes apply equally to the other collective module functions below (``connect``, ``disconnect``, and ``group``). direct_modex ^^^^^^^^^^^^ ``direct_modex`` (``pmix_server_dmodex_req_fn_t``) |mdash| Requests that the host contact the remote server hosting a given process and retrieve that process's modex blob on demand (used when a :ref:`PMIx_Get(3) ` cannot be satisfied from local data). The blob is returned through a ``pmix_modex_cbfunc_t``. A timeout directive may be supplied to bound the wait. The request may carry a ``PMIX_REQUIRED_KEY`` (char*) naming the specific key the requester is waiting on, allowing the host to defer its response until that key has been posted by the target process. publish ^^^^^^^ ``publish`` (``pmix_server_publish_fn_t``) |mdash| Services :ref:`PMIx_Publish(3) `. Stores the provided key/value data in the host's data store subject to the requested range and persistence directives, recording the publisher's identity for later lookup. Completion is reported through a ``pmix_op_cbfunc_t``. lookup ^^^^^^ ``lookup`` (``pmix_server_lookup_fn_t``) |mdash| Services :ref:`PMIx_Lookup(3) `. Retrieves previously published data for a NULL-terminated array of string keys, honoring any wait/timeout directives, and returns the results through a ``pmix_lookup_cbfunc_t``. unpublish ^^^^^^^^^ ``unpublish`` (``pmix_server_unpublish_fn_t``) |mdash| Services :ref:`PMIx_Unpublish(3) `. Deletes previously published data matching a NULL-terminated array of keys (within any specified range) and reports completion through a ``pmix_op_cbfunc_t``. spawn ^^^^^ ``spawn`` (``pmix_server_spawn_fn_t``) |mdash| Services :ref:`PMIx_Spawn(3) `. Launches the given array of applications. A failure to start any process causes the entire request to be terminated and an error returned. The namespace of the spawned job is returned through a ``pmix_spawn_cbfunc_t``. The job-level information accompanying the request may include ``PMIX_REQUESTOR_IS_TOOL`` or ``PMIX_REQUESTOR_IS_CLIENT`` (bool), indicating whether the process that issued the spawn request is a tool or a client, respectively; a host may use this to apply different policies to tool-initiated launches. connect ^^^^^^^ ``connect`` (``pmix_server_connect_fn_t``) |mdash| Services :ref:`PMIx_Connect(3) `. Records the specified processes as "connected" so the host treats the failure of any of them as a reportable event. This is a client-side collective, so the callback fires once all participants have contributed; completion is reported through a ``pmix_op_cbfunc_t``. As with ``fence_nb``, the ``info`` array may carry ``PMIX_LOCAL_COLLECTIVE_STATUS``, ``PMIX_LOCAL_PARTICIPANTS``, and ``PMIX_SORTED_PROC_ARRAY`` describing the local portion of the collective. disconnect ^^^^^^^^^^ ``disconnect`` (``pmix_server_disconnect_fn_t``) |mdash| Services :ref:`PMIx_Disconnect(3) `. Reverses a prior ``connect`` of the same set of processes; an error is returned if the set was not previously connected. Completion is reported through a ``pmix_op_cbfunc_t``. As with ``fence_nb``, the ``info`` array may carry ``PMIX_LOCAL_COLLECTIVE_STATUS``, ``PMIX_LOCAL_PARTICIPANTS``, and ``PMIX_SORTED_PROC_ARRAY`` describing the local portion of the collective. register_events ^^^^^^^^^^^^^^^ ``register_events`` (``pmix_server_register_events_fn_t``) |mdash| Part of the support for :ref:`PMIx_Register_event_handler(3) `. Tells the host that the library wishes to receive notification of the specified (typically environmental) event codes; the host translates its internal codes to the corresponding PMIx codes when it later notifies. Completion is reported through a ``pmix_op_cbfunc_t``. deregister_events ^^^^^^^^^^^^^^^^^ ``deregister_events`` (``pmix_server_deregister_events_fn_t``) |mdash| The companion to ``register_events`` (see :ref:`PMIx_Deregister_event_handler(3) `). Cancels a prior registration for the specified event codes. The host remains obligated to report job-related events regardless. listener ^^^^^^^^ ``listener`` (``pmix_server_listener_fn_t``) |mdash| Optionally lets the host own the rendezvous listening socket. The host monitors the descriptor, accepts incoming client connections, and passes each accepted socket to the provided ``pmix_connection_cbfunc_t``. Leaving this member NULL causes the library to spawn its own internal listener thread. notify_event ^^^^^^^^^^^^ ``notify_event`` (``pmix_server_notify_event_fn_t``) |mdash| Services :ref:`PMIx_Notify_event(3) ` when an event generated by the server or one of its local clients must be delivered beyond the local node. The host propagates the event (identified by ``code``, ``source``, and ``range``) and reports completion through a ``pmix_op_cbfunc_t``. query ^^^^^ ``query`` (``pmix_server_query_fn_t``) |mdash| Services :ref:`PMIx_Query_info(3) `. The host answers an array of ``pmix_query_t`` requests on behalf of the identified process and returns the results through a ``pmix_info_cbfunc_t``. tool_connected ^^^^^^^^^^^^^^ ``tool_connected`` (``pmix_server_tool_connection_fn_t``) |mdash| **DEPRECATED**; use ``tool_connected2``. Invoked when a tool connects and needs a namespace/rank assignment. The host returns the assigned identifier through a ``pmix_tool_connection_cbfunc_t``. This original form returns ``void`` and so cannot synchronously reject the connection. log ^^^ ``log`` (``pmix_server_log_fn_t``) |mdash| **DEPRECATED**; use ``log2``. Services :ref:`PMIx_Log(3) ` on behalf of a client. The host must not call `PMIx_Log` from within this callback (doing so would create an infinite loop); instead it forwards the data to a gateway/logging channel or returns a not-supported error. Completion is reported through a ``pmix_op_cbfunc_t``. This original form returns ``void``. allocate ^^^^^^^^ ``allocate`` (``pmix_server_alloc_fn_t``) |mdash| Services :ref:`PMIx_Allocation_request(3) `. The host acts on the requested allocation modification (identified by a ``pmix_alloc_directive_t``) and returns any results through a ``pmix_info_cbfunc_t``. The directives may include ``PMIX_REQUESTOR`` (pmix_proc_t*) identifying the process on whose behalf the request was made when it was relayed by another process. job_control ^^^^^^^^^^^ ``job_control`` (``pmix_server_job_control_fn_t``) |mdash| Services :ref:`PMIx_Job_control(3) `. The host executes the requested control action (e.g., pause, resume, signal, terminate) against the target processes and returns results through a ``pmix_info_cbfunc_t``. The directives may include ``PMIX_REQUESTOR`` (pmix_proc_t*) identifying the process on whose behalf the request was made when it was relayed by another process. monitor ^^^^^^^ ``monitor`` (``pmix_server_monitor_fn_t``) |mdash| Services :ref:`PMIx_Process_monitor(3) `. The host arranges the requested monitoring (e.g., heartbeat or file-based) of the requestor, raising the specified error status on failure, and returns results through a ``pmix_info_cbfunc_t``. get_credential ^^^^^^^^^^^^^^ ``get_credential`` (``pmix_server_get_cred_fn_t``) |mdash| Services :ref:`PMIx_Get_credential(3) `. The host requests a security credential from the SMS on behalf of the identified process and returns it through a ``pmix_credential_cbfunc_t``. validate_credential ^^^^^^^^^^^^^^^^^^^ ``validate_credential`` (``pmix_server_validate_cred_fn_t``) |mdash| Services :ref:`PMIx_Validate_credential(3) `. The host asks the SMS to validate the supplied credential and returns the result through a ``pmix_validation_cbfunc_t``. iof_pull ^^^^^^^^ ``iof_pull`` (``pmix_server_iof_fn_t``) |mdash| Services :ref:`PMIx_IOF_pull(3) `. Registers the local server to receive the specified IO channels (a ``pmix_iof_channel_t`` bitmask) forwarded from a set of source processes; the ``PMIX_IOF_STOP`` directive instead removes the server from the distribution list. Note that ``stdin`` cannot be pulled with this call. Completion is reported through a ``pmix_op_cbfunc_t``. push_stdin ^^^^^^^^^^ ``push_stdin`` (``pmix_server_stdin_fn_t``) |mdash| Services :ref:`PMIx_IOF_push(3) `. Passes a ``stdin`` byte object from a source process to the host for delivery to the named target processes (a WILDCARD rank targets all processes in a namespace). Completion is reported through a ``pmix_op_cbfunc_t``. group ^^^^^ ``group`` (``pmix_server_grp_fn_t``) |mdash| Services :ref:`PMIx_Group_construct(3) ` and the related group operations. The ``op`` argument (a ``pmix_group_operation_t``) selects construct, destruct, or cancel of the named group across its member processes; directives may request, for example, assignment of a group context ID. Results are returned through a ``pmix_info_cbfunc_t``. As with ``fence_nb``, the ``info`` array may carry ``PMIX_LOCAL_COLLECTIVE_STATUS``, ``PMIX_LOCAL_PARTICIPANTS``, and ``PMIX_SORTED_PROC_ARRAY`` describing the local portion of the collective. fabric ^^^^^^ ``fabric`` (``pmix_server_fabric_fn_t``) |mdash| Services :ref:`PMIx_Fabric_register(3) ` and related fabric operations. The host retrieves fabric-related information from the server supporting the system scheduler, per the ``pmix_fabric_operation_t`` op, and returns it through a ``pmix_info_cbfunc_t``. client_connected2 ^^^^^^^^^^^^^^^^^ ``client_connected2`` (``pmix_server_client_connected2_fn_t``) |mdash| Replaces the deprecated ``client_connected``. Invoked when a registered client connects; the library prefers this member and only falls back to ``client_connected`` when this is NULL. Adds an ``info[]``/``ninfo`` array carrying additional information about the connecting client. The client is held blocked until the host invokes ``cbfunc``. tool_connected2 ^^^^^^^^^^^^^^^ ``tool_connected2`` (``pmix_server_tool_connection2_fn_t``) |mdash| Replaces the deprecated ``tool_connected``. Invoked when a tool connects and needs a namespace/rank assignment; the library prefers this member over ``tool_connected``. Unlike the original, it returns a ``pmix_status_t`` so the host can synchronously reject the connection. The assigned identifier is returned through a ``pmix_tool_connection_cbfunc_t``. log2 ^^^^ ``log2`` (``pmix_server_log2_fn_t``) |mdash| Replaces the deprecated ``log`` and services :ref:`PMIx_Log(3) `; the library prefers this member and falls back to ``log`` only when it is NULL. Identical in parameters to ``log`` but returns a ``pmix_status_t``. As with ``log``, the host must not call `PMIx_Log` from within this callback. session_control ^^^^^^^^^^^^^^^ ``session_control`` (``pmix_server_session_control_fn_t``) |mdash| Services :ref:`PMIx_Session_control(3) `. The host executes the requested control operation against the identified session (``sessionID``) and returns results through a ``pmix_info_cbfunc_t``. The directives may include ``PMIX_REQUESTOR`` (pmix_proc_t*) identifying the process on whose behalf the request was made when it was relayed by another process. resource_block ^^^^^^^^^^^^^^ ``resource_block`` (``pmix_server_resource_block_fn_t``) |mdash| Services :ref:`PMIx_Resource_block(3) `. The host defines, deletes, extends, or removes resources from a named resource block per the ``pmix_resource_block_directive_t`` and the supplied array of ``pmix_resource_unit_t`` units. Completion is reported through a ``pmix_op_cbfunc_t``. .. seealso:: :ref:`PMIx_server_init(3) `, :ref:`PMIx_server_register_nspace(3) `, :ref:`PMIx_Abort(3) `, :ref:`PMIx_Fence(3) `, :ref:`PMIx_Publish(3) `, :ref:`PMIx_Lookup(3) `, :ref:`PMIx_Unpublish(3) `, :ref:`PMIx_Spawn(3) `, :ref:`PMIx_Connect(3) `, :ref:`PMIx_Disconnect(3) `, :ref:`PMIx_Register_event_handler(3) `, :ref:`PMIx_Deregister_event_handler(3) `, :ref:`PMIx_Notify_event(3) `, :ref:`PMIx_Query_info(3) `, :ref:`PMIx_Log(3) `, :ref:`PMIx_Allocation_request(3) `, :ref:`PMIx_Job_control(3) `, :ref:`PMIx_Process_monitor(3) `, :ref:`PMIx_Get_credential(3) `, :ref:`PMIx_Validate_credential(3) `, :ref:`PMIx_IOF_pull(3) `, :ref:`PMIx_IOF_push(3) `, :ref:`PMIx_Group_construct(3) `, :ref:`PMIx_Fabric_register(3) `, :ref:`PMIx_Session_control(3) `, :ref:`PMIx_Resource_block(3) `, :ref:`pmix_info_t(5) `, :ref:`pmix_proc_t(5) `