.. _man3-PMIx_Allocation_request: PMIx_Allocation_request ======================= .. include_body ``PMIx_Allocation_request``, ``PMIx_Allocation_request_nb`` |mdash| Request an allocation operation from the host resource manager or scheduler. SYNOPSIS -------- .. code-block:: c #include pmix_status_t PMIx_Allocation_request(pmix_alloc_directive_t directive, pmix_info_t *info, size_t ninfo, pmix_info_t **results, size_t *nresults); pmix_status_t PMIx_Allocation_request_nb(pmix_alloc_directive_t directive, pmix_info_t *info, size_t ninfo, pmix_info_cbfunc_t cbfunc, void *cbdata); Python Syntax ^^^^^^^^^^^^^ .. code-block:: python3 from pmix import * foo = PMIxClient() # ... after a successful foo.init() ... # the directives is a list of Python ``pmix_info_t`` dictionaries pyinfo = [{'key': PMIX_ALLOC_NUM_NODES, 'value': {'value': 4, 'val_type': PMIX_UINT64}}] rc, results = foo.allocation_request(PMIX_ALLOC_EXTEND, pyinfo) INPUT PARAMETERS ---------------- * ``directive``: An allocation directive of type ``pmix_alloc_directive_t`` identifying the requested operation (see `DIRECTIVES`_). * ``info``: Pointer to an array of :ref:`pmix_info_t(5) ` structures describing and qualifying the request (see `ATTRIBUTES`_). A ``NULL`` value is supported when no attributes are desired. * ``ninfo``: Number of elements in the ``info`` array. OUTPUT PARAMETERS ----------------- * ``results`` (blocking form): Address where a pointer to an array of :ref:`pmix_info_t(5) ` structures containing the results of the request is returned. Set to ``NULL`` when no data is returned. The caller is responsible for releasing the returned array with ``PMIX_INFO_FREE`` when done. * ``nresults`` (blocking form): Address where the number of elements in the returned ``results`` array is stored. The non-blocking form replaces ``results`` and ``nresults`` with a callback: * ``cbfunc``: Callback function of type :ref:`pmix_info_cbfunc_t ` invoked with the final status and any returned data once the request has been processed. * ``cbdata``: Opaque pointer that is passed, unmodified, to ``cbfunc``. DESCRIPTION ----------- Request an allocation operation from the host resource manager or scheduler. Several broad categories of operation are envisioned, including the ability to: * Request allocation of additional resources, including memory, bandwidth, and compute. Note that the new allocation will be disjoint from (i.e., not affiliated with) the allocation of the requestor |mdash| thus the termination of one allocation will not impact the other. * Extend the reservation on currently allocated resources, subject to scheduling availability and priorities. This includes extending the time limit on current resources and/or requesting that additional resources be allocated to the requesting job. Any additional allocated resources will be considered part of the current allocation, and thus will be released at the same time. * Release currently allocated resources that are no longer required. This is intended to support partial release of resources, since all resources are normally released upon termination of the job. * "Lend" resources back to the scheduler with an expectation of getting them back at some later time in the job, together with the corresponding ability to "reacquire" resources previously released. ``PMIx_Allocation_request`` is the blocking form: it does not return until the operation is complete. If successful, results are returned in the ``results`` and ``nresults`` parameters. ``PMIx_Allocation_request_nb`` is the non-blocking form: it returns immediately, and the provided ``cbfunc`` is invoked with the final status and any returned data once the operation has been processed. If the request is for additional resources and it succeeds, the returned results include the host environment's allocation identifier (``PMIX_ALLOC_ID``) that the requester can use to reference the resulting resources in, for example, a later spawn request. As with all non-blocking PMIx APIs, callers of ``PMIx_Allocation_request_nb`` **must** keep the ``info`` array valid until ``cbfunc`` is invoked. DIRECTIVES ---------- The ``directive`` argument is a value of type ``pmix_alloc_directive_t`` that identifies the requested operation: * ``PMIX_ALLOC_NEW`` |mdash| a new allocation is being requested. The resulting allocation will be disjoint (i.e., not connected in a job sense) from the requesting allocation. * ``PMIX_ALLOC_EXTEND`` |mdash| extend the existing allocation, either in time or by adding resources. * ``PMIX_ALLOC_RELEASE`` |mdash| release part or all of the existing allocation. Attributes in the accompanying ``info`` array may be used to specify "lending" of those resources for some period of time. * ``PMIX_ALLOC_REAQUIRE`` |mdash| reacquire resources that were previously "lent" back to the scheduler. * ``PMIX_ALLOC_REQ_CANCEL`` |mdash| cancel the indicated allocation request. Values at or above ``PMIX_ALLOC_EXTERNAL`` are reserved for implementer-defined directives. ATTRIBUTES ---------- PMIx libraries are not required to directly support any attributes for this operation. However, any provided attributes must be passed to the host environment for processing, and the PMIx library is required to add the ``PMIX_USERID`` and ``PMIX_GRPID`` attributes of the client process making the request. Host environments that implement support for this operation are required to support the following attributes: * ``PMIX_ALLOC_REQ_ID`` (char*) |mdash| user-provided string identifier for this allocation request, which can later be used to query its status. * ``PMIX_ALLOC_NUM_NODES`` (uint64_t) |mdash| number of nodes being requested. * ``PMIX_ALLOC_NUM_CPUS`` (uint64_t) |mdash| number of CPUs being requested. * ``PMIX_ALLOC_TIME`` (char*) |mdash| time for which the resources are being requested, specified in the usual ``months:days:hours:minutes:seconds`` format, scanned from right to left (i.e., a value of ``"2"`` equates to two seconds). The following attributes are optional for host environments that support this operation. * ``PMIX_REQUESTOR`` (pmix_proc_t\*) |mdash| identifier of the process on whose behalf the request is being made, used when one process relays the operation to the PMIx library on behalf of another (the API itself takes no requestor parameter). Identifying an existing allocation (for the ``PMIX_ALLOC_EXTEND``, ``PMIX_ALLOC_RELEASE``, ``PMIX_ALLOC_REAQUIRE``, and ``PMIX_ALLOC_REQ_CANCEL`` directives): * ``PMIX_ALLOC_ID`` (char*) |mdash| the host-provided string identifier of the allocation to be operated upon (used both as a qualifier on input and, for a new-resource request, returned on output |mdash| see below). Describing the resources being requested: * ``PMIX_ALLOC_NODE_LIST`` (char*) |mdash| regular expression of the specific nodes being requested. * ``PMIX_ALLOC_EXCLUDE`` (char*) |mdash| regular expression of nodes to exclude from scheduling consideration. * ``PMIX_ALLOC_NUM_CPU_LIST`` (char*) |mdash| regular expression of the number of CPUs being requested on each node. * ``PMIX_ALLOC_CPU_LIST`` (char*) |mdash| regular expression of the specific CPUs being requested on each node. * ``PMIX_ALLOC_MEM_SIZE`` (float) |mdash| amount of memory being requested, in Mbytes. * ``PMIX_ALLOC_RES_BLOCK`` (char*) |mdash| name of a previously-defined resource block to be allocated. * ``PMIX_ALLOC_NUM_BLOCKS`` (uint64_t) |mdash| number of the specified resource blocks to allocate. * ``PMIX_ALLOC_MAU`` (pmix_data_array_t*) |mdash| minimum allocatable unit for the system, expressed as an array of ``pmix_resource_unit_t`` structures. * ``PMIX_ALLOC_IMAGE`` (char*) |mdash| name of the image that the requested nodes are to have on them. * ``PMIX_MEM_ALLOC_KIND`` (char*) |mdash| comma-delimited list of memory kinds being allocated (e.g., ``system,mpi,rocm:device``). * ``PMIX_GPU_SUPPORT`` (bool) |mdash| direct the application to enable (true) or disable (false) its internal library's GPU support. Describing fabric resources being requested: * ``PMIX_ALLOC_FABRIC`` (pmix_data_array_t*) |mdash| array of :ref:`pmix_info_t(5) ` structures describing a fabric resource request. It must include at least ``PMIX_ALLOC_FABRIC_ID``, ``PMIX_ALLOC_FABRIC_TYPE``, and ``PMIX_ALLOC_FABRIC_ENDPTS``, plus any other desired descriptors. * ``PMIX_ALLOC_FABRIC_ID`` (char*) |mdash| key to be used when accessing this requested fabric allocation. * ``PMIX_ALLOC_FABRIC_TYPE`` (char*) |mdash| type of desired transport (e.g., ``tcp``, ``udp``). * ``PMIX_ALLOC_FABRIC_PLANE`` (char*) |mdash| identifier string for the NIC (fabric plane) to be used for this allocation. * ``PMIX_ALLOC_FABRIC_ENDPTS`` (size_t) |mdash| number of endpoints to allocate per process. * ``PMIX_ALLOC_FABRIC_ENDPTS_NODE`` (size_t) |mdash| number of endpoints to allocate per node. * ``PMIX_ALLOC_FABRIC_QOS`` (char*) |mdash| quality-of-service level being requested. * ``PMIX_ALLOC_BANDWIDTH`` (float) |mdash| bandwidth being requested, in Mbits/sec. * ``PMIX_ALLOC_FABRIC_SEC_KEY`` (pmix_byte_object_t) |mdash| fabric security key. Controlling scheduling, timing, and lifecycle: * ``PMIX_ALLOC_QUEUE`` (char*) |mdash| name of the queue being referenced. * ``PMIX_ALLOC_PREEMPTIBLE`` (bool) |mdash| whether the jobs in the resulting allocation are to be considered preemptible (overridable at the per-job level). * ``PMIX_ALLOC_BEGIN`` (char*) |mdash| direct the scheduler to defer the allocation until the specified time (see the attribute definition in ``pmix_common.h`` for the accepted time formats). * ``PMIX_ALLOC_DEPENDENCY`` (char*) |mdash| defer the start of the allocation until the specified dependencies have successfully completed. * ``PMIX_ALLOC_WAIT_ALL_NODES`` (bool) |mdash| whether to wait for all nodes to be scheduled before beginning to initialize and release nodes for use. * ``PMIX_ALLOC_WARN_TIMEOUT`` (uint32_t) |mdash| request that the scheduler provide advance warning of the impending expiration of the allocation. The value is the number of seconds prior to expiration at which the warning is to be delivered via a ``PMIX_ALLOC_TIMEOUT_WARNING`` event. * ``PMIX_ALLOC_LEND`` (char*) |mdash| estimated time before the lent resources shall be available for return, in the same time format as ``PMIX_ALLOC_TIME`` (used with the ``PMIX_ALLOC_RELEASE`` directive). * ``PMIX_ALLOC_RELEASABLE`` (bool) |mdash| true if the entire allocation may be released. * ``PMIX_ALLOC_NOSHELL`` (bool) |mdash| immediately exit after allocating resources, without running a command. * ``PMIX_ALLOC_NOT_WAITING`` (bool) |mdash| the requestor is not waiting for the allocation to be issued; it will discover the allocation by monitoring for the "allocated" event or by querying the system. * ``PMIX_ALLOC_PROPERTY`` (char*) |mdash| name of an allocation property. Controlling ownership, sharing, and inheritance: * ``PMIX_ALLOC_TARGET`` (char*) |mdash| namespace to which the allocated resources are to be assigned; the host reserves the resources to members of that namespace. * ``PMIX_ALLOC_SHARE`` (bool) |mdash| if true, make the allocated resources generally available within the requestor's session rather than reserving them solely for the requestor's namespace (false by default). * ``PMIX_ALLOC_CHILD_SEP`` (bool) |mdash| treat the resulting allocation as independent from its parent |mdash| i.e., do not terminate the allocation upon termination of the parent. * ``PMIX_ALLOC_INHERITANCE`` (pmix_alloc_inheritance_t) |mdash| inheritance rules to be applied to the allocated resources. If not provided, defaults to ``PMIX_ALLOC_INHERIT_DEFAULT``. On successful completion of a request for additional resources, the returned data includes ``PMIX_ALLOC_ID`` (char*), a host-provided string identifier for the resulting allocation. RETURN VALUE ------------ For the blocking form, ``PMIX_SUCCESS`` indicates that the request was processed and any results were returned in ``results`` and ``nresults``. For the non-blocking form, a return of ``PMIX_SUCCESS`` indicates only that the request was accepted for processing; the final status and any data are delivered to ``cbfunc``. * ``PMIX_SUCCESS`` |mdash| the request was successfully processed. * ``PMIX_ERR_NOT_SUPPORTED`` |mdash| the operation is not supported in the caller's role or environment |mdash| for example, the caller is itself the scheduler, or is a system controller with no scheduler attached. * ``PMIX_ERR_UNREACH`` |mdash| the caller is not connected to a server capable of servicing the request. * ``PMIX_ERR_NOT_AVAILABLE`` |mdash| the operation cannot be serviced because the library's progress engine has been stopped. * ``PMIX_ERR_INIT`` |mdash| the PMIx library has not been initialized. Any other negative value indicates an appropriate error condition. PMIx error constants are defined in ``pmix_common.h``. NOTES ----- This operation is only meaningful when the caller can reach a scheduler, either because the caller's server is the scheduler or because the caller is a server whose host environment provides an allocation entry point that can forward the request. A process hosted directly by the scheduler cannot issue an allocation request against itself and will receive ``PMIX_ERR_NOT_SUPPORTED``. Upon completion of an allocation request, the host environment may generate an event to notify processes other than the requestor |mdash| for example, the members of a job affected by the change |mdash| of the result. The result is conveyed in that event through the ``PMIX_ALLOC_STATUS`` (pmix_status_t) attribute, which carries the completion status of the allocation request. .. seealso:: :ref:`PMIx_Init(3) `, :ref:`PMIx_Resource_block(3) `, :ref:`PMIx_Session_control(3) `, :ref:`pmix_info_t(5) `, :ref:`pmix_status_t(5) `, :ref:`pmix_alloc_directive_t(5) `