6.7. PMIx_Log Support

This document describes how the PMIx_Log / PMIx_Log_nb API is implemented inside the PMIx library, from the public API entry points down through the MCA plog framework and its components, including the special pmix_show_help duplicate-suppression use-case.

6.7.1. Public API

Two functions are exported from include/pmix.h:

pmix_status_t PMIx_Log(const pmix_info_t data[], size_t ndata,
                       const pmix_info_t directives[], size_t ndirs);

pmix_status_t PMIx_Log_nb(const pmix_info_t data[], size_t ndata,
                          const pmix_info_t directives[], size_t ndirs,
                          pmix_op_cbfunc_t cbfunc, void *cbdata);

PMIx_Log is a thin blocking wrapper: it constructs a pmix_cb_t, calls PMIx_Log_nb, then sleeps on the condition variable inside the callback object until the non-blocking path completes.

The data array carries one entry per logging channel (what to log and where), and the directives array carries behavioral control attributes (timestamps, aggregation keys, PMIX_LOG_ONCE, etc.).

Both functions are implemented in src/common/pmix_log.c.

6.7.2. Channel Constants

Log channel names are defined in include/pmix_common.h.in. The key ones are:

Constant	Key string	Component that handles it
PMIX_LOG_STDERR	pmix.log.stderr	plog/stdfd
PMIX_LOG_STDOUT	pmix.log.stdout	plog/stdfd
PMIX_LOG_SYSLOG	pmix.log.syslog	plog/syslog
PMIX_LOG_LOCAL_SYSLOG	pmix.log.lsys	plog/syslog
PMIX_LOG_GLOBAL_SYSLOG	pmix.log.gsys	plog/syslog (gateway only)
PMIX_LOG_EMAIL	pmix.log.email	plog/smtp

Directive constants (passed in the directives array) include:

• PMIX_LOG_SOURCE — (pmix_proc_t*) originator of the request

• PMIX_LOG_TIMESTAMP — (time_t) explicit timestamp

• PMIX_LOG_GENERATE_TIMESTAMP — (bool) auto-generate a timestamp

• PMIX_LOG_ONCE — (bool) stop after the first successful component

• PMIX_LOG_AGG — (bool) aggregate (suppress) duplicate messages

• PMIX_LOG_KEY / PMIX_LOG_VAL — (char*) key/value pair used for duplicate detection (see pmix_show_help Aggregation below)

• PMIX_LOG_SYSLOG_PRI — (int) syslog priority level

• PMIX_LOG_TAG_OUTPUT — (bool) prefix output with channel name

• PMIX_LOG_XML_OUTPUT — (bool) emit output in XML format

6.7.3. Call Flow

The diagram below traces the full path for a log request.

PMIx_Log(data, directives)
    └─► PMIx_Log_nb(data, directives, opcbfunc, &cb)  [src/common/pmix_log.c]
             │
             ├─ [Client / tool, connected to server]
             │       Pack PMIX_LOG_CMD into wire message
             │       Optionally pack timestamp (v3+)
             │       Pack data[], directives[]
             │       PMIX_PTL_SEND_RECV → log_cbfunc()
             │           │
             │           │  Server receives PMIX_LOG_CMD
             │           │  pmix_server_log()  [src/server/pmix_server_ops.c]
             │           │      Unpack timestamp, data[], directives[]
             │           │      Append PMIX_LOG_SOURCE (requester's proc)
             │           │      Optionally append PMIX_LOG_TIMESTAMP
             │           │      ──────────────────────────────────────
             │           │      if (NOT gateway  OR  pmix_log_host_only)
             │           │          Upcall: pmix_host_server.log2()
             │           │                 or pmix_host_server.log()
             │           │      else (gateway and !pmix_log_host_only)
             │           │          pmix_plog.log()  ─► plog framework
             │           │      ──────────────────────────────────────
             │           │
             │           └─ If server returned ERR_NOT_AVAILABLE:
             │                  fall through to local plog processing
             │
             ├─ [Client / tool, NOT connected]  ──────────────────────────┐
             │                                                             │
             └─ [Server, not gateway or pmix_log_host_only set]           │
                     Upcall host via log2() / log()                       │
                     If neither available → fall through ──────────────────┤
                                                                           │
             [Server, gateway and !pmix_log_host_only] ───────────────────┤
                                                                           ▼
                                                           PMIX_THREADSHIFT → pmix_log_local_op()
                                                               pmix_plog.log()
                                                               └─► pmix_plog_base_log()
                                                                   [src/mca/plog/base/plog_base_stubs.c]

6.7.4. The MCA plog Framework

The plog framework lives under src/mca/plog/. Its module interface (src/mca/plog/plog.h) exposes:

typedef struct {
    char  *name;
    char **channels;   /* NULL-terminated list of channel key substrings */
    pmix_plog_base_module_init_fn_t   init;
    pmix_plog_base_module_fini_fn_t   finalize;
    pmix_plog_base_module_log_fn_t    log;
} pmix_plog_module_t;

typedef pmix_status_t (*pmix_plog_base_module_log_fn_t)(
    const pmix_proc_t *source,
    const pmix_info_t data[],  size_t ndata,
    const pmix_info_t directives[], size_t ndirs);

Each component declares the channel key substrings it handles. The base framework matches an incoming data entry’s key against those substrings with a simple strstr call, so a key of "pmix.log.stderr" matches a component whose channels[] contains "stderr".

6.7.4.1. Component Selection

At framework open time, src/mca/plog/base/plog_base_select.c queries all available components, sorts them by priority, and builds pmix_plog_globals.actives — a pmix_pointer_array_t of pmix_plog_base_active_module_t objects.

The MCA parameter pmix_plog_base_order can force a specific ordering. If a channel is suffixed with :req, the framework will abort initialization if no component can satisfy it (error via pmix_show_help("help-pmix-plog.txt", "reqd-not-found", ...).

6.7.4.2. Dispatch (pmix_plog_base_log)

pmix_plog_base_log() in src/mca/plog/base/plog_base_stubs.c:

1. Scans directives for PMIX_LOG_ONCE, PMIX_LOG_AGG, PMIX_LOG_KEY, and PMIX_LOG_VAL.

2. Runs the pmix_show_help Aggregation duplicate check (see below).

3. For each non-completed data entry, finds every active module whose channels[] list contains the entry’s key as a substring, and appends that module to a temporary ordered list (preserving the order in which they appear in the data array — the first-listed channel gets first priority for PMIX_LOG_ONCE).

4. Iterates through the assembled list, calling module->log() for each:

   • PMIX_SUCCESS or PMIX_OPERATION_SUCCEEDED — success; if PMIX_LOG_ONCE is set, stop.

   • PMIX_ERR_NOT_AVAILABLE or PMIX_ERR_TAKE_NEXT_OPTION — this module cannot handle the request; skip to next.

   • Any other error — abort processing.

6.7.5. plog Components

6.7.5.1. stdfd (src/mca/plog/stdfd/)

Handles PMIX_LOG_STDERR and PMIX_LOG_STDOUT.

• When called on a client: writes directly to stderr / stdout via fprintf.

• When called on a server: delivers the output to the client’s IOF stream via PMIx_server_IOF_deliver() so it appears on the originating process’s stdio.

The component also honours PMIX_LOG_TAG_OUTPUT (prepend channel name) and PMIX_LOG_XML_OUTPUT (wrap in XML tags).

6.7.5.2. syslog (src/mca/plog/syslog/)

Handles PMIX_LOG_SYSLOG, PMIX_LOG_LOCAL_SYSLOG, and (on gateway servers only) PMIX_LOG_GLOBAL_SYSLOG.

MCA parameters on the component (set at open time) configure the default syslog facility and priority level. The per-call PMIX_LOG_SYSLOG_PRI directive overrides the priority. Unrecognised facility or level strings cause a pmix_show_help warning and the component falls back to sensible defaults.

6.7.5.3. smtp (src/mca/plog/smtp/)

Handles PMIX_LOG_EMAIL. Requires libesmtp at build time.

Additional directives used only by this component:

• PMIX_LOG_EMAIL_ADDR — (char*) recipient address(es)

• PMIX_LOG_EMAIL_SENDER_ADDR — (char*) sender address

• PMIX_LOG_EMAIL_SUBJECT — (char*) subject line

• PMIX_LOG_EMAIL_SERVER — (char*) SMTP relay hostname

A failed libesmtp send emits a pmix_show_help warning from help-pmix-plog.txt topic smtp:send_email failed.

6.7.6. pmix_show_help Aggregation

pmix_show_help (src/util/pmix_show_help.c) is PMIx’s general facility for displaying parameterised, multi-line help and error messages stored in *.txt files under $pkgdatadir. Callers pass a filename and a topic name:

pmix_show_help("help-pmix-plog.txt", "syslog:unrec-level", true, level);

The library locates the [syslog:unrec-level] block in the file, performs printf-style substitutions, and emits the result to stderr.

The plog framework uses pmix_show_help’s duplicate-tracking infrastructure to implement the PMIX_LOG_AGG directive. Specifically, the function:

pmix_status_t pmix_help_check_dups(const char *filename, const char *topic);

returns PMIX_SUCCESS if the (filename, topic) pair has been seen before and records the pair on first call.

When pmix_plog_base_log() sees PMIX_LOG_AGG=true together with both PMIX_LOG_KEY and PMIX_LOG_VAL in the directives, it calls pmix_help_check_dups(key, val) before dispatching to any component. If the pair has already been logged, every data entry is marked PMIX_INFO_OP_COMPLETED and the dispatch loop skips them all — the duplicate message is silently suppressed.

This is how callers (typically inside the PMIx library itself) prevent the same error or status message from flooding the output when many processes encounter the same condition. The typical pattern is:

pmix_info_t data[1], dirs[3];
PMIX_INFO_LOAD(&data[0], PMIX_LOG_STDERR, message_string, PMIX_STRING);

PMIX_INFO_LOAD(&dirs[0], PMIX_LOG_AGG,  &(bool){true}, PMIX_BOOL);
PMIX_INFO_LOAD(&dirs[1], PMIX_LOG_KEY,  filename_key,  PMIX_STRING);
PMIX_INFO_LOAD(&dirs[2], PMIX_LOG_VAL,  topic_key,     PMIX_STRING);

PMIx_Log(data, 1, dirs, 3);

The first call goes through; subsequent calls with the same PMIX_LOG_KEY / PMIX_LOG_VAL pair are no-ops.

6.7.7. Gateway Servers vs. Non-Gateway Servers

The split between “upcall to host” and “process locally” appears in two places: PMIx_Log_nb for server callers, and pmix_server_log for messages arriving from a client. Both use identical logic:

• Non-gateway server (ordinary daemon) — always upcall to the host RM via pmix_host_server.log2() (preferred) or pmix_host_server.log() (legacy). If neither is implemented, the server falls through to local plog processing.

• Gateway server (the server that touches the outside world, e.g., the head-node daemon in PRRTE) — processes the request locally via the plog framework so it can reach global syslog, send email, etc.

The pmix_log_host_only MCA parameter (bool, default false) overrides this: when set, all log requests are forwarded to the host regardless of gateway status, and local plog processing is bypassed entirely. If the host provides no log or log2 upcall and pmix_log_host_only is set, PMIx_Log returns PMIX_ERR_NOT_SUPPORTED.

6.7.8. The pmix_log_host_only MCA Parameter

Defined and registered in src/runtime/pmix_params.c:

bool pmix_log_host_only = false;   /* exported via src/runtime/pmix_rte.h */

pmix_mca_base_var_register("pmix", "pmix", NULL, "log_host_only",
    "Direct all PMIx_Log handling to the host environment;"
    " the library's plog framework will not be invoked [default: no]",
    PMIX_MCA_BASE_VAR_TYPE_BOOL,
    &pmix_log_host_only);

Set via the standard MCA parameter mechanism, e.g.:

export PMIX_MCA_pmix_log_host_only=1

6.7.9. Thread Safety

PMIx_Log_nb does not call the plog framework directly from the calling thread. When local processing is required, it packages the request into a pmix_shift_caddy_t and uses PMIX_THREADSHIFT to deliver it to pmix_log_local_op() on the PMIx progress thread. Individual plog components may thread-shift further internally if they need to perform blocking I/O.

When a client sends a log request to a server and the server cannot satisfy it (returns PMIX_ERR_NOT_AVAILABLE), the client-side log_cbfunc() receives the reply and falls through to local plog processing on the client side — ensuring a best-effort result even when the server is not a gateway.

6.7.10. Key Source Files

File	Purpose
include/pmix.h	Public API declarations
include/pmix_common.h.in	Channel and directive constants
src/common/pmix_log.c	PMIx_Log / PMIx_Log_nb implementation
src/server/pmix_server_ops.c	pmix_server_log() handler
src/mca/plog/plog.h	plog MCA module interface
src/mca/plog/base/plog_base_select.c	Component selection at init
src/mca/plog/base/plog_base_stubs.c	pmix_plog_base_log() dispatcher
src/mca/plog/stdfd/plog_stdfd.c	stdout / stderr component
src/mca/plog/syslog/plog_syslog.c	syslog component
src/mca/plog/smtp/plog_smtp.c	email (SMTP) component
src/util/pmix_show_help.c	show-help subsystem & dup tracking
src/runtime/pmix_params.c	pmix_log_host_only MCA parameter