Merge pull request flux-framework#5527 from garlick/man7_env

flux-environment(7): add new man page

doc/Makefile.am

@@ -319,7 +319,8 @@ MAN7_FILES = $(MAN7_FILES_PRIMARY)
 
 MAN7_FILES_PRIMARY = \
 	man7/flux-broker-attributes.7 \
-	man7/flux-jobtap-plugins.7
+	man7/flux-jobtap-plugins.7 \
+	man7/flux-environment.7
 
 
 RST_FILES  = \

doc/man1/common/submit-environment.rst

@@ -54,9 +54,9 @@ via a set of *RULE* expressions. The currently supported rules are
    simple substitution supports the following syntax:
 
      * ``$$`` is an escape; it is replaced with ``$``
-     * ``$var`` will substitute ``var`` from the current environment,
+     * ``$var`` will substitute :envvar:`var` from the current environment,
        falling back to the process environment. An error will be thrown
-       if environment variable ``var`` is not set.
+       if environment variable :envvar:`var` is not set.
      * ``${var}`` is equivalent to ``$var``
      * Advanced parameter substitution is not allowed, e.g. ``${var:-foo}``
        will raise an error.
@@ -66,12 +66,12 @@ via a set of *RULE* expressions. The currently supported rules are
 
  * Otherwise, the rule is considered a pattern from which to match
    variables from the process environment if they do not exist in
-   the generated environment. E.g. ``PATH`` will export ``PATH`` from the
+   the generated environment. E.g. ``PATH`` will export :envvar:`PATH` from the
    current environment (if it has not already been set in the generated
    environment), and ``OMP*`` would copy all environment variables that
-   start with ``OMP`` and are not already set in the generated environment.
-   It is important to note that if the pattern does not match any variables,
-   then the rule is a no-op, i.e. an error is *not* generated.
+   start with :envvar:`OMP` and are not already set in the generated
+   environment.  It is important to note that if the pattern does not match
+   any variables, then the rule is a no-op, i.e. an error is *not* generated.
 
    Examples:
        ``PATH``, ``FLUX_*_PATH``, ``/^OMP.*/``
@@ -80,7 +80,7 @@ Since we always starts with a copy of the current environment,
 the default implicit rule is ``*`` (or :option:`--env=*`). To start with an
 empty environment instead, the ``-*`` rule or :option:`--env-remove=*` option
 should be used. For example, the following will only export the current
-``PATH`` to a job:
+:envvar:`PATH` to a job:
 
 ::
 
@@ -111,6 +111,6 @@ This works particularly well when specifying rules in a file:
     BAR=${FOO}/baz
 
 The above file would first clear the environment, then copy all variables
-starting with ``OMP`` from the current environment, set ``FOO=bar``,
+starting with :envvar:`OMP` from the current environment, set ``FOO=bar``,
 and then set ``BAR=bar/baz``.
 

doc/man1/common/submit-job-environment-variables.rst

@@ -1,29 +1,33 @@
 JOB ENVIRONMENT VARIABLES
 =========================
 
-Flux creates several environment variables for every job task.  They are as follows:
+The job environment is described in more detail in the :man7:`flux-environment`
+:ref:`job_environment` section.  A summary of the most commonly used variables
+is provided below:
 
-**FLUX_JOB_ID**
-    The jobid for this job
+.. list-table::
+   :header-rows: 1
 
-**FLUX_URI**
-    The URI of the enclosing Flux instance
+   * - Name
+     - Description
 
-**FLUX_JOB_TMPDIR**
-    Path to temporary directory created for job
+   * - :envvar:`FLUX_JOB_ID`
+     - the current jobid
 
-**FLUX_TASK_LOCAL_ID**
-    The task id local to the node
+   * - :envvar:`FLUX_JOB_SIZE`
+     - the number of tasks in the current job
 
-**FLUX_TASK_RANK**
-    The global rank for this task
+   * - :envvar:`FLUX_JOB_NNODES`
+     - the total number of nodes hosting tasks on behalf of the job
 
-**FLUX_JOB_NNODES**
-    The total number of nodes in the job
+   * - :envvar:`FLUX_TASK_RANK`
+     - the global task rank (zero origin)
 
-**FLUX_JOB_SIZE**
-    The count of job shells in the job, typical the same value as FLUX_JOB_NNODES
+   * - :envvar:`FLUX_TASK_LOCAL_ID`
+     - the local task rank (zero origin)
 
-**FLUX_KVS_NAMESPACE**
-    The identifier of the KVS guest namespace of the job. This redirects KVS command and
-    API calls to the job guest namespace instead of the root namespace.
+   * - :envvar:`FLUX_JOB_TMPDIR`
+     - path to temporary, per-job directory, usually in ``/tmp``
+
+   * - :envvar:`FLUX_URI`
+     - the URI of the enclosing Flux instance

doc/man1/common/submit-other-options.rst

@@ -92,11 +92,11 @@ OTHER OPTIONS
      within the following search path, in order of precedence:
 
      - ``XDG_CONFIG_HOME/flux/config`` or ``$HOME/.config/flux/config`` if
-       ``XDG_CONFIG_HOME`` is not set
+       :envvar:`XDG_CONFIG_HOME` is not set
 
      - ``$XDG_CONFIG_DIRS/flux/config`` or ``/etc/xdg/flux/config`` if
-       ``XDG_CONFIG_DIRS`` is not set. Note that ``XDG_CONFIG_DIRS`` may
-       be a colon-separated path.
+       :envvar:`XDG_CONFIG_DIRS` is not set. Note that :envvar:`XDG_CONFIG_DIRS`
+       may be a colon-separated path.
 
 .. option:: --begin-time=+FSD|DATETIME
 
@@ -184,16 +184,16 @@ OTHER OPTIONS
 .. option:: --cc=IDSET
 
    *(submit,bulksubmit)* Replicate the job for each ``id`` in ``IDSET``.
-   ``FLUX_JOB_CC=id`` will be set in the environment of each submitted job
-   to allow the job to alter its execution based on the submission index.
-   (e.g. for reading from a different input file). When using :option:`--cc`,
-   the substitution string ``{cc}`` may be used in options and commands
-   and will be replaced by the current ``id``.
+   :envvar:`FLUX_JOB_CC` will be set to ``id`` in the environment of each
+   submitted job to allow the job to alter its execution based on the
+   submission index.  (e.g. for reading from a different input file). When
+   using :option:`--cc`, the substitution string ``{cc}`` may be used in
+   options and commands and will be replaced by the current ``id``.
 
 .. option:: --bcc=IDSET
 
    *(submit,bulksubmit)* Identical to :option:`--cc`, but do not set
-   ``FLUX_JOB_CC`` in each job. All jobs will be identical copies.
+   :envvar:`FLUX_JOB_CC` in each job. All jobs will be identical copies.
    As with :option:`--cc`, ``{cc}`` in option arguments and commands will be
    replaced with the current ``id``.
 

doc/man1/common/submit-shell-options.rst

@@ -46,6 +46,7 @@ overridden in some cases:
 
 .. option:: -o stage-in
 
-   Copy files previously mapped with :man1:`flux-filemap` to $FLUX_JOB_TMPDIR.
-   See :man1:`flux-shell` for more *stage-in* options.
+   Copy files previously mapped with :man1:`flux-filemap` to the directory
+   referred to by :envvar:`FLUX_JOB_TMPDIR`.  See :man1:`flux-shell` for more
+   *stage-in* options.
 

doc/man1/flux-config.rst

@@ -120,32 +120,36 @@ The following configuration keys may be printed with
    The configured ``${libexecdir/flux/cmd`` directory.
 
 **lua_cpath_add**
-   Consulted by :man1:`flux` when setting the LUA_CPATH environment variable.
+   Consulted by :man1:`flux` when setting the :envvar:`LUA_CPATH` environment
+   variable.
 
 **lua_path_add**
-   Consulted by :man1:`flux` when setting the LUA_PATH environment variable.
+   Consulted by :man1:`flux` when setting the :envvar:`LUA_PATH` environment
+   variable.
 
 **python_path**
-   Consulted by :man1:`flux` when setting the PYTHONPATH environment variable.
+   Consulted by :man1:`flux` when setting the :envvar:`PYTHONPATH` environment
+   variable.
 
 **man_path**
-   Consulted by :man1:`flux` when setting the MANPATH environment variable.
+   Consulted by :man1:`flux` when setting the :envvar:`MANPATH` environment
+   variable.
 
 **exec_path**
-   Consulted by :man1:`flux` when setting the FLUX_EXEC_PATH environment
-   variable.
+   Consulted by :man1:`flux` when setting the :envvar:`FLUX_EXEC_PATH`
+   environment variable.
 
 **connector_path**
-   Consulted by :man1:`flux` when setting the FLUX_CONNECTOR_PATH environment
-   variable.
+   Consulted by :man1:`flux` when setting the :envvar:`FLUX_CONNECTOR_PATH`
+   environment variable.
 
 **module_path**
-   Consulted by :man1:`flux` when setting the FLUX_MODULE_PATH environment
-   variable.
+   Consulted by :man1:`flux` when setting the :envvar:`FLUX_MODULE_PATH`
+   environment variable.
 
 **pmi_library_path**
-   Consulted by :man1:`flux` when setting the FLUX_PMI_LIBRARY_PATH environment
-   variable.
+   Consulted by :man1:`flux` when setting the :envvar:`FLUX_PMI_LIBRARY_PATH`
+   environment variable.
 
 **cmdhelp_pattern**
    Used by :man1:`flux` to generate a list of common commands when run without

doc/man1/flux-filemap.rst

@@ -47,7 +47,8 @@ is not on a network file system without considering the ramifications.
 :option:`flux filemap unmap` unmaps mapped files.
 
 The ``stage-in`` shell plugin described in :man1:`flux-shell` may be used to
-extract previously mapped files into $FLUX_JOB_TMPDIR or another directory.
+extract previously mapped files into the directory referred to by
+:envvar:`FLUX_JOB_TMPDIR` or another directory.
 
 OPTIONS
 =======
@@ -132,7 +133,7 @@ by multiple jobs.
   flux filemap unmap
 
 Example 2: a batch script that maps two data sets with tags, then uses the
-``stage-in`` shell plugin to selectively copy them to $FLUX_JOB_TMPDIR,
+``stage-in`` shell plugin to selectively copy them to :envvar:`FLUX_JOB_TMPDIR`
 which is automatically cleaned up after each job.
 
 .. code-block:: console

doc/man1/flux-job.rst

@@ -58,8 +58,8 @@ display a status line while the job is pending, e.g
 
     flux-job: ƒJqUHUCzX9 waiting for resources                 00:00:08
 
-This status line may be suppressed by setting ``FLUX_ATTACH_NONINTERACTIVE``
-in the environment.
+This status line may be suppressed by setting
+:envvar:`FLUX_ATTACH_NONINTERACTIVE` in the environment.
 
 .. option:: -l, --label-io
 

doc/man1/flux-jobs.rst

@@ -78,10 +78,10 @@ OPTIONS
    Specify a named output format *NAME* or a format string using Python's
    format syntax. See `OUTPUT FORMAT`_ below for field names. Named formats
    may be listed via :option:`--format=help`.  An alternate default format can
-   be set via the FLUX_JOBS_FORMAT_DEFAULT environment variable.  Additional
-   named formats may be registered with :program:`flux jobs` via configuration.
-   See the `CONFIGURATION`_ section for more details. A configuration snippet
-   for an existing named format may be generated with
+   be set via the :envvar:`FLUX_JOBS_FORMAT_DEFAULT` environment variable.
+   Additional named formats may be registered with :program:`flux jobs` via
+   configuration.  See the `CONFIGURATION`_ section for more details. A
+   configuration snippet for an existing named format may be generated with
    :option:`--format=get-config=NAME`.
 
 .. option:: --json
@@ -503,12 +503,13 @@ The :program:`flux jobs` command supports registration of named output formats
 in configuration files. The command loads configuration files from
 ``flux-jobs.EXT`` from the following paths in order of increasing precedence:
 
- * ``$XDG_CONFIG_DIRS/flux`` or ``/etc/xdg/flux`` if ``XDG_CONFIG_DIRS`` is
-   not set. Note that ``XDG_CONFIG_DIRS`` is traversed in reverse order
-   such that entries first in the colon separated path are highest priority.
+ * ``$XDG_CONFIG_DIRS/flux`` or ``/etc/xdg/flux`` if :envvar:`XDG_CONFIG_DIRS`
+   is not set. Note that :envvar:`XDG_CONFIG_DIRS` is traversed in reverse
+   order such that entries first in the colon separated path are highest
+   priority.
 
- * ``XDG_CONFIG_HOME/flux`` or ``$HOME/.config/flux`` if ``XDG_CONFIG_HOME``
-   is not set
+ * ``$XDG_CONFIG_HOME/flux`` or ``$HOME/.config/flux`` if
+   :envvar:`XDG_CONFIG_HOME` is not set
 
 where ``EXT`` can be one of ``toml``, ``yaml``, or ``json``.
 

doc/man1/flux-kvs.rst

@@ -37,7 +37,7 @@ Different KVS namespaces can be created in which kvs values can be
 read from/written to. By default, all KVS operations operate on the
 default KVS namespace "primary". An alternate namespace can be
 specified in most kvs commands via the *--namespace* option, or by
-setting the namespace in the environment variable FLUX_KVS_NAMESPACE.
+setting the namespace in the environment variable :envvar:`FLUX_KVS_NAMESPACE`.
 
 :program:`flux kvs` runs a KVS *COMMAND*. The possible commands and their
 arguments are described below.

doc/man1/flux-pgrep.rst

@@ -72,12 +72,12 @@ OPTIONS
 
    Specify a named output format *NAME* or a format string using Python's
    format syntax.  An alternate default format can be set via the
-   FLUX_PGREP_FORMAT_DEFAULT environment variable.  For full documentation of
-   this option, including supported field names and format configuration
-   options, see :man1:flux-jobs. This command shares configured named formats
-   with *flux-jobs* by reading *flux-jobs* configuration files. Supported
-   builtin named formats include *default*, *full*, *long*, and *deps*. The
-   default format emits the matched jobids only. (pgrep only)
+   :envvar:`FLUX_PGREP_FORMAT_DEFAULT` environment variable.  For full
+   documentation of this option, including supported field names and format
+   configuration options, see :man1:flux-jobs. This command shares configured
+   named formats with *flux-jobs* by reading *flux-jobs* configuration files.
+   Supported builtin named formats include *default*, *full*, *long*, and
+   *deps*. The default format emits the matched jobids only. (pgrep only)
 
 .. option:: -n, --no-header
 

doc/man1/flux-proxy.rst

@@ -18,7 +18,7 @@ DESCRIPTION
 .. program:: flux proxy
 
 :program:`flux proxy` connects to the Flux instance identified by *TARGET*,
-then spawns a shell with FLUX_URI pointing to a local:// socket
+then spawns a shell with :envvar:`FLUX_URI` pointing to a local:// socket
 managed by the proxy program. As long as the shell is running,
 the proxy program routes messages between the instance and the
 local:// socket. Once the shell terminates, the proxy program
@@ -72,7 +72,7 @@ OPTIONS
 EXAMPLES
 ========
 
-Connect to a job running on the localhost which has a FLUX_URI
+Connect to a job running on the localhost which has a :envvar:`FLUX_URI`
 of ``local:///tmp/flux-123456-abcdef/0/local`` and spawn an interactive
 shell:
 

doc/man1/flux-queue.rst

@@ -130,10 +130,10 @@ OUTPUT FORMAT
 The :option:`--format` option can be used to specify an output format using
 Python's string format syntax or a defined format by name. For a list of
 built-in and configured formats use :option:`-o help`.  An alternate default
-format can be set via the FLUX_QUEUE_LIST_FORMAT_DEFAULT environment variable.
-A configuration snippet for an existing named format may be generated with
-:option:`--format=get-config=NAME`.  See :man1:`flux-jobs` *OUTPUT FORMAT*
-section for a detailed description of this syntax.
+format can be set via the :envvar:`FLUX_QUEUE_LIST_FORMAT_DEFAULT` environment
+variable.  A configuration snippet for an existing named format may be
+generated with :option:`--format=get-config=NAME`.  See :man1:`flux-jobs`
+*OUTPUT FORMAT* section for a detailed description of this syntax.
 
 The following field names can be specified:
 

doc/man1/flux-resource.rst

@@ -175,15 +175,14 @@ OUTPUT FORMAT
 The :option:`--format` option can be used to specify an output format using
 Python's string format syntax or a defined format by name. For a list of
 built-in and configured formats use :option:`-o help`.  An alternate default
-<<<<<<< HEAD
-format can be set via the FLUX_RESOURCE_STATUS_FORMAT_DEFAULT,
-FLUX_RESOURCE_DRAIN_FORMAT_DEFAULT, and FLUX_RESOURCE_LIST_FORMAT_DEFAULT
-environment variables (for :program:`flux resource status`,
-:program:`flux resource drain`, and :program:`flux resource list`
-respectively).  A configuration snippet for an existing named format may be
-generated with :option:`--format=get-config=NAME`.  See :man1:`flux-jobs`
-:ref:`flux_jobs_output_format` section for a detailed description of this
-syntax.
+format can be set via the :envvar:`FLUX_RESOURCE_STATUS_FORMAT_DEFAULT`,
+:envvar:`FLUX_RESOURCE_DRAIN_FORMAT_DEFAULT`, and
+:envvar:`FLUX_RESOURCE_LIST_FORMAT_DEFAULT` environment variables (for
+:program:`flux resource status`, :program:`flux resource drain`, and
+:program:`flux resource list` respectively).  A configuration snippet for an
+existing named format may be generated with :option:`--format=get-config=NAME`.
+See :man1:`flux-jobs` :ref:`flux_jobs_output_format` section for a detailed
+description of this syntax.
 
 Resources are combined into a single line of output when possible depending on
 the supplied output format.  Resource counts are not included in the