dragon.globalservices.process

Global Services API for the Process primary resource component.

Functions

`create`(exe, run_dir, args, env[, user_name, ...])	Asks Global Services to create a new process.
`create_with_argdata`(exe, run_dir, args, env)	Asks Global services to create a new process and deliver starting args to it thru messaging.
`get_argdata_from_immediate_handshake`(...)
`get_create_message`(exe, run_dir, args, env)	Return a GSProcessCreate object.
`get_create_message_with_argdata`(exe, ...[, ...])	Return a GSProcessCreate object with starting args.
`get_list`()	Asks Global Services for a list of the p_uids of all managed processes.
`get_multi_join_failure_puids`(statuses)	Go through list of processes that have been joined on to isolate non-zero exits
`get_multi_join_success_puids`(statuses)	Go through list of processes that have been joined on to isolate successful zero exits
`join`(identifier[, timeout])	Asks Global Services to join a specified managed process.
`kill`(identifier[, sig])	Asks Global Services to kill a specified managed process with a specified signal.
`multi_join`(identifiers[, timeout, join_all, ...])	Asks Global Services to join a list of specified managed processes.
`prepare_argdata_for_immediate`(argdata)
`query`(identifier)	Asks Global Services for the ProcessDescriptor of a specified managed process
`start_capturing_child_mp_output`()
`stop_capturing_child_mp_output`()

Classes

StreamDestination

An enumeration.

Exceptions

ProcessError

class StreamDestination: An enumeration.

exception ProcessError

get_create_message(exe, run_dir, args, env, user_name='', options=None, stdin=None, stdout=None, stderr=None, group=None, user=None, umask=-1, pipesize=-1, pmi_required=False, policy=None)

Return a GSProcessCreate object.

Parameters

exe – executable to run
run_dir – directory to run it in
args – argv for the process
env – environment for the new process, may be added to for infrastructure
user_name – Requested user specified reference name
options – ProcessOptions object, what options to apply to creation
stdin – If stdin=PIPE, a Dragon Connection will be returned to enable sending data to the stdin of the child process.
stdout – Capture stdout from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stdout. Using DEVNULL will cause the stdout data to be thrown away.
stder – Capture stderr from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stderr. Using DEVNULL will cause the stdout data to be thrown away. Using STDOUT will cause the stderr data to be combined with the stdout data.
group – Not used
user – Not used
umask – Not used
pipesize – Set the channel capacity. Default = -1.
pmi_required – This process is part of a Dragon managed MPI/PMI application group.
policy – If a policy other than the global default is to be used for this process.

Returns

GSProcessCreate message object

get_create_message_with_argdata(exe, run_dir, args, env, argdata=None, user_name='', options=None, stdin=None, stdout=None, stderr=None, group=None, user=None, umask=-1, pipesize=-1, pmi_required=False, policy=None)

Return a GSProcessCreate object with starting args.

This is an extension of the ‘get_create_message’ method that encapsulates our scheme for getting starting arguments to the new process. This depends on how big the arguments turn out to be, and is tightly bound with logic in global services and in the api_setup module.

Parameters

exe – executable to run
run_dir – directory to run it in
args – argv for the process
env – environment for the new process, may be added to for infrastructure
argdata – bytes to deliver
user_name – Requested user specified reference name
options – ProcessOptions object, what options to apply to creation
stdin – If stdin=PIPE, a Dragon Connection will be returned to enable sending data to the stdin of the child process.
stdout – Capture stdout from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stdout. Using DEVNULL will cause the stdout data to be thrown away.
stder – Capture stderr from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stderr. Using DEVNULL will cause the stdout data to be thrown away. Using STDOUT will cause the stderr data to be combined with the stdout data.
group – Not used
user – Not used
umask – Not used
pipesize – Set the channel capacity. Default = -1.
pmi_required – This process is part of a Dragon managed MPI/PMI application group.

Returns

GSProcessCreate message object

create(exe, run_dir, args, env, user_name='', options=None, soft=False, stdin=None, stdout=None, stderr=None, group=None, user=None, umask=-1, pipesize=-1, pmi_required=False, policy=None)

Asks Global Services to create a new process.

Parameters

exe – executable to run
run_dir – directory to run it in
args – argv for the process
env – environment for the new process, may be added to for infrastructure
user_name – Requested user specified reference name
options – ProcessOptions object, what options to apply to creation
soft – If process already exists with given name, do not create and return descriptor instead.
stdin – If stdin=PIPE, a Dragon Connection will be returned to enable sending data to the stdin of the child process.
stdout – Capture stdout from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stdout. Using DEVNULL will cause the stdout data to be thrown away.
stder – Capture stderr from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stderr. Using DEVNULL will cause the stdout data to be thrown away. Using STDOUT will cause the stderr data to be combined with the stdout data.
group – Not used
user – Not used
umask – Not used
pipesize – Set the channel capacity. Default = -1.
pmi_required – This process is part of a Dragon managed MPI/PMI application group.
policy – If a policy other than the global default is to be used for this process.

Returns

ProcessDescriptor object

create_with_argdata(exe, run_dir, args, env, argdata=None, user_name='', options=None, soft=False, pmi_required=False, stdin=None, stdout=None, stderr=None, policy=None)

Asks Global services to create a new process and deliver starting args to it thru messaging.

This is an extension of the ‘create’ method that encapsulates our scheme for getting starting arguments to the new process. This depends on how big the arguments turn out to be, and is tightly bound with logic in global services and in the api_setup module.

Parameters

exe – executable to run
run_dir – directory to run it in
args – argv for the process
env – environment for the new process, may be added to for infrastructure
argdata – bytes to deliver
user_name – Requested user specified reference name
options – ProcessOptions object, what options to apply to creation
soft – If process already exists with given name, do not create and return descriptor instead.
pmi_required – This process is part of a Dragon managed MPI/PMI application group.
stdin – If stdin=PIPE, a Dragon Connection will be returned to enable sending data to the stdin of the child process.
stdout – Capture stdout from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stdout. Using DEVNULL will cause the stdout data to be thrown away.
stder – Capture stderr from the child process. If using PIPE, a Dragon Connection object will be returned which can be used to read from stderr. Using DEVNULL will cause the stdout data to be thrown away. Using STDOUT will cause the stderr data to be combined with the stdout data.

Returns

ProcessDescriptor object

get_list()

Asks Global Services for a list of the p_uids of all managed processes.

This is the moral equivalent of a very simple unix ‘ps’ for managed processes.

TODO: add some options to this, and to the message itself, to have a finer control over which processes you get back.

Returns: list of the p_uids of all processes, alive and dead

query(identifier)

Asks Global Services for the ProcessDescriptor of a specified managed process

Note you can query processes that have already exited; this is how one can find out the process’s exit code.

Parameters: identifier – string indicating process name or integer indicating a p_uid
Returns: ProcessDescriptor object corresponding to specified managed process
Raises: ProcessError if there is no such process

kill(identifier, sig=Signals.SIGKILL)

Asks Global Services to kill a specified managed process with a specified signal.

Note that this is like the unix ‘kill’ command - the signal given to the process might not necessarily be intended to cause it to terminate.

Parameters

identifier – string indicating process name or integer indicating a p_uid
sig – signal to use to kill the process, default=signal.SIGKILL

Returns

Nothing if successful

Raises

ProcessError if there is no such process, or if the process has not yet started.

join(identifier, timeout=None)

Asks Global Services to join a specified managed process.

Returns when the process has exited.

Parameters

identifier – string indicating process name or integer indicating a p_uid
timeout – Timeout in seconds for max time to wait. None = default, infinite wait

Returns

The unix exit code from the process, or None if there is a timeout.

Raises

ProcessError if there is no such process or some other error has occurred.

get_multi_join_success_puids(statuses: Dict[str, List[int]]) → Tuple[List[Tuple[int, int]], bool]

Go through list of processes that have been joined on to isolate successful zero exits

Parameters: statuses (Dict[str, List[int, int]]) – Dict of puid keys pointing to error status and exit codes of multi_join function
Returns: List comprised of puids and exit code tuples, if exit code was zero. And whether there was a timeout
Return type: Tuple[List[Tuple[int, int]], bool]

get_multi_join_failure_puids(statuses: Dict[str, List[int]]) → Tuple[List[Tuple[int, int]], bool]

Go through list of processes that have been joined on to isolate non-zero exits

Parameters: statuses (Dict[str, List[int, int]]) – Dict of puid keys pointing to error status and exit codes of multi_join function
Returns: Tuple made up of List comprised of puids and exit code tuples, if exit was no-zero. And whether there was a timeout
Return type: Tuple[List[Tuple[int, int]], bool]

multi_join(identifiers: List[int], timeout: Optional[bool] = None, join_all: bool = False, return_on_bad_exit: bool = False) → Tuple[List[Tuple[int, int]], Dict]

Asks Global Services to join a list of specified managed processes.

If join_all is False, it returns when ‘any’ process has exited or there is a timeout. If join_all is True, it returns when ‘all’ processes have exited or there is a timeout.

Parameters

identifiers (List[int, str]) – list of process identifiers indicating p_uid (int) or process name (string)
timeout (bool, optional) – Timeout in seconds for max time to wait. defaults to None, infinite wait
join_all (bool, optional) – indicates whether we need to wait on all processes in the list or not, defaults to False
return_on_bad_exit (bool, optional) – If join_all is True, multi_join will still return if there was a non-zero exit, defaults to False

Returns

If join_all is False, return a list of tuples (p_uid, unix_exit_code) for any processes exited, or None if none exited and there is a timeout, along with a dictionary with the status of each process. If join_all is True, return a list of tuples (p_uid, unix_exit_code) when all processes exited, or None if none exited and there is a timeout or some exited and some errored/timed out, along with a dictionary with the status of each process.