dragon.native.process

The Dragon native process object provides process management across one or multiple distributed systems. ProcessTemplate can hold a blueprint for a process that can be used to generate many similar processes.

Functions

current()

Get the PythonProcess object of the current process.

Classes

Popen

Process

This object describes a process managed by the Dragon runtime.

ProcessTemplate

This class provides a template for a Dragon process.
class Popen

Bases: object

PIPE = -1
STDOUT = -2
DEVNULL = -3
__init__(args, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, shell=False, cwd=None, env=None, group=None, user=None, proc_name=None, umask=-1, pipesize=-1)

Emulate the subprocess.Popen interface using Dragon infrastructure.

The DragonPopen constructor mirrors the interface of the subprocess.Popen class. Not every field is used, but is provided for signature compatability. Unsupported options may be ignored. See subprocess.Popen description for a description of arguments. Here are notable differences.

When PIPE is specified, stdout and stderr will be a Connection. When STDOUT is specified for stderr, they will both be set to the same Connection object. If DEVNULL is specified for stdout or stderr, then the output is thrown away.

When bufpool is provided it will be used for any allocations of connections for this process.

Parameters:

  • args (_type_) – _description_

  • executable (_type_, optional) – _description_, defaults to None

  • stdin (_type_, optional) – _description_, defaults to None

  • stdout (_type_, optional) – _description_, defaults to None

  • stderr (_type_, optional) – _description_, defaults to None

  • preexec_fn (_type_, optional) – _description_, defaults to None

  • shell (bool, optional) – _description_, defaults to False

  • cwd (_type_, optional) – _description_, defaults to None

  • env (_type_, optional) – _description_, defaults to None

  • group (_type_, optional) – _description_, defaults to None

  • user (_type_, optional) – _description_, defaults to None

  • proc_name (_type_, optional) – _description_, defaults to None

  • umask (int, optional) – _description_, defaults to -1

  • pipesize (int, optional) – _description_, defaults to -1

Raises:

ex – _description_

poll()

Check if process has terminated

wait(timeout=None)

wait for process to terminate

communicate(input=None, timeout=None)

receive all output and transmit all input and wait for process to terminate.

send_signal(signal)

just like it sounds

terminate()

Send SigTERM

kill()

Send SigKILL

property args

actual args passed to process

property stdin

the stdin connection when PIPE was specified. None otherwise.

property stdout

The stdout connection when PIPE was specified. None otherwise.

property stderr

The stderr connection when PIPE was specified. None otherwise.

property puid

Dragon process puid. Globally unique

property pid

OS process pid. Node unique

property returncode

When it has terminated, exit code. None otherwise.

class ProcessTemplate

Bases: object

This class provides a template for a Dragon process.

__init__(target: str, args: tuple | None = None, kwargs: dict | None = None, cwd: str = '.', env: dict | None = None, stdin: int | None = None, stdout: int | None = None, stderr: int | None = None, policy: Policy | None = None, options: ProcessOptions | None = None)

Generic Dragon process template object defining a process based on a binary executable or a Python callable.

If the target is a callable, define the Python process executing the callable. If the target is a string, assume it’s the filename of a binary to be started. If the target file is not found, the object tries to resolve the command via the PATH environment variable. If that fails, it assumes the file is in the current working directory.

This template cannot be started, but it can be used to create many similar processes like this:

```Python

t = ProcessTemplate(myfunc) p1 = Process.from_template(t, ident=”NewProcess1”) p2 = Process.from_template(t, ident=”NewProcess2”)

```

The template stores the absolute path of the target if the target is an executable. If the target is a Python function, the template stores the Python executable in its target attribute, the command line parameters in its args attribute and the Python data in its argdata attribute. To obtain the original attributes in Python use

```Python

target, args, kwargs = p.get_original_python_parameters()

```

Parameters:

  • target (str or callable) – The binary or Python callable to execute.

  • args (tuple, optional) – arguments to pass to the binary or callable, defaults to None

  • kwargs (tuple, optional) – keyword arguments to pass to the Python callable, defaults to None

  • cwd (str, optional) – current working directory, defaults to “.”

  • env (dict, optional) – environment variables to pass to the process environment, defaults to None

  • stdin (int, optional) – Stdin file handling. Valid values are PIPE and None.

  • stdout (int, optional) – Stdout file handling. Valid values are PIPE, STDOUT and None.

  • stderr (int, optional) – Stderr file handling. Valid values are PIPE, STDOUT and None.

  • policy (dragon.infrastructure.policy.Policy, optional) – determines the placement and resources of the process

  • options (dragon.infrastructure.process_desc.ProcessOptions, optional) – process options, such as allowing the process to connect to the infrastructure

property sdesc
get_sdict()
classmethod from_sdict(sdict)
get_original_python_parameters() → tuple[callable, tuple, dict]

Return the original parameters of a Python process from the object.

Returns:

function, args and kwargs of the templated process

Return type:

tuple[callable, tuple, dict]

start() → None
class Process

Bases: ProcessTemplate

This object describes a process managed by the Dragon runtime.

__init__(target: str, args: tuple | None = None, kwargs: dict | None = None, cwd: str = '.', env: dict | None = None, ident: str | None = None, _pmi_enabled: bool = False, stdin: int | None = None, stdout: int | None = None, stderr: int | None = None, policy: Policy | None = None, options: ProcessOptions | None = None)

Generic Dragon process object executing a binary executable or a Python callable.

Processes are named and can be looked up by specifying the ident keyword: Process(None, ident="Apple")

If the target is a callable, start a new Python process executing the callable. The class uses cloudpickle to transfer the callable to the child process. See the cloudpickle documentation for possible limitations.

If the target is a string, assume it’s the filename of a binary to be started. If the target file is not found, the object tries to resolve the command via the PATH environment variable. If that fails, it assumes the file is in the current working directory.

The Process class should not be subclassed, because the Dragon runtime only holds on to attributes of the Process super class, breaking the name lookup.

Parameters:

  • target (str or callable) – The binary or Python callable to execute.

  • args (tuple, optional) – arguments to pass to the binary or callable, defaults to None

  • kwargs (tuple, optional) – keyword arguments to pass to the Python callable, defaults to None

  • cwd (str, optional) – current working directory, defaults to “.”

  • env (dict, optional) – environment variables to pass to the process environment, defaults to None

  • ident (int or str) – unique id or name of this process

  • stdin (int, optional) – Stdin file handling. Valid values are PIPE and None.

  • stdout (int, optional) – Stdout file handling. Valid values are PIPE, STDOUT and None.

  • stderr (int, optional) – Stderr file handling. Valid values are PIPE, STDOUT and None.

  • policy (dragon.infrastructure.policy.Policy) – determines the placement and resources of the process

  • options (dragon.infrastructure.process_desc.ProcessOptions, optional) – process options, such as allowing the process to connect to the infrastructure

classmethod from_template(template: ProcessTemplate, ident: str | None = None, _pmi_enabled: bool = False) → object

A classmethod that creates a new process object from a template.

Parameters:

  • template (ProcessTemplate) – the template to base the process on

  • ident (str, optional) – intended name of the process, defaults to None

Returns:

The new process object

Return type:

dragon.native.Process

start() → None

Start the process represented by the underlying process object.

property is_alive: bool

Check if the process is still running

Returns:

True if the process is running, False otherwise

Return type:

bool

join(timeout: float | None = None) → int

Wait for the process to finish.

Parameters:

timeout (float, optional) – timeout in seconds, defaults to None

Returns:

exit code of the process, None if timeout occurs

Return type:

int

Raises:

ProcessError

terminate() → None

Send SIGTERM signal to the process, terminating it.

Returns:

None

Return type:

NoneType

kill() → None

Send SIGKILL signal to the process, killing it.

Returns:

None

Return type:

NoneType

send_signal(sig) → None

Send sig to the process, killing it.

Parameters:

sig (signal.Signals) – [description]

Returns:

None

Return type:

NoneType

property name: str
property puid: int

Process puid. Globally unique

property node: int

Return the unique host id of the node the process is running on.

Returns:

huid of host node.

Return type:

int

property returncode: int

When the process has terminated, return exit code. None otherwise.

children() → list[object]

Find all active children of the current process.

Returns:

a list of p_uids of active children of the current process

Return type:

list(int)

parent() → object

Get the process object of the parent process. :returns: The puid or the parent of the current process :return type: int

property stdin_conn
property stdout_conn
property stderr_conn
current() → Process

Get the PythonProcess object of the current process.

Returns:

The current puid

Return type:

int