dragon.workflows.runtime.Proxy

class Proxy

Bases: object

Object providing API to manage Dragon runtime connections for remote execution.

The Proxy class enables seamless switching between local and remote Dragon runtime environments. It manages infrastructure connections and environment variables to allow processes and resources created in one runtime to be accessed from another.

A typical workflow using Proxy may look like:

import dragon.workflows.runtime as runtime

def howdy(q):
    q.put(
        f"howdy from {socket.gethostname()} - local num cores is {os.cpu_count()}, "
        f"runtime available cores is {mp.cpu_count()}"
    )

def signal_exit(exit_path):
    file = open(exit_path, "w")
    file.close()

def shutdown_remote_runtime(exit_path, remote_working_dir):
    exit_proc = Process(target=signal_exit, args=(exit_path,), cwd=remote_working_dir)
    exit_proc.start()
    exit_proc.join()

if __name__ == '__main__':
    # Set up paths for remote runtime discovery

    system = "my.remote.system"
    runtime_name = "proxy_runtime"
    publish_dir = "/my/remote/publish/dir"
    exit_path = "/my/remote/publish/dir/exit_client"

    # Configure Dragon to use multiprocessing API
    mp.set_start_method("dragon")

    # Look up and attach to remote runtime
    runtime_sdesc = runtime.lookup(system, runtime_name, 30, publish_dir=publish_dir)
    proxy = runtime.attach(runtime_sdesc, remote_cwd=remote_working_dir)

    # Enable proxy to create queue in remote runtime
    proxy.enable()
    q = mp.Queue()

    # Launch processes in remote runtime
    print("Launching remote runtime processes...", flush=True)
    procs = []
    for _ in range(2):
        p = Process(target=howdy, args=(q,))
        p.start()
        procs.append(p)


    # Wait for all processes to complete.
    # The queue must have a maxsize greater than the number of processes for this ordering to work.
    for p in procs:
        p.join()

    proxy.disable()

    # Collect results from remote processes
    # The remote q can still be accessed outside the proxy environment
    for p in procs:
        msg = q.get()
        print(f"Message from remote runtime: {msg}", flush=True)

    # Clean up resources
    for p in procs:
        del p
    del q

    # signal client's exit
    proxy.enable()
    shutdown_remote_runtime(exit_path, remote_working_dir)
    proxy.disable()

__init__(sdesc, oob_net, cwd=None, remote_python_executable=None)

Initialize a Proxy object for managing remote runtime connections.

Parameters:

• sdesc (dragon.infrastructure.messages.RuntimeDesc) – Serialized descriptor of the remote runtime containing connection information

• oob_net (dragon.transport.oob.OutOfBand) – Out-of-band network connection to remote runtime for communication

• cwd (str , optional) – Current working directory to use on remote runtime, defaults to None

• remote_python_executable (str , optional) – Path to Python executable on remote runtime, defaults to None

Methods

__init__(sdesc, oob_net[, cwd, ...])	Initialize a Proxy object for managing remote runtime connections.
disable()	Disable the proxy by restoring local runtime environment.
enable()	Enable the proxy by switching to remote runtime environment.
get_env()	Get the remote runtime environment variables.
get_remote_cwd()	Get the current working directory configured for the remote runtime.
get_remote_python_executable()	Get the Python executable path configured for the remote runtime.
set_remote_cwd(cwd)	Set the current working directory to use on the remote runtime.
set_remote_env(remote_env)	Set the environment variables to use on the remote runtime.
set_remote_python_executable(...)	Set the Python executable to use on the remote runtime.

__init__(sdesc, oob_net, cwd=None, remote_python_executable=None)

Initialize a Proxy object for managing remote runtime connections.

Parameters:

• sdesc (dragon.infrastructure.messages.RuntimeDesc) – Serialized descriptor of the remote runtime containing connection information

• oob_net (dragon.transport.oob.OutOfBand) – Out-of-band network connection to remote runtime for communication

• cwd (str , optional) – Current working directory to use on remote runtime, defaults to None

• remote_python_executable (str , optional) – Path to Python executable on remote runtime, defaults to None

enable()

Enable the proxy by switching to remote runtime environment.

Saves the current local environment and reconfigures the Dragon infrastructure to use the remote runtime’s channels and environment variables. This allows subsequent Dragon operations to target the remote runtime.

disable()

Disable the proxy by restoring local runtime environment.

Restores the original local infrastructure connections and environment variables, switching back from remote to local runtime.

get_env()

Get the remote runtime environment variables.

Returns:

Dictionary of environment variables from the remote runtime

Return type:

dict

get_remote_cwd()

Get the current working directory configured for the remote runtime.

Returns:

Path to the remote working directory, or None if not set

Return type:

str or None

get_remote_python_executable()

Get the Python executable path configured for the remote runtime.

Returns:

Path to the remote Python executable, or None if not set

Return type:

str or None

set_remote_cwd(cwd)

Set the current working directory to use on the remote runtime.

Configures the working directory that will be used when launching processes on the remote runtime.

Parameters:

cwd (str ) – Path to the desired working directory on the remote system

set_remote_python_executable(remote_python_executable)

Set the Python executable to use on the remote runtime.

Configures which Python interpreter will be used when launching Python processes on the remote runtime.

Parameters:

remote_python_executable (str ) – Path to the Python executable on the remote system

set_remote_env(remote_env)

Set the environment variables to use on the remote runtime.

Configures the environment that will be used when launching processes on the remote runtime.

Parameters:

remote_env (dict ) – Dictionary of environment variables for the remote runtime