dragon.workflows.batch.batch.Job

class Job[source]

Bases: Task

__init__(batch, process_templates: list [ProcessTemplate], reads: list | None = None, writes: list | None = None, name: str | None = None, timeout: float = 1000000000.0) → None [source]

Creates a new job task. Arguments for a process passed using ProcessTemplate.args that are of type AsyncDict will create a dependency for this task on the output specified by the AsyncDict. Further, the output specified by the AsyncDict will be passed in place of the AsyncDict when the job executes.

Parameters:

  • batch (Batch) – The batch in which this function task will execute.

  • process_templates – List of pairs of the form (nprocs, process_template), where nprocs is the number

of processes to create using the specified template. :type process_templates: list :param reads: A list of Read objects created by calling Batch.read. :type reads: Optional[list] :param writes: A list of Write objects created by calling Batch.write. :type writes: Optional[list] :param name: A human-readable name for the task. :type name: Optional[str]

Returns:

Returns None.

Return type:

None

Methods

__init__(batch, process_templates[, reads, ...])

Creates a new job task.

dump_dag(file_name)

Dump a PNG image of the dependency DAG associated with a compiled program.

get()

Performs a batch fence to guarantee the completion of all tasks batched in the background, and then gets the result for this task.

read(obj, *channels)

Indicates READ accesses of a specified set of channels on a communication object, and associates these accesses with this task.

run([timeout])

Starts a task and waits for it to complete.

start()

Start this task by sending its work chunks to the managers.

wait([timeout])

Wait for this Task to complete.

write(obj, *channels)

Indicates WRITE accesses of a specified set of channels on a communication object, and associates these accesses with this task.

Attributes

result

Handle for the task's result.

stderr

Handle for the task's stderr.

stdout

Handle for the task's stdout.

uid

Provides the unique ID for this task.
__init__(batch, process_templates: list [ProcessTemplate], reads: list | None = None, writes: list | None = None, name: str | None = None, timeout: float = 1000000000.0) → None [source]

Creates a new job task. Arguments for a process passed using ProcessTemplate.args that are of type AsyncDict will create a dependency for this task on the output specified by the AsyncDict. Further, the output specified by the AsyncDict will be passed in place of the AsyncDict when the job executes.

Parameters:

  • batch (Batch) – The batch in which this function task will execute.

  • process_templates – List of pairs of the form (nprocs, process_template), where nprocs is the number

of processes to create using the specified template. :type process_templates: list :param reads: A list of Read objects created by calling Batch.read. :type reads: Optional[list] :param writes: A list of Write objects created by calling Batch.write. :type writes: Optional[list] :param name: A human-readable name for the task. :type name: Optional[str]

Returns:

Returns None.

Return type:

None

dump_dag(file_name: str ) → None [source]

Dump a PNG image of the dependency DAG associated with a compiled program.

Parameters:

file_name – Name for the new PNG file.

Returns:

Returns None.

Return type:

None

get()[source]

Performs a batch fence to guarantee the completion of all tasks batched in the background, and then gets the result for this task.

Returns:

Returns the result of this task.

Return type:

Any

read(obj, *channels) → None [source]

Indicates READ accesses of a specified set of channels on a communication object, and associates these accesses with this task. Associating READ accesses with a task allows the Batch service to infer dependencies between subtasks in a compiled task, but has no effect on individual (non-compiled) tasks.

Parameters:

  • obj – The communication object being accessed.

  • *channels –

    A tuple of channels on the communcation object that will be read from.

Returns:

Returns None.

Return type:

None

property result

Handle for the task’s result. This should not be accessed until the task is started. result only applies to individual tasks and compiled tasks with a single subtask. The handle has type AsyncDict.

run(timeout: float = 1000000000.0) → Any [source]

Starts a task and waits for it to complete. Currently, a task can only be started once and cannot be restarted after it completes.

Parameters:

timeout (float ) – The timeout for waiting. Defaults to 1e9.

Raises:

  • TimeoutError – If the specified timeout is exceeded.

  • Exception – If this Task raised an exception while running. The exception raised by the task is propagated back to the host that started the task so it can be raised here.

Returns:

Returns the result of the operation being waited on.

Return type:

Any

start() → None [source]

Start this task by sending its work chunks to the managers. Currently, a task can only be started once and cannot be restarted after it completes. If a task is a subtask of a compiled task, then start must be called for the compiled task and not the subtask.

Returns:

Returns None.

Return type:

None

property stderr

Handle for the task’s stderr. This should not be accessed until the task is started. stderr only applies to individual tasks and compiled tasks with a single subtask. The handle has type AsyncDict.

property stdout

Handle for the task’s stdout. This should not be accessed until the task is started. stdout only applies to individual tasks and compiled tasks with a single subtask. The handle has type AsyncDict.

property uid

Provides the unique ID for this task.

wait(timeout: float = 1000000000.0) → None [source]

Wait for this Task to complete. This can only be called after start has been called. This function does not return the task’s result; instead, use get for that purpose.

Parameters:

timeout – The timeout for waiting. Defaults to 1e9.

Raises:

TimeoutError – If the specified timeout is exceeded.

Returns:

Returns None.

Return type:

None

write(obj, *channels) → None [source]

Indicates WRITE accesses of a specified set of channels on a communication object, and associates these accesses with this task. Associating WRITE accesses with a task allows the Batch service to infer dependencies between subtasks in a compiled task, but has no effect on individual (non-compiled) tasks.

Parameters:

  • obj – The communication object being accessed.

  • *channels –

    A tuple of channels on the communcation object that will be written to.

Returns:

Returns None.

Return type:

None