Workflows

The following reference provides details about the Dragon Workflows API. Tools for graph-based execution of tasks, data proessing, and integrations with other workflow systems are included.

Python Reference

Batch for Graph-based Execution

Run functions, executables, and parallel applications through a high-level, graph-based tasking API.

Batch

Graph-based distributed scheduling for functions, executables, and parallel applications

`BatchDDict`
`BatchFile`
`Function`
`Job`
`MakeTask`	Note on terminology: we use the terms "process" and "resolve" in a number of places in this class and in helper functions for this class.
`SubmitAfterCloseError`	Exception for submitting work to a batch instance after it's been closed.
`Task`

Data Processing Tools

Prototype API for creating dataflow processing pipelines.

data_mover

Classes for efficiently moving data between a host and devices

Other Workflow Tools

Dragon can integrate into other workflow tools and enhance their scalability and performance. These APIs are integrations into other tools. These are currently works-in-progress and are useful primarily for experimentation.

`parsl_batch_executor`	An expiremental executor for parsl that uses Dragon's multiprocessing pool to submit batches of tasks to workers.
`parsl_executor`
`parsl_mpi_app`	Dragon's experimental `@mpi_app` decorator and `DragonMPIExecutor` for `parsl`.