dragon.mpbridge.queues

Dragon’s replacement of Multiprocessing Queue objects based on Channels.

Functions

JoinableQueue([maxsize, ctx, use_base_impl])

Queue([maxsize, ctx, use_base_impl])

SimpleQueue(*[, ctx, use_base_impl])

restrict_public_api(public_api)

"Decorator that removes public methods of the decorated class not in public_api.

Classes

BaseImplJoinableQueue

BaseImplQueue

BaseImplSimpleQueue

DragonJoinableQueue

DragonQueue

DragonSimpleQueue

FakeConnectionHandle

A placeholder connection handle for a queue that has been reset

PatchedDragonNativeQueue

The Dragon native queue including patches for Python Multiprocessing The API has to exactly match the Multiprocessing.queues.Queue API.
restrict_public_api(public_api: list) → callable

“Decorator that removes public methods of the decorated class not in public_api. To do so it changes the namespace the first parent & grandparent class. Breaks inheritance.

Parameters:

public_api (list) – contains the names of all valid public attributes and methods

Returns:

returns the cleaned class

Return type:

type

class FakeConnectionHandle

Bases: object

A placeholder connection handle for a queue that has been reset

class PatchedDragonNativeQueue

Bases: Queue

The Dragon native queue including patches for Python Multiprocessing The API has to exactly match the Multiprocessing.queues.Queue API.

__init__(*args, **kwargs)

Init method

Parameters:

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:
qsize() → int

Return the approximate size of the queue. This number may not be reliable.

Raises:

ValueError – If the queue is closed

Returns:

approximate number of items in the queue

Return type:

int

put(obj, block=True, timeout=None) → None

Puts the serialization of an object onto the queue. This version includes patches for the multiprocessing unit tests

Parameters:

  • obj – object to serialize and put

  • block – Whether to block

  • timeout – Timeout, if blocking. None means infinity, default

Returns:

None

close() → None

Indicate that no more data will be put on this queue by the current process. The refcount of the background channel will be released and the channel might be removed, if no other processes are holding it. This version includes a patch for the multiprocessing unit tests.

join_thread() → None

This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

cancel_join_thread()

This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

class DragonQueue

Bases: PatchedDragonNativeQueue

__init__(*args, ctx=None, **kwargs)

Init method

Parameters:

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:
class DragonJoinableQueue

Bases: PatchedDragonNativeQueue

__init__(*args, ctx=None, **kwargs)

Init method

Parameters:

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:
class DragonSimpleQueue

Bases: PatchedDragonNativeQueue

__init__(*args, ctx=None, **kwargs)

Init method

Parameters:

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:
class BaseImplQueue

Bases: Queue

__init__(*args, **kwargs)
class BaseImplJoinableQueue

Bases: JoinableQueue

__init__(*args, **kwargs)
class BaseImplSimpleQueue

Bases: SimpleQueue

__init__(*args, **kwargs)
Queue(maxsize=0, *, ctx=None, use_base_impl=True)
JoinableQueue(maxsize=0, *, ctx=None, use_base_impl=True)
SimpleQueue(*, ctx=None, use_base_impl=True)