dragon.mpbridge.queues

Dragon’s replacement of Multiprocessing Queue objects based on Channels.

Functions

JoinableQueue([maxsize, ctx, use_base_impl])

Queue([maxsize, ctx, use_base_impl])

SimpleQueue(*[, ctx, use_base_impl])

restrict_public_api(public_api)

"Decorator that removes public methods of the decorated class not in public_api.

Classes

BaseImplJoinableQueue

BaseImplQueue

BaseImplSimpleQueue

DragonJoinableQueue

DragonQueue

DragonSimpleQueue

FakeConnectionHandle

A placeholder connection handle for a queue that has been reset

PatchedDragonNativeQueue

The Dragon native queue including patches for Python Multiprocessing The API has to exactly match the Multiprocessing.queues.Queue API.
restrict_public_api(public_api: list) → callable

“Decorator that removes public methods of the decorated class not in public_api. To do so it changes the namespace the first parent & grandparent class. Breaks inheritance.

Parameters

public_api (list) – contains the names of all valid public attributes and methods

Returns

returns the cleaned class

Return type

type

class FakeConnectionHandle

A placeholder connection handle for a queue that has been reset

class PatchedDragonNativeQueue

The Dragon native queue including patches for Python Multiprocessing The API has to exactly match the Multiprocessing.queues.Queue API.

__init__(*args, **kwargs)

Init method

Parameters

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises
qsize() → int

Return the approximate size of the queue. This number may not be reliable.

Raises

ValueError – If the queue is closed

Returns

approximate number of items in the queue

Return type

int

put(obj, block=True, timeout=None) → None

Puts the serialization of an object onto the queue. This version includes patches for the multiprocessing unit tests

Parameters

  • obj – object to serialize and put

  • block – Whether to block

  • timeout – Timeout, if blocking. None means infinity, default

Returns

None

close() → None

Indicate that no more data will be put on this queue by the current process. The refcount of the background channel will be released and the channel might be removed, if no other processes are holding it. This version includes a patch for the multiprocessing unit tests.

join_thread() → None

This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

cancel_join_thread()

This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

class DragonQueue
__init__(*args, ctx=None, **kwargs)

Init method

Parameters

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises
class DragonJoinableQueue
__init__(*args, ctx=None, **kwargs)

Init method

Parameters

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises
class DragonSimpleQueue
__init__(*args, ctx=None, **kwargs)

Init method

Parameters

  • maxsize (int, optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100

  • m_uid (int, optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID

  • block_size (int, optional) – Block size for the underlying channel, defaults to 64 kbytes

  • joinable (bool, optional) – If this queue should be joinable, defaults to False

  • policy (object, optional) – policy object, defaults to POLICY_USER

  • _ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises