dragon.mpbridge.queues.DragonJoinableQueue

class DragonJoinableQueue

Bases: PatchedDragonNativeQueue

A jonable queue co-located on the same node by default as the creating process

__init__(*args, ctx=None, **kwargs)

Init method

Parameters:

maxsize (int , optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100
m_uid (int , optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID
block_size (int , optional) – Block size for the underlying channel, defaults to 64 kbytes
joinable (bool , optional) – If this queue should be joinable, defaults to False
policy (object , optional) – policy object, defaults to POLICY_USER
_ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:

QueueError – If something goes wrong during creation
ValueError – If a parameter is wrong
NotImplementedError – If a joinable queue is used with an external channel

Methods

`__init__`(*args[, ctx])	Init method
`cancel_join_thread`()	This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.
`close`()	Indicate that no more data will be put on this queue by the current process.
`empty`()	Return True if the queue is empty, False otherwise.
`full`()	Return True if the queue is full, False otherwise.
`get`([block, timeout])	Remove and return an item from the queue.
`get_nowait`()	Equivalent to get(False).
`join`(*args[, timeout])	Block until all items in the queue have been gotten and processed.
`join_thread`()	This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.
`put`(obj[, block, timeout])	Puts the serialization of an object onto the queue.
`put_nowait`(obj)	Equivalent to put(obj, False).
`qsize`()	Return the approximate size of the queue.
`task_done`()	Indicate that a formerly enqueued task is complete.

__init__(*args, ctx=None, **kwargs)

Init method

Parameters:

maxsize (int , optional) – sets the upperbound limit on the number of items that can be placed in the queue, defaults to 100
m_uid (int , optional) – The m_uid of the memory pool to use, defaults to _DEF_MUID
block_size (int , optional) – Block size for the underlying channel, defaults to 64 kbytes
joinable (bool , optional) – If this queue should be joinable, defaults to False
policy (object , optional) – policy object, defaults to POLICY_USER
_ext_channel (instance of dragon.channel, optional) – non-local externally managed channel for testing purposes only, defaults to None

Raises:

QueueError – If something goes wrong during creation
ValueError – If a parameter is wrong
NotImplementedError – If a joinable queue is used with an external channel

cancel_join_thread(): This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

close() → None : Indicate that no more data will be put on this queue by the current process. The refcount of the background channel will be released and the channel might be removed, if no other processes are holding it. This version includes a patch for the multiprocessing unit tests.

empty() → bool 

Return True if the queue is empty, False otherwise. This might not be reliable

Raises:: ValueError – If the queue is closed
Returns:: Wether or not the queue is empty
Return type:: bool

full() → bool 

Return True if the queue is full, False otherwise.

Raises:: ValueError – If the queue is closed
Returns:: Wether or not the queue is full
Return type:: bool

get(block: bool = True, timeout: float = None) → object 

Remove and return an item from the queue.

Parameters:

block (bool , optional) – Make this call blocking, defaults to True
timeout (float , optional) – number of seconds to block, defaults to None

Raises:

ValueError – If queue is closed
queue.Empty – If queue is empty

Returns:

The next item in the queue

Return type:

object

get_nowait() → object 

Equivalent to get(False).

Returns:: The next item in the queue
Return type:: object

join(*args, timeout: float = None, **kwargs) → None 

Block until all items in the queue have been gotten and processed.

If a join() is blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Parameters:: timeout (float ) – time to wait on the join in sec, optional
Returns:: None

join_thread() → None : This method is a no-op for Dragon based multiprocessing and exists solely for compatibility with the queues.Queue API.

put(obj, block=True, timeout=None) → None 

Puts the serialization of an object onto the queue. This version includes patches for the multiprocessing unit tests

Parameters:

obj – object to serialize and put
block – Whether to block
timeout – Timeout, if blocking. None means infinity, default

Returns:

None

put_nowait(obj) → None 

Equivalent to put(obj, False).

Parameters:: obj (object ) – object to serialize and put
Returns:: None

qsize() → int 

Return the approximate size of the queue. This number may not be reliable.

Raises:: ValueError – If the queue is closed
Returns:: approximate number of items in the queue
Return type:: int

task_done() → None 

Indicate that a formerly enqueued task is complete. Used by queue consumers. Only for joinable queues.

Raises:

QueueError – If this queue is not joinable or there were no pending tasks.
ValueError – If called more times than there were items in the queue.