dragon.fli
The file-like-interface (FLI) is a class that abstracts dragon.channels to a file-like API.
This is a lower-level API for efficient data transfer over dragon.channels, especially for situations
where the data being communicated is not contiguous. The most common use-case is for adapting into Pickle for
object serialization.
Classes
Receiving handle for FLInterfaces. |
|
Sending handle for FLInterfaces. |
|
Cython wrapper for the File-Like-Interface |
|
Exceptions
The DragonFLIError is an exception that can be caught that explicitly targets those errors generated by the FLI code. |
|
The FLIEOT Exception is used to indicate the end of stream for an FLI conversation. |
- exception DragonFLIError
Bases:
ExceptionThe DragonFLIError is an exception that can be caught that explicitly targets those errors generated by the FLI code. The string associated with the exception includes any traceback avaialable from the C level interaction.
- __init__(lib_err, msg)
- exception DragonFLIOutOfMemoryError
Bases:
DragonFLIError,MemoryError
- exception DragonFLIRecvdMsgDestroyedError
Bases:
DragonFLIError,MemoryError
- exception DragonFLITimeoutError
Bases:
DragonFLIError,TimeoutError
- exception FLIEOT
Bases:
DragonFLIError,EOFErrorThe FLIEOT Exception is used to indicate the end of stream for an FLI conversation. This Exception inherits from EOFError so applications using the FLI may choose to catch EOFError instead.
- class FLIRecvH
Bases:
objectReceiving handle for FLInterfaces.
- __init__()
If specifying that the main channel is to be used as a stream channel then both sender and receiver must agree to this. Both send and receive handle would need to be specified using the use_main_as_stream_channel in that case.
- Parameters:
adapter – An FLI over which to create a send handle.
stream_channel – Default is None. The receiver may supply a stream channel when opening a receive handle. If the FLI is created with stream channels, then the value of the argument may be None. If supplied by a user then the manager channel of the FLI must exist. If use_main_as_stream_channel is True, this argument must be None.
use_main_as_stream_channel – Default is False. If True, then both send handle and receive handle must be true. This would indicate that both sender and receiver are agreeing they are the only sender and the only receiver and they wish to use the single main channel as the stream channel. This can be useful in some restricted circumstances but must only be used when there is exactly one sender and one receiver on the FLI.
destination_pool – Default is None. If provided, it is the pool which should contain the message after it is received. This makes sense mainly for receiving memory, but other receive methods will use the pool as a transient storage space while receiving a message.
timeout – Default is None. None means to block forever. Otherwise the timeout should be some number of seconds to wait for the operation to complete. The operation could timeout when not supplying a stream channel and there is no channel available during the specified amount of time in the manager channel. The timeout provided here also becomes the default timeout when used in the context manager framework.
- Returns:
An FLI send handle.
- close(timeout=None)
Close the receive handle and discard any remaining messages in the stream. If EOT is found, discard it. If more data is found than the EOT marker, raise an exception to indicate data was discarded.
- create_fd(timeout=None)
Creates a readable file-descriptor and returns it.
- finalize_fd()
Flushes a file-descriptor and waits until all buffers are read and the file descriptor is closed.
- is_closed
Return True if the receive handle is closed and False otherwise.
- no_close_on_exit()
Call this to avoid closing the receive handle when exiting the context manager. By default, the receive handle is closed when the context manager exits.
- recv_bytes(size=-1, timeout=None)
Receive at most size bytes from the stream with the given timeout, which is given in seconds. If timeout is None, wait forever. If size is -1 (the default) then read all available bytes.
- recv_bytes_into(bytes_buffer=None, timeout=None)
Receive bytes into the bytes_buffer with timeout given in seconds. If timeout is None (the default) then wait forever for data. The receive handle must be open to call this.
- recv_mem(timeout=None)
Receive the next memory allocation/message from the stream. The timeout if given is in seconds. None means to wait forever.
- stream_received
Return True if the stream has been entirely received.
- class FLISendH
Bases:
objectSending handle for FLInterfaces. A send handle is needed when sending data. Proper use of a send handle includes creating it (which also opens it for sending), sending data with one or more to the send operations, and closing it once data transmission is complete.
- __init__()
When creating a send handle an application may provide a stream channel to be used. If specifying that the main channel is to be used as a stream channel then both sender and receiver must agree to this. Both send and receive handle would need to be specified using the use_main_as_stream_channel in that case.
- Parameters:
adapter – An FLI over which to create a send handle.
stream_channel – Default is None. The sender may supply a stream channel when opening a send handle. If the FLI is created with stream channels, then the value of the argument may be None. If supplied by a user then the main channel of the FLI must exist. If use_main_as_stream_channel is True, this argument must be None.
use_main_as_stream_channel – Default is False. If True, then both send handle and receive handle must be true. This would indicate that both sender and receiver are agreeing they are the only sender and the only receiver and they wish to use the single main channel as the stream channel. This can be useful in some restricted circumstances but must only be used when there is exactly one sender and one receiver on the FLI.
destination_pool – Default is None. This is used to indicate that messages that are located elsewhere should end up in this pool. This can also be a remote pool on a different node so long as the channel being sent to also resides on the same node.
allow_strm_term – Default is False. When True is provided a receiver that closes a receive handle before the end of stream will cause the sender to receive an EOFError on any send operation. This allows the stream to be terminated by a receiver and a sender must then handle the EOFError exception when sending.
timeout – Default is None. None means to block forever. Otherwise the timeout should be some number of seconds to wait for the operation to complete. The operation could timeout when not supplying a stream channel and there is no channel available during the specified amount of time in the manager channel. The timeout provided here also becomes the default timeout when used in the context manager framework.
- Returns:
An FLI send handle.
- close(timeout=None)
When the conversation is complete the send handle should be closed. In the case of a buffered FLI, no data is sent until the send handle is closed. In all cases, closing the send handle indicates the end of the stream for the receiver.
- create_fd(buffered=False, chunk_size=0, arg=0, timeout=None)
Opens a writable file-descriptor and returns it.
- finalize_fd()
Flushes a file-descriptor and waits until all buffers are written and the file descriptor is closed.
- send_bytes(data, arg=0, buffer=False, timeout=None)
When sending bytes it is possible to specify the bytes to be sent. In addition, you may specify a user specified argument or hint to be sent. If buffer is true, then data is not actually sent on this call, but buffered for future call or until the send handle is closed.
If the receiver closes the receive handle early, sending bytes may result in raising EOFError.
- send_mem(mem, arg=0, transfer_ownership=True, timeout=None)
Send a memory allocation with a hint (provided in arg). When True (the default) transfer_ownership will transfer ownership of the memory allocation to the sender. If a timeout is provided, then it will wait for that time in seconds to send it. If timeout is None it will wait forever if needed.
If the receiver closes the receive handle early, sending memory may result in raising EOFError.
- class FLInterface
Bases:
objectCython wrapper for the File-Like-Interface
- __init__(*args, **kwargs)
- classmethod attach(serialized_bytes, mem_pool=None)
Given a serialized descriptor (in serialized_bytes), attach to an FLI.
- classmethod create_buffered(main_ch=None, pool=None)
Helper function to more easily create a simple buffered FLInterface Does not require any internal function, it’s simply limiting the number of options for the user in order to make it more straightforward to make an explicitly buffered FLI
- destroy()
Free the resources of the FLI. The underlying channels are emptied (manager and main) and detached.
- detach()
Detach from an FLI without altering it in any way.
- is_buffered
Returns True if the FLI is a buffered FLI and False otherwise.
- num_available_streams(timeout=None)
Return the number of available streams.
- recvh(*args, **kwargs)
Return a new FLI Recv Handle object
- sendh(*args, **kwargs)
Return a new FLI Send Handle object
- serialize()
Return a serialized representation of the FLI that can be used to attach to it later.