dragon.managed_memory
Classes
TBD |
|
Cython wrapper object for memory pool allocations. |
|
Cython wrapper object to provide access to lists of existing memory allocations in a pool |
|
Cython wrapper for managed memory pools and related structures |
|
Cython wrapper for managed memory attributes Currently unused |
|
TBD |
Exceptions
- exception DragonMemoryError
Bases:
DragonException- __init__(lib_err, msg)
- exception DragonMemoryTypeError
Bases:
DragonMemoryError,TypeError- __init__(lib_err, msg)
- exception DragonPoolAllocationNotAvailable
Bases:
DragonPoolError- __init__(lib_err, msg)
- exception DragonPoolAttachFail
Bases:
DragonPoolError- __init__(lib_err, msg)
- exception DragonPoolCreateFail
Bases:
DragonPoolError- __init__(lib_err, msg)
- exception DragonPoolDetachFail
Bases:
DragonPoolError- __init__(lib_err, msg)
- exception DragonPoolError
Bases:
DragonMemoryError- __init__(lib_err, msg)
- class MemoryAlloc
Bases:
objectCython wrapper object for memory pool allocations.
- classmethod attach(ser_bytes)
Attach to a serialized memory allocation
- Parameters:
ser_bytes – Bytes-like object (memoryview, bytearray, bytes) of a serialized memory descriptor
- Returns:
MemoryAlloc object
- clear(start=0, stop=None)
- clone(offset=0, length=None)
Clone this memory allocation with an offset into it.
- Parameters:
size – offset in bytes into this allocation
- Returns:
New MemoryAlloc object
- Raises:
RuntimeError
- copy(pool: MemoryPool, timeout=None)
Copy an allocation into a pool
- Parameters:
pool – The pool in which to copy it.
- Returns:
A new memory allocation
- Raises:
DragonMemoryError
- detach()
Detach from memory previously attached to.
- free()
Free the managed memory allocation.
- Returns:
None
- Raises:
DragonMemoryError
- get_memview()
Get a memoryview of the underlying memory
- Returns:
Memoryview object
- id
Returns the unique identifier of this memory allocation.
- Returns:
the unique allocation identifier
- Raises:
DragonMemoryError
- is_the_same_as(other)
- pool
Return the pool of this memory allocation.
- Returns:
pool where this resides.
- Raises:
DragonMemoryError
- serialize()
Serialize the memory allocation for storage or communication
- Returns:
Memoryview of the serialized data
- size
Return the size of the allocation
return: the size in bytes of the memory allocation
- class MemoryAllocations
Bases:
objectCython wrapper object to provide access to lists of existing memory allocations in a pool
- alloc_id(idx)
- alloc_type(idx)
Get the type of a particular allocation id.
- Returns:
Enum of the allocation type (if it exists)
- Raises:
RuntimeError if allocation not found
- num_allocs
Number of allocations in pool.
- class MemoryPool
Bases:
objectCython wrapper for managed memory pools and related structures
- __init__()
Create a new memory pool and return a MemoryPool object.
- Parameters:
size – Minimum size (in bytes) of the pool. The actual size will be at least this size.
fname – Filename of the pool to use.
uid – Unique pool identifier to use.
pre_alloc_blocks – A list of integers indicating the number of pre-allocated blocks to allocate of each power of two starting with the minimum block size.
min_block_size – The minimum block size of the pool. The maximum block size is determined by the pool size. The default minimum block size is 4KB.
max_allocations – The maximum number of concurrent allocations that can exist among all processes at a given time. The default is 1048576 (1024^2) but can be set higher if needed. Space is reserved for a manifest of all allocations in shared memory and the default results in approximately 128MB of reserved space. It can be adjusted, but adjustments will increase or decrease the reserved space proportionally.
- Returns:
MemoryPool object
- Raises:
DragonPoolCreateFail
- alloc(size, alloc_type=None)
Allocate a memory block within this pool. Please note that the internal memory manager allocates to nearest powers of 2.
- Parameters:
size – Size (in bytes) to allocate
- Returns:
New MemoryAlloc object
- Raises:
RuntimeError
- alloc_blocking(size, timeout=None, alloc_type=None)
Allocate a memory block within this pool. Please note that the internal memory manager allocates to nearest powers of 2.
- Parameters:
size – Size (in bytes) to allocate
- Returns:
New MemoryAlloc object
- Raises:
RuntimeError
- alloc_from_id(id)
Attach to a memory allocation given its unique identifier
- Returns:
New MemoryAlloc object
- allocation_exists(id)
Scan the pool to determine if a given allocation exists
- Parameters:
id – Integer ID of the allocation
- Returns:
True if allocation exists, False otherwise
- classmethod attach(pool_ser, *, existing_memory_pool=None)
Attach to an existing pool through a serialized descriptor.
- Parameters:
pool_ser – Bytes-like object of a serialized pool descriptor.
- Returns:
MemoryPool object
- Raises:
DragonPoolAttachFail
- classmethod attach_default()
Attach to the default pool. :return: default memory pool object :raises: DragonPoolAttachFail
- current_allocations
- deregister(timeout=None)
Deregister a process local pool which is a pool that exists for the lifetime of the current process. By deregistering it the current process is claiming responsibility for the management of the lifetime of this pool. This only applies to pools created with the make_process_local path. Those pools created through Global Services are managed exclusively by Global Services.
- Parm timeout:
The amount of time the process is willing to wait for a response. A timeout of None means to wait without timing out.
- Raises:
DragonMemoryError if there was an error. Note that the Dragon run-time must be running to use this function as it interacts with Local Services on the node on which it is called.
- destroy()
Destroy the pool created by this object.
- detach(serialize=False)
Detach from a previously attached to pool by this object.
- Parameters:
serialize – Boolean to optionally store a serializer before detaching
- classmethod empty_pool()
- free_blocks
Return the number of free blocks with different block sizes.
- Returns:
A dictionary with keys as block sizes and values as the number of blocks with the size.
- free_space
- get_allocations(alloc_type=None)
Get a list of allocations in this pool
- Returns:
New MemoryAllocations object
- is_local
- classmethod make_process_local(name, size, min_block_size=None, pre_allocs=None, timeout=None)
Create a process local pool which is a pool that exists for the lifetime of the current process. The pool may be shared with other processes, but must be managed in such a way that all shared allocations in the pool are freed prior to this process terminating since the pool will be destroyed when the process exits. The pool may live beyond the process by deregistering the pool from the process (see deregister) and a pool may be registered as a local pool if desired. When registering a pool you should make sure that its lifetime is not being managed by Global Services. Locally created pools are not managed by Global Services.
This is especially useful for processes that need a pool for their own application’s use and don’t need the pool anymore once the process exits.
- Parameters:
name – The name to be used as part of the name of the mmap that is used for the shared memory.
size – The size of the managed memory pool. It will be at least that large.
min_block_size – The minimum allocatable block size. The min block size will be at least this big, possibly bigger. The current implementation will make it the next smallest power of 2. If no min_block_size is provided a default will be used.
pre_allocs – A list of integers corresponding to pre-allocated blocks within the pool where each index within the list corresponds to a power of 2 starting with the min_block_size. The value at each location in the list is the number of pre-allocated blocks of that size.
timeout – Default is None which means to block without timeout until the pool is made. This should not timeout and should be processed quickly. If a timeout value is specified, it is the number of seconds to wait which may be a float.
- Returns:
A new pool object.
- Raises:
DragonMemoryError if there was an error. Note that the Dragon run-time must be running to use this function as it interacts with Local Services on the node on which it is called.
- max_allocations
- max_used_allocations
- muid
- num_block_sizes
Return the total number of block sizes.
- Returns:
number of block sizes of the pool.
- register(timeout=None)
Register a process local pool which is a pool that exists for the lifetime of the current process. By registering it the current process is turning over responsibility for the management of the lifetime of this pool to local services. This only applies to pools created with the make_process_local path or those pools created entirely by a process through the MemoryPool constructor (be careful if doing this - the muid must be unique). Local Services will guarantee unique muids for pools created through the create_process_local method of MemoryPool as will Global Services through its API. Those pools created through Global Services are managed exclusively by Global Services and should not be registered with Local Services.
- Parm timeout:
The amount of time the process is willing to wait for a response. A timeout of None means to wait without timing out.
- Raises:
DragonMemoryError if there was an error. Note that the Dragon run-time must be running to use this function as it interacts with Local Services on the node on which it is called.
- rt_uid
- serialize()
Serialize the pool held by this object. Will store a copy of the serialized data as part of the object after first call.
- Returns:
Memoryview of serialized pool descriptor.
- classmethod serialized_uid_fname(pool_ser)
- size
- utilization