The Distributed Dictionary C Client

The C DDict client api uses the same underlying messaging that is used by the Python client. The C Client cannot create a DDict, but it can attach to one that was created in Python. Python code can serialize a DDict (i.e. create a serialized handle to it) and pass that handle to C code which can then attach to it. After attaching, a C DDict client has full access to the DDict.

Because objects are serialized differently between Python and C, without special handling, C and Python are not going to be able to share key/value pairs. A custom pickler would have to be used in Python and similar code would have to be written in C to have cross-language compatibility of key/value pairs. Both languages provide full access to the DDict, so there is nothing preventing cross-language sharing. It is a serialization and deserialization problem to solve for cross-language sharing.

The design of the C DDict client revolves around the idea of a request for continuity between multi-step operations requiring more than one API call to complete. A DDict Request object is created at the beginning of multi-step DDict interactions. That request object is used on the multi-step calls until the request has been completed and finalized.

Listing 44 Storing a key/value pair in the DDict using C

dragonDDictDescr_t ddict;
dragonDDictRequestDescr_t req;
dragonError_t err;
dragonFLISendHandleDescr_t value_sendh;
dragonFLISendHandleDescr_t key_sendh;

// Attach to the instance of the dictionary. The ddict_ser could be
// provided as a command-line argument or it could receive it via some
// other means (like a Dragon Queue for instance).
dragonError_t err = dragon_ddict_attach(ddict_ser, &ddict, &TIMEOUT);
if (err != DRAGON_SUCCESS)
    err_fail(err, "Could not attach");

err = dragon_ddict_create_request(&ddict, &req);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not create DDict put request.");

err = dragon_ddict_request_key_sendh(&req, &key_sendh);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not access the request key send handle.");

/* One or more key send_bytes can be called to send the key. The key is buffered
   as send_bytes is called so it can be used to select a manager where this
   key will be/is stored. The bytes is a pointer to num_bytes of data.
   Timeout can be specified as needed. NULL for timeout will block
   until the key can be written. The arg and buffer arguments are
   ignored on the send_bytes called when used with the DDict.
*/

err = dragon_fli_send_bytes(&key_sendh, num_key_bytes, key_bytes, 0, false, timeout);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not send the key.");

/* Calling put selects put as the operation. Notice the key is written first, then
   put it called. */
err = dragon_ddict_put(&req);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not send DDict put message.");

/* Getting a value send handle will allow the code to send the value on a put
   operation. */
err = dragon_ddict_request_value_sendh(&req, &value_sendh);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not access the request send handle.");

/* One or more value send_bytes can be called. The buffer arg can be true or false. Using
   true will do one send across the network (if going to another node) but either will work.
   When buffer is false, the value is streamed to the manager, which for values that take a
   large amount of memory may be advantageous. */
err = dragon_fli_send_bytes(&value_sendh, num_value_bytes, value_bytes, VALUE_HINT, buffer, timeout);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not send the value.");

/* Finally, the request is finalized once it is complete. */
err = dragon_ddict_finalize_request(&req);
if (err != DRAGON_SUCCESS)
    throw DragonError(err, "Could not finalize the DDict put request.");

This code captures attaching to a DDict and creating a request and using it to store a key/value pair in the DDict. When a DDict operation can be completed with one API call, a request object is not created and required as in calling something like the length function which returns the number of keys stored in the DDict. However, when the operation extends across multiple API calls, a request object ties the calls together into one operation.

For sample code for other operations, you can consult the C code used to implement the C++ DDict client in dictionary.hpp. Better yet, use the C++ DDict client!

C API Reference

Structures

struct dragonDDictKey_t

#include <ddict.h>

A Key Structure.

When retrieving keys from a dictionary, this structure is used to provide a pointer to the serialized data and its length.

struct dragonDDictDescr_t

#include <ddict.h>

An opaque DDict descriptor object.

When a using a distributed dictionary from C, this serves as the handle to the dictionary. Attaching to a distributed dictionary intializes a dragonDDictDescr_t.

struct dragonDDictRequestDescr_t

#include <ddict.h>

An opaque handle to a request object.

This is used when doing any interaction with the distributed dictionary. Operations on the dictionary may involve multiple call, such as a put or a get operation, and this request descriptor helps maintain the state of the request and response to this request.

Lifecycle Management

dragonError_t dragon_ddict_serialize(const dragonDDictDescr_t *obj, const char **serial)

Serialize a DDict object for sharing with another process. When sharing a DDict object with another process you may use this function to create a shareable serialized descriptor. This creates an ASCII compliant string that may be shared. This call mallocs the space occupied by the serial pointer that is returned.

NOTE: You must call free to free the serialized descriptor after calling this function to free the space allocated by this function once you are done with the serialized descriptor.

Parameters:

• obj – is a valid DDict descriptor that has previously been created or attached.

• serial – is a serialized descriptor that will be initialized with the correct byte count and serialized bytes for so it can be passed to another process.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_attach(const char *serial, dragonDDictDescr_t *obj, timespec_t *timeout)

Attach to a DDict object.

Calling this attaches to a DDict object by using a serialized DDict descriptor that was passed to this process. The serialized DDict descriptor must have been created using the dragon_ddict_serialize function.

Parameters:

• serial – is a pointer to the serialized DDict descriptor.

• obj – is a pointer to a DDict descriptor that will be initialized by this call.

• timeout – is a timeout in every request and operation.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_detach(dragonDDictDescr_t *obj)

Detach from a DDict object.

All internal, process local resources are freed by making this call.

Parameters:

obj – is a descriptor and opaque handle to the DDict object.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

Request Management

dragonError_t dragon_ddict_create_request(const dragonDDictDescr_t *obj, dragonDDictRequestDescr_t *req)

Create a request descriptor for making a request to the distributed dictionary.

All internal state of the connection to the distributed dictionary is maintained by this request object. Details of particular operations may be stored in the private data structure for this object but are not accessible directly by the user. The user uses associated API calls that use this object. Not every request requires a request object. Requests that are accomplished by one API call are not dependent on a request object. When a request object is required it will be evident in the API.

Parameters:

• obj – is a valid DDict descriptor that has previously been created or attached.

• req – is a pointer to a request object that will be initialized by this call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_request_value_sendh(const dragonDDictRequestDescr_t *req_descr, dragonFLISendHandleDescr_t *sendh)

Return the DDict request send handle.

Use of this means that you are reaching inside a request object and obtaining the send handle for object serialization. This function is low-level and not of use unless you are writing a DDict client like the C++ client. This provides access to a send handle where a DDict value should be sent.

Parameters:

• req_descr – is a valid DDict request descriptor.

• sendh – is a pointer to a send handle. This send handle will only be valid while the request is in process and error free. Once there is an error or the request is complete, the send handle should not be referenced.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_request_key_sendh(const dragonDDictRequestDescr_t *req_descr, dragonFLISendHandleDescr_t *sendh)

Return the DDict request key send handle.

Use of this means that you are reaching inside a request object and obtaining the key send handle for key object serialization. This function is low-level and not of use unless you are writing a DDict client like the C++ client. This provides access to a send handle where a DDict key should be sent.

Parameters:

• req_descr – is a valid DDict request descriptor.

• sendh – is a pointer to a send handle. This send handle will only be valid while the request is in process and error free. Once there is an error or the request is complete, the send handle should not be referenced.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_request_recvh(const dragonDDictRequestDescr_t *req_descr, dragonFLIRecvHandleDescr_t *recvh)

Return the DDict request receive handle.

Use of this means that you are reaching inside a request object and obtaining the receive handle for object deserialization. This function is low-level and not of use unless you are writing a DDict client like the C++ client.

Parameters:

• req_descr – is a valid DDict request descriptor.

• recvh – is a pointer to a receive handle. This recv handle will only be valid while the request is in process and error free. Once there is an error or the request is complete, the pointer should not be referenced.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_free_required(const dragonDDictRequestDescr_t *req, bool *free_mem)

Return the value free memory variable from the request.

In some cases, we don’t want to free the memory after receiving as other processes are still using it. This API provides a guide to user if the memory should be cleaned up or not.

Parameters:

• req – is an initialized request object.

• free_mem – is a boolean flag that determines if the memory should be released.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_finalize_request(const dragonDDictRequestDescr_t *req)

This finalizes a request by completing any operation that was still pending for this request. When a request object is required it will be indicated in the API.

Parameters:

req – is a valid request object.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

Send/Recv Functions

These functions are depracated. Instead, please use the dragon_ddict_request_value_sendh, dragon_ddict_request_key_sendh, and the dragon_ddict_request_recvh methods functions along with FLI send and receive functions for sending and receiving data.

dragonError_t dragon_ddict_write_bytes(const dragonDDictRequestDescr_t *req, size_t num_bytes, uint8_t *bytes)

Writes either key or value data to the distributed dictionary.

The client may call this multiple times to put the parts of a key or value to the distributed dictionary. Internally, all key writes are buffered so the key can then be used to determine where the data is to be placed in the distributed dictionary. All value writes are streamed immediately to the distributed dictionary. All Key writes must come first for a request, followed by value writes. Key writes are terminated by an API call to the actual operation that requires a key as part of its request. Value writes, for a put, follow the API call for the operation until the request is finalized. All clients use the same selection algorithm for data placement so data put by one client can be found by all other clients.

Parameters:

• req – is an initialized request object.

• num_bytes – is the number of bytes on this put request. There may be additional bytes sent using this call as well.

• bytes – is a pointer to a byte array (continguous bytes) with num_bytes size.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_read_bytes(const dragonDDictRequestDescr_t *req, size_t requested_size, size_t *received_size, uint8_t **bytes)

Waits to receive streamed data from a distributed dictionary manager.

If all data has been read, then DRAGON_EOT will be returned as the return code. This should be called after a get operation has been performed by calling dragon_ddict_get. Note that before calling the get operation, the key should have been written using the dragon_ddict_write_bytes operation.

Parameters:

• req – is a valid request object that has been used to initiate reading data from the distributed dictionary. For example, a key should have been written and dragon_ddict_get should have been called.

• requested_size – is the number of requested bytes. The actual size will be equal to or less than the requested_size.

• received_size – is a pointer to the number of bytes that have been read on the call (assuming DRAGON_SUCCESS was returned).

• bytes – is a pointer pointer that will be initialized to the bytes that were read. The space is malloc’ed and should be freed by the user once the data has been processed.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_read_bytes_into(const dragonDDictRequestDescr_t *req, size_t requested_size, size_t *received_size, uint8_t *bytes)

Receive streamed data.

Calling this waits to receive streamed data from a distributed dictionary manager. If all data has been read, then DRAGON_EOT will be returned as the return code. This should be called after a get operation has been performed by calling dragon_ddict_get. Note that before calling the get operation, the key should have been written using the dragon_ddict_write_bytes operation.

Parameters:

• req – is a valid request object that has been used to initiate reading data from the distributed dictionary. For example, a key should have been written and dragon_ddict_get should have been called.

• requested_size – is the number of requested bytes. The actual size will be equal to or less than the requested_size.

• received_size – is a pointer to the number of bytes that have been read on the call (assuming DRAGON_SUCCESS was returned).

• bytes – is a pointer to valid space where the data should be placed. It must be at least requested_size in size.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_read_mem(const dragonDDictRequestDescr_t *req, dragonMemoryDescr_t *mem)

Receive streamed data.

Calling this waits to receive streamed data from a distributed dictionary manager but instead of copying it into malloced memory, returns the underlying managed memory object to the user. If all data has been read, then DRAGON_EOT will be returned as the return code. This should be called after a get operation has been performed by calling dragon_ddict_get. Note that before calling the get operation, the key should have been written using the dragon_ddict_write_bytes operation.

Parameters:

• req – is a valid request object that has been used to initiate reading data from the distributed dictionary. For example, a key should have been written and dragon_ddict_get should have been called.

• mem – is a managed memory allocation containing the packet of streamed data. The size of the memory allocation is available as part of the object and the managed memory API provides a means to get a pointer to the data. The managed memory allocation should be freed using the managed memory API once it is no longer needed.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

Dictionary Operations

dragonError_t dragon_ddict_which_manager(const dragonDDictRequestDescr_t *req_descr, uint64_t *manager_id)

Find which manager the written key would be sent to.

The key is not sent, but the manager id is returned that would house this key and an associated value. To call this method you must create a request, write a key, and then call this method. After calling this function you may do a ddict operation or you may finalize the request.

Parameters:

• req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

• manager_id – is a pointer to a variable that will hold the identifier of the manager to which this key would be sent should it be sent. The key is not sent.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_contains(const dragonDDictRequestDescr_t *req)

Calling this tells the ddict client to take the key already written via the dragon_ddict_write_bytes call(s) to be posted to the correct manager and wait for a response.

Parameters:

req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_get(const dragonDDictRequestDescr_t *req)

Calling this tells the ddict client to take the key already written via the dragon_ddict_write_bytes call(s) to be posted to the correct manager and wait for a response.

Parameters:

req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_put(const dragonDDictRequestDescr_t *req)

Calling this tells the ddict client to take the key already written via the dragon_ddict_write_bytes call(s) to be posted to the correct manager. The key must be written before calling put. All writes to this request, following the call to this function are written to the correct manager as the value for the put on the distributed dictionary manager.

Parameters:

req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_pput(const dragonDDictRequestDescr_t *req)

Calling this tells the ddict client to take the key already written via the dragon_ddict_write_bytes call(s) to be posted to the correct manager. The key must be written before calling put. All writes to this request, following the call to this function are written to the correct manager as the value for the put on the distributed dictionary manager.

Parameters:

req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_length(const dragonDDictDescr_t *dd_descr, uint64_t *length)

Calling this retrieves the number of keys in the distributed dictionary.

Parameters:

• dd_descr – is a serialized descriptor of the dictionary.

• length – is a pointer to an uint64_t that will hold the number of keys up return from the function call when DRAGON_SUCCESS is returned as the return code. Otherwise, the field will not be initialized.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_clear(const dragonDDictDescr_t *dd_descr)

This removes all key/value pairs from the distributed dictionary.

Parameters:

dd_descr – is a serialized descriptor of the dictionary.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_pop(const dragonDDictRequestDescr_t *req)

Calling this tells the ddict client to take the key already written via the dragon_ddict_write_bytes call(s) to be posted to the correct manager. The key must be written before calling put. All writes to this request, following the call to this function are written to the correct manager as the value for the put on the distributed dictionary manager.

Parameters:

req – is a valid created request object. It must have already had a key written to it via the dragon_ddict_write_bytes call.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_keys(const dragonDDictDescr_t *dd_descr, dragonDDictKey_t ***keys, size_t *num_keys)

Calling this tells the ddict client to send request to all managers to get all keys.

Parameters:

• dd_descr – is a serialized descriptor of the dictionary.

• keys – is a pointer to an byte array that store an array of keys following the response from managers.

• num_keys – is a pointer to the number of keys received.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_stats(const dragonDDictDescr_t *dd_descr)

Calling this to get the stats of all managers.

Parameters:

dd_descr – is a serialized descriptor of the dictionary.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_checkpoint(const dragonDDictDescr_t *dd_descr)

Calling this to move to the next checkpoint.

Parameters:

dd_descr – is a valid DDict descriptor that has previously been created or attached.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_rollback(const dragonDDictDescr_t *dd_descr)

Calling this to rollback to one earlier checkpoint.

Parameters:

dd_descr – is a valid DDict descriptor that has previously been created or attached.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_sync_to_newest_checkpoint(const dragonDDictDescr_t *dd_descr)

Calling this to get latest checkpoints from all managers and set current checkpoint to the newest among all managers.

Parameters:

dd_descr – is a valid DDict descriptor that has previously been created or attached.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_checkpoint_id(const dragonDDictDescr_t *dd_descr, uint64_t *chkpt_id)

Calling this to get client’s current checkpoint ID.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• chkpt_id – is a pointer to the checkpoint ID

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_set_checkpoint_id(const dragonDDictDescr_t *dd_descr, uint64_t chkpt_id)

Set the client’s current checkpoint ID.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• chkpt_id – a non-negative checkpoint ID.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_local_manager(const dragonDDictDescr_t *dd_descr, uint64_t *local_manager_id)

Get client’s local manager if there is one.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• local_manager_id – is a pointer to the local manager ID.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_main_manager(const dragonDDictDescr_t *dd_descr, uint64_t *main_manager)

Get client’s main manager.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• main_manager – is a pointer to the main manager ID.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_manager(const dragonDDictDescr_t *dd_descr, dragonDDictDescr_t *dd_new_descr, uint64_t id)

Create a copy of original client object and assign a chosen manager. The client will only interact with the chosen manager.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• dd_new_descr – is a DDict descriptor that holds a copy of client from dd_descr with the chosen manager.

• id – is a pointer to the chosen manager ID.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_empty_managers(const dragonDDictDescr_t *dd_descr, uint64_t **manager_ids, size_t *num_empty_managers)

Return a list of empty managers during restart.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• manager_ids – is the address of an array of empty managers’ IDs.

• num_empty_managers – is the number of empty managers.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_local_managers(const dragonDDictDescr_t *dd_descr, uint64_t **local_manager_ids, size_t *num_local_managers)

Return local manager IDs.

Parameters:

• dd_descr – is a valid DDict descriptor that has previously been created or attached.

• local_manager_ids – is an array of local manager IDs.

• num_local_managers – is the number of local managers.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_local_keys(const dragonDDictDescr_t *dd_descr, dragonDDictKey_t ***local_keys, size_t *num_local_keys)

Return the DDict keys that are located on the same node as the caller.

Parameters:

• dd_descr – is a serialized descriptor of the dictionary.

• keys – is a pointer to an byte array that store an array of local keys following the response from local managers.

• num_keys – is a pointer to the number of local keys received.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_synchronize(const char **serialized_ddicts, const size_t num_serialized_ddicts, timespec_t *timeout)

Calling this tells the ddict client to synchronize dictionaries.

Parameters:

• serialized_ddicts – is a list of serialized descriptor of the dictionaries to synchronize.

• num_serialized_ddicts – is the number of dictionaries to synchronize.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.

dragonError_t dragon_ddict_clone(const dragonDDictDescr_t *dd_descr, const char **serialized_ddicts, const size_t num_serialized_ddicts)

Calling this tells the ddict client to clone dictionaries.

Parameters:

• dd_descr – is a serialized descriptor of the source dictionary.

• serialized_ddicts – is a list of serialized descriptor of the destination dictionaries.

• num_serialized_ddicts – is the number of destination dictionaries.

Returns:

DRAGON_SUCCESS or a return code to indicate what problem occurred.