The Distributed Dictionary C++ Client

Dragon Distributed Dictionary supports both C and C++ user APIs, allowing both C and C++ clients to interact with the dictionary like a map. The internal messaging of both clients is shared with the Python client. In fact, a Python DDict manager handles the interaction from C and C++ clients.

A user must create the DDict in Python, serialize the DDict, and pass the serialized descriptor to C++ code which can then attach to it. See dragon.native.Popen for the means to start a C++ program within Dragon. Please note that creating multiple client processes inside a single C/C++ program using fork() is not supported in Dragon Distributed Dictionary. You must create client processes from the Python API and attach to the dictionary in the C++ program.

When using the C++ DDict, keys and values must implement the Serializable Interface which makes writing serialization/deserialization code for objects in C++ easily done by using the FLI send and receive operations. See The FLI API. Some example Serializables are provided and can be found in the header file and the implementation of Serializable . The template provided here is not mean to be instantiated, but provides an outline of what you would want to write. Documentation for Serializables can be found here.

Listing 45 A Serializable Example
 1 * Use this documentation as an outline for writing your own subclasses of Serializable. DO NOT
 2 * instantiate this class and expect it to do anything.
 3 */
 4template<class Type>
 5class DerivedSerializable: public Serializable {
 6public:
 7    /**
 8     * @brief Constructor for DerivedSerializable
 9     *
10     * Write your own subclass of Serializable and a constructor for it. You may pass multiple
11     * arguments. The constructor is for your own program's use and is not used by Dragon.
12     *
13     * @param x A Type of object to wrap.
14     */
15    DerivedSerializable(Type obj) {
16        throw DragonError(DRAGON_INVALID_OPERATION, "This class should not be instantiated. Inherit from Serializable instead.");
17    }
18
19    /**
20     * @brief Serialize a C++ object.
21     *
22     * This method should be overridden in the implementing derived subclass. It should
23     * write the bytes of the serialized object to the FLI send handle using the
24     * FLI send_bytes interface defined in fli.h. The arg argument should simply be passed
25     * through from the serialize function call to the FLI API call for sending bytes. The
26     * buffer argument should typically just be passed through to FLI send_byte operations. It will be
27     * determined by the context in which serialize is called. For DDict keys, the writes
28     * are buffered. For DDict values, the writes are not. But in some cases you may wish to buffer
29     * serialized objects and specify true to cause the writes to be consilidated into one
30     * network communication.
31     *
32     * @param sendh is an FLI send handle used for writing
33     * @param arg is a provided hint. You may override this in some circumstances to create your own hint.
34     * @param buffer is provided or you can override. A value of true on FLI sends will cause written data to be
35     * consolidated into one network transfer.
36     * @param timeout A value of nullptr will wait forever to serialize/transfer data. If value of {0,0} will
37     * try once. Otherwise, the timeout specifies how long to wait for the serialization/transfer to be completed.
38     */
39    virtual void serialize(dragonFLISendHandleDescr_t* sendh, uint64_t arg, const bool buffer, const timespec_t* timeout) const {
40        throw DragonError(DRAGON_INVALID_OPERATION, "This class should not be instantiated. Inherit from Serializable instead.");
41    }
42
43    /**
44     * @brief Deserialize a serialized C++ object.
45     *
46     * This method should be written in the implementing derived subclass and should
47     * return the derived subtype of Serializable. It may throw a
48     * DragonError exception when a byte stream is not deserializable. It
49     * should throw a EmptyError when EOT is received while reading bytes
50     * as it deserializes a value. Assuming that the deserialization
51     * succeeds, the deserialize method should return a deserialized value
52     * of the derived type after it has read the serialized object's
53     * bytes. The arg argument should be passed through to the FLI API
54     * calls for reading bytes or pool memory and will be set according to
55     * what was sent when it was serialized. This function relies on NRVO
56     * in C++17 and above. This optimization means that the object is
57     * initialized in the caller's space so when the value is returned, it
58     * is already in-place. This means we can return a value without
59     * making an extra copy.
60     *
61     * @param recvh An FLI receive handle. The receive handle is used to read
62     * the data of the object. You can read the data using any FLI recvh methods.
63     * @param arg A pointer to a variable to hold the received arg value.
64     * @param timeout A value of nullptr will wait forever. A value of {0,0} will
65     * try once. Otherwise, wait for the specified time to receive the object.
66     *
67     * @returns A DerivedSerializable instance.
68     */
69    static DerivedSerializable<Type> deserialize(dragonFLIRecvHandleDescr_t* recvh, uint64_t* arg, const timespec_t* timeout) {
70        throw DragonError(DRAGON_INVALID_OPERATION, "This class should not be instantiated. Inherit from Serializable instead.");
71    }
72
73    /**
74     * @brief Get the wrapped value for the object.
75     *
76     * This method may be named whatever you like. It is not part of the Serializable class. And you may
77     * define more than one accessor method like this to retrieve parts of your object. You will need
78     * something like this to access your deserialized object in your program. The wrapped value (i.e.
79     * Type) may also be more than one value which would then be passed to the constructor and you

Listing 46 demonstrates how a DDict can be attached and values can be stored and retrieved to/from it. Other example code can be found in test code for the C++ client .

Listing 46 Storing a key/value pair in a DDict using C++
 1void my_fun(const char * ddict_ser) {
 2   uint64_t manager_id;
 3   SerializableInt x(6); // key
 4   SerializableInt y(42); // value
 5   // This demonatrates attaching to a serialized DDict. The ddict_ser
 6   // would be provided to the program via a command-line argument or some
 7   // other means (like a Queue).
 8   DDict<SerializableInt, SerializableInt> dd(ddict_ser, &TIMEOUT);
 9   manager_id = dd.which_manager(x); // Sample code, not needed here.
10   dd[x] = y; // Stores the key x and maps it to y in the DDict dd.
11   SerializableInt z = dd[x]; //Looks up key x to find its value.
12   assert (z.getVal() == 42);
13}

C++ API Reference

template<class SerializableKey, class SerializableValue>
class DDict

A Dragon Distributed Dictionary Client.

This class provides the functionality to attach to and interact with a DDict. The DDict should be created from Python code. After it is serialized, the serialized descriptor can then be passed to a C++ program/code via some means (e.g. a command-line argument). Then it can be attached to from this C++ client. To fully interact with a DDict from C++, Serializable keys and values must be defined. See the Serializable abtract class for details on implementing one or more of those classes.

Public Functions

inline DDict(dragonDDictDescr_t *cDict)

A Constructor for DDict.

This constructor takes a C DDict descriptor as an argument and constructs a C++ DDict over the C descriptor. Not likely used, it is provided since it is a way of moving to C++ from C. Using this constructor will imply that operations on the DDict will not timeout.

Parameters:

cDict – A C DDict descriptor.

inline DDict(dragonDDictDescr_t *cDict, const timespec_t *timeout)

A Constructor for DDict with a timeout.

This constructor takes a C DDict descriptor as an argument and constructs a C++ DDict over the C descriptor. Not likely used, it is provided since it is a way of moving to C++ from C.

Parameters:

  • cDict – A C DDict descriptor.

  • timeout – A default timeout to be applied to operations.

inline DDict(const char *serialized_dict, const timespec_t *timeout)

Attach to a serialized DDict.

This constructor will attach to a DDict using its serialized descriptor as passed to it from Python or another C++ client. A serialized DDict is a valid string variable and can be passed as a string to other instances that want to access the DDict and store values to it from C++.

Parameters:

  • serialized_dict – is a base64 encoded serialized descriptor.

  • timeout – is a timeout to apply to all operations on the DDict.

inline ~DDict()

The DDict Destructor.

The destructor is applied automatically when the object is declared on the stack and gets called when delete is called on a heap allocated DDict.

inline const char *serialize()

Return a serialized handle to the DDict.

Return a Base64 encoded string that can be shared with other clients that want to attach to the same dictionary.

Returns:

A DDict serialized handle that can be handed off to other clients.

inline uint64_t size()

Get the number of key/value pairs in the DDict.

This method cannot be counted on to return an exact number of key/value pairs. Due to the the nature of a parallel distributed dictionary the value may have changed by the time this method returns to the user.

Returns:

The number of key/value pairs at a moment in time.

inline void clear()

Clear the DDict.

Removes all key/value pairs as it is executed from all managers of the DDict. This does not mean that other clients couldn’t immediately start adding in key/value pairs again.

inline KeyRef operator[](SerializableKey &key)

Get a KeyRef.

This method constructs a key reference which can then be used in either a lookup of a value in the DDict or or used to set a key/value pair in the DDict.

inline void pput(SerializableKey &key, SerializableValue &value)

Put a persistent key/value pair to the DDict.

Persistent key/value pairs persist across checkpoints. These pairs remain in the DDict until explicitly deleted.

inline bool contains(SerializableKey &key)

Check that the DDict contains the key.

Returns true if the key is in the DDict. They key provided must be serializable and the binary equivalent of a key in the DDict.

Returns:

A bool to indicate membership.

inline uint64_t which_manager(SerializableKey &key)

Return which manager will hold this key should it be stored in the DDict.

This does not check to see if the key is already in the DDict. It will tell you which manager will be used to store this key. The manager_id will be 0 <= manager_id < num_managers for the DDict.

Parameters:

key – A serializable key for the DDict

Returns:

the manager_id to be used for this key

inline SerializableValue erase(SerializableKey &key)

Remove a key from the DDict.

Delete a key/value pair from the DDict. If the key does not exist a DragonError is raised.

Parameters:

A – DDict Key

Throws:

DragonError – if the key does not exist.

inline std::vector<SerializableKey> keys()

Return the keys of the DDict.

This returns a vector of keys of the DDict. The RVO (Return Value Optimization) that C++ provides you can assign this result to a stack allocated vector and it will be initialized in place. Note that you get all the keys of the DDict at once.

Returns:

A vector of all DDict keys

inline void checkpoint()

Increment the client checkpoint id.

This is a local only operation that increments the client’s checkpoint id. Subsequent operations will work with the next checkpoint in any manager’s working set.

inline void rollback()

Decrement the client checkpoint id.

This is a local only operation that decrements the client’s checkpoint id. Subsequent operations will work with the previous checkpoint in any manager’s working set.

inline void sync_to_newest_checkpoint()

Go to the latest checkpoint.

This will query all managers to find the latest checkpoint id and then set this client’s checkpoint id to it.

inline uint64_t checkpoint_id()

Retrieve the checkpoint id of this client.

Returns the client’s checkpoint id.

Returns:

The client’s current checkpoint id.

inline uint64_t set_checkpoint_id(uint64_t chkpt_id)

Set the client’s checkpoint id.

This allows the checkpoint id of the client to be set to a particular value. The chosen checkpoint id if older than the working set will be rejected on DDict operations. If newer than any checkpoint in the working set, the result will be to advance the working set in affected managers.

Returns:

The checkpoint id that was set.

inline uint64_t local_manager()

Get a manager id for a local manager.

Returns the manager id of a local manager. if none exist, a DragonError will be thrown. If more than one exists, one will be chosen at random.

Returns:

0 <= manager_id < number of managers.

inline uint64_t main_manager()

Return a manager id.

Calling this will always return a manager id that can be used in certain operations. It will be a local manager if one exists. If the node has no local managers, then it will be a random manager from a different node.

Returns:

0 <= manager_id < number of managers.

inline DDict manager(uint64_t manager_id)

Get a manager directed copy of the DDict.

Calling this will return a new DDict reference that targets only the given manager. All put and get operations will be directed at this manager only. All other operations that interact with the DDict will also only query or modify the given manager.

Parameters:

manager_id – The manager id of the chosen manager.

Returns:

A new DDict reference that targets only the chosen manager.

inline std::vector<uint64_t> empty_managers()

Get the empty managers of the DDict.

Calling this returns a vector of all empty manager ids in this DDict.

Returns:

A vector of integer manager ids.

inline std::vector<uint64_t> local_managers()

Get the local managers of the DDict.

Calling this returns a vector of all local manager ids in this DDict.

Returns:

A vector of integer manager ids.

inline std::vector<SerializableKey> local_keys()

Return a vector of all keys stored on this node.

Calling this queries only local managers to find the keys that are stored on this node. Using this can help optimize code around minimizing data movement across the network.

Returns:

A vector of all local keys.

inline void clone(std::vector<std::string> &serialized_ddicts)

Copy the current DDict to one or more existing DDicts.

This clones the current DDict to each of the serialized descriptors that is passed to it. It will not create new DDicts. It will clone the contents of this DDict to the DDicts passed in this vector.

Parameters:

serialized_ddicts – A vector of DDicts that will get cloned into from this one.

Public Static Functions

static inline void synchronize(std::vector<std::string> &serialized_ddicts)

Duplicate one or more DDicts.

Calling this assumes that all DDicts in the vector are exact duplicates of each other. Some managers within the set of duplicates may be empty due to a previous error and a restart. If this occurs, then this synchronize method will use non-empty managers to re-populate the empty managers in these parallel dictionaries.

Parameters:

serialized_ddicts – A list of the serialized descriptors of the parallel dictionaries. Each parallel dictionary must have the same number of managers and same data across all of them.