Creating and Using a Queue in Dragon Native

The Dragon Native Queue implementation is Dragon’s specialized implementation of a Queue, working in both single and multi node settings. It is interoperable in all supported languages. The API is similar to Python’s Multiprocessing.Queue in many ways, but has a number of extentions and simplifications. In particular, the queue can be initialized as joinable, which allows to join on the completion of an item.

Using a Queue with Python

TBD

Using a Queue with C

To use a Dragon queue it’s as simple as creating it and then sending data. Managed queues incur some startup cost, but no performance cost while using and they make creation and disposal extremely easy. Here is sample code for creating a queue and sending some binary data using it.

Listing 28 Creating a Queue using C and the stream interface.
 1#include <dragon/global_types.h>
 2#include <dragon/queue.h>
 3
 4int main(int argc, char* argv[]) {
 5  dragonError_t err;
 6  dragonQueueDescr_t queue;
 7  dragonQueuePutStream_t put_str;
 8  uint64_t my_data[VERY_BIG]; /* assumed initialized */
 9  int val = 100;
10
11  err = dragon_managed_queue_create("my_unique_queue_name", 100, true, NULL, NULL, NULL, &queue);
12
13  if (err != DRAGON_SUCCESS) {
14    fprintf(stderr, "The managed queue could not be created. Error Code=%lu\n", err);
15    return 1;
16  }
17
18  err = dragon_queue_put_open(&queue, &put_str);
19  if (err != DRAGON_SUCCESS) {
20    fprintf(stderr, "The queue putstream could not be opened. Error Code=%lu\n", err);
21    return 1;
22  }
23
24  /* some work is done and results need to be shared */
25
26  err = dragon_queue_put_write(&put_str, my_data, VERY_BIG*sizeof(uint64_t), NULL);
27  if (err != DRAGON_SUCCESS) {
28    fprintf(stderr, "The queue putstream write failed. Error Code=%lu\n", err);
29    return 1;
30  }
31
32  err = dragon_queue_put_write(&put_str, &val, sizeof(int), NULL);
33  if (err != DRAGON_SUCCESS) {
34    fprintf(stderr, "The queue putstream write failed. Error Code=%lu\n", err);
35    return 1;
36  }
37
38  return 0;
39}

Using a Queue with C++

Both reading and writing to a Queue in C++ is handled via the std::streambuf protocol. An istream can be constructed in a similar way to read data from a stream buffer. The left and right shift operators can be defined for user-defined data types so arbitrarily complex data can be read and written in this fashion.

Listing 29 Creating a C++ Stream over a Queue
 1std::string greeting = "Hello World";
 2int id = 42;
 3char action = 'a';
 4
 5DragonManagedQueueBuf strm_buf("holly", 1024, 1024, true, NULL, NULL);
 6std::ostream out(strm_buf, 1024);
 7
 8out << greeting << id << action << std::endl;
 9
10out.close()

Using a Queue with Fortran

TBD