Rapid prototyping and teaching ZeroMQ in C++ with biicode.

ZMQ or ZeroMQ is one of the coolest open source libraries out there. It fills a real necessity in a new, brilliant way. It is very active, has bindings to many languages… that’s why it is so popular.

ZeroMQ is already in biicode

I was a professor in university, teaching C/C++, software engineering, distributed systems (plain old sockets, SOAP, web…), and I really enjoyed keeping my classes up to date and teaching last trends and technologies. If I was still a teacher, I would definitely teach some sessions using ZeroMQ.

But I suffered many problems when using third party source code in the classroom, and not only there, but also in my real projects. In the classroom, most of the time could be spent configuring and installing external dependencies, and the problem would became worse when students tried to repeat the process at home or in their laptops, maybe in another OS. This is one of the reasons we started the biicode project.

Today, if you try to build the basic C++ client-server example that ZeroMQ provides in their site, you might encounter some problems. You have to guess that the C++ binding is not in the library, instead, it’s inside another repo (zmqcpp). I had to google it myself to find it. You have to get, configure and build the library, then setup your own project to use it.

The question is: Given some example source code snippets that use zmq, can anyone, even unexperienced developers build them quickly and easily, even without writing a single line of configuration, in any of the major OS? We think that it can be done, in a few simple steps.

A simple client-server example with ZeroMQ

We have the source code we want to build:

hwclient.cpp

hwserver.cpp

Note that these are exactly the examples hosted in http://zeromq.org/, except for the #include directives.

Let’s build the examples

Building this example is straightforward with biicode. If you haven’t installed it yet, you might want to try the C++ getting started first.

The first step is to create a project (e.g. myzmqproject) and a block (a folder inside your project) named myuser/zmqexamples. You don’t need to be actually registered, you can type exactly “myuser”, or any other name.

Copy the files above, inside the folder blocks/myuser/zmqexamples

This command checks your #includes and tries to find a compatible zmq version in biicode, and then it will automatically generate a requirements.bii file, with the contents:

This file defines your block’s requirements, in this case, the block required is www.biicode.com/diego/zmqcpp, version number 0, which holds the ZeroMQ C++ binding. Of course, you can also write this file manually instead of relying on the finder, and this file can also be used to resolve dependencies conflicts. In turn, this zmqcpp biicode block has a dependency to version 0 of libzmq in http://www.biicode.com/diego/libzmq

You might want to check what biicode has retrieved into your project “deps” folder. There you can find the zmqcpp binding, as well as the libzmq library. Interestingly for the library, it has only retrieved the necessary files but skipped, for example, unnecessary test code. This is an advantage of biicode, and I will show interesting use cases for this feature in forthcoming posts.

Building and running the project is also fairly simple:

We use the de-facto standard CMake build system,  you can pass also other generators to the -G option. Building the project will create 2 executables inside the “bin” folder.

Run first  your server, then your client, you can find both inside the bin folder

How was libzmq uploaded?

This functionality is great, but before taking advantage of it, it is necessary to upload the necessary code to biicode. The code that has been retrieved in this example can be found in :

Uploading code is absolutely trivial for simple code (e.g. the zmqcpp C++ binding), but can take some more time to do for configuration-intensive projects. I will summarize here the steps I did to upload libzmq.

The first step is usually to create a fork of the origin project, to manage changes and versions and keep synchronized with the original project.  You can check the fork of libzmq here: https://github.com/drodri/libzmq, and you can compare with the upstream repo to see the differences involved.

Usually the required changes are just limited to the CMakeLists.txt file. Biicode provides a variable that can be used to leave the original CMakeLists.txt unaltered in functionality:

The unique major changes ZeroMQ required in the main CMakeLists.txt are:

This code removes (do not build), executables not intended to be build in that platform. Note the check for BII_BLOCK_EXES (this is a variable defined by biicode) to exist because when we depend on this code, executables might not exist, just the library.

Also the  build directory has to be included, as config.h is generated there, and some definitions have to be added in case of building with MSVC.

Finally, the tests subdirectory is added only if it exists (it might not when this block is being reused).

You can check the file tests/CMakeLists.txt to see required changes, that are very simple in this case.

This CMakeLists.txt changes are not only for managing ZeroMQ as a dependency, but also for full development. At any time you can “bii open” a block you depend on, and you will get that block inside your blocks folder, with full source code, and all executables including tests.

In this ocassion, I had to manually add some dependencies. This can be done in the file “bii/dependencies.bii”:

Biicode parses and detect dependencies via #includes and automatically detecting implementations (a header implemented in a source file), but CMakeLists.txt are not parsed yet. If we want auxiliary cmake and config files to be detected as a dependency of CMakeLists.txt, and always be retrieved when we depend on libzmq, we just add the file names in that file.

Ongoing work

This example has been tested in Mac, Win (with VS 2012) and Ubuntu 14, but further testing might be necessary, and also checking other compilers as MinGW. Included unit tests pass also except a single test in VS2012, which I am investigating.

Libsodium dependency is not handled now. I have disabled the default use of TweetNaCL, so it is not really required in Linux. I am working in uploading Libsodium to biicode, so it can also be managed easily. Also OpenPGM is not available yet.

It is easy to set-up a travis-ci  to automatically pull last github version, download biicode (and a more modern version of Cmake, as the one included is a little bit outdated), build the project, run tests, and finally publish to biicode, so everybody else can easily upgrade to the most recent version if they want to. I will post about all these things soon, stay tuned.

ZeroMQ books by Pieter Hintjens

Conclusion

I have shown how easy is to build C++ ZeroMQ examples with the use of biicode, as it manages to retrieve and configure it in your project, and only requiring very few changes to the CMakeLists.txt while also avoiding any lock-in.

But biicode is a little bit more than a plain dependency manager. You can develop in one of your projects a server, a client and some serialization classes, and let’s say that your server has some functionality only available in Linux. You don’t have to worry about creating different projects/repositories/libraries for client, server and serialization. A windows application can just directly #include the client and it will be able to build correctly very easily.

 

I will show it in next posts, meanwhile I would love to hear your feedback, please try this example and tell me!

Stay tuned


Related Posts