Meta-configuration with CMake of C/C++ projects

This blog post is out dated.

If you’re interested on knowing the updates, check our docs for more information.

Project configuration and setup

Project configuration in software development is the process of setting up how your code will be built, managed and/or run. It varies depending on the programming language and tools, as the IDE, used. For instance, when programming in C/C++, the project setup usually comes with defining which artifacts (executables, libraries) will be built from certain source code files, with given compiling and linking options. In Microsoft Visual, this can be mostly done with wizards, menu and contextual commands; e.g. you can configure your directories containing external libraries in a dialog (Project Properties).

 

Meta-configuration with CMake

C/C++ project meta-configuration with CMake

Other languages, especially interpreted ones, as node or python, typically require much less project configuration or setup. In languages, such as Java, typical project setup can be done with IDEs like Eclipse or Netbeans, nevertheless, the use of Maven is probably more extended. Maven allows you to define not only how the project has to be built, packaged, executed or tested, but also, to specify dependencies to binaries artifacts (jars), which can be retrieved from external servers.

In the C/C++ ecosystem, the best tool for project configuration is CMake. CMake allows to specify the build of a project in files named CmakeLists.txt with a simple syntax (it is simpler than writing Makefiles). From those files it can generate projects for the most popular IDEs and build systems in different OS. It is a must have tool. It is the de-facto standard in the industry for the C/C++ multiplatform and even for single OS development. We love it. We have used it for a long time in our own projects, and, as professors, we have taught it from the first day in our Software Engineering courses at university since 2005. We obviously began using it in the biicode platform from the very beginning of the company in 2012.

What is the project meta-configuration? Probably you already know something about meta-programming. Meta-programming is a process in which the code you write (as in a C++ template) is the specification or instructions of how the real code will be generated by a system (in the case of C++ templates, the compiler).

In biicode, meta-configuration is the process in which the project setup is done (most of the time automatically) by collecting information about the project and user intentions from different origins. For example, the source code itself is a great source of information that can be exploited for this purpose.

Let’s see how it works with the well-known example “Hello World”:

Imagine someone starts to write code, and writes the following three (simplified) files for a Hello World application.
Analyzing the source code, it is clear that the user wants to build an executable (from the main function, in green), that includes the file “hello.h” (red). Cross-checking declared and defined symbols in “hello.h” and “hello.cpp” it can be easily deduced that the implementation (blue) of the function hello() is found in the “hello.cpp” file, and so it is required to build the application. Accordingly, a CMakeLists.txt like the following could be automatically generated:

What makes this automatic generation of the CMakeLists.txt file interesting? If a couple of new files are added and included by “main.cpp” or “hello.cpp”, they will automatically be added to the executable. If the user wants to create a new executable, all they have to do is to write a file (with whatever name) with a main() function inside, i.e. the user just has to focus on writing code, practically the rest of the process can be automated. Although real large scale multiplatform projects usually have a complex building process which probably cannot be automatically fully deduced, this approach can be very valuable for students, programming courses, rapid prototyping and testing, etc. And for such large projects, they can also benefit from this approach. Lets see how.

Biicode approach to C++ projects meta-configuration with Cmake

Biicode is a free tool (and it will always be free for OSS) created by C/C++ lovers to manage source code files dependencies in a novel way, facilitating the process of sharing and reusing source code.

In biicode, each project is called a hive, and it has the following simplified layout:

The user’s source code is arranged in blocks, each one in the form username/blockname. These blocks are the units that are published and reused, somewhat (but different, really) similar to repositories in a version control system.

Let’s assume that the username for this example is maya, the hive is called hello, and the blockname in this case (this is not necessary), is also hello. The layout will be:

Generated CMake files

The files inside the cmake folder are automatically generated by biicode. The CMakeLists.txt is created just once, so the user can modify its contents if necessary.

It includes two other files, also generated by biicode. These two files are overwritten everytime the bii tool is called and there are changes in the project. The first one, bii_vars.cmake creates variables that define the targets to build and their properties, but does not create those targets yet. Some variables might be empty, but they are declared here for convenience to the user.

The other bii_targets.cmake file, is the one that actually define the targets to be built, based on the variables declared in the previous file.

This way makes it very simple to define or customize the building process. Users can edit the used variables in the CMakeLists.txt between the two included .cmake files. Note that this approach is not exclusive with the possibility of the user having their own CMakeLists inside their blocks along with his source code, or even other cmake files that could be included from the biicode generated CMakeList.txt one.

Integrating dependencies

Biicode allows a very simple publication and sharing of your source code to the biicode cloud. You (or anyone else in the world) can later very easily reuse code in another project. All you have to do to reuse previously published code is to write in your code a #include directive, in the form “username/block/path/to/file.h”. If biicode does not find such file locally, it will look for it in biicode cloud and retrieve it into your project along with other files (included by or implementing such file). What happens with such source code?

It is retrieved as source code, not as binaries, so it has to be built locally. The source files could be just directly added to the executable, but it seems more intuitive to define a library that contains such files, as they wont be usually edited by the user, and link the executable to such library.

For example, imagine that the user willy has developed a similar application that says goodbye instead of hello, with the following layout:

Willy can very easily publish and share his code, with the command (see HYPERLINK to DOCU):

Maya can very easily reuse that code, writing in her code:

and issuing the command:

The source code files “bye.cpp” and “bye.h” are retrieved and written in the deps folder. Note that willy’s main.cpp file is not retrieved as it is not necessary as indicated by the dependency graph.

The generated CMake bii_vars.cmake file will now also contain:

and the bii_targets.cmake file:

Advanced configuration

Why we call it meta-configuration instead of automated configuration? Because, is the user who specifies the configuration of the project in a higher level. The automatic dependencies detection in C/C++ can sometimes fail, most of the times due to the usage of macros that a normal parser cannot handle. The programmer could also want to specify his own dependencies to do reflection, feature toggling, etc. Custom dependencies can be set in a file called “dependencies.bii” in which the user can add, remove or redefine dependencies between files.

Biicode has also a way to define custom build options. These options are propagated to the affected targets automatically, following the dependency graph. For example, imagine the user willy uses in the bye block some mathematical functions from <math.h>, that requires to link with the “m” library under linux (the typical -lm link flag). How can user willy specify such behaviour? It is true that #pragma directives can sometimes be used to define libraries to link with, but biicode defines a more general approach that can be used not just for linking libraries, but for many building properties.

The user willy can create a file cpp_rules.bii in his block bye that could contain something similar to:

This defines that under Linux, the library ‘m’ has to be added to the current target (in this case, the STATIC library that will be created, named willy_bye. Although it is a static library and it does not make much sense to link it with another library (linking is only actually performed for exes and shared libraries), biicode knows that this setting has to be transitively propagated, that is, executables and shared libraries that link to willy_bye will have in turn to link with m, so biicode adds it to the list of required libraries for such executables and shared libraries.

Editing CMakeLists.txt

Suppose that you need to use a library, let’s say Boost. If such library was already in biicode, it would be enough to just #include it. But boost is still not in biicode (and it is not likely to be in the short term, for many reasons, as being too large or having many complex interdependencies among its libraries). Fortunately, as explained above, it is fairly straightforward to modify the CMakeLists.txt in order to account for it.

For example, if the required library is lambda, which is only composed by headers, you just need to specify your boost installation directory as:


Of course, you can also use FIND_PACKAGE features of CMake for such purpose. In fact, we are already using it to find and configure projects with well known dependencies, large and massively used libraries as WxWidgets or Boost, so it will be enough for users to just #include what they want and biicode is able to fully configure the project if a local installation of such libraries is found.

Conclusion

In this post we have introduced a new approach to C/C++ projects configuration: meta-configuration with CMake from user information, mainly source code. This is a powerful approach, that can lower barriers for students and new users of the C/C++ language, but that can be also very interesting for many users used in conjunction with a dependency manager based on source code instead of binary artifacts.

Biicode is such novel dependency manager for the C/C++ ecosystem in which source code files can be easily reused among projects and very simply shared with the OSS community. Biicode is in Beta stage, it has currently few contents, so it could happen that your typical requirements are not available in it yet, and it is still not very stable. But it is a solid proof of the power of this approach, and it is evolving quickly, iterating on user feedback. The fact that we use CMake has been widely accepted by our users, we are so convinced about its power that can assure that we will always use it. We are even using it for our experimental Fortran biicode tools!

Would you like to give it a try? Sign up for free at www.biicode.com

Stay tuned


Related Posts