The Importance of C++ Distributed Build Systems

As C++ developer if you have already build a C# or a java project you could be surprised by the speed of the build compared to C++. For some C++ projects it could take few minutes and for others, it could take many hours. It depends on the project size.
And even if the compilation phase is parallelized by using the cores available, the build still take too long compared to the other languages. Here are some reasons from this good stack overflow answer:

  • Header files: Every single compilation unit requires hundreds or even thousands of headers to be 1: loaded, and 2: compiled. Every one of them typically has to be recompiled for every compilation unit, because the preprocessor ensures that the result of compiling a header might vary between every compilation unit. (A macro may be defined in one compilation unit which changes the content of the header).This is probably the main reason, as it requires huge amounts of code to be compiled for every compilation unit, and additionally, every header has to be compiled multiple times (once for every compilation unit that includes it)
  • Linking: Once compiled, all the object files have to be linked together. This is basically a monolithic process that can’t very well be parallelized and has to process your entire project.
  • Parsing: The syntax is extremely complicated to parse, depends heavily on context, and is very hard to disambiguate. This takes a lot of time
  • Templates: In C#, List<T> is the only type that is compiled, no matter how many instantiations of List you have in your program. In C++, vector<int> is a completely separate type from vector<float>, and each one will have to be compiled separately.

What happens if the build process is slow?

If a build takes 10 mins, the company will lose each day the equivalent of  10 min * number of builds done per day, and it depends on the number of developers. If there are many developers, there are many builds and the lost become bigger.

The IncrediBuild ROI calculator gives us an approximation of how much we lose if we have X developers and the build takes Y minutes. However, this calculator is optimistic because it can’t also count the hidden cost. Indeed, if a developer launch a build and he must wait for 10 mins, he can go do something else like googling on the web where he can be easily distracted and the 10 mins could grow to 15 mins or 20 mins.

A fast build help the developer to more focus on what he was coding.

How can we optimize the build process of the C++ projects?
There are some  techniques that can help optimize the process, some of them are enumerated here

  • Pimpl Idiom
  • Forward Declarations
  • Guard Conditions
  • Reduce interdependency
  • And of course the most used one: Precompiled Headers

However, using the techniques below has some drawbacks:

  •  It’s difficult to ensure that these solutions are really applied, a code review is needed regularly.
  •  Even if these solutions are applied as expected we will not gain a lot of time,  and it could take many hours for some big projects.
  •  Changing our code just to resolve the build time issue is probably a very bad idea, the code is designed and implemented independently of the build process used.

C++ Distributed build systems to the rescue

The distributed build system is a program to distribute compilation of code across several machines on a network. it should always generate the same results as a local compile.

There are some interesting C++ distributed build systems. And personally, I used only IncrediBuild.

IncrediBuild is a suite of grid computing software developed by IncrediBuild Ltd. IncrediBuild is designed to help accelerate computationally-intensive tasks by distributing them over the network, with notable applications including compiling source code, building software generally, and other software development-related tasks. Jobs can be distributed to several computers over a network, giving both the possibility of accelerating the work by using more resources than were available on the initiating computer alone.

IncrediBuild helped us a lot to reduce the build time and focus more on productive tasks.

If in your case your C++ build process takes too long, consider using a distributed build system from now. Just search on the web all the tools available and choose the right one that fulfills your needs.