Speeding up the analysis of C/C++ code through distributed build systems (IncrediBuild)


To speed up the analysis, you can use a distributed build system, for example, IncrediBuild. The analysis of C/C++ code in PVS-Studio can be divided into 2 stages: preprocessing and analysis itself. Each of these steps can be executed remotely by the distributed build system. To analyze each C/C++ compiled file, PVS-Studio first launches an external preprocessor, and then the C++ analyzer itself. Each such process can be executed remotely.

Depending on the type of a checked project, the analysis of PVS-Studio is launched either through the PVS-Studio_Cmd.exe (for MSBuild projects) utility, or using the utility for monitoring the calls of the compiler - CLMonitor.exe \ Standalone.exe (for any build system). Further, one of these utilities will first run the preprocessor (cl.exe, clang.exe for Visual C++ projects, for the rest – the same process that was used for compilation) for each checked file, and then - the C++ analyzer PVS-Studio.exe.

Setting the value of 'ThreadCount' option to more than '16' (or more than a number of processor cores, if processor possesses more than 16 cores) is available only in PVS-Studio Enterprise license. Please contact us to order a license.

These processes run concurrently, depending on the 'PVS-Studio|Options...|Common AnalyzerSettings|ThreadCount' setting. By increasing the number of concurrently scanned files, with the help of this setting, and distributing the execution of these processes to remote machines, you can significantly (several times) reduce the total analysis time.

An example of IncrediBuild configuration

Here is an example of speeding up the PVS-Studio analysis by using the IncrediBuild distributed system. For this we'll need an IBConsole management utility. We will use the Automatic Interception Interface, which allows remotely executing any process, intercepted by this system. Launching of the IBConsole utility for distributed analysis using PVS-Studio will look as follows:

ibconsole /command=analyze.bat /profile=profile.xml

The analyze.bat file must contain a launch command for the analyzer, PVS-Studio_Cmd.exe or CLMonitor.exe, with all the necessary parameters for them (more detailed information about this can be found in the relevant sections of analyzer documentation). Profile.xml file contains the configuration for the Automatic Interception Interface. Here is an example of such a configuration for the analysis of the MSBuild project using PVS-Studio_Cmd.exe:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<Profile FormatVersion="1">
  <Tools>
    <Tool Filename="PVS-Studio_Cmd" AllowIntercept="true" />
    <Tool Filename="cl" AllowRemote="true" />
    <Tool Filename="clang" AllowRemote="true" />
    <Tool Filename="PVS-Studio" AllowRemote="true" />
  </Tools>
</Profile>

Let's see, what each record in this file means. We can see that the AllowIntercept attribute with the 'true' value is specified for PVS-Studio_Cmd. This means that a process with such a name will not be executed itself in a distributed manner, but the system of automatic interception will track the child processes generated by this process.

For the preprocessor cl and clang processes and the C/C++ analyzer PVS-Studio process, the AllowRemote attribute is specified. This means that processes with such names, after being intercepted from the AllowIntercept processes, will be potentially executed on other (remote) IncrediBuild agents.

Before running IBConsole, you must specify the 'PVS-Studio|Options...|Common AnalyzerSettings|ThreadCount' setting, according to the total number of cores available on all of IncrediBuild agents. If it's not done, there will be no effect from using IncrediBuild!

Note: during the analysis of Visual C++ projects, PVS-Studio uses clang.exe supplied in the PVS-Studio distribution for preprocessing C/C++ files before the analysis, instead of the cl.exe preprocessor. This is done to speed up the preprocessing, as clang is doing it faster than cl. Some older versions of Incredibuild performs a distributed launch of the clang.exe preprocessor not quite correctly, resulting in errors of preprocessing. Therefore, clang should not be specified in the IBConsole configuration file, if your version of IncrediBuild handles clang incorrectly.

The used type of preprocessor during the analysis is specified with the 'PVS- Studio|Options...|Common AnalyzerSettings|Preprocessor' setting. If you choose the 'VisualCpp' value for this setting, PVS-Studio will use only cl.exe for preprocessing, which will be executed in a distributed manner, but slower than clang, which cannot be executed in a distributed manner. You should choose this setting depending on the type of the project and the number of agents available to analyze - when having a large numbers of agents, the choice of VisualCpp will be reasonable. With a small numbers of agents, local preprocessing with clang might prove to be faster.

For a distributed analysis using CLMonitor / Compiler Monitoring UI (Standalone.exe), you must change the configuration file as follows: replace PVS-Studio_Cmd with CLMonitor or Standalone (depending on whether the check is triggered from the UI or from the command line); cl, if necessary, should be replaced with the type of the preprocessor which is used during build (gcc, clang). For example:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<Profile FormatVersion="1">
  <Tools>
    <Tool Filename="CLMonitor" AllowIntercept="true" />
    <Tool Filename="gcc" AllowRemote="true" />
    <Tool Filename="PVS-Studio" AllowRemote="true" />
  </Tools>
</Profile>

When specifying the ThreadCount settings, please note, that the coordinator machine of the analysis (i.e. the one, which runs the PVS-Studio_Cmd/CLMonitor/Standalone) will be responsible for processing the results coming from all of the PVS-Studio.exe processes. This job cannot be distributed - therefore, especially when ThreadCount is set to a very high value (more than 50 processes simultaneously), it is worth thinking about how to "unload" the coordinator machine from the analysis tasks (i.e., from performing the processes of the analyzer and preprocessor). This can be done by using the '/AvoidLocal' IBConsole flag, or in the settings of local IncrediBuild agent on the coordinator machine.


Bugs Found

Checked Projects
363
Collected Errors
13 495