Integrating PVS-Studio into the Continuous Integration Process


This article discusses integration of PVS-Studio into the continuous integration process on Windows. Integration into the CI process on Linux is discussed in the article "How to run PVS-Studio on Linux".

How to use static analysis most efficiently

Before talking on the subject of this article, it would be useful for you to know that running PVS-Studio solely on the build server is effective yet inefficient. A better solution is to build a system that could perform source code analysis at two levels: locally on the developers' machines and on the build server.

This concept stems from the fact that the earlier a defect is detected, the less expensive and difficult it is to fix. For that reason, you want to find and fix bugs as soon as possible, and running PVS-Studio on the developers' machines makes this easier. We recommend using the incremental analysis mode, which allows you to have analysis automatically initiated only for recently modified code after the build.

However, this solution does not guarantee that defects will never get to the version control system. It is to track such cases that the second security level - regular static analysis on the build server - is needed. Even if a bug does slip in, it will be caught and fixed in time. With the analysis integrated into night builds, you will get a morning report about the errors made the day before and be able to fix the faulty code quickly.

Note. It is not recommended to have the analyzer check every commit on the server, as the analysis process may take quite a long time. If you do need to use it in this way and your project is built with MSBuild build system, use the incremental analysis mode of command line module 'PVS-Studio_Cmd.exe'. For details about this mode, see the section "Incremental analysis in command line module 'PVS-Studio_Cmd.exe'" of this paper. You can also use utility 'CLMonitor.exe' (for C and C++ code only) to analyze your source files in this mode (regardless of the build system). To learn more about the use of 'CLMonitor.exe' utility, see the section "Compiler monitoring system" of this paper.

Preparing for CI

Preparing for integration of PVS-Studio into the CI process is an important phase that will help you save time in the future and use static analysis more efficiently. This section discusses the specifics of PVS-Studio customization that will make further work easier.

Unattended deployment of PVS-Studio

You need administrator privileges to install PVS-Studio. Unattended installation is performed by running the following command from the command line (in one line):

PVS-Studio_setup.exe /verysilent /suppressmsgboxes 
/norestart /nocloseapplications

Executing this command will initiate installation of all available PVS-Studio components. Please note that PVS-Studio may require a restart to complete installation if, for example, the files being updated are locked. If you run the installer without the 'NORESTART' flag, it may restart the computer without any prior notification or dialogue.

The package includes utility 'PVS-Studio-Updater.exe', which checks for analyzer updates. If there are updates available, it will download and install them on local machine. To run the utility in 'silent' mode, use the same options as with installation:

PVS-Studio-Updater.exe /verysilent /suppressmsgboxes

Settings and license files are generated automatically and can then be edited or copied to other machines. The default directories of these files are:

%AppData%\PVS-Studio\PVS-Studio.lic
%AppData%\PVS-Studio\Settings.xml

To learn more about unattended deployment of PVS-Studio, see the article "Unattended deployment of PVS-Studio".

Preliminary configuration of the analyzer

Before running the analyzer, you need to configure it to optimize handling of the warning list and (if possible) speed up the analysis process.

Note. The options discussed below can be changed by manually editing the settings file or through the settings page's interface of the Visual Studio plug-in or Standalone version.

It may often be helpful to exclude certain files or even entire directories from analysis - this will allow you to keep the code of third party libraries unchecked, thus reducing the overall analysis time and ensuring that you will get only warnings relevant to your project. The analyzer is already configured by default to ignore some files and paths such as the boost library. To learn more about excluding files from analysis, see the article "Settings: Don't Check Files".

At the phase of analyzer integration, you also want to turn off those PVS-Studio diagnostics that are irrelevant to the current project. Diagnostics can be disabled both individually and in groups. If you know which diagnostics are irrelevant, turn them off right away to speed up the check. Otherwise, you can turn them off later. To learn more about disabling diagnostic rules, see the article "Settings: Detectable Errors".

Suppression of the warnings related to the old code

When integrating static analysis into an existing project with a large codebase, the first check may reveal multiple defects in its source code. The developer team may lack the resources required for fixing all such warnings, and then you need to hide all the warnings triggered by the existing code so that only warnings triggered by newly written/modified code are displayed.

To do this, use the mass warning suppression mechanism, described in detail in the article "Mass Suppression of Analyzer Messages".

Note 1. If you need to hide only single warnings, use the false positive suppression mechanism described in the article "Suppression of false alarms".

Note 2. Using SonarQube, you can specify how warnings issued within a certain period are displayed. You can use this feature to have the analyzer display only those warnings that were triggered after the integration (that is, turn off the warnings triggered by old code).

Integrating PVS-Studio into the CI process

Integrating PVS-Studio into the CI process is relatively easy. In addition, it provides means for convenient handling of analysis results.

Integration of PVS-Studio with the SonarQube platform is possible only if you own an Enterprise license. You can order one by emailing us.

The principles of analysis of the projects, based on different build systems will be described below, as well as the utilities for working with the results of analysis.

Analyzing the source code of MSBuild / Visual Studio projects

This section discusses the most effective way of analyzing MSBuild / Visual Studio solutions and projects, i.e. Visual Studio solutions (.sln) and Visual C++ (.vcxproj) and Visual C# (.csproj) projects.

General information

Project types listed above can be analyzed from the command line by running the 'PVS-Studio_Cmd.exe' module, located in PVS-Studio's installation directory. The default location is 'C:\Program Files (x86)\PVS-Studio\'.

You can modify analysis parameters by passing various arguments to 'PVS-Studio_Cmd.exe'. To view the list of all available arguments, enter the following command:

PVS-Studio_Cmd.exe --help

The analyzer has one obligatory argument, '--target', which is used to specify the target object for analysis (a .sln, .vcxproj, or .csproj file). The other arguments are optional; they are discussed in detail in the article "Analyzing Visual C++ (.vcxproj) and Visual C# (.csproj) projects from the command line".

The following example demonstrates how to start analysis of a .sln file (in one line):

PVS-Studio_Cmd.exe --target "targetsolution.sln" --platform "Any CPU" 
--output "results.plog" --configuration "Release"

Executing this command will initiate analysis of .sln file 'targetsolution.sln' for platform 'Any CPU' in 'Release' configuration. The output file ('results.plog') will be created in the directory of the solution under analysis. The check will be performed with the standard analyzer settings since no specific settings have been specified.

The 'PVS-Studio_Cmd.exe' module employs a number of non-zero exit codes, which it uses to report the final analysis status. An exit code is a bit mask representing all states that occurred while the utility was running. In other words, a non-zero exit code does not necessarily indicate an error in the utility's operation. For a detailed description of exit codes, see the above-mentioned article "Analyzing Visual C++ (.vcxproj) and Visual C# (.csproj) projects from the command line".

Displaying analysis results only for newly written / modified code

If you use the analyzer regularly, you may want it to issue warnings triggered only by newly written/modified code. With night builds on the build server, this would allow you to view only those warnings that were triggered by mistakes made on the previous day.

To turn on this mode, run the 'PVS-Studio_Cmd.exe' module with the command line argument '--suppressAll'. When this flag is present, the utility will add all the messages to the database of suppressed warnings (.suppress files of the corresponding projects) after saving the analysis results. This will prevent those messages from appearing at the next check. In case you need to view the old warnings again, the complete analysis log can be found in the same directory where the .plog file with new messages is located.

To learn more about the mass warning suppression mechanism, see the article "Mass Suppression of Analyzer Messages".

Note. When using the SonarQube platform, you can keep track of new messages without applying the suppression mechanisms. To do this, configure it to display changes only for the past day.

Incremental analysis in command line module 'PVS-Studio_Cmd.exe'

PVS-Studio's incremental analysis mode allows you to check only those files that have been modified/affected since the last build. This mode is available in both the Visual Studio plug-in and the command line module. With incremental analysis, only warnings triggered by modified code will be displayed, thus reducing the analysis time by excluding unaffected parts of the solution from analysis.

This mode is useful when your continuous integration system is configured to run an automatic incremental build every time changes in the version control system are detected; that is, when the project is built and analyzed on the build server many times during the day.

The use of incremental analysis in the 'PVS-Studio_Cmd.exe' module is controlled by the flag '--incremental'. The following modes are available here:

  • Scan - analyze dependencies to determine which files must be included into incremental analysis. The analysis process itself is not initiated.
  • Analyze - run incremental analysis. This step must be performed after the Scan step and can be performed both before and after building the solution/project. Only those files that have changed since the last build will be analyzed.
  • ScanAndAnalyze - analyze dependencies to determine which files must be included into incremental analysis, and immediately start incremental analysis.

To learn more about PVS-Studio's incremental analysis, see the article "PVS-Studio's incremental analysis mode".

Note. There are a few details to keep in mind about this mode. Specifically, you could encounter a file locking issue when PVS-Studio uses Visual C++'s preprocessor ('cl.exe'). It has to do with the fact that the 'cl.exe' compiler may lock a file while preprocessing it, causing writing of this file to fail. When the Clang preprocessor is used, this issue is much rarer. Please keep this in mind when configuring the server to run incremental analysis rather than full-fledged analysis at night.

Analysis of CMake projects

If you need to analyze CMake projects, it is recommended that you convert them into Visual Studio solutions and continue to work with these. This will allow you to use the 'PVS-Studio_Cmd.exe' module's capabilities in full.

Analyzing projects that use uncommon build systems

If your project uses a build system other than MSBuild, you will not be able to analyze it with the command line module 'PVS-Studio_Cmd.exe'. The package, however, includes utilities to make it possible to analyze such projects too.

Compiler monitoring system

The PVS-Studio Compiler Monitoring system, or CLMonitoring, is designed to provide 'seamless' integration of PVS-Studio into any build system under Windows that employs one of the preprocessors supported by the command line module 'PVS-Studio.exe' for compilation.

The monitoring server (CLMonitor.exe) monitors the launches of processes corresponding to the target compiler and collects information about these processes' environment. The server monitors only those processes that run under the same user profile where it has been launched.

Supported compilers:

  • Microsoft Visual C++ compilers (cl.exe);
  • C/C++ compilers of the GNU Compiler Collection (gcc.exe, g++.exe);
  • Clang compiler (clang.exe) and Clang-based compilers.

Before integrating the monitoring server into the build process, start the 'CLMonitor.exe' module with the argument 'monitor':

CLMonitor.exe monitor

This command will tell the monitoring server to call itself in monitoring mode and terminate, while the build system will be able to continue with its tasks. Meanwhile, the second CLMonitor process (called by the first) will be still running and monitoring the build process.

Once the build is complete, you will need to launch the 'CLMonitor.exe' module in client mode to generate preprocessed files and start static analysis proper:

CLMonitor.exe analyze -l "c:\ptest.plog" -u "c:\ptest.suppress" -s

This command contains the following arguments:

  • analyze - run the 'CLMonitor.exe' module for analysis;
  • -l - full path to the file the analysis results will be saved to;
  • -u - path to suppress file;
  • -s - append all new messages of the current check to the suppress file.

To learn more about the use of the compiler monitoring system, see the article "Compiler Monitoring System in PVS-Studio".

Note. The compiler monitoring system has a number of drawbacks stemming from the natural limitations of this approach, namely the impossibility to guarantee a 100% intercept of all the compiler launches during the build process (for example, when the system is heavily loaded). Another thing to remember is that when several build processes are running in parallel, the system may intercept compiler launches related to another build.

Direct integration into build automation systems

Note. In direct integration mode, the analyzer can check only C/C++ code.

Direct integration may be necessary when you cannot use the command line module 'PVS-Studio_Cmd.exe' (since the project is built with a system other than MSBuild) and the compiler monitoring system (see the note in the corresponding section).

In that case, you need to integrate a direct call of the analyzer ('PVS-Studio.exe') into the build process and provide it with all the arguments required for preprocessing. That is, the analyzer must be called for the same files that the compiler is called for.

To learn more about direct integration into build automation systems, see the article "Direct integration of the analyzer into build automation systems (C/C++)".

Handling analysis results

Once the check has finished, the analyzer outputs a .plog file in the XML format. This file is not intended to be handled manually (read by the programmer). The package, however, includes special utilities whose purpose is to provide a convenient way to handle the .plog file.

PlogConverter

'PlogConverter.exe' is used to convert the analyzer report into one of the formats that could be read by the programmer:

  • text file with analysis results. It may be convenient when you want the analysis results (for example, new diagnostic messages) output into the log of the build system or CI server;
  • HTML file with hyperlinks to the source files. This format is convenient for sending the report via email to the developers involved;
  • CSV table with analysis results;
  • Tasks file to be viewed in QtCreator;
  • text file with a summary table showing the number of messages across severity levels and groups of diagnostics.

This example demonstrates how to use 'PlogConverter.exe' utility (in one line):

PlogConverter.exe test1.plog -o "C:\Results" -r "C:\Test" 
-a GA:1 -t Html

This command converts the 'test1.plog' file into an .html file that will include the first-level diagnostic messages of the GA (general-analysis) group. The resulting report will be written to 'C:\Results', while the original .plog file will stay unchanged.

To see full help on 'PlogConverter' utility's parameters, run the following command:

PlogConverter.exe --help

Note. 'PlogConverter' utility comes with the source files (in C#), which can be found in the archive 'PlogConverter_src.zip'. You can adopt the algorithm of parsing a .plog file's structure to create your own output format.

To learn more about 'PlogConverter', see the article "Managing the Analysis Results (.plog file)".

SonarQube

Analysis results can be imported into the SonarQube platform, which performs continuous code quality inspection. To do this, use the 'sonar-pvs-studio-plugin' included into the package. This plugin allows you to add warnings issued by PVS-Studio to the SonarQube server's message database. This, in its turn, enables you to view bug occurrence/fixing statistics, navigate the warnings, view the documentation on diagnostic rules, and so forth.

Once added to SonarQube, all PVS-Studio messages are assigned type Bug. SonarQube's interface keeps the same layout of message distribution across diagnostic groups as in the analyzer.

To learn more about integrating analysis results into SonarQube, see the article "Integration of PVS-Studio analysis results into SonarQube".

PVS-Studio plugin for Jenkins

At the moment, PVS-Studio plugin for Jenkins is only supported on Windows operating systems.

The distribution package includes 'pvs-studio.hpi' plugin, designed for publishing the results of PVS-Studio analyzer work in the Jenkins continuous integration system.

Note. Plugin for Jenkins allows to publish results of static analysis, for which it uses the reports of the analyzer (.plog-files). This allows to configure the publication of various analysis results: full project analysis, incremental analysis (analysis of only modified files), etc. Accordingly, to publish the results of various types of analysis, it is only necessary to correctly configure the analyzer itself and pass the paths of one or more related .plog files to plugin (about this it will be written below). Information about the complete analysis is described in the sections "Analysis of the source code of MSBuild / Visual Studio projects" and "Analyzing projects that use uncommon build systems", incremental analysis mode is described in the article "PVS-Studio's incremental analysis mode".

To install the plugin you should upload it from the 'Advanced' tab at the plugin setup menu (Manage Jenkins - Manage Plugins); select a plugin in the 'Upload Plugin' section and perform the installation by clicking the 'Upload'. Jenkins will install the plugin and notify you if it is necessary to reboot the server.

In the server settings ('Configure System') in the section 'PVS-Studio Plugin Settings', you must specify the installation directory of PVS-Studio. Default directory is 'C:\Program Files (x86)\PVS-Studio\'.

To publish the results of analysis in project settings, you must add the post-build step ('Post-build Actions' section) 'Publish PVS-Studio analysis result'. Mandatory field 'Path(s) to PVS-Studio analysis report(s)', which sets the path to the .plog file (PVS-Studio result), the contents of which will be displayed on the build page. The absolute and relative (for project workspace) paths are supported. Jenkins environment variables are also supported.

If it is necessary to display integrated results of several .plog files, they must be separated by '|' punctuation mark.

The example of setting paths to several reports to publish an integrated result:

$REPORT_PATH | D:\analysis reports\testLog.plog | .\additionalLog.plog

Additional parameters ('Analyzers and levels settings', 'Excluded codes', 'Settings path') allow performing supplementary filtration of analyzer warnings. A brief description of these parameters is available directly in the form of their task. The meaning of these parameters and possible values are similar to the parameters 'analyzer', 'excludedCodes' and 'settings' of PlogConverter utility. They are described in the article "Managing the Analysis Results (.plog file)".

After a successful execution of publishing PVS-Studio analysis results, they will be displayed on build page. The results of the analysis are stored for each build that allows you to view the analyzer report that corresponds to a specific build.

If the resulting analyzer report is large, its preview will be displayed in the build page. The full report will be available for viewing when clicking the links 'View full report' over/under report preview, or 'Full PVS-Studio analysis report' in the menu on the left.

Figure 1 - Build page look, displaying the results of PVS-Studio analysis

Figure 1 - Build page look, displaying the results of PVS-Studio analysis

Sending analysis results via email

Sending analysis report copies to developers is an effective way to inform them about the results. It can be done with the help of special utilities such as SendEmail. SonarQube provides this option as well.

Another way to inform the developers is to use 'BlameNotifier' utility, which also comes with the PVS-Studio package. This application allows you to form reports in a flexible way. For example, you can configure it so that it will send individual reports to the developers who submitted faulty code; team leaders, development managers, etc. will get a complete log with the data about all the errors found and developers responsible for them.

For basic information about the utility, run the following command:

BlameNotifier.exe --help

To learn more about 'BlameNotifier', see the article "Managing the Analysis Results (.plog file)", section "Notifying the developer team".

Conclusion

If you have any questions, please feel free to contact us at support@viva64.com.


Do you make errors in the code?

Check your code
with PVS-Studio

Static code analysis
for C, C++, and C#

goto PVS-Studio;