Using PVS-Studio with continuous integration systems
- Using the "Mark As False Alarm" function during testing automation
- Integrating PVS-Studio into build automation process
- Launching the analyzer in batch mode through command line
- An example of integrating PVS-Studio plug-in for Visual Studio with CruiseControl .NET
- Integrating with Hudson
This article illustrates techniques required to employ the use of PVS-Studio static code analyzer together with continuous integration systems. Also provided are configuration guidelines for PVS-Studio launching modes.
Continuous integration (CI) is the practice of software development implying frequent building and subsequent testing for the most recent versions of designed application. Generally, continuous integration systems interact directly with revision control and allow for a significant increase in integration process's reliability while decreasing its laboriousness by automating entire building-testing phase.
For the purpose of deployment within continuous integration, PVS-Studio can launch the analysis for its target project in "silent" mode from command line shell. As such, integrating PVS-Studio into periodic build process allows for an effective utilization of analysis's results during collaborate project development process.
While launching from command line shell, PVS-Studio will always analyze the whole target project. The reason for such an approach is that modifications in a single header file can affect multiple cpp files, thus checking only modified files will not be sufficient.
Keep in mind that static analysis will always generate a lot of so called "false alarms", so to suppress that "noise" it is imperative to employ "Mark as False Alarm" functionality while using PVS-Studio during regular source code verification. Thus PVS-Studio operation can be subdivided into 2 steps: integration and exploitation. During the integration stage developer should review all generated error messages and either correct corresponding code or mark it as false alarm. Afterwards, the analyzer would generate error messages only for newly added code.
In the light of PVS-Studio being an extension to Microsoft Visual Studio IDEs, it is possible to integrate the analyzer into any CI system capable of executing builds for these IDEs via command line commands with arguments. Also, direct support for IDE building process is not required from the CI system, the ability to pass commands to operating system's shell and process stdout stream will suffice.
Launching the analyzer from command line is described in detail in this separate article.
The resulting XML-log file can be loaded into IDE (by double-clicking it directly from file manager or through PVS-Studio/Load Analysis Report menu item) and used to navigate through errors in analyzed source code. As the log file itself is not conveniently formatted to be reviewed manually, analyzer will also create the plain text file containing a list of all errors (not marked as false alarms). This file is intuitive and can be conveniently included into continuous integration system's general logs, which in turn can be published, for example, by e-mail to all concerned developers.
In case the verification of all the files from all projects is not an appropriate option, PVS-Studio is able to analyze only the files which were modified during the predefined time interval. This time interval can be defined at the SpecificAnalyzerSettings option page. Alternatively one can also explicitly specify the files for analysis using the batch mode.
Launching through command line will by default make the analyzer check all the files included into the selected projects. If, however, the verification of only a predefined set of files is required, it is possible to utilize the batch mode which will subject to analysis only files that were explicitly defined by their paths. This launching mode is also mentioned in this article.
PVS-Studio integration with CruiseControl.NET can be achieved by addition of "Executable" task into build project. This task will launch cmd.exe Windows shell and pass the required build arguments into it. Below is the fragment of XML server configuration file, which contains such a task.
<exec> <description>PVS-Studio check solution example</description> <executable>&CMD_PATH;</executable> <baseDirectory>&DEVENV_PATH;<baseDirectory> <buildArgs> <item> /c devenv.exe "C:\sample\VSSol.sln" /command "PVSStudio.CheckSolution Win32|Release|C:\sample\result-log.plog" </item> </buildArgs> </exec>
Besides the XML analysis log (plog), PVS-Studio will create additional plain text and html log files, containing only newly discovered messages (excluding messages marked as false positives and suppressed messages). These analysis results can be published by any available publisher tasks, for example by being moved into common builder directory or by e-mailing build server logs by any third-party tool. The inclusion of these results into general builder logs can be performed by such Executable task:
<exec> <description>PVS-Studio load error log</description> <executable>&CMD_PATH;</executable> <baseDirectory>&PROJECT_ROOT;</baseDirectory> <buildArgs> <item> /c type result-log.plog.txt </item> </buildArgs> <buildTimeoutSeconds>0</buildTimeoutSeconds> </exec>
Integrating with Hudson can be performed by adding "execute Windows shell command" build step into build project (Job). An example of the command to be added, as follows:
devenv.exe "C:\sample\VSSol.sln" /command "PVSStudio.CheckSolution Win32|Release|C:\sample\result-log.plog"
Text file holding the list of generated errors (not marked as false alarms) can be included into build server general log by addition of another command:
Afterwards the resulting build log can be published with the help of any available Hudson publisher tools. For example, to publish results by e-mail, you can use ext-mail plug-in, adding $BUILD_LOG token into message body: