- Using the "Mark As False Alarm" function during testing automation
- Integrating PVS-Studio into build automation process
- Launching the analyzer in batch mode through command line
- An example of integrating PVS-Studio plug-in for Visual Studio with CruiseControl .NET
- Integrating with Hudson
- Integrating with Microsoft Team Foundation Server 2005/2008/2010
This article illustrates techniques required to employ the use of PVS-Studio static code analyzer together with continuous integration systems. Also provided are configuration guidelines for PVS-Studio launching modes.
Continuous integration (CI) is the practice of software development implying frequent building and subsequent testing for the most recent versions of designed application. Generally, continuous integration systems interact directly with revision control and allow for a significant increase in integration process's reliability while decreasing its laboriousness by automating entire building-testing phase.
For the purpose of deployment within continuous integration, PVS-Studio can launch the analysis for its target project in "silent" mode from command line shell. As such, integrating PVS-Studio into periodic build process allows for an effective utilization of analysis's results during collaborate project development process.
While launching from command line shell, PVS-Studio will always analyze the whole target project. The reason for such an approach is that modifications in a single header file can affect multiple cpp files, thus checking only modified files will not be sufficient.
Keep in mind that static analysis will always generate a lot of so called "false alarms", so to suppress that "noise" it is imperative to employ "Mark as False Alarm" functionality while using PVS-Studio during regular source code verification. Thus PVS-Studio operation can be subdivided into 2 steps: integration and exploitation. During the integration stage developer should review all generated error messages and either correct corresponding code or mark it as false alarm. Afterwards, the analyzer would generate error messages only for newly added code.
In the light of PVS-Studio being an extension to Microsoft Visual Studio/Embarcadero RAD Studio IDEs, it is possible to integrate the analyzer into any CI system capable of executing builds for these IDEs via command line commands with arguments. Also, direct support for IDE building process is not required from the CI system, the ability to pass commands to operating system's shell and process stdout stream will suffice.
Launching the analyzer from command line is described in detail in this separate article.
The resulting XML-log file can be loaded into IDE (by double-clicking it directly from file manager or through PVS-Studio/Load Analysis Report menu item) and used to navigate through errors in analyzed source code. As the log file itself is not conveniently formatted to be reviewed manually, analyzer will also create the plain text file containing a list of all errors (not marked as false alarms). This file is intuitive and can be conveniently included into continuous integration system's general logs, which in turn can be published, for example, by e-mail to all concerned developers.
In case the verification of all the files from all projects is not an appropriate option, PVS-Studio is able to analyze only the files which were modified during the predefined time interval. This time interval can be defined at the SpecificAnalyzerSettings option page. Alternatively one can also explicitly specify the files for analysis using the batch mode.
Launching through command line will by default make the analyzer check all the files included into the selected projects. If, however, the verification of only a predefined set of files is required, it is possible to utilize the batch mode which will subject to analysis only files that were explicitly defined by their paths. This launching mode is also mentioned in this article.
PVS-Studio integration with CruiseControl.NET can be achieved by addition of "Executable" task into build project. This task will launch cmd.exe Windows shell and pass the required build arguments into it. Below is the fragment of XML server configuration file, which contains such a task.
<exec> <description>PVS-Studio check solution example</description> <executable>&CMD_PATH;</executable> <baseDirectory>&DEVENV_PATH;<baseDirectory> <buildArgs> <item> /c devenv.exe "C:\sample\VSSol.sln" /command "PVSStudio.CheckSolution Win32|Release|C:\sample\ result-log.plog" </item> </buildArgs> </exec>
The analysis results contained within "result-log.plog.only_new_messages.txt" text file can be published by any available publisher tasks, for example by being moved into common builder directory or by e-mailing build server logs. The inclusion of these results into general builder logs can be performed by such Executable task:
<exec> <description>PVS-Studio load error log</description> <executable>&CMD_PATH;</executable> <baseDirectory>&PROJECT_ROOT;</baseDirectory> <buildArgs> <item> /c type result-log.plog.only_new_messages.txt </item> </buildArgs> <buildTimeoutSeconds>0</buildTimeoutSeconds> </exec>
Integrating with Hudson can be performed by adding "execute Windows shell command" build step into build project (Job). An example of the command to be added, as follows:
devenv.exe "C:\sample\VSSol.sln" /command "PVSStudio.CheckSolution Win32|Release|C:\sample\ result-log.plog"
Text file holding the list of generated errors (not marked as false alarms) can be included into build server general log by addition of another command:
Afterwards the resulting build log can be published with the help of any available Hudson publisher tools. For example, to publish results by e-mail, you can use ext-mail plug-in, adding $BUILD_LOG token into message body:
Using PVS-Studio analyzer together with Team Foundation build server is complicated by the presence of strong integration between Visual Studio projects and Microsoft's own source control system (SCS) thus requiring additional steps compared to other companies' continuous integration systems in case Microsoft Team Foundation SCS is being used. You should also take into account that TFS build server uses MSBuild utility for its builds and the presence of Visual Studio IDE is not required for build server's operations. In the light of PVS-Studio being an extension plug-in to Visual Studio IDE, the access of TFS build server to Visual Studio environment (devenv.exe in particular) with PVS-Studio installed in it is a mandatory requirement.
A typical TFS build process using PVS-Studio analyzer should include the following minimal set of stages:
- The server downloads required projects' sources into a defined local directory.
- Program source codes are verified by PVS-Studio analyzer (to do this you should call devenv.exe with "/command PVSStudio.Chechsolution" command line switch), the results are saved into defined local directory.
- The server builds the projects using MSBuild.
- Build and verification results are published into local directory predefined for this particular build.
To integrate PVS-Studio analysis (stage 2) into Team Foundation 2010 server's build process you should add the InvokeProcess build activity to build script defined in xaml file(Edit Build Definition menu item for required build definition, Process tab, Build Process Template field), pass a path to PVS-Studio.exe and this line as an Arguments property:
--sln-file VSSol.sln --plog-file result-log.plog --vcinstalldir "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC" --platform "Win32" --configuration "Release"
To integrate PVS-Studio analysis (stage 2) into Team Foundation 2005/2008 server's build process you should modify MSBuild project (file with the proj extension) belonging to the required build definition, adding an Exec build task to it. (The Exec task must be a descendant of one of the Target nodes in which MSBuild tasks are grouped) You should also pass the following string as the command attribute:
PVS-Studio.exe --sln-file VSSol.sln --plog-file result-log.plog --vcinstalldir "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC" --platform "Win32" --configuration "Release"
Team Foundation System build server executes its builds using Local System Account (NT AUTHORITY\SYSTEM) or Local Service account in non-interactive mode by default, i.e. works as Windows service and it is unable to interact with the user's desktop. It is highly recommended to use interactive desktop mode for TFS builds to ensure the correct operation of Visual Studio environment. To launch TFS build in interactive mode you should start Team Foundation Build service from under the account which has privileges to access the desktop (for example your current user account). For Team Foundation 2008 this service can be launched manually (C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\ TFSBuildService.exe). For Team Foundation 2010 the launch properties of this service can be modified using the server configuration utility.
After the first launch of devenv.exe from under the Local System Account (NT AUTHORITY\SYSTEM), Visual Studio will produce a dialog window, allowing you to select one of default environment settings templates. This dialog will also block the operation of PVS-Studio analyzer, thus it should be closed manually at least once, but it is impossible to accomplish for an application which was started in non-interactive mode. Moreover, starting from Windows Vista, the applications launched from under the Local System Account are not allowed to be launched in interactive mode. To bypass this restriction you could use the PsExec utility which is a part of PSTools utility set. With its help, you can start devenv.exe process from under the Local System Account in interactive mode using the following arguments: psexec.exe –i –s %devenvPath%.
If Microsoft Team Foundation (TFS versions 2008/2010) is used as source control system, this modus operandi makes it impossible to open and analyze Visual Studio solution files that contain TFS source control metainformation because the dialog windows produced by the IDE will block the operations of PVS-Studio extension. PVS-Studio does not support command line launched verification of projects containing TFS 2008/2010 source control metainformation.
To begin analyzing Visual C++ projects using PVS-Studio analyzer from Team Foundation System build server which utilizes Team System source control, you should erase all TFS source control metainformation from sln and projects (vcxproj) files. You can accomplish this without interrupting build process and in fully automated mode with the help of TFSRipper command line utility. To accomplish this you should add the call to this utility before and after calling devenv.exe, also using InvokeProcess build activity. The utility should receive the %LocalSourcePath% – path to sources to be analyzed downloaded by the build server – as argument while it is launched the first time, and %LocalSourcePath% \restore argument while it is launched second time. The utility will process all sln an vcxproj files, deleting the source control metainformation and creating the copies of these files beforehand (with the pvsbak extension), and it will restore all files from earlier created copies after being called second time with /restore argument.