Code analysis by PVS-Studio from the command line using Microsoft Visual Studio or Embarcadero RAD Studio (when project files vcproj/vcxproj for devenv.exe or cbproj for bds.exe are available)

23.08.2013

Introduciton

In this article we describe how to run project analysis by PVS-Studio not directly from IDE (Visual Studio/RAD Studio), but through the command line. Note that the check is still performed under an IDE application (in particular, devenv.exe in Visual Studio or bds.exe in RAD Studio) using the project files (.vcproj/.vcxproj or cbproj) and solutions/project groups (.sln/. groupproj), but the launch itself will be performed from the command line instead of IDE. This way of running the tool is accepted in developer teams who use continuous integration systems (like Cruise Control, Draco.NET, or Team Foundation Build by Visual Studio Team System).

We have a special utility to assist in preparing the command line parameters. See the PVS-Studio Command Generator section below to read the description.

Running analysis from the command line under Visual Studio

You can use the Visual Studio command "PVSStudio.CheckSolution" to run analysis from the command line. This is a command integrated into IDE by the PVS-Studio plugin, and, like any other standard command, it can be called from the command line either with the argument /command, or directly from the development environment in the Commands window. You also need to pass the path to the sln file into Visual Studio which is to be checked (it's an individual argument). After opening and loading the development environment, this command passes control directly into the PVS-Studio plugin. Let's look at the format of this command:

%devenvPath% "%solution%" /command "PVSStudio.CheckSolution %platform%|%configuration%|%plog%|%filelist%|%preprocessor%"

%devenvPath% is the path to the Visual Studio executable file (devenv.exe) with the PVS-Studio plugin installed.

"%solution%" is the path to the solution file (.sln) containing the projects to be checked

%platform% is the project platform to be checked

%configuration% is the project configuration to be checked

%plog% is the location and the name of the XML PVS-Studio report-file that will be generated when all the files are checked

%filelist% is the location and the name of the text file containing the list of source C/C++ files to be checked. It will allow you to check particular files explicitly specified in this list (instead of checking all the files of the solution projects). If a file specified in the list is absent in the project being checked, it is ignored. This parameter is optional; if you don't define it, PVS-Studio will check all the files of the solution projects. We will describe this batch mode in detail, in one of the next paragraphs.

%preprocessor% is the preprocessor to be used while checking the source files. PVS-Studio can use 2 preprocessors: the native IDE preprocessor (cl.exe for Visual Studio, the parameter's value is VisualCPP) or the faster third-party preprocessor clang (the parameter's value is Clang). This parameter is optional; if it is not defined, PVS-Studio by default tries to preprocess the files with clang at first, and then with cl if errors occur (clang sometimes fails to process MSVC header files).

The last two parameters (%preprocessor% and %filelist%) are optional, but you still should type in the separating character | when omitting one of them (i.e. if you omit the %filelist% parameter, the command line will look like this: %platform%|%configuration%|%plog%||%preprocessor%; note the repeating separating character ||).

An example of running analysis from the command line with the Visual Studio window's UI displayed

For the purpose of demonstrating the launch of PVS-Studio we advise that you install the OmniSample project from the PVS-Studio distribution kit and use it as a test sample for analysis. Let's assume that the OmniSample project is installed into the C:\Users\evg\Documents\OmniSample folder. We need to check it in Visual Studio 2012. Then we'll just need to pass the following parameters in the command line (note that arguments containing paths to directories shouldn't end with the \ character, as the \" character combination is sometimes handled incorrectly by the operating system):

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\devenv.exe "C:\Users\evg\Documents\OmniSample\OmniSample (vs2012).sln" /command "PVSStudio.CheckSolution x64|Release|C:\Users\evg\Documents\result.plog"

Once we execute this command, the necessary devenv.exe version is launched, the "OmniSample (vs2012).sln" solution is opened, the x64 architecture is selected, as well as the Release configuration, and thus PVS-Studio starts analysis of our solution. When it is over, PVS-Studio will save the report into the file C:\Users\evg\Documents\result-log.plog and quit. You can then open the saved report in IDE with the "Load Analysis Report" command of the PVS-Studio menu or by double-clicking it directly from the file manager and view it in the convenient environment with code navigation, filters and all the PVS-Studio functionality enabled. That is, it's very convenient to handle the report-file under PVS-Studio itself because it has a special format.

If the project is correct and all the parameters are set properly, analysis will terminate some time later, and the window will close, while you will be able to open the result-log.plog file. But if something goes wrong, the corresponding error message will be saved into a text file named plog.exceptions (plog is the name of the log file defined through the command line) into the same folder as the log file. Keep in mind that sometimes messages generated during night builds by Visual Studio itself may cause the analysis process to "hang", awaiting the user's reaction to the message (although we try to avoid these situations). That's why we recommend that you always set a timeout period when launching devenv.exe from the command line.

An example of using the batch mode when running the analyzer from the command line

When launching analysis from the command line, PVS-Studio by default checks all the files included into the projects of the specified Visual Studio solution-file. But if you need to check only a particular set of files, you may use the batch mode when only those files are analyzed the paths to which are defined explicitly. To run analysis in this mode, you need to define the path to the list of files to be checked as a command line parameter (this parameter is optional).

To check files in the batch mode from the command line, you need to run analysis with the following arguments:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\devenv.exe "C:\Users\evg\Documents\OmniSample\OmniSample (vs2012).sln" /command "PVSStudio.CheckSolution x64|Release|C:\Users\evg\Documents\result.plog| C:\Users\evg\Documents\ OmniSample\ExtFilelist.txt "

The necessary requirement is that all the C++ files listed inside the filelist-file be included into the projects of the defined .sln-file. The list must contain full paths to the C++ files separated by the newline characters (\r\n). Admissible file extensions are c, cc, cpp, and cxx, i.e. compilable files.

Here's an example of the text file ExtFilelist.txt (file per line):

c:/Users/evg/Documents/OmniSample/OmniSample/../OmniSample/OmniSample.cpp

"c:\\Users\\evg\\Documents\\OmniSample\\OmniSample\\..\\OmniSample\\ParallelSample.cpp"

The PVS-Studio Command Generator (auxiliary) utility

PVS-Studio Command Generator is a utility to generate a command to run project analysis with PVS-Studio indirectly, through a command line, instead of from within IDE (Visual Studio/RAD Studio). You may either construct the command line manually, as described above, or with the help of this utility.

Figure 1 shows the application's graphical interface.

Figure 1 - Interface of the command line generating application

Figure 1 - Interface of the command line generating application

All the parameters necessary to create a command line are evident and were described earlier in this article.

PVS-Studio Command Generator is designed to work on a Windows platform with .NET Framework 2.0 installed, but it doesn't require the PVS-Studio analyzer or any of the supported development environments (Microsoft Visual Studio or Embarcadero RAD Studio) to be installed. In other words, you can generate a command line on a computer with PVS-Studio absent and use it on a computer with PVS-Studio installed. If there is a development environment installed, the actual path to IDE's executable file will be added into the command line. Otherwise, the default path is taken so that the analyzer is not run only in test mode, while the generated command line could also be executed on another computer with one of the necessary development environments installed; you will only need to fix the path to IDE's executable file, if necessary.

Running analysis from the command line under RAD Studio

To launch analysis using the RAD Studio plugin (the bds.exe file), you need to pass 2 arguments to it: the path to the groupproj file to be checked and the PVSStudio.CheckSolution command. The format of the PVSStudio.CheckSolution command is almost identical to its version in Visual Studio (see the description in the previous paragraph), save some of the arguments.

In particular, the BDS plugin doesn't support specification of an active platform (%platform%) for the project being checked as well as selection of the external preprocessor Clang.exe (%preprocessor%). If these parameters are defined, they will be ignored. Note that the separating character after the %platform% platform parameter is obligatory even if it contains an empty line, i.e. this parameter cannot be skipped (although it will be ignored):

%bdsPath% "%groupproj%" "PVSStudio.CheckSolution:|%configuration%|%plog%|%filelist%"

%bdsPath% is the path to the RAD Studio executable file with the PVS-Studio plugin installed.

"%groupproj% is the path to the group of C++ Builder projects (the .groupproj file).

Here's an example of how to run analysis by PVS-Studio from the command line using the RAD Studio plugin with all the possible arguments defined:

c:\Program Files (x86)\Embarcadero\RAD Studio\10.0\bin\bds.exe "c:\Users\Evg\Sample\Sample.groupproj" "PVSStudio.CheckSolution:|Debug|c:\Users\Evg\Sample\Sample.plog "

The PVS-Studio plugin doesn't currently allow you to choose a platform to be checked when launching bds.exe from the command line; it is the platform saved as the active one before the last closing of the project group being checked that will be analyzed. This limitation is caused by some peculiarities of RAD Studio environment extension modules' API.

Running analysis from the command line with the IDE window hidden

If you need to hide the IDE window, you may launch analysis using a small script file written in Visual Basic Script (vbs). vbs is supported by all the contemporary Windows versions, so you may view this method as a "native" one.

Create the file run-PVS-Studio.vbs consisting of the following five lines (this is an example for Visual Studio):

Set oShell = CreateObject("WScript.Shell")

DevEnv8 = """c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\devenv.exe """

SolFile = """C:\Users\evg\Documents\OmniSample\OmniSample (vs2012).sln"""

Cmd = " /command ""PVSStudio.CheckSolution Win32|Debug|C:\Users\evg\Documents\result-log.plog"""

strCommand = oShell.Run(DevEnv8+SolFile+Cmd, 0, True)

All the parameters here are similar to those of a common batch file (see the previous paragraph). The difference is that in this case analysis is run through a special function Run which provides the possibility to hide the window as one of its parameters.

After saving the file, open it. You won't see any Visual Studio window, but you will see the devenv.exe process in Task Manager. In the same way as described before, the file result-log.plog will appear in the folder you've defined.

How PVS-Studio settings affect analysis launched from the command line

When running code analysis from the command line, all the settings used are the same as in case of running under IDE (Visual Studio/RAD Studio). In fact, it is IDE that we run analysis from. The number of processor cores being used is the number specified in the settings.

Regarding, for instance, the filters system (Message Suppression and Detectable Error), it is NOT used when analyzing code from the command line: it is not used in the sense that you'll get all the error messages in the report file regardless of the parameters you've specified. But when loading the log file in IDE, filters will be applied to it. It happens because the filters are dynamically applied to results (when launching from IDE as well). It's very convenient because when you get a message list, you may want to turn off some of them (for example, V201). You can just turn them off in the settings and the corresponding messages will be excluded from the list WITHOUT relaunching the analysis.

Regular use and integration of PVS-Studio into the process of "daily builds"

When using PVS-Studio regularly, it's very important that you use the "Mark as False Alarm" function. This function allows you to mark those diagnostic messages generated by the PVS-Studio analyzer which are false positives in order not to see them the next time you run the analyzer. For example, the analyzer used to generate the V101 message for the following code fragment but stopped doing it after adding a special comment:

size_t bufferSize = imageWidth * imageHeght * //-V101
                    bytePerPixel * maxFrameCountInBuffer;

This comment can be added manually, but we recommend that you use a special command of the PVS-Studio menu.

Handling the PVS-Studio code analyzer consists of two stages: deployment and constant use. At the stage of deploying PVS-Studio into an existing large project, programmers review diagnostic messages and either fix the code, or mark it through the "Mark as False Alarm" function. Having sorted it out, they run the final analysis of the whole code to get 0 messages (if displaying of false positives is turned off). It means that the deployment stage is over and the stage of regular use is starting.

From now on all the fresh code fragments to be added into the project will be checked with PVS-Studio. In fact, the WHOLE code will be checked, but only new errors will be detected: they will be found in the freshly written/fixed code as well as in old code that wasn't marked as false positives.

Developers need the possibility of running analysis from the command line to perform regular (for instance, daily) code check. This is what the procedure looks like:

You run code analysis from the command line.

You send the resulting report file to all the developers involved.

When running PVS-Studio from the command line, a text file is created (besides the xml-file named, for instance, result-log.plog) whose name ends with ".only_new_messages.txt" (for instance, result-log.plog.only_new_messages.txt). This file contains new messages generated by PVS-Studio (the files not marked as False Alarm):

c:\users\evg\documents\omnisample\x64Sample.cpp(17): error V101: Implicit assignment type conversion to memsize type.

c:\users\evg\documents\omnisample\x64Sample.cpp(50): error V201: Explicit type conversion. Type casting to memsize.

c:\users\evg\documents\omnisample\x64Sample.cpp(54): error V102: Usage of non memsize type for pointer arithmetic.

Or, if there are no new messages, the file result-log.plog.only_new_messages.txt contains the phrase:

New diagnostic messages are not produced.

The xml-file will anyway contain the identical number of messages, while the txt-file only those which weren't marked as "False Alarm".

The existing txt-file containing the new PVS-Studio-generated messages won't be large if you run PVS-Studio and handle the analyzer-generated messages daily. You can send it to all the developers involved via e-mail. Sending the xml-file, in its turn, doesn't seem reasonable, as it contains full paths to the files. But if these paths coincide in different developers' computers, you may send the xml-files as well.

Thus, running PVS-Studio daily will help you to avoid new errors in your code.

Running PVS-Studio from the command line without using Visual Studio/RAD Studio

The way of running PVS-Studio from the command line we've just discussed will work ONLY if you have correctly set project files (.vcproj/cbproj) and solution/project groups files (.sln/groupproj). If you don't have these files but still want to check your code, see the article "Direct integration of the analyzer into build automation systems (in the absence of .vcproj/.vcxproj/.cbproj project files)".

Conclusion

Running PVS-Studio daily from the command line will help you to significantly increase the quality of your code, especially if this code is marked through the "Mark As False Alarm" function. In this case you will see 0 new messages every day if the code is correct. If some old code (that was reviewed earlier) is modified incorrectly, the next PVS-Studio check will at once reveal the defects introduced. The freshly written code will be, of course, also checked regularly in automated (independent from the programmer) mode.