Analyzing Visual Studio projects from the command line


In addition to using PVS-Studio directly from Visual Studio, you can also run analysis of MSBuild (i.e. Visual C++ and Visual C#) projects from the command line. This can be useful for setting up regular automatic analyzer runs. For example, during the "night builds" on the build server.

Starting analysis on sln and csproj/vcxproj files

To analyze C++/C# projects or solutions (sln) files that contain such projects, you can use the command line version of the PVS-Studio analyzer directly without having to start the IDE (devenv.exe) process and open in it projects that you are intending to check.

The 'PVS-Studio_Cmd.exe' tool can be found in PVS-Studio installation directory (default path is ' c:\Program Files (x86)\PVS-Studio\'). The '--help' command displays all available arguments of cmd analyzer:

PVS-Studio_Cmd.exe --help

The main arguments of the analyzer:

  • --target (-t): required parameter. Allows you to specify the target for analysis (sln or csproj/vcxproj file);
  • --sourceFiles (-f): a path to an XML file which can be used to specify a list of files and\or filters to be analyzed. Only the files specified by this list will be analyzed in the project(s). The detailed description of this file's syntax is available in the corresponding subsection below.
  • --output (-o): path to the plog file, where the analysis results will be written. If this parameter is missing, a plog file will be created next to the file indicated in the target;
  • --platform (-p) and --configuration (-c): platform and configuration of the target which will be analyzed. If these parameters are not specified, then the first available "platform|configuration" pair will be chosen (when checking the sln file) or "Debug|AnyCPU" (when analyzing a single csproj project) or "Debug|Win32" (when analyzing a single vcxproj project);
  • --settings (-s): path to the configuration file of PVS-Studio. If the parameter is missing, IDE settings of PVS-Studio plug-in will be used, located in the folder "c:\Users\%UserName%\AppData\Roaming\PVS-Studio\Settings.xml". Note that for the C# analyzer to work properly, your settings, passed through this flag, should contain your registration information (if you are using the default settings from the AppData, your registration information can be entered via the PVS-Studio plug-in in Visual Studio);
  • --progress (-r): allows to enable the detailed logging of the analysis progress to stdout (disabled by the default);
  • --suppressAll (-a): append all un-suppressed messages to the 'suppress' files of corresponding projects (disabled by default). If this flag is used, all the diagnostic messages will be added into the database of suppressed messages after saving the analysis log file. The flag supports two modes:
    • SuppressOnly adds analyzer messages from the input plog to the suppress files without starting the analysis;
    • AnalyzeAndSuppress will start the analysis, save the resulting plog, and only then the messages from this plog will be added to suppress files. Using this mode, you will only see messages generated for newly written/changed code at each analyzer run; that is, new messages will be written into the new log and get immediately suppressed so that you don't see them at the next run. However, if you need to view old messages (without having to re-run the analysis on the project), you can view them anytime by opening the complete-log file which is automatically saved in the same folder with the new-messages log file. To learn more about the message suppression mode, see this documentation section;
  • --sourceTreeRoot (-e): the root of a source tree. PVS-Studio will use this value to generate relative paths in its messages. Settings this option will override the 'SourceTreeRoot' value in PVS-Studio settings;
  • --incremental (-i): enable the incremental analysis mode. Refer to the "PVS-Studio's incremental analysis mode" topic for more details about incremental analysis in PVS-Studio. The following operating modes of incremental analysis are available:
    • Scan – scan all the dependencies to determine source files for incremental analysis. Note that at this step the incremental analysis will not be started. The previous history of edits will be removed, the analyzer will take into account only the modifications made after the last build.
    • AppendScan – analyze all the dependencies to determine source files for incremental analysis. Note that at this step the incremental analysis will not be started. The analyzer will take into account the modifications made after the last build and all the previous modifications.
    • Analyze – run incremental analysis on the target. This step should be executed after executing the 'Scan' and 'AppendScan' steps, and can be executed both before and after building the target. Static analysis will be performed only for the files from the list, received in the result of the execution of 'Scan' and 'AppendScan' commands.
    • ScanAndAnalyze – scan dependencies to determine source files for incremental analysis and analyze the files that were modified since last build.
  • --msBuildProperties (-m): set or override the specified project-level properties. Use a vertical bar "|" to separate multiple project-level properties: --msBuildProperties WarningLevel=2|OutDir=bin\OUT32\

Here is an example of running the analysis for Visual Studio solution "My Solution":

PVS-Studio_Cmd.exe --target "mysolution.sln" --platform "Any CPU" 
--configuration "Release" --output "mylog.plog"
--settings "pvs.xml" --progress

The command line version of the PVS-Studio analyzer supports all settings on filtering/disabling messages available in the IDE plugin for Visual Studio. You can either set them manually in the xml file, that is passed through the '--settings' argument, or use the settings specified through the UI plugin, without passing this argument. Note that the IDE plug-in of PVS-Studio uses an individual set of settings for each user in the system.

Specifying individual files for analysis

PVS-Studio_Cmd allows you to analyze individual files specified in a list, a path to which can be passed to the analyzer with the --sourceFiles (-f) flag. This file list should be in XML format and it can contain a list of absolute paths to the source files and\or a list of file masks that define such source file paths.

<SourceFilesFilters>
  <SourceFiles>
    <Path>C:\Projects\Project1\source1.cpp</Path>
    <Path>C:\Projects\Project1\source1.h</Path>
    <Path>\Project2\source2.cpp</Path>
    <Path>source_*.cpp</Path>
    <Path>stdAfx.h</Path>
  </SourceFiles>
  <TlogDirs>
    <Dir>D:\Build\Tlogs\</Dir>
  </TlogDirs>
  <SourcesRoot>C:\Projects\</SourcesRoot>
</SourceFilesFilters>

Let us review in detail all of available fields and their possible values.

  • The SourceFiles element allows you to specify a list of files and\or filters for analysis. Each file\filter is specified in the Path sub-element. These filters will be applied to all files from all projects in the solution under analysis. Only the files that are either specified directly or conform to one of the filters will be queued for analysis. You can specify both source files (c/cpp for C++ and cs for C#) and header files (h/hpp for C++). Important note: analysis of header files for C/C++ is only possible when compiler tracing logs are available. The way to specify them is described for the TLogDirs element below. Directory separators in the values specified in the Path elements will be normalized. However, the special directory traversal directives, such as '.' or '..' will not be expanded. The file path masks support '*' and '?' wildcard characters. Possible values of the Path sub-elements are the following:
    • An absolute path to a file will be directly compared with source files within projects
    • A relative path to a file (for example, \Project1\source2.cpp) or any mask that does not start with a root directory (i.e. 'C:\' or '\\' for UNC paths) will be interpreted as a mask starting with '*' wildcard. For example, 'stadAfx.h' will be interpreted as '*stdAfx.h'.
  • The TlogDirs element specifies a list of directories that will be used to search for tlog build artifacts - compiler tracking logs. These files are required for analyzing individual C/C++ header files - as the analyzer works only with compilation units (i.e. c/cpp source files), it is unable to analyze a header file 'by itself', separate from a source file that includes it. To determine the dependency between source and header files, analyzer utilizes tlog files. Tlog files are generated by MSBuild build system while building a project, inside the build output directory, which also contains other build artifacts (such as, for example, compiled binaries). Analyzer needs tlog files with a following name pattern: CL.read.*.tlog, where '*' can be any number. Usually MSBuild generates one such tlog file for each project. You can use the Dir sub-elements to either specify several directories for each separate tlog file, or specify a single root directory, which will contain all of the tlogs - the analyzer will search for tlogs in all of the sub-directories in each of the Dir element. Important node: if tlog files were not generated by building the analyzed project locally (i.e. the build was performed either in a different directory from which the analysis is ran, or on a different machine altogether), then for a correct analysis of individual header files you should additionally specify the current root directory for the project you are analyzing using the SourcesRoot element. The reason for this is that the tracing logs (tlogs) contain file paths in an absolute format.
  • The SourcesRoot element allows you to specify the base root directory containing the sources of project under analysis. If you are using tlog (TLogDirs element) files that were not generated locally, under the analyzed project, specify this root directory to allow the analyzer correctly identify the source files from the analyzed project among the paths that are present in the tlog files. Only the current root directory of the analyzed project should be specified in the SourcesRoot element. For example, if the sources of the analyzed projects are located in the C:\Projects\Project1, C:\Projects\Project2, etc. folders, then you should specify C:\Projects\ as the SourcesRoot element.

When running analysis with the --sourceFiles (-f) flag, the analyzer, contrary to the 'normal' run on the whole project\solution, will generate its warning messages only for the files that are explicitly specified through the SourceFiles filters. This means that when analyzing a source cpp file, the analyzer will only generate messages for this file alone - the messages on the header (h/hpp) files that are included into this source file will be filtered out, unless these header files are also explicitly specified by the SourceFiles filters.

Command line tool exit codes

The PVS-Studio_Cmd utility defines several non-zero exit codes, which do not necessarily indicate some issue with the operation of the tool itself, i.e. even when the tool returns something other when zero it does not always mean that the tool has 'crashed'. The exit code is a bit mask that represents all states that occurred during the PVS-Studio_Cmd utility operation. For example, the tool will return non-zero code (it will be 256 actually) when the analyzer finds some potential issues in the code being analyzed. Such behavior allows you to handle this situation individually, for example, when the policy of using the analyzer on a build server does not allow analyzer issues to be present in the code that was committed to a revision control system. Consider another example: during analysis there were found some issues in code, and one of the source files is missing on disk. In this case the exit code of the PVS-Studio_Cmd utility will be 264 (8 - some of the analyzed source files or project files were not found, 256 - some issues were found in the source code), or, in binary representation 100001000.

Next, let's examine all possible PVS-Studio_Cmd state codes that form a bit mask exit code.

  • '0' - analysis was successfully completed, no issues were found in the source code;
  • '1' - error (crash) during analysis of some source file(s);
  • '2' - general (nonspecific) error in the analyzer's operation, a possible handled exception. Usually, this means an error inside the analyzer and is followed by a stack trace. If you encounter this error, please help us improve the analyzer by sending it to us;
  • '4' - some of the command line arguments passed to the tool were incorrect;
  • '8' - specified project, solution or analyzer settings file were not found;
  • '16' - specified configuration and (or) platform were not found in a solution file;
  • '32' - solution file or project is not supported or contains errors;
  • '64' - incorrect extension of analyzed project or solution;
  • '128' - incorrect or out-of-date analyzer license;
  • '256' - some issues were found in the source code;
  • '512' - some issues were encountered while performing analyzer message suppression;
  • '1024' - indicates that analyzer license will expire in less than a month;

Let us provide an example of a Windows batch script that decodes the PVS-Studio_Cmd utility exit code:

@echo off

"C:\Program Files (x86)\PVS-Studio\PVS-Studio_Cmd.exe"
-t "YourSolution.sln" -o "YourSolution.plog"

set /A FilesFail = "(%errorlevel% & 1) / 1"
set /A GeneralExeption = "(%errorlevel% & 2) / 2"
set /A IncorrectArguments = "(%errorlevel% & 4) / 4"
set /A FileNotFound = "(%errorlevel% & 8) / 8"
set /A IncorrectCfg = "(%errorlevel% & 16) / 16"
set /A InvalidSolution = "(%errorlevel% & 32) / 32"
set /A IncorrectExtension = "(%errorlevel% & 64) / 64"
set /A IncorrectLicense = "(%errorlevel% & 128) / 128"
set /A AnalysisDiff = "(%errorlevel% & 256) / 256"
set /A SuppressFail = "(%errorlevel% & 512) / 512"
set /A LicenseRenewal = "(%errorlevel% & 1024) / 1024"


if %FilesFail% == 1 echo FilesFail
if %GeneralExeption% == 1 echo GeneralExeption
if %IncorrectArguments% == 1 echo IncorrectArguments
if %FileNotFound% == 1 echo FileNotFound
if %IncorrectCfg% == 1 echo IncorrectConfiguration
if %InvalidSolution% == 1 echo IncorrectCfg
if %IncorrectExtension% == 1 echo IncorrectExtension
if %IncorrectLicense% == 1 echo IncorrectLicense
if %AnalysisDiff% == 1 echo AnalysisDiff
if %SuppressFail% == 1 echo SuppressFail
if %LicenseRenewal% == 1 echo LicenseRenewal

Running analysis from the command line for C/C++ projects built in build systems other than Visual Studio's

If your C/C++ project doesn't use Visual Studio's standard build system (MSBuild) or even uses a custom build system \ make files through Visual Studio NMake projects, you won't be able to analyze this project with the PVS-Studio_Cmd.

If this is the case, you can use the compiler monitoring system that allows you to analyze projects regardless of the build systems they use by "intercepting" compilation process invocations. The compilation monitoring system can be used both from the command line and through the C and C++ Compiler Monitoring UI client (Standalone.exe).

You can also integrate the invocation of command line analyzer kernel directly into your build system. Notice that this will require you to define the PVS-Studio.exe analyzer kernel call for each file to be compiled, just like in the case when calling the C++ compiler.

How PVS-Studio settings affect the command line launch; analysis results (plog file) filtering and conversion

When running project analysis from the command line, by default, the same settings are used as in the case when it is run from the IDE (Visual Studio). The number of processor cores to be used for analysis depends on the number specified in the analyzer settings. You can also specify the settings file directly with '--settings' argument, as was described above.

As for the filtering systems (Keyword Message Filtering and Detectable Errors), they will not be used when running the analysis from the command line. It means you'll see all the messages in the log file anyway, regardless of the parameters you have specified. However, when loading the log file in the IDE, the filters will be applied. This is because filters are applied dynamically to the results (including the case when you analyze your project from the IDE as well). It's very convenient since you may want to switch off some of the messages you've got (for example V201). You only need to disable them in the settings to have the corresponding messages disappear from the list WITHOUT you having to rescan the project.

The plog file format (XML) is not intended for directly displaying to or read by a human. However, if you need to filter off the analysis results and convert them into a "readable" format, you can use the PlogConverter utility available within the PVS-Studio distribution. You can also download the source code of the utility.

PlogConverter allows you to convert one or more plog files into the following formats:

  • Light-weight text file with analysis results. It can be useful when you need to output the analysis results (for example new messages) into the log of the build system or continuous integration server.
  • HTML report with a short description of the analysis results. It suits best for the e-mailing of the notifications;
  • HTML report with sorting of the analysis results according to the different parameters and navigation along the source code;
  • Tasks - format that can be opened in QtCreator;
  • CSV table with analysis results.
  • Text file with a summary table of the number of messages across different levels and diagnostic rule sets.

The format of the log file is defined by the command line parameters. You can also use them to filter the results by rule sets, levels, and individual error codes.

Here's an example of the PlogConverter command line launch (in one line):

PlogConverter.exe test1.plog test2.plog -o "C:\Results" -r "C:\Test" 
-a GA:1,2,3;64:1 -t Html,Txt,Totals -d V101,V105,V122

PlogConverter will be launched for the 'test1.plog' and 'test2.plog' files in the current folder and the error messages from all specified plog files will be merged and saved into the Results folder on disk C; only messages from the General Analysis (GA) rule set of levels 1, 2, and 3, and from the 64-bit diagnostic rule set (64) of level 1 will be used. Diagnostics V101, V105, and V122 will be filtered off from the list. The results will be saved into a text and HTML files as well as a summary text file for all the diagnostics (with the mentioned parameters taken into account). The original plog file will remain unchanged.

Detailed description of the levels of certainty and sets of diagnostic rules is given in the documentation section "Getting Acquainted with the PVS-Studio Static Code Analyzer".

A detailed help on all the parameters of the PlogConverter utility can be accessed by executing the following command:

PlogConverter.exe --help

If you need to convert the plog file into some specific format, you can do it by parsing the file as an XML document yourself.

Note that you can reuse the algorithm of sorting out a structure of a plog file from our utility to create a customized output format of analysis results.

Regular use of PVS-Studio and integration with "daily builds"

The process of long-term use of the PVS-Studio code analyzer is comprised of two stages: integration and regular use. When integrating PVS-Studio into an existing large-scale project, programmers will usually read the analyzer messages and either fix the code or mark it using the "Mark as False Alarm" and "Message Suppression" features. Having sorted out all the messages, they will then re-analyze the code once again to get 0 warnings (given that the messages marked as false positives or suppressed through suppress files are set to stay hidden). It means that the integration stage is over and the stage of regular use sets in.

From this point on, all the new code added into the project will be analyzed by PVS-Studio. Actually, the ENTIRE code base will be analyzed, but you'll be seeing only new messages. Errors will be found in freshly written/fixed code or old code that has been left unmarked.

The option of running the analysis from the command line is useful when developers need to launch it regularly (daily, for instance). The procedure looks as follows:

  • Running the analysis from the command line.
  • Sending the resulting log file to all the developers involved.

This way, regularly running PVS-Studio on your project will help you avoid new bugs in the code.

Automatic update and installation of the PVS-Studio distribution

To install the PVS-Studio distribution from the command line in a "quiet" mode (i.e. without displaying the UI and dialog boxes), you need to pass the following parameters into it (in one line):

PVS-Studio_setup.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART
/COMPONENTS=Core,Standalone,MSVS,MSVS\2010,
MSVS\2012,MSVS\2013,MSVS\2015,MSVS\2017,MSVS\2019

The /COMPONENTS argument allows specifying the components to be installed: Standalone (C and C++ Compiler Monitoring), plugins for different IDEs and is optional.

PVS-Studio may require a reboot if, for example, files that require update are locked. To install PVS-Studio without reboot, use the 'NORESTART' flag. Please also note that if PVS-Studio installer is started in a silent mode, the computer may be rebooted without any warnings or dialogs.

Keep in mind that it is required to avoid running Visual Studio (the devenv.exe process) while installing PVS-Studio.

The PVS-Studio-Updater.exe utility can check for analyzer updates and download and install them on a local machine. To run the update utility in the 'quiet' mode, you can use the same parameters as for the distribution installation process:

PVS-Studio-Updater.exe /VERYSILENT /SUPPRESSMSGBOXES

If there are no updates available on the server, the utility will terminate with '0' exit code. Since PVS-Studio-Updater.exe performs local installation, you must avoid running the devenv.exe process while it is working as well.

Conclusion

Running PVS-Studio daily from the command line can help you significantly increase the quality of your code. Sticking with this approach, you will get 0 diagnostic messages every day if your code is correct. And should there be any incorrect edits of the old (already scanned and fixed) code, the next run of PVS-Studio will reveal any fresh defects. Newly written code will be regularly scanned in automatic mode (i.e. independently of the programmer).


Bugs Found

Checked Projects
344
Collected Errors
12 970