When we publish new articles about our static code analyzer for C++ code PVS-Studio, people will often comment on them with one of the following questions or statements.
Now we must give a detailed reply to each of these items.
Comparing code analysis tools is a difficult task (I wrote about that in one article). You cannot compare static analyzers only by the sets of diagnostic rules they include, for they are used in everyday life to check real projects, not artificial tests. It is at this point that you figure out that using a static analyzer involves two stages: integration into a project under development and regular use.
Imagine a programmer who tries Clang for the first time. He takes a few small test files with bugs, feeds them to the analyzer and finds that it has detected some of the bugs and missed some others. Suppose the programmer has liked the diagnostics that were triggered. Then he tries the analyzer on his real project and gets hundreds, if not thousands, of warning messages. He can't fix them all at once, yet he doesn't know what to do with them because analyzers like Clang simply output them into the console. It is at this point that the stage of analyzer integration begins.
The stage of static analysis integration into a current project (which means it is pretty large) is meant to get the analyzer to generate 0 diagnostic messages on the code. Now we find out that Clang has nothing to offer regarding this aspect; it simply doesn't have any mechanisms to handle a big list of diagnostic messages. The only way is to manually review and revise them, if necessary, one by one.
What does PVS-Studio have to offer here? Lots:
The crucial thing is that all these mechanisms can be applied real-time, without you having to relaunch the analysis. It is very crucial because analysis of large projects takes considerable time.
Why does PVS-Studio have such features to simplify the integration stage and Clang doesn't? The reason is PVS-Studio displays analysis results in a table (see the figure):
Figure 1 - Representation of analysis results in PVS-Studio.
To be more exact, the table is a visual representation of a database behind it, which makes possible all the mentioned mechanisms of data filtering and processing. On the other hand, clang outputs its results into the console. If you integrate clang into an IDE, you'll get the navigation function, but that's all. So, here we come to the basic difference between PVS-Studio and Clang.
PVS-Studio provides means of tool integration (i.e. complete elimination of diagnostic messages generated for the current project), whereas Clang doesn't. You don't care about it much when you just read articles on code analysis on the Internet, but it gets very important once you start dealing with static analysis in everyday practice and try to integrate it into your own project you've been working on for a few years.
You may suspect that I consciously avoid comparing PVS-Studio and Clang by the sets of diagnostics. Well, yes and no. It's not an easy task to compare tools' diagnostic capabilities as such. But what's more, results of such a comparison get obsolete very quickly. Both we and the Clang developers regularly add new diagnostic rules into our analyzers.
When one says, "I run clang and get 0 messages, though I didn't perform any integration", it means that one just uses an already integrated tool. That's it.
Now let's discuss why Clang is free and PVS-Studio is not. Programmers don't usually wonder about the source they get their money from. Clang is developed by guys from Apple, Google and Intel, while PVS-Studio is developed as an independent project, so we need to provide us with salaries on our own - that's why PVS-Studio is a paid product. We don't insist on everyone using it, of course. Our customers are the users who need PVS-Studio and understand "what they pay for".
Although Clang's analyzer is an open source project, a programmer who is not an expert in static analysis will have a hard time trying to add a self-made diagnostic rule into the tool. But you shouldn't rely on my saying so; just get the tool and try it yourself if it is urgent for you.
And, finally, are we afraid of Clang becoming a considerable rival to us? Not yet, as Clang's static analyzer is currently just one more feature among many others provided by the tool, whereas ours is a specialized product. Besides, moving an incomplete project to another compiler (and you'll have to use Clang's compiler if you want to use Clang's static analyzer) is a difficult task.
However, us having written this text doesn't mean at all that we despise Clang. It's a very nice project, and PVS-Studio even utilizes a part of it (as a preprocessor), and the large team working on it are highly-skilled developers.
We have a practice of occasionally re-analyzing projects we have already checked with PVS-Studio. There are several reasons why we do so. For example, we want to know if we have managed to eliminate false positives for certain diagnostics. But the most interesting thing is to see how new diagnostic rules work and what errors they can find. It is ...