While communicating with programmers who ponder over the idea of using a static code analyzer, you may often here them say: "You don't need any special interface for a tool of that kind. A simple command line tool will do. And you can always save stdout into a file". I want to show you what is wrong about this way of thinking by the example of PVS-Studio's Output Window.
So, this is the minimum information you can see in a report file of any static code analyzer or compiler that generates a list of error messages:
Really, it seems that you can well arrange this information in a form of a text file or stdout. Most compilers actually do it that way. And what about static analyzers? Maybe they can "spit it all into stdout" too? And how do you tell the difference?
The difference between static analyzers and compilers regarding results representation is only one yet crucial parameter. This is the number of messages output.
When you deal with a compiler, you don't get too many messages from it. Usually there are several error messages (not more than ten of them) and a bit more (about several dozens at most) warnings. Complete absence of warnings in a project is considered a good style. Of course there are cases when there are much more compilation errors but they are rare compared to the total number (when porting the code to a different platform or when libraries being used undergo global changes). So, once again, a user gets just a few error messages in the usual operation mode.
Handling a static analyzer is quite a different thing. Usually a static analyzer is "being set on" an already existing project. And the larger the project is, the greater urge the team feels to use a static analyzer. What follows from that? A static analyzer will generate hundreds of messages for a small project, thousands of messages for a medium project, and a large project (the team has been working for years on) will cause even dozens of thousands of messages. You have to handle all these messages somehow, analyze them and so on.
So what is the difference between the contemporary output window of a static analyzer (for example, in PVS-Studio) and plain text stdout? I will first enumerate common things many of which can be found in the Error List window in Visual Studio (although stdout cannot do even that):
And now let's see what capabilities significant especially for a static analyzer are provided by a window.
Even when static analysis gets integrated into a project and the number of messages generated is few, it's still much more convenient to handle them through a specialized window.
The five advantages of message output window I have listed show it very well that it's much more convenient to handle a static analyzer when it has such an interface. Phrases like "stdout is enough" result from lack of understanding. It's "enough" only in theory. But in practice it will be impossible to use a tool until you create a similar infrastructure for yourself (the capability to navigate on the code, sort and filter messages, etc.).
Many programmers think that the more error messages a static code analyzer produces, the better. It would be true if all the messages hit the bull's eye, as they say. But this is impossible: the same warnings may be considered both true and false by different programmers depending on the project type. There is also one more important and interesting ...
As a PVS-Studio's developer, I am often asked to implement various new diagnostics in our tool. Many of these requests are based on users' experience of working with dynamic code analyzers, for example Valgrind. Unfortunately, it is usually impossible or hardly possible for us to implement such diagnostics. In this article, I'm going to explain briefly why static code analyzers ...