Verification and validation

Andrey Karpov
Articles: 371



The terms verification and validation are related to software quality checking. We use these terms in our articles and reports. Very often we have heard various comments and debates concerning the question if static analysis of program source code should be referred to verification and validation and in what way these notions differ. On a whole, it seems that everyone means something different by these terms and it leads to a mutual misunderstanding.

We decided to make it clear in order to stick to the most proper understanding of these notions. During the investigation we came across a work by V.V. Kulyamin "Software verification methods" [1]. It gives a thorough description of the terms and we decided to further rely upon the definitions provided in this work. Here are some extracts from it related to verification and validation.

Verification and validation are the types of activity intended for checking the quality of software and detecting errors in it. Having the same goal, they differ in the origin of the properties, rules and limitations (whose violation is considered an error) being tested during these processes.

Before going on we must introduce the term "an artefact of software life-cycle". By artefacts of software life-cycle we understand various information entities, documents and models created or used during the process of software development and maintenance. Thus, requirements specification, architecture description, domain model in some graphics language, source code, user documentation etc. are artefacts. Various models used by single developers during software creation and analysis but not fixed in the form of documents available to other people are not considered artefacts.

Verification checks if some artefacts being created while developing and maintaining software correspond to some other artefacts created earlier or being used as source data and also if these artefacts and processes of their development correspond to the rules and standards. In particular, verification checks if standards, software requirements specification, design decisions, source code, user documentation and operation of the software itself correspond to each other. Besides, it checks if the requirements, design decisions, documentation and the code are arranged in correspondence with the software development rules and standards accepted in a particular country, branch and organization, and also if all the operations stated in the standards are executed and in the appropriate sequence. Defects and errors detected during verification are divergences between some of the listed documents, between documents and actual program operation or between standards and actual processes of software development and maintenance. Making a decision what document must be corrected (if not both of them) is a separate task.

Validation checks if any artefacts being created or used in the process of software development and maintenance correspond to the needs of the users and customers of this software, with taking into account the laws of the domain and limitations of software usage context. These needs are usually not fixed in the form of a document - when they are, they turn into a requirements specification, i.e. one of the artefacts of the software development process. That is why validation is a less formal activity than verification. It is always carried out in the presence of customers', users', business-analysts' or domain experts' representatives - those whose opinion can be considered a rather good expression of the needs of users, customers and other persons concerned. The methods of its execution often involve specific techniques of discovering knowledge and actual needs of the participants.

The difference between verification and validation is illustrated in Figure 1.

Figure 1 - Relation between verification and validation

Figure 1 - Relation between verification and validation

The definitions given above are drawn through some extension of the definitions of IEEE 1012 standard on the processes of verification and validation [2]. In the standard software engineering glossary IEEE 610.12 of 1990 [3], the definition of verification is nearly the same but the definition of validation is different - it is said there that validation must check if the result software corresponds to the requirements set to it in the beginning of the development process. In this case, validation would be an instance of verification but it is not mentioned anywhere in literature on software engineering, and this is the reason, together with the fact that this definition was corrected in IEEE 1012 of 2004, that it should be considered incorrect. This phrase by B. Boehm which is used very frequently [4]:

Verification answers the question "Are we making the product properly?" while validation - "Are we making a proper product?"

also contributes to the confusion because its aphoristic character, unfortunately, combines with ambiguity. But multiple works of this author allow us to think that he meant by verification and validation nearly the same notions that were defined above. The described discrepancies can be traced in the content of software engineering standards. Thus, ISO 12207 standard [5] considers testing a kind of validation but not verification and it seems to result from sticking to the incorrect definition of the standard glossary [3].

In conclusion, I would like to note that according to the definitions given above static source code analysis corresponds to software verification being the method of checking correspondence between program code and various coding standards. Static analysis checks if the results of the stage of designing a program system correspond to the requirements and limitations stated before.

References

  • V.V. Kulyamin, "Software verification methods". RAS System Programming Institute, 25 B. Kommunisticheskaya st., Moscow, 109004.
    http://www.ict.edu.ru/ft/005645/62322e1-st09.pdf
  • IEEE 1012-2004 Standard for Software Verification and Validation. IEEE, 2005.
  • IEEE 610.12-1990 Standard Glossary of Software Engineering Terminology, Corrected Edition. IEEE, February 1991.
  • B. W. Boehm. Software Engineering; R&D Trends and Defense Needs. In R. Wegner, ed. Research. Directions in Software Technology. Cambridge, MA:MIT Press, 1979.
  • ISO/IEC 12207 Systems and software engineering - Software life cycle processes. Geneva, Switzerland: ISO, 2008.


Use PVS-Studio to search for bugs in C, C++, C# and Java

We offer you to check your project code with PVS-Studio. Just one bug found in the project will show you the benefits of the static code analysis methodology better than a dozen of the articles.

goto PVS-Studio;

Andrey Karpov
Articles: 371


Bugs Found

Checked Projects
344
Collected Errors
12 970