This is the second of our two-part blog concerning verification and validation. In part 1 we discussed the basic concepts of verification and validatlion in terms of the context in which they are used within the system development lifecycle.
In this second part of the blog, we discuss these concepts in terms of the Systems Engineering “V-model.
While the previous blog presented definitions that are useful in helping address the issues of ambiguity in the use of the terms verification and validation, the system engineering process is more complex. Systems Engineering is an iterative and recursive process. Requirements development and design occur top-down as shown on the left side of the SE “V” as shown in Figure 2.
Figure 2: Verification and Validation and the Systems Engineering “V” Model
Systems engineering starts with the concept stage where stakeholder needs, expectations, and requirements are elicited, documented, and baselined. This is part of defining the scope of the system to be developed. Next, the stakeholder needs, expectations and requirements are transformed into a set of system requirements which are baselined via requirements verification and requirements validation as discussed above. Once the system requirements are baselined, design results in a system architecture in which the subsystems are defined. The design at this level is baselined via the design verification and design validation process discussed above.
For each subsystem, the above cycle is repeated with the definition of stakeholder subsystem needs, expectations, and stakeholder requirements, and then transformed into subsystem requirements, which are baselined via requirements verification and requirements validation. Once the subsystem requirements are baselined, design results in a subsystem architecture in which the units/components are defined. This design is baselined via the design verification and design validation process. This process repeats until the organization makes a buy, build, code, or reuse decision.
System integration, system verification, and system validation (IV&V) occur bottom-up as shown on the right side of the SE “V” as shown in Figure 2.
Once all the components that make up the subsystems are developed, unit/component verification and unit/component validation take place as described above. Once these activities are complete, the units/components are integrated together and then the resulting subsystems are verified and validated as described above. Once the subsystem verification and subsystem validation activities are complete, the subsystems are integrated together and then system verification activities are completed. In the end, proof will be documented that can be evaluated by the customer to determine system verification activities have been completed successfully showing that the stakeholder requirements have been met (both organizational/people requirements as well as the technical requirements).
Following system verification activities, system validation activities are performed. This could be done in the form of acceptance and/or operations evaluation and validation activities. In the end, proof will be documented that can be evaluated by the customer to determine whether system validation activities have been completed successfully, stakeholder needs have been met, and the system will operate as intended in its operational environment.
Following the customer evaluation, the system can be accepted and ownership transferred to the customer.
Note: It is possible to pass verification but fail system validation. This occurs when you have a defective set of requirements that are not correct, are incomplete, inconsistent, and unclear. By having a good requirement verification and requirement validation process in place, you increase the chances of passing both system verification and system validation and thus reducing the risk of cost and schedule overruns due to rework.
Conclusions and parting thoughts on verification and validation
While the concepts of verification and validation are commonly used, the words are often used interchangeably and the meaning of the concepts is often misunderstood resulting in confusion. This paper addressed the use of the terms verification and validation and presented their various meanings, particularly in the relevant standards and literature, as well as everyday usage. In particular, it was identified that both terms are very ambiguous unless a modifier is included in front of the word clearly indicating what the context in which term is referring to, specifically: requirements, the design, or to the system under development. The meaning is also ambiguous unless it is clear whether the terms are used to refer to an activity as part of a process or to an outcome of that process.
Before closing, there are two very important points that need to be made concerning system verification and system validation:
1) Project managers often don’t understand these concepts nor the resources needed to accomplish the activities associated with the product lifecycle phases. We have seen cases where the cost to do integration, system verification, and system validation activities ended up costing between 50%-70% of the total development cost. The higher percentages occur due to having a poor set of requirements and thus the resulting rework. Project managers who only budget for maybe 20% of the development costs are in for a rude awaking and are setting their project up for failure.
2) The concept of “system validation” discussed above is part of the formal product development lifecycle addressed in most textbooks. What most people don’t recognize is that there is also an “informal” system validation that takes place after a product is released for use. This informal system validation is communicated to the developing organization is various forms including product failures during operations, product recalls, product returns, warranty work, negative customer comments and reviews on social media, declining reputation, decreased sales, sales projections not being met, etc.
All of these things have a direct impact on profits and mission success. In today’s world of limited funding, profit margins are getting tighter and tighter. If organizations took the formal verification and validation activities and processes seriously during product development, the number of issues that occur after the product is released for use would be significantly reduced – avoiding the negative “informal” system validation we frequently see. Companies cannot afford to fail informal system validation any more than they can afford to fail formal system validation.
I go into a lot more detail on the topics of verification and validation in my paper “Thinking ahead to verification and validation”. Send me an email at info@argondigital.com and I will send you a copy of the paper.
Comments to this blog are welcome.
If you have any other topics you would like addressed in our blog, feel free to let us know via our “Ask the Experts” page and we will do our best to provide a timely response.
References
INCOSE-TP-2003-002-04, 2015, Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities. Version 4, Edited by Walden, David D., et al.
INCOSE-TP-2010-006-02, 2015, Guide to Writing Requirements, Prepared by the Requirement Working Group, INCOSE
ISO/IEC 15288, ISO/IEC 15288:2015, 2015, Systems Engineering—System Life Cycle Processes.
ISO/IEC 29148, 2011, ISO/IEC 29148 FDIS Systems and Software Engineering—Life Cycle Processes—Requirements Engineering.
Ryan, M.J., Wheatcraft, L.S.; Dick, J.; and Zinni, R., 2015. “An Improved Taxonomy for Definitions Associated with a Requirement Expression”, Systems Engineering and Test and Evaluation Conference SETE 2014, Adelaide, Australia, 28-30 April 2014 and INCOSE IS 2015, Seattle, WA 12-17 July 2015.
Ryan, M., Wheatcraft, L., “On the Use of the Terms Verification and Validation”, November 2016.
Wheatcraft, L.S., 2012, Thinking Ahead to Verification and Validation, presented at NASA’s PM Challenge 2012, February 2012, Orlando, Florida.
