A question we are often asked is “What is difference between verification and validation?”
This two-part blog answers that question. In part 1 we discuss the basic concepts of verification and validation in terms of the context in which they are used within the system development lifecycle. In part 2 of the blog, these concepts are discussed in terms of the Systems Engineering V-model.
Now to the question: “What is the difference between verification and validation?”
In general, verification refers to the basics (structure) of the item (requirements, design, system) being verified, making sure it meets requirements that drive the creation of the item, whether it be rules on writing well- formed requirements, standards and best practices (external and internal) on the design, or requirements on the coding or manufacturing of the system. Validation goes beyond the basics (structure) to how well the item communicates or addresses stakeholder needs and expectations while operating in the intended operational environment.
While these terms are commonly used, the true meaning of the concepts represented in each are often misunderstood and the terms are often used interchangeably without making clear the context in which they are used – resulting in ambiguity. This is true particularly when the terms are combined into the generic terms verification and validation (V&V); independent V&V (IV&V); or integration, verification and validation (IV&V).
Using the terms interchangeably with the assumed or implied meaning and context leads to confusion and misunderstanding. When referring to IV&V, the ambiguity in the general use of the terms verification and validation creates even greater difficulties, especially if an organization is being contracted to perform “IV&V”. First, the ‘I’ in IV&V is often used to mean ‘independent’ V&V in which an outside organization is called in to undertake ‘V&V’ as an activity. However, what is the assumption concerning what each term do the “Vs” refers to: “validation and verification” or “verification and validation”? Verification of what? Validation of what? Without a qualifying adjective, context is assumed and it is therefore not clear what ‘V&V’ refers to: requirements, design, or the system that has already been designed and built in accordance with the requirements.
Sometimes, however, the ‘I’ means ‘integration’ when referring to the right-hand side of the systems engineering ‘Vee’ model that depicts the processes of integration, verification, and validation (see the discussion concerning the Vee Model below for explanation). Clearly, by not being precise in the use of the terms and not indicating the context intended, confusion can result.
Figure 1. Verification and validation are the processes of confirming that artifacts generated during the transformation processes are acceptable.
To avoid this ambiguity, each term needs to be preceded by a modifier (i.e., the subject) which clearly denotes the proper context in which the term is being used, specifically requirement verification or requirement validation; design verification or design validation; system verification or system validation as shown in Figure 1. The concepts of verification and validation are very different depending on the modifier. When using these terms, it should be clear as to which concept is intended.
Requirement Verification and Validation Defined
Context example: As shown in Figure 1, a requirement set results from a formal transformation of stakeholder needs and expectations. Correspondingly, design is a result of formal transformation of the requirement set in to an agree-to design, and a system is a formal transformation of the design into that system.
The process of creating a requirement set involves:
- analyzing stakeholder needs and expectations to obtain the necessary elements to be included in the requirement set;
- selecting a format for the requirement expression and an organization of the requirement set;
- identifying the characteristics of the desired result against the organizational guidelines and rules by which the requirement statements and requirement set is to be written, and
- transforming the stakeholder needs and expectations into a set of requirements that unambiguously communicates these stakeholder needs and expectations to the design organization.
In this context, Requirement Verification confirms, by inspection, that the requirements contain the necessary elements and possess the characteristics of a well-formed requirement, and that the requirement set conforms to the rules set forth in the organization’s requirement development guidelines. Requirement Validation confirms, by inspection and analysis, that the resulting requirement set meets the intent of the stakeholder needs from which the requirements and requirement set was decomposed or derived. Thus, the requirement statements and the requirement set are confirmed by both verification and validation activities.
Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions of these terms are included in terms of a product life cycle context:
– Requirement Verification: the process of ensuring the requirement meets the rules and characteristics defined for writing a good requirement. The focus is on the wording and structure of the requirement. “Is the requirement worded or structured correctly in accordance with the organization’s standards, guidelines, rules, and checklists?”.
– Requirement Validation: confirmation that the requirements and requirement set is an agreed-to transformation that clearly communicates the stakeholder needs and expectations in a language understood by the developers. The focus is on the message the requirements and requirement set is communicating. “Does the requirements and requirements set clearly and correctly communicate the stakeholder expectations and needs?” “Are we doing the right things?” or “Are we building the right thing [as defined by the requirement set]?”
Requirement verification and requirement validation activities should be done continuously as one develops the requirements at each level and as part of baseline activities of the requirement set performed during the System Requirements Review (SRR) or similar type of gate review at each level.
Note: Most organizations do not make a distinction between requirement verification vs requirement validation. Rather they use only the phrase “requirement validation” to mean both. Using the phrase “requirement verification” often confuses the conversation in that many interpret “requirement verification” to have the same meaning as “system verification” as defined later.
Design Verification and Validation Defined
Figure 1 also carries these concepts forward to design and realization of the system under development.
Once the requirements set is baselined, the requirements are transformed into a design of the system. Most organizations have a set of “design guidelines” or “golden rules” that guide the design process. These represent best practices and lessons learned the design team is expected to follow. As part of the design process, the design team may develop prototypes or engineering units. They will use these to run tests to fine tune their design.
After the system design is complete, there is usually a gate review or series of reviews where the design is both verified and validated (e.g. System Design Review (SDR), Preliminary Design Review (PDR), Critical Design Review (CDR). In this context, design verification has two aspects: 1) Does the design clearly represent the requirement set that drove the design? and 2) Did the design team follow the organization’s guidelines for design? Also as part of the gate review, design validation is addressed to determine whether the resulting design for the system, when implemented, will result in the intended purpose being met in the operational environment and the stakeholder’s expectations and needs being met.
Frequently, during the gate reviews, the design team “pushes back” on requirements that proved difficult to meet or were deemed not feasible for reasons of cost, schedule, and/or technology. This results in proposed changes to the requirements, which are submitted to the configuration control authority for the system for approval. When this happens, not only do the requirements needed to be changed, but also the stakeholder expectations and needs from which the requirement were derived may need to be examined, which could result in a scope change.
Design verification and design validation activities should be done as part of a continuous process during the design phase as well as during the base-lining of the design in the gate review(s).
Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions for design verification and design validation are included in terms of a product life cycle context.
– Design Verification: the process of ensuring the design meets the rules and characteristics defined for the organization’s best practices associated with design. The focus is on the design process. “Did we follow our organizations guidelines for doing the design correctly?” The design process also includes ensuring the design reflects the design-to requirements. Thus, design verification is also a confirmation the design is an agreed-to transformation of the design-to requirements into a design that clearly implements those requirements correctly. “Does the design clearly and correctly represent the requirement set?” “Did we design the thing right?”
– Design Validation: confirmation the design will result in a system that meets its intended purpose in its operational environment. Will the design result in a system that will meet the stakeholder expectations (needs) that were defined during the scope definition phase? The focus is on the message the design is communicating. “How well does the design meet the intent of the requirements?” “Do we have the right design?” “Are we doing the right things?” “Will this design result in the stakeholder expectations and needs being met?”
System Verification and Validation Defined
Once the design is baselined, the design is transformed; via build, code, buy, or reuse; into the system of interest. Similar to the discussion for the design process, most organizations have a set of guidelines or “golden rules” that guide the build (manufacture or code) process. These include workmanship and quality control requirements for the organization.
After the system has been built or coded, there will be a gate review where the system is both verified and validated. At this stage of the system lifecycle, the concepts of system verification and system validation take on a more formal meaning. Thus, the Systems Engineering (SE) lifecycle processes include the processes of System Verification and System Validation. Each process represents a set of activities (test, demonstration, inspection, analysis) that cumulate with one or more gate reviews associated with the acceptance of the system by the customer. Thus, System Verification is a formal SE process and has a legal aspect where the developer is proving the system reflects the baselined requirements have been meet. From a contracting perspective, the baselined requirements are a type of contract and are legally binding.
In this context, system verification has two aspects: 1) Does the built or coded system of interest clearly represent the requirements that drove the design? Did we build the right thing? and 2) Did the build or code team follow the organizations guidelines for manufacturing and coding?
Following system verification, system validation is performed. Again, System Validation is a formal SE process and has a legal aspect where the developer is proving whether or not the built or coded and verified system, results in the intended purpose being met in the operational environment and the stakeholder’s expectations and needs being met. Like the system requirements, the baselined stakeholder needs and requirements defined during the scope definition phase can also be considered part of a contract and are legally binding.
Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions for system verification and system validation are included in terms of a product life cycle context.
– System Verification: a process done after design and build or coding, making sure the designed and built or coded system meets its requirements. The focus is on the built or coded system and how well it meets the agreed-to requirement set that drove the design and fabrication. Methods used for system verification include: test, demonstration, inspection, or analysis. “Did we build the thing right?” Also included in system verification is a determination that the team responsible for building or coding the system of interests followed the organization’s rules, guidelines, and best practices associated with manufacturing and coding. The focus is on the manufacturing or coding processes. “Did we follow our organizations guidelines for manufacturing or coding correctly?”
– System Validation: a process that occurs after system verification that makes sure the designed, built, and verified system meets its intended purpose in its operational environment. The focus is on the completed system and how well it meets stakeholder expectations (needs) that were defined during the scope definition phase that should have occurred at the beginning of the project. “Did we build the right thing?”
System verification and system validation processes are directly related to the contractual obligation concept for a requirement statement and set of requirements. It is through these process activities that we prove we have met both the agreed-to requirements and the agreed-to needs of the entities who are the source of or own them. This is often accomplished as part of certification and acceptance activities.
You may also be interested in my blog: “Use of multiple verification methods.”
In part 2, these concepts are discussed in terms of the Systems Engineering V-model.
Comments to this blog are welcome.
If you have any other topics you would like addressed in our blog, feel free to let us know via our “Ask the Experts” page and we will do our best to provide a timely response.
