We received the following question via our “Ask the Experts” page:
“Can (should) a requirement have more than one verification method?”
The short answer is no, you should state the primary verification method you will use to show the system of interest meets the stated requirement.
However the question is a common one and I have seen organizations that list multiple verification methods for the same requirement. The reasons behind this practice are due to a lack of understanding of each of the verification methods, the use of the same terms in different contexts, as well as the limitations of the English Language.
Each organization should clearly define each of the four primary verification methods: Test, Demonstration, Inspection, and Analysis. (I often see sub-categories of Analysis that include “by simulation”, “by model”, by “similarity”, etc.)
Some organizations say they are in the “Test” phase of the project when they really mean they are in the Verification Phase, and Test is just one of the four primary methods.
Conducting a test (activity) is not the same as using Test as a method of verification. We use one word for each of two very different concepts. Test as an activity is used to say you are running a procedure, conducting a test, etc. to exercise or inspect the system in order to collect data needed for verification.
The truth of the matter is that test as an “activity” is part of the verification methods of Test, Demonstration, and Analysis.
Verification by Test involves special test equipment and/or instrumentation. It also involves running multiple “tests” (activities) to collect data on the system that will show proof the requirement has been met. Because there are multiple sets of data collected, we then have to analyze (activity) the data to make a determination whether or not the verification success criteria has been met to say the verification by Test (method) was successful. Prior to running these tests (activities) we may inspect (activity) the verification (method) results of the children requirements to this requirement. In this case, the method of verification is Test.
Verification by Demonstration also involves running at least one “test” (activity) (usually more than once) without special test equipment or instrumentation to collect the sets of data that we analyze (activity) to prove the system meets a requirement. Again, prior to running these tests (activities) we may inspect (activity) the verification (method) results of the children requirements to this requirement. In this case, the method of verification is Demonstration.
Verification by Analysis is used when one of the other methods is not appropriate or you can’t afford to use Verification by Test (method) for every requirement or you can’t do an end-to-end test (activity) but only on parts of the system. In this case you collect data on parts of the system via test (activity). Then based on this data and your knowledge of the system design, you make an engineering judgment whether or not the defined success criteria has been met in order to conclude (prove) that verification by Analysis (method) was successful. In this case, even though I may have run some tests (activities) to collect the data, the method of verification is Analysis.
Verification by Inspection is when you can use one of your senses to prove if the system meets a requirement. You don’t have to exercise the system, you can observe the results of your observations or inspections (activities) to prove the system meets a requirement.
The word inspection is often misused (activity vs method) when there is a need to examine paperwork from other verification activities as a prerequisite to completing one of the other verification methods. For example, I may be verifying the system meets a requirement that involves the interaction of two of the subsystems. Before running a test (activity) as part of verification by Test (method), I may want to examine the verification paperwork for each of the subsystem children requirements to make sure each subsystem was proven to meet its respective interface requirement prior to integrating the two subsystems together and then verifying that the integrated system meets the parent requirement. In this case, even though I inspected (examined) (activity) the lower level verification paperwork, the verification method is Test (or one of the other methods of verification).
However, if I determine that a parent requirement is met if all the verification activities have been completed and show that the subsystems have met the children requirements and no other verification activity is needed, then the question is what was the method of verification of that parent requirement? Inspection because I examined the verification paperwork for the children requirements? Or perhaps Analysis because based on my knowledge that the children requirements verification activities were successfully completed and overall knowledge of the system, I can conclude that the set of children requirements were necessary and sufficient such that their implementation results in the parent requirement being met? To me, in this case, the verification method would be Analysis. In the end you have to make the determination and then be consistent within your organization.
Conclusion
While the topic seems to be confusing, it is the English language that gets us in trouble, especially when we use the same words to have different meanings. The confusion goes away once we accept the fact that the words associated with system verification and system validation can refer to either the “method” of verification OR an “activity” performed as part of proving the system meets a requirement (verification).
By making a distinction between verificaton “method” vs “activity” and decoupling verification methods from activities incidental in accomplishing a particular method, there is really only one verification method being used, while there may be a mixture of activities needed to provide the data and show proof the system meets or will meet the requirement.
It is also important to understand just what the terms “verification” and “validation” mean based on the context in which they are used. To understand this see my two part blog: “What is the difference between verification and validation?”
I have written a comprehensive paper called “Thinking Ahead to Verification and Validation” that goes in to more detail. If you want a copy of the paper, email me at info@argondigital.com and I will send you a copy.
If you have any other questions on requirements, feel free to ask your question on our “Ask the Experts” page and we will do our best to provide a timely response.
