We were recently asked the following question: “For a system that has both “threshold” and “objective” requirements, what is the best way to delineate them? Our initial thought was to use “should”, but then we don’t want to get them confused with non-verifiable design goals.”
The answer depends a lot on how you are define “threshold” and “objective”. The answer also depends on whether the system is being developed in-house or will be contracted out to a vendor. Another closely related concept is “threshold/baseline”. The key question to address is what value will be the one the project or vendor is going to be held accountable to meet, i.e., what value will you verify? To better understand, I will go through a scenario.
Using the threshold/objective approach. A stakeholder has a need for a system with a certain performance parameter. Current, state-of-the-art performance is not good enough to meet this person’s needs. So the stakeholder with the need states a threshold value that is good enough for the planned system. However, this stakeholder would get value if the new system did better than the threshold value. So this stakeholder does some research and based on their understanding of technology readiness levels (TRLs) and their own efforts, they come up with an objective value for the parameter that they think could be achieved. In this case the threshold value is the minimum acceptable value for the parameter, however the objective value is what the person would really like to have the system achieve.
Alternatively, you could use the threshold/baseline approach. Using the threshold/baseline approach, the project determines what they think is technically feasible given the TRL, the available time, and available budget. They also have to determine what the minimum acceptable, threshold, value is in case the baseline can’t be achieved. The value they think is achievable becomes the “baseline” value that will be written as a requirement (shall) that must be met and will be verified. It is what the project thinks is feasible and they are willing to pay for. This is the “baseline level of performance ” and what is believed to be achievable. This then becomes a “verifiable” design baseline. The threshold is the minimum acceptable value and represents the descope position. If contracting out, this value may not be shared with the Contractor.
How to document these values. You have to decide whether you want to follow the threshold/objective approach or the threshold/baseline approach. If in-house, you could include both threshold and objective values in the same requirement statement (shall), or separate requirement/goal statements (shall/should) explaining in the rationale that the threshold value is the minimum acceptable (from a verification standpoint) but the project would like to get closer to the objective “goal” value. “The system shall obtain the [parameter threshold value] of xxxx, with an objective [or goal] value of yyyy.” Or “The system shall obtain the [parameter threshold value] of xxxx. and “The system should obtain an objective [or goal] value of yyyy.”
Alternately, you could use the threshold/baseline approach and state: “ The system shall obtain the [parameter baseline value] of xxxxx. “This is the value that will be verified to. In this case the threshold value is not stated in the requirement, but is used by the project if there are problems in meeting the baseline value and the requirement needs to be “relaxed” or “descoped”. If this happens, a change request would need to be submitted getting permission to change to the new value the team thinks is achievable. This new value cannot be worse than the threshold value.
Communicating “threshold/objective” values in RFPs. When issuing a Request for Proposal (RFP) to vendors, you may want to state the threshold/objective as: “The system shall obtain the [parameter threshold value] of xxxx, with an objective value of yyyy.” You ask the vendors to propose a value they will commit to. Some vendors may bid the threshold value, others the objective value, and others somewhere in between. You evaluate each of the proposals. Everything else being equal, you pick the proposing organization that bids the best value for the parameter (closest to the objective). Now, from a contract standpoint, this value becomes the baseline and there will be a performance requirement that states this baseline value. The threshold value stays the threshold value. “ The system shall obtain the [parameter baseline value] of xxxxx.“ The project keeps the threshold value in their back pocket for descope options.
The tricky part when contracting this out, is determining whether or not a vendor is “gaming” the system and stating a value they really can’t meet. This is where you use the baseline value the project determined was “feasible” and check to see if the vendor bid is better than your “baseline value”. If so, you may question the vendor’s capability to meet the requirement. It may be that they have a capability you didn’t know about and their value really is achievable (and thus becomes the contracted “baseline” value). However, you don’t want to make any assumptions.
Comments to this blog are welcome.
If you have any other topics you would like addressed in our blog, feel free to let us know via our “Ask the Experts” page and we will do our best to provide a timely response.
