– In the first part of this blog, I introduced the benefits and need for organizations to include CMLs in their systems engineering processes.

– In the second part of this blog, I went into more detail defining and explaining the activities and outcomes for each CML.

– In the third part of this blog, I showed how CMLs are an implementation of an older concept referred to as “The Doctrine of Successive Refinement.

– In the fourth part of this blog, I presented some tools to help implement, manage, and mature the project through the various CMLs i

In this fifth part of this blog, I discuss the applicability CMLs to other product development approaches including incremental and spiral development.

The discussion concerning CMLs to this point assume a scenario for system development where the complete system is to be delivered with all the capabilities, functionality, performance, security, quality, etc. per the design-to requirements baselined at CML 6.  This is the case for a project like the NASA space science projects managed at JPL where the CMLs were originally defined.  This approach is used when the end system requirements can be baselined and all the capabilities, functionality, performance, and quality needs to be realized in one delivery.  This approach is often referred to as the “waterfall” development approach.

However, one size doesn’t fit all.  There are other approaches that may be more appropriate depending on the domain and needs of your organization.  For example, in the US Department of Defense, different philosophies to system development and acquisition are gaining popularity including Rapid Acquisition Program (RAP) and Quick Reaction Capability (QRC). RAP and QRC promise the ability to:

– quickly develop, communicate and validate requirements to resolve unforeseen threats to mission and/or casualties;

– identify solutions that can be rapidly fielded with no, or very limited, development; and the willingness to accept less than a 100% solution;

– immediately apply funding to selected solutions and begin executing a program; and

– execute a tailored acquisition that accepts “acquisition risk” but delivers solutions in a time frame consistent with the urgency of the mission requirements, thus reducing the warfighter’s operations risk.

Other organizations are using approaches like Rapid Application Development (RAD), agile development, rapid technology infusion cycles, and middle-out or bottom-up design.  In consumer products, time to market is a key consideration.  For example, the Apple watch.  The first hardware release provided functionality and a high degree of quality, but it was a “push” product not driven by consumer needs for the watch, but rather what Apple thought consumers would want.  Apple’s vision for the Apple watch is far greater than what was delivered in the first increment.  Some functionality and performance was left out due to technologies available at the time as well as the need to get it to market given the smart watches being release by others.  From a software perspective, iWatch 1.0 was clearly a “beta” version.  iWatch 2.0 fixed problems and added some new features and performance.  However, many feel that iWatch 3.0 is the first real consumer release based on lessons directly from the consumers when using the earlier versions.  It seems to be a common practice to let not only developers but consumers beta test software products.  Incremental releases that consumers must pay for also form many organization’s business model. When using this approach, the organization can apply CMLs to mature their system concept for each of the incremental release as well as the final system release.

One key issue is risk.  When is a product concept mature enough to proceed with development and release the product for use?  If you release too early, bugs, poor performance or poor quality, can result in negative consumer/operator feedback and can doom the product to failure.  If you wait too long, could mean a competitor may introduce a product that dominates the market, making it difficult to break into this market or gain market share.  There is no easy answer.

Because systems engineering is a knowledge based process, CMLs provide an excellent roadmap to follow no matter which approach an organization chooses to follow.  No matter the system to be developed and the development approach used, project managers need to understand the problem, the need, goals, drivers, and constraints.  They need to define a feasible system concept that will address the problem/opportunity and understand user expectations.   These expectations need to be transformed in to a set of requirements that represent an agreed-to obligation (contract) for the developers and to which the designed and built system will be proven to meet (system verification and system validation.)

In addition to the classic waterfall development approach, two historical develop approaches that can be used to meet the evolving need for more rapid processes for system development and acquisition are incremental and spiral development.

Incremental Development

Incremental Development is similar to Waterfall – where the organization can define a core set of system requirements upfront — however, rather than delivering the whole system at one time, as shown in Figure 5 the organization delivers the system in increments, with each subsequent increment delivery adding additional capabilities, functionality, performance, quality, etc. based on stakeholder priorities and changing needs. The first increment results in a useable system that is delivered in a shorter time that meets the stakeholders top priority needs and expectations.  Subsequent increments add to the first delivered system lower priority needs/requirements, additional capabilities, functionality, performance, quality, etc.

Figure 5. Incremental Development and Delivery

Incremental Development can be combined with iterative approaches as well.   Where Incremental Development focuses on planned deliveries, adding new capabilities with each delivery, iterative focuses on making improvements on existing deliveries as well as updating requirements for future increment deliveries.  With an iterative focus, an organization can also use the subsequent increments to include updates to the previous increments based on lessons learned during testing and operations of the system “in the field” or infuse new technologies that were not mature enough to be included in the first increment.  This allows the organization to correct problems found during the previous increment delivery and use this knowledge to improve the system capabilities, functionality, performance, quality, etc.

The Incremental Development approach supports organizations that have adopted a rapid development philosophy as discussed earlier.  Key provisions that enable Incremental Development include using a modular design philosophy and defining the interfaces (external and internal) at the beginning of the project so that modules added or upgraded in subsequent increments will be able to be integrated into the system.  A key tenant of Incremental Development is that a 75% solution is acceptable and useful.  A product delivered in 1 year, that provides 75% of the needed capabilities, is more useful and beneficial to the current users, than a product delivered in 2 years later with 100% of the needed capabilities.

Spiral Development

Spiral Development is a good approach when the requirements are difficult to define upfront. As shown in Figure 6, each spiral can result in a prototype that can be shown to the stakeholders to get their feedback. It often takes several spirals to nail down the requirements.  Once the stakeholder needs, expectations, and desired outcomes are understood, the requirements can be defined and the organization can transition into either a waterfall approach and deliver the whole thing or an incremental approach and deliver the system in increments driven by stakeholder priorities and changing operational needs.

Spiral Development is very similar to the Doctrine of Successive Refinement discussed earlier. The Spiral Development model results in the same fundamental outcome.  While the Doctrine of Successive Refinement started on the outside and spiraled inward, zeroing in on a solution, Spiral Development is shown as starting in the middle and spiraling outward toward the same outcome (See Figure 3). With each spiral our knowledge increases and so does the maturity of our concept and requirements to the point we are ready to build the final product.  The Spiral Development approach is very applicable to the loop nature of CML 2-4 as discussed earlier. The key difference is that the NGOs, KPPs, MOEs, etc. defined during CML 1 activities are very preliminary at the beginning where the stakeholders have a problem but, initially, a concept to address that problem is illusive, requiring more work to develop an acceptable solution. Once an initial concept is able to be formulated, then it can be matured it into a feasible concept that addresses the problem and the stakeholders agreed-to NGOs.  Thus, for projects like this, the loop is between CML 1 and 4.

Figure 6: Spiral Development derived from [Boehm 2001]

Spiral Development was developed by Dr. Barry Boehm, primarily addressing software development projects.  Spiral Development addresses projects where the requirements aren’t clearly understood at the beginning, users often don’t know what they want until they see a prototype, and based on what they learn, the user requirements often change.  Dr. Boehm defines spiral development as ^{[Boehm 2001]}:

“The spiral development model is a risk–driven process model used to guide multi-stakeholder concurrent engineering of software-intensive systems. It has two main distinguishing features. One is a cyclic approach for incrementally growing a system’s degree of definition and implementation while decreasing its degree of risk. The other is a set of anchor point milestones [life-cycle objectives (LCO) – what should the system accomplish, life-cycle architecture (LCA) – what is the structure of the system, and initial operational capability (IOC)- first release] that are key decision points for ensuring stakeholder commitment to feasible and mutually satisfactory system solutions.”

Prototypes are used to support the risk analysis process that results in a better understanding of the user’s needs and the challenges in meeting those needs. Once a prototype has been validated it will often become the simulation, model, or benchmark for the next step in the spiral.

Each trip around the spiral starts with determining and refining the NGOs, MOEs, and KPPs as well as one or more system concepts that will result in meeting the NGOs within the defined drivers and constraints.  Multiple spirals (CML 1-4 loops) may be required until the NGOs are finalized and a feasible system concept has been defined to meet those NGOs.  Prototypes are developed, matured, and evaluated by the stakeholders. Trades are completed.   Once the team has a feasible concept (CML 5) they can proceed to finalize the design-to requirements (CML 6), and proceed with development of the end product (CMLs 7-9).  With each spiral stakeholder needs, expectations, drivers and constraints are better understood, alternative concepts are identified and accessed, and the system concept matures.

Each spiral considers these main elements: 1) critical-stakeholder objectives and constraints, 2) system concept alternatives, 3) risk identification and resolution 4) stakeholder review, and 5) commitment to proceed.  These elements are very similar to what is defined for CMLs 1-5. There is enough flexibility in the spiral model enabling the study/design team to complete several spirals before going on to the next CML – it depends on the results of the risk analysis and the decisions made along the review/commitment partition shown in the diagram.

Boehm’s inclusion of anchor point milestones is especially important. Each milestone is a stakeholder commitment point. The purpose of the LCO review is to make sure that there is at least one feasible system concept to meet the stated NGOs of the project. The purpose of the LCA review is to align the stakeholders to a common vision and agree on the scope of the project (CML 5). Boehm states: “The LCA milestone is particularly important, as its pass/fail criteria enables stakeholders to hold up projects attempting to proceed into evolutionary or incremental development without a life cycle architecture. ^{[Boehm 2001]}”   Risk analysis is a key activity for each spiral and the project must have a risk management plan in place.  At each milestone the project risks are reviewed and the resolution of each risk is agreed to by the stakeholders.

To better understand how a project can use the anchor point milestones as part of the CML approach, Boehm provides the following Stud Poker Analogy^{[Boehm 2001]}: “A valuable aspect of the spiral model is its ability to support incremental commitment of corporate resources rather than requiring a large outlay of resources to the project before its success prospects are well understood. Funding a spiral development can thus be likened to the game of stud poker. In that game, you put a couple of chips in the pot and receive two cards, one hidden and one exposed. If your cards don’t promise a winning outcome, you can drop out without a great loss. This corresponds to canceling a project at or before LCO. If your two cards are both aces, you will probably bet on your prospects aggressively (or less so if you see aces among other players’ exposed cards). Dropping out of the second or third round of betting corresponds to canceling at or before LCA. In any case, based on information available, you can decide during each round whether it’s worth putting more chips in the pot to buy more information or whether it’s better not to pursue this particular deal or project.”

In the final part of this blog, I discuss the need for organizations to establish a collaborative work environment to help concept maturation and study/design teams to quickly mature system concepts through the early CMLs.

References:

– Association of the United States Army, Rapid Equipping and the U.S. Army’s Quick-Reaction Capability, October 2015, https://www.ausa.org/file/518/download?token=cpmV_uNh

– Boehm, B., Spiral Development: Experience, Principles, and Refinements, CMU/SEI-00-SR-008, Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2000.

– Boehm, B. and Hansen, W., The Spiral Model as a Tool for Evolutionary Acquisition, CrossTalk – The Journal of Defense Software Engineering, May 2001: 4-11

– Romero, Pia, Quick wins show the benefits of DoD’s Rapid Acquisition Program, Federal News Radio, June 2012, https://federalnewsradio.com/defense/2012/06/quick-wins-show-the-benefits-of-dods-rapid-acquisition-program/

– Wikipedia, Rapid application development, https://en.wikipedia.org/wiki/Rapid_application_development