– In this first part of this blog, I introduced the benefits and need for organizations to include CMLs in their systems engineering processes.
– In the second part of this blog, I went into more detail defining and explaining the activities and outcomes for each CML.
In this part of this blog, I show how CMLs are an implementation of an older concept referred to as “The Doctrine of Successive Refinement”.
The CMLs defined previously show how a system concept matures as a function of time. Note that between CMLs 2 – 4 there is a feedback loop, indicating an iterative nature between initial feasibility, trade space, and point design/architecture selection based on the trades completed. This highlights the fact that system concept definition is not a linear process, especially early on in a concept’s development and maturation. Some organizations refer to this loop as Design Analysis Cycles (DACs). Management should not expect the team will get it “correct” with only one iteration. To be successful, there will often be multiple iterations or DACs. The idea of multiple interactions, or DAC cycles, is referred to as the “Doctrine of Successive Refinement” defined in the NASA Systems Engineering Handbook [NASA 1995] shown in Figure 2.
Figure 2: Doctrine of Successive Refinement
In many cases, at the beginning of concept development there are many uncertainties making it difficult to define a feasible system concept in one attempt. Projects must invest the time it takes, in the early phases of the project, to understand the problem and recognize the need/opportunity. The study/design team works with the stakeholders to identify and quantify the stakeholder needs and expectations as well as develop a plan to refine and mature the system concept. The result is a feasible concept to meet the agreed-to NGOs within the defined drivers and constrains.
The project starts with a clear problem statement and recognized need/opportunity which remains constant throughout the system development lifecycle. Goals, objectives, and system concepts start out at a somewhat ambiguous level, but are refined with increasing resolution and precision with each inward spiral. With each iteration the study/design team gains the knowledge they need to “zero-in” on a feasible approach. The first attempts at gaining the knowledge may be a little fuzzy. Candidate system architectures and enabling technologies are identified and trade studies completed. The study/design team will assess the maturity (TRL) of the enabling technologies. Prototypes are developed and evaluated by the users. With each iteration, the prototypes will be modified based on feedback from the users. The study/design team may start with key performance statements containing threshold values as well as objective values. (See my blog “How to Communicate Threshold vs Objective Requirements.)
From these activities, the study/design team increases their knowledge, zeroing in on a feasible solution, and refining the goals, objectives, drivers, and constraints. If uncertainties still exist, the study/design team can then start the cycle over at the next level of detail. With each iteration, the study/design team hones the system concept. As the enabling technologies are matured and the study/design team understands what is possible, within the stated drivers and constraints, they will be able to select a feasible concept, define the scope, develop requirements for the final product, and baseline a design.
This feedback loop is a key advantage of using the CML approach to mature a project’s system concepts. This loop allows study/design teams to return to an earlier stage of concept development if system implementation issues are encountered. Typically, study teams return from a point design or architectural selection to trade space exploration to find an alternate approach for meeting the NGOs, maximizing ROI within the defined drivers and constraints. Trade space exploration is a key part of concept maturation and is needed to provide an increased likelihood that a global optimum is identified in selection of a viable system concept architecture.
In the JPL reference paper, the authors note that “prior to the development of CMLs, some study teams would move directly into a specific system architecture to implement their system concept without first having done sufficient trade space exploration. This resulted in system concepts that 1) did not have a maximum ROI, 2) had inefficient system and support system designs, and 3) had a less efficient overall system concept because trades between NGOs, organizational ROI expectations, the system, risks, and support system design for a particular cost point never occurred.” Another issue I commonly see is projects failing to assess the TRL of the enabling technologies at the beginning of the project, yet relying on those technologies for project success. In my experience this, together with the other issues stated in the JPL reference paper, are major causes of program and project failure or significant cost and schedule slips.
CMLs provide an efficient methodology for describing the maturity of a system concept and achieving the level of maturity needed for a particular project lifecycle.
Systems engineering processes are knowledge based. “The devil is in the details.” Your system concepts are continually evolving as your knowledge increases during the system development lifecycle. Your system concept document is a living document that needs to be kept current as your knowledge increases and system development process progresses and matures through each of the CMLs. Also, if you are using a MBSE approach, your models will also continue to evolve and get more detailed as you progress through each of the CMLs and product development lifecycle stages.
In this fourth part of this blog, I present some tools to help implement, manage, and mature the project through the various CMLs.
References:
– NASA System Engineering Handbook, SP6105, June 1995.
– NASA, Systems Engineering Processes and Requirements, NPR7123.1B, April 2013, NASA Online Directives Information System (ODIS) Library: https://nodis3.gsfc.nasa.gov
– Wessen, Randii R., Borden, Chester, Ziemer, John, Kwok, Johnny, Space mission concept development using concept maturity levels, NASA JPL, AIAA SPACE 2013 Conference & Exposition, San Diego, California, September 10-12, 2013 https://hdl.handle.net/2014/44299
