– In this first part of this blog, I introduced the benefits and need for organizations to include CMLs in their systems engineering processes.

– In the second part of this blog, I went into more detail defining and explaining the activities and outcomes for each CML.

– In the third part of this blog, I showed how CMLs are an implementation of an older concept referred to as “The Doctrine of Successive Refinement”.

In this fourth part of this blog, I present some tools to help implement, manage, and mature the project through the various CMLs

In the JPL reference paper, the authors define two key tools that will help the study/design team to advance from one CML to another: the CML Matrix and the CML checklists. The CML Matrix and CML checklists allow the study/design team and management to assess and know the current state of a system concept, enabling them to compare the state of the system concept to that required to pass a specific gate review and proceed with the next set of activities to further mature the system concept and proceed with the next development lifecycle phase. The result is an explicit list of areas where the concept is at the desired level of maturity and a list of deficiencies (more work and resources are needed to address these deficiencies).  Armed with that information, the study/design team can efficiently plan their work to get their mission concept up to the desired level of maturity.

Note:  There is an overlap of the CMLs 5-9 corresponding to a major gate review occurring at the end of major system development lifecycle phases. Because of this, standard systems engineering and project management processes and guidelines can be referred to when generating the CML Matrix and corresponding CML Checklists for these levels. For example, NASA’s SE process definition standards (NPR 7123.1) define specific entrance and success criteria for each of the lifecycle gate reviews corresponding to CMLs 5-9.  As stated previously, some organizations may combine the MDR (CML 7) with either the SRR (CML 6) or PDR (CML 8).  In this case, the CML Matrix and associated checklists described below will need to be tailored to reflect the organization’s specific gate reviews.

CML Matrix

Figure 3. Example CML Matrix described in the JPL reference paper.

The CML matrix defined in the JPL reference paper contains columns corresponding to each CML (they show CML 1-7) and rows for key attributes associated with a concept. The rows are grouped into areas of interest: science, technical, management, cost, and “other”.   (Figure 3 shows the science area of interest.)  Instead of science, the first area could be labeled MOEs or KPPs, whatever best fits your project domain.  Because each domain is different, I leave it up to your organization to develop a CML matrix tailored to your organization, domain, and system using the JPL reference paper CML matrix and my updated descriptions as examples.

The intent of the CML matrix is to serve as a high-level guide for study/design and proposal teams through the stages of system concept maturation and architecture selection. The matrix can be used by management and the study/design team in several ways:

– To determine the maturity of a system concept at the time of a particular gate review. As an example, by looking at the contents of the cells in the CML 5 column, a system architect can quickly see the material that is needed for a study/design team to pass their Mission Concept or Scope Review. Of course, the matrix only identifies what is needed at each CML, not the quality of the deliverable nor what is necessary to achieve a winning proposal.

– To understand the deliverables and their maturity required as a function of time.

– As discussed below, the contents of each column can be used to generate a CML checklist.

CML Checklists

Figure 4. Example portion of a CML Checklist described in the JPL reference paper.

The CML Checklists defined in the JPL reference paper provide study/design teams and management a tool to assist in CML evaluation. (An example of a CML checklist is shown in Figure 4.) The CML Checklists can be applied quickly to help identify gaps for the various CMLs. The CML Checklists:

– Allow management and the study/design team to quickly measure the system concept’s maturity,

– Are reusable, i.e., the checklists can be applied to any project that is maturing their concepts, providing the same level of maturity score for concepts with the same level of maturity and,

– Identify deficiencies and provide clear information as to what areas of the concept need additional work to get to the overall mission concept to the desired level of maturity.

The checklists are based on the activities and products identified in the specified column of the CML Matrix. Again, because each domain is different, I leave it up to your organization to develop CML checklists tailored to your organization, domain, and system, using the JPL reference paper CML checklists, the expanded descriptions in this write-up as examples along with the CML matrix you developed specifically for your organization.

The CML checklists are intended to provide guidelines for management and the study/design team to correct areas that are found deficient. The CML checklists provide an independent check on where the system concept is weak and where resources should be applied to make the system concept implementable and robust. Once the system concept maturity is assessed, management and the study/design team is faced with a choice. Of the areas that are identified as “not at the desired maturity level,” where should the team’s limited resources be applied to help achieve their system concept’s desired maturity? In many cases, study teams do not have the time, resources, and funds to correct all deficiencies identified by the CML checklists. It is a decision for management, the study/design team lead, project’s lead systems engineer and/or any other key team personnel to decide how best to apply the team’s limited resources to most efficiently correct the most critical omissions.  This should be a risk-based decision.

In the fifth part of this blog, I discuss the applicability CMLs to other product development approaches including incremental and spiral development.

References:

– NASA, Systems Engineering Processes and Requirements, NPR7123.1B, April 2013, NASA Online Directives Information System (ODIS) Library:  https://nodis3.gsfc.nasa.gov

– Wessen, Randii R., Borden, Chester, Ziemer, John, Kwok, Johnny, Space mission concept development using concept maturity levels, NASA JPL, AIAA SPACE 2013 Conference & Exposition, San Diego, California, September 10-12, 2013 https://hdl.handle.net/2014/44299