In a previous blog, I discussed the need to define and baseline your scope before writing requirements.  In that blog, I discussed how organizations can use a Scope Review, defined success criteria, and assessment of risk to determine whether or not the project is ready to proceed to the next product development lifecycle phase.  (The Scope Review may also be referred to as a Scope Baseline Review, System Concept Review, Mission or Project Concept Review, etc.)

In this blog I introduce Concept Maturity Levels (CMLs) as a way of measuring the maturity of a project’s system concept for all gate reviews defined by an organization, including the Scope Review.

Because of the complexity of the topic, I have divided up this blog into a six parts.

In this first part of this blog, I introduce the benefits and need for organizations to include CMLs in their systems engineering processes.

• In the second part of this blog, I go into more detail defining and explaining the activities and outcomes for each CML.

• In the third part of this blog, I show how CMLs are an implementation of an older concept referred to as “The Doctrine of Successive Refinement”.

• In the fourth part of this blog, I present some tools to help implement, manage, and mature the project through the various CMLs

• In the fifth part of this blog, I discuss the applicability CMLs to other product development approaches including incremental and spiral development.

• In the final part of this blog, I discuss the need for organizations to establish a collaborative work environment to help concept maturation and study/design teams to quickly mature system concepts through the early CMLs.

Introduction

Some projects may have their funding approved from the beginning, but must pass specific gate reviews at the end of each phase of product development.  When this is the case, the organization defines specific entrance criteria that needs to be met to perform the review, as well as success criteria that the project has to demonstrate have been met before passing the gate review and proceeding to the next lifecycle phase.

Other organizations may use a two-step, knowledge based process for funding projects.  Step 1 involves the submission of new project ideas and proposals to obtain funding and resources needed to mature the system concept to a specified level.  Step 2 projects that successfully accomplished the activities associated with Step 1 are evaluated for continuation in terms of concept maturity, feasibility, risk, and return on investment (ROI).  Based on this evaluation, additional funding required to complete development of the proposed system is approved.

With both funding approaches, a dilemma faced by many program/project managers when deciding whether to fund a project or schedule a gate review is how to:

• evaluate the feasibility (cost, schedule, technology) of the proposed system concept (project, product, system, mission) and its fulfillment of the project’s Need, Goals, and Objectives (NGOs) and stakeholder expectations within the defined drivers and constraints, and

• access whether or not the project will deliver an acceptable return on investment (ROI) with acceptable risk, and

• determine if the maturity of the concept, critical technologies, available resources, and associated planning are sufficient to approve the project for additional funding or conduct the gate review, baseline the deliverables associated with the review (scope, requirements, design, project and technical plans), and proceed with the next phase of product development.

(Note: I am using the term ROI generically to refer to the benefits to the organization associated with a given project.)

The answer? “Concept Maturity Levels (CMLs)”   I was recently made aware that Mark Adler at NASA’s Jet Propulsion Laboratory (JPL), had developed CMLs as described in a paper you can access at: https://hdl.handle.net/2014/44299.

JPL introduced the idea of CMLs so that system architects, proposal teams, and system engineers have a way to measure, assess, and communicate the fidelity, accuracy, technical progress, and maturity of a system (science mission) concept through the system’s lifecycle.

Note: The CML definitions and other descriptive information in the JPL paper is very domain specific in terms of NASA science space missions.  In this blog I have revised and expanded the definitions to be more generic and applicable to other domains.  Added to the CML definitions are Technology Readiness Assessments (TRAs) as described in the recently released GAO Technology Readiness Assessment Guide.  I have also aligned the CMLs to be consistent with the scope definition and requirements development activities we present in our classes and seminars and defined in NASA’s NPR7123.1B, Systems Engineering Processes and Requirements.  While for NASA space missions the traditional waterfall development approach is used, I have defined CMLs in this blog so they are applicable to other development approaches including incremental, spiral development, quick prototyping.

Figure 1: CML summary

As shown in Figure 1, CMLs are expressed in a scale from 1-9 that provides a repeatable way to assess and describe the maturity of system concepts and provides a single numerical scale, comparable to the Technology Readiness Level (TRL) scale, to document the maturity of different system concepts. (To learn more about TRLs, see my blog “Using Technology Readiness Levels to Manage Risk”.)

CMLs provide a standardized method to allow management to:

• determine how much work (funding, resources, and effort) has been placed into the definition and maturation of a system concept;

• compare competing project system concepts in terms of relevance to meeting the organization’s strategic goals, objectives, and ROI with acceptable risk;

• determine which system concepts have had the same level of work and can be compared on the same terms;

• understand the maturity of critical technologies needed to meet the project’s goals and objectives

• understand how much future work a system concept will require to get to a subsequent level of maturity; and

• have the information needed to determine when a proposed project’s system concept is mature enough to proceed to the next stage of system development.

The CML organizing structure corresponds to an increasing level of maturity as the system concept, planning (project and technical), design, architecture, and risks are analyzed and evolve. CML(s) provide the ability to measure a system concept’s maturity guided by an incremental set of maturity criteria.  This defined maturity criteria can be tailored to correspond to the processes specific to a particular organization, domain, and project within that domain.

References

NASA, Systems Engineering Processes and Requirements, NPR7123.1B, April 2013, NASA Online Directives Information System (ODIS) Library:  https://nodis3.gsfc.nasa.gov

Government Accountability Office, Technology Readiness Assessment Guide, draft, August 2016, https://www.gao.gov/assets/680/679006.pdf

Wessen, Randii R., Borden, Chester, Ziemer, John, Kwok, Johnny, Space mission concept development using concept maturity levels, NASA JPL, AIAA SPACE 2013 Conference & Exposition, San Diego, California, September 10-12, 2013 https://hdl.handle.net/2014/44299