Navigating Concept Maturity Levels in the Systems Engineering Process
In the realm of systems engineering, the journey from ideation to realization involves a systematic approach that ensures successful project outcomes. One crucial aspect of this approach is understanding and managing concept maturity levels. Concept maturity levels play a pivotal role in guiding the development process, fostering innovation, and mitigating risks. Here, we’ll explore what concept maturity levels are and why they hold immense importance in the systems engineering process.
Understanding Concept Maturity Levels
Concept maturity levels, often referred to as technology readiness levels (TRLs), are a standardized framework used to assess the maturity of a technology, idea, or concept. This framework allows engineers and project managers to gauge the progress of a concept through its various stages of development. The concept maturity level provides a clear snapshot of the readiness of a concept for implementation, indicating how close it is to becoming a tangible product or solution.
The TRL framework consists of several levels, typically ranging from TRL 1 (Basic Principles Observed) to TRL 9 (Actual System Proven in Operational Environment). Each level represents a specific phase of development, from the initial theoretical exploration to the successful deployment and operation of the system.
The Importance of Concept Maturity Levels
1. Risk Management
Understanding the concept maturity level is crucial for assessing and managing risks. In the early stages of development (lower TRLs), there is a higher degree of uncertainty and technical challenges. As the concept progresses to higher TRLs, the associated risks become more manageable. This allows project managers to allocate resources effectively and make informed decisions to mitigate potential setbacks.
2. Resource Allocation
Resource allocation is optimized when concept maturity levels are accurately assessed. Allocating substantial resources to a concept that is in its infancy can be wasteful. Instead, resources should be allocated based on the current maturity level. As the concept advances to higher TRLs, the allocation of resources can be increased with confidence.
3. Innovation and Collaboration
Concept maturity levels encourage innovation and collaboration. Engineers and researchers are motivated to explore new ideas, push boundaries, and advance concepts to higher TRLs. Additionally, collaboration between different teams and organizations becomes more effective when all parties have a shared understanding of the concept’s maturity level.
4. Decision-Making
Informed decision-making relies on accurate information about the concept’s maturity level. Whether it’s selecting a concept for further development, seeking funding, or pursuing partnerships, decision-makers can rely on the TRL framework to guide their choices. This ensures that decisions align with the current state of the concept’s development.
5. Communication
Clear communication is facilitated by concept maturity levels. When discussing projects with stakeholders, investors, or regulatory bodies, using TRLs provides a common language for understanding the progress of a concept. This transparency enhances credibility and fosters trust among all involved parties.
Defining and Explaining the Activities and Outcomes for Each Concept Maturity Level (CML)
The idea for a CML scale (*shown in Figure 1) is to address the common path of progression through the project initiation phase from initial idea through Critical Design Review (CDR) or comparable gate review. Varying levels of concept maturity may entail differing levels of fidelity of project planning and engineering analysis needed for broader vs. more localized trade studies, and varying techniques for cost, schedule, and risk analysis.
CML 1 - "Cocktail Napkin"- Overview and Advocacy
In the overview, the problem (or opportunity) is defined, communicating why this project is worthwhile. The essence of what makes the concept unique and meaningful is clearly stated. The project vision and “Need” statement is defined, showing the top level reason for developing the system and providing a focus for the team. The overview lays out a broad outline of the intention of the project, high level benefits, overall advantages, but provides limited details of cost, schedule, or other programmatic or design information. Think of the overview as the “30 second” elevator pitch addressing the problem being solved, or opportunity being pursued, is and what the ‘Need’ is- or in other words, the desired end result at the end of the project once the system is operational.
From an advocacy viewpoint, information needed to sell the idea is documented. Key stakeholders are identified, the benefits, advantages, and ROI that result from pursuing this concept are defined for the stakeholders and for the organization. Needed capabilities are defined in terms of people, process, and products along with what needs to be done to provide these capabilities. A rudimentary “back of the napkin” sketch of the system concept is developed.
The advocacy perspective expands on the overview, going into more detail as to both what needs to be done to meet the ‘Need’, and how management will know if the project is successful. Preliminary goals and objectives are defined including preliminary measures of effectiveness (MOEs) and Key Performance Parameters (KPPs), including preliminary key figures of merit concerning safety, reliability, sustainability, affordability, extensibility/evolvability, agility/robustness, effectiveness, and innovation.
Assumptions are documented and system level drivers and constraints are identified, including cost, schedule, standards, regulations, technology, resource availability, interaction with existing systems, and higher level requirements allocated to the system of interest.
CML 2 - Initial Feasibility
The system concept is expanded and assessed on the basis of feasibility (cost, schedule, technology) from ROI, technical, and programmatic viewpoints. Preliminary system life-cycle concepts are documented, including concepts for development, procurement, system verification, system validation, manufacturing, transportation, deployment, operations, maintenance, upgrades, and disposal. Enabling systems required during development and operations are identified.
The system concept is expanded to address how the proposed system will fit within the macro system of which it is a part. A preliminary functional architecture of the macro system and system of interest is developed. How the systems that make up the macro system are related (connectivity, interaction, and flow of information) to each other and to your system of interest is defined. External interfaces are identified. High-level schedule, resource, and cost estimates are developed. Key risks are identified.
Outside influences the system needs to take into account are addressed. Need, Goals and Objectives (NGOs), MOEs, and KPPs are refined based on the information gained at this level.
CML 3 - Trade Space
The system functional architecture is transformed into an initial physical architecture. This physical architecture is used to access the options of the system to be developed, meeting the NGOs within the defined drivers and constraints.
The study/design team will perform architecture trades between the system, subsystems, support systems, critical technologies, and enabling systems with elaboration to better understand the relationship between the parts of the architecture and impacts to ROI, cost, schedule, and risk.
The system (product, device, application, etc.) concept is demonstrated to be viable within the defined drivers and constraints, technology maturity, proposed macro architecture, and operating environment.
Objectives, MOEs, and KPPs are refined based on the information gained at this level. Measures of Performance (MOPs) are defined. Key risks are investigated, updated, and possible mitigation strategies outlined.
A key consideration at this stage is understanding the partials- that is how ROI changes as a function of some system parameter (e.g. mass, data volume, communication bandwidth, computing power, operating environment, technologies used, etc.). How will the ability of the system to meet the NGOs, MOEs, MOPs, and KPPs be affected as a function of changes in cost, schedule, technology, or other key parameter? How will the cost, schedule, technology, and schedule be effected by a change in a KPP?
For complex systems where many of the parameters across the system architecture have a dependency (directly proportional or inversely proportional), it is very hard to track and understand the partials “manually”. For complex systems, we recommend the study/design team use Systems Engineering (SE) software tools to develop a model of the system including the subsystems that make up the system, as well as the macro environment in which the system will operate. With such a model, the team can make changes to the MOPs and KPPs as well as external constraint parameters to access the impact of the change to meeting the NGOs and other key system parameters. This knowledge will be of great benefit in choosing or optimizing the system architecture within the defined drivers and constraints.
Another key consideration when performing architectural trade studies is the TRL of the candidate parts of the architecture. Trade studies will guide which technologies are needed to best meet the project’s objectives.
Critical technologies needed to meet the goals and objectives, MOEs, and KPPs are identified. An initial Technology Readiness Assessment (TRA) is performed per the processes outlined in the GAO Technology Readiness Assessment Guide and the resulting TRA Report is generated.
If the current capability to meet an MOP or KPP is beyond current state-of-the-art for that part, then new parts/technology with the potential to provide the desired capability will have to be considered. Best practices and guidance from the Government Accountability Office (GAO) state that the system architecture should not rely on parts/technology with a TRL less than 3 at the time of scope baseline (CML 5) and a plan needs to be in place to mature the technology to at least TRL 6 by the time of the Preliminary Design Review (CML 8), and TRL 7 by the time of the Critical Design Review (CML 9). For more information on TRLs, see our blog, “Using Technology Readiness Levels to Manage Risk”.
CML 4 – Point Design - Architecture Selected Within Trade Space
Candidate system architectures are identified that will implement the system concept selected to meet the project NGOs and achieve the desired ROI providing the needed capabilities, functionality, performance, and quality within the specified drivers and constraints within the trade space. These architectures are defined to the level of major subsystems with acceptable cost, schedule, risk, and resource margins and reserves. (For a detailed discussion on the importance of defining development and operational margins and management reserves at this stage of the development lifecycle, see my four-part blog: Resource Margins and Reserves.) Subsystems’ trades are performed. Descope and backup options are defined. The objectives, MOEs, MOPs, KPPs, cost, schedule, and risks are further refined based on the information gained at this level.
Operational scenarios and/or use cases are defined (nominal, alternate nominal, and off-nominal) and the system concept and candidate architectures are assessed based on their ability to successfully address each of the operational scenarios and/or use cases. Stakeholder roles and functions are defined and how various stakeholders (operators, maintenance and software update personnel, etc.) will interact with the system during operations are documented. The focus in on roles and functions – what the stakeholders do with it, and how they will use the system once it is operational. A “day-in-the-life” of the system from the stakeholder’s viewpoint is addressed. How does data and information flow between architectural elements? How are the capabilities of the overall system used to accomplish its intended purpose in the operational environment?
Organizations may use “rapid prototyping” of potential physical solutions of a single component, subsystem, or system. With advances in 3D printing, study/design teams can develop prototypes quickly to be used as part of the architectural trade activities. When prototypes are developed, various configurations can be made available to the actual users. These prototypes can be functional, non-functional, or just “form and fit”. A prototype is useful to help the study/design team understand the user needs and expectations which will drive the system requirements. Prototypes are developed based on the currently known requirements. By using prototypes, the users can get an “actual feel” of the system in representative operational environments.
Also referred to as “discovery learning”, user interactions with the prototypes can enable both the user and study/design team to better understand the requirements for the desired system. Rapid prototyping allows for user feedback much earlier in the development lifecycle concerning errors and problems, such as missing functions, substandard performance, safety issues, and confusing or difficult user interfaces. This is an interactive process that allows users to understand, modify, and eventually approve a working physical model of the system that meets their needs.
From the operational scenarios and user interactions with system prototypes, a draft set of system requirements are developed including core functional requirements and associated performance requirements, requirements to implement defined objectives and MOPs, requirements addressing defined drivers and constraints, and system level interface requirements.
The TRA and resultant TRA report are updated. Based on the results of the TRA, a Technology Maturation Plan (TMP) is created per the processes outlined in the GAO Technology Readiness Assessment Guide to mature the critical technologies needed to meet the goals and objectives, MOEs, and KPPs.
Based on the knowledge gained up to this point and the activities completed, the study/design team needs to determine whether or not they are ready to proceed with the activities associated with CML 5 or go back and repeat activities associated with CML 2-4 to further refine the system concept.
Note: it is a myth that design work shouldn’t start until after the scope and requirements are baselined. A key outcome of the Scope Review (CML 5) is that there is at least one feasible concept that will result in the NGOs being achieved within the given drivers and constraints with acceptable risk. Lessons learned from key programs indicate that design work during the early concept maturation phases can identify important issues that are difficult to determine until a design concept is of sufficient maturity to evaluate a more complete set of development, test, operations, and evaluation considerations. Using the knowledge gained from prototypes and trade studies, a design concept should be developed, as early as possible in the project formulation phase, to allow a more complete understanding of programmatic implications and development challenges and risks. The design work at this stage enables validation of the draft requirements, providing analysis to determine where something is missing or perhaps not needed. This validation is a key part in accessing feasibility of the concept. Prototyping results in a system concept that is much more than just an artist rendering or a parametric model.
CML 5 – Scope/Concept Review Baseline
The implementation approach is defined including project management, systems engineering, contracting mode, integration, system verification and system validation approaches, cost and schedule. Relationships and dependencies, partnering, key risks, mitigation plans, and system “buy (from an external source), build/code (make internally), reuse/modify (existing systems)” or even buy/try/decide approaches are defined. The TRA is completed, verifying a minimum of TRL 3 for all parts of the architecture and a plan for advancing the TRLs to at least TRL 6 by PDR is complete, for systems going operational. (The purpose of some projects may be to advance the TRL. In this case, advancing to TRL 6 may be the goal of the project.) A viable, feasible, and stable system concept is defined that will meet the NGOs, MOEs, MOPs, and KPPs within the defined drivers and constraints with acceptable risk.
The proposed system concept along with operational scenarios/and or use cases is documented in a system concept document or model. Project management and systems engineering documentation is developed to the maturity required by the organization for this lifecycle stage of the product development. The TMP is baselined. (For a more detailed discussion on baselining scope, the entrance criteria, exit criteria, and risk identification, see my blog: Baseline Your Scope Before Writing Requirements.) For a two-step project funding process, this is the maturity that the system concept needs to be at for a Step 1 funding proposal as discussed earlier. By baselining scope, the project has demonstrated a system concept that is mature enough to proceed to the next lifecycle stage and document its requirements.
Note: In the above text I referred to a “system concept document” rather than an Operational Concept or Concept of Operations Document. See my blog: ConOps vs OpsCon – What’s the Difference?) For those moving away from a document-centric approach and towards a data-centric systems engineering approach, the system concept may be documented in a model developed using a systems engineering modeling tool. Using this approach, requirements and other systems engineering lifecycle artifacts can be linked to the model. With a data-centric systems engineering approach, documents are generated from the data in the form of “reports’, where the “ground truth” data is maintained and managed within the model. These documents can then be shared with other internal and external organizations. For more details on data-centric systems engineering and selection SE tools to implement this approach, see my blog: Features an SE Tool Set Should Have.
Traditionally, projects focus on developing an operational concept or concept of operations document that is baselined as part of the scope review. Too often the focus is on writing this document. This is a mistake. The focus shouldn’t be on the document but on maturing your system concept to the point you can document a feasible concept that has reached CML 5 and is ready to be baselined.
Developing and documenting a system concept is not an exercise in writing, rather it is an exercise in systems engineering!
CML 6 – System Requirements Review (SRR)
Requirements are finalized. Design and planning commensurate for a SRR is complete. From the baselined system concept, stakeholder needs and expectations are transformed into “design-to” requirements. In this context, requirements resulting from this transformation represent an agree-to obligation of what is expected by the customer and communicates clearly these expectations to the design/development team.
Using SE tools, requirements are traced to their source and allocated to the subsystem level, schedules for the subsystem development defined, and system verification and validation approach defined. Interfaces are identified and reflected in interface requirements documented in the set of requirements being baselined. Expanded details on the technical, management, cost, risks, and other elements of the system concept have been defined and documented.
Project management and systems engineering documentation is developed to the maturity required by the organization for this lifecycle stage of the product development. The system concept document or model is updated as necessary. The TRA, resultant TRA report, and TMP are updated. The requirement set is baselined.
CML 7 – System Design Review (SDR)
Design and planning commensurate for an SDR is complete. System architecture trades have been completed and a design approach to meet the baselined system requirements has been defined and is ready for baseline. Preliminary subsystem-level requirements and analyses, demonstrated (and acceptable) margins and reserves, prototyping and technology demonstrations, risk assessments and mitigation plans are completed. Preliminary integrated cost-schedule-design is baselined.
The system concept document or model is updated as necessary. The Preliminary Project Plan is ready for review. Other project management and systems engineering documentation is developed to the maturity required by the organization for this lifecycle stage of the product development. The TRA, resultant TRA report, and TMP are updated. For a two-step project funding process, this is the maturity that your concept needs be at for a Step 2 project funding proposal as discussed earlier.
Note: Some organizations do not have a standalone SDR. Some may combine the SDR with the SRR or with the PDR. For those cases, criteria for determining whether or not your concept is at CML 7 would combined with criteria for either CML 6 or CML 8, depending on how your organization defines their gate reviews.
CML 8 – Preliminary Design Review (PDR)
Design and planning commensurate for a PDR is complete. System design documentation, “build-to” requirements and drawings are 10%-20% complete. Interfaces are defined in Interface Control Documents (ICDs). Final integrated cost-schedule-design is baselined. All parts/technologies have reached an TRL of at least 6. The system concept document or model is updated as necessary. The Project Plan is baselined including budget and schedule. Other project management and systems engineering documentation is developed to the maturity required by the organization for this lifecycle stage of the product development. The TRA, resultant TRA report, and TMP are updated.
CML 9 – Critical Design Review (CDR)
Design and planning commensurate for a CDR is complete. Detailed system design documentation, “build-to” requirements and drawings are 80%-90% complete. ICDs are complete. The system concept document or model and other project management and systems engineering documentation is updated as necessary.
Notice that the CMLs apply to the left side of the Systems Engineering “Vee” Model. Ensuring all the activities are completed successfully on the left side of the SE Vee Model go a long way in minimizing problems that often occur on the right side of the SE Vee Model during system integration, verification, and validation. This is a major reason, and benefit for using CMLs to manage your system development activities. The TRA, resultant TRA report, and TMP are updated.
Harnessing Concept Maturity Levels for Effective Systems Engineering
Concept maturity levels serve as a vital compass in the systems engineering process, guiding projects from ideation to realization. These levels offer a structured way to assess the readiness of concepts and technologies, enabling effective risk management, resource allocation, innovation, decision-making, and communication. By understanding and leveraging the power of concept maturity levels, engineers and project managers can navigate the complex landscape of systems engineering with confidence, ensuring successful project outcomes.
Ready to Dive Deeper into Concept Maturity Levels? Connect with the Experts at ArgonDigital Today!
References
- Government Accountability Office, Technology Readiness Assessment Guide, draft, August 2016, https://www.gao.gov/assets/680/679006.pdf
- NASA, Systems Engineering Processes and Requirements, NPR7123.1B, April 2013, NASA Online Directives Information System (ODIS) Library: https://nodis3.gsfc.nasa.gov
- Wessen, Randii R., Borden, Chester, Ziemer, John, Kwok, Johnny, Space mission concept development using concept maturity levels, NASA JPL, AIAA SPACE 2013 Conference & Exposition, San Diego, California, September 10-12, 2013 https://hdl.handle.net/2014/44299
- Wikipedia, Rapid application development, https://en.wikipedia.org/wiki/Rapid_application_development
