Note: I originally came up with the list of features with a focus on features that a Requirement Management Tool (RMT) should have to better manage an organization’s requirements throughout the product development lifecycle.  During the discussion of these features during INCOSE IW 2016 and IW 2017 RWG sessions, it became clear that the list was more a list of features the organization’s Systems Engineering (SE) toolset needs to have in order for the organization to have the capabilities they need to better manage their systems engineering projects.   Many of these features need a “tool” to realize that capability.   The original list has been expanded to include suggestions made at the INCOSE IW 2016 and IW 2017 RWG sessions.

Currently, in order to meet the intent of the INCOSE’s MBSE Initiative, many organizations want to increase their capability to practice SE from a data-centric perspective.  Individual SE tools tend to focus on specific needs and types of work products.  Organizations (at the enterprise level) need to perform trade-studies to see if the expense of purchasing a specific SE tool or toolset, maintaining the licenses, training people to use the tool(s), maintaining the tool(s), maintaining models and other work products and their underlying data and information developed by the tool(s), etc. are going to provide a positive ROI, improved time to market, reduce the number of product defects, reduce the amount of rework, or reduce the number of warranty work and recalls.

Note: the concept of “data-centric SE” is discussed in my blog series: Integrated Data as the Foundation of Systems Engineering. Figure 1 summarizes the overall concept.

Figure 1: Integrated Data is the foundation of SE
(Developed by the RWG at INCOSE IW 2016 & IW2017)

If you need a detailed analytical model that will allow you to run simulations, then you will need the proper SE toolset and knowledge to allow you to do so.  However, you may not need this fidelity.  Maybe you just want to better manage requirements throughout the product development lifecycle process activities.  Maybe you want to use functional flow diagrams and context diagrams to better understand your interfaces, interactions, dependencies, and to better understand the functionality and flow of information of a complex system and the interactions of the parts that make up that system.  Maybe you want to be able to develop requirements from these diagrams and link your requirements to these diagrams as well as link data and information between work products, and link work products and their underlying data and information across lifecycle processes.  To do so, you need to choose an SE toolset that has features to provide this level of functionality.

Given today’s systems are increasingly complex, an SE toolset, including requirement management tools, diagraming tools, modeling tools, budgeting tools, scheduling tools are needed to help manage the challenges associated with these increasingly complex systems.

An SE toolset provides a more effective way of carrying out portions, or in some cases, all the SE lifecycle development process using the data-centric perspective. To more effectively develop systems, the SE toolset needs to be tailored to your organization’s needs, as evidenced by statements such as the following in NASA’s NPR 7123.1, NASA Systems Engineering Processes and Requirements   “...technical teams and individuals should use the appropriate and available sets of tools and methods to accomplish required common technical process activities. This would include the use of modeling and simulation as applicable to the product-line phase, location of the WBS model in the system structure, and the applicable phase exit criteria.”

Going beyond requirements, there are SE tools that can support the entire system lifestyle including budgeting, scheduling, defining, designing, building/coding, verifying, validating, and sustaining engineering activities.  These tools are used to collect, link, visualize, analyze, manage, and communicate data and information across the system lifecycle.  These more robust SE tools allow the organization to produce various views of the system under development and create and maintain the various work products (documents, databases, reports, diagrams, drawings, models, etc.) and their underlying data and information needed to more effectively manage the system development efforts.

What capabilities are needed from an SE toolset depends on the product line, its complexity, issues the organization is having and wants to address, and the workforce knowledge and experience. Organizations need to understand what data and information best meets their needs and which set of SE tools they need to work with and manage this data. SE tools are like any other software application…one size does not fit all.  The SE toolset that is best for your organization is the toolset that meets your organization’s requirements management, systems engineering, and modeling needs.  Consider the outcomes needed as a result of using SE tools and the ROI resulting from these outcomes.

Before embarking on an SE toolset evaluation and selection initiative, work with management, project teams, engineering staff, and other key stakeholders to determine what the organization needs to help better manage the development of the systems in their domain.   What features and functionality are needed in an SE toolset so the projects can effectively and efficiently manage their requirements, design, and other SE lifecycle process activities throughout the system development lifecycle?  Specifically, choose the SE toolset that supports the SCL the project has decided to strive for.  Above all, select SE tools that support the concept of SE from a data-centric perspective using the project’s common, integrated dataset. For more details on SCLs read my blog Levels of SE Capability

As stated earlier, it is advisable to choose SE tools that support the generation and management of multiple lifecycle work products and their underlying data and information and especially SE tools that support interoperability standards for compatible tools, schemas, and databases.  The perfect case would be to procure a single SE tool that “does it all”, i.e., the one tool would result in having an integrated project dataset by default.  That would help to ensure all data and information is current and consistent across all lifecycle stages.  Again, currently, such a tool does not exist.  See also Vitech’s blog on SE tools .

It is important to understand, that for an organization to provide a capability three things need to be addressed: people, products/tools (that have the needed features and functionality), and policy/processes/procedures.  You have to have all three to provide the capability.  To realize the overall capability and to effectively implement a successful systems development project using this SE tool set, the organization needs to clearly define organizational policies, processes, and work instructions for managing a SE project using the tool set AND provide training for their people in not only in the use of the SE tool set, but the associated organizational policies, processes, and work instructions. This training is critical so that the people not only have the knowledge but they understand the culture associated with implementing the organization’s SE processes.)

Features and Functions your SE toolset should include

Below are the features to be considered for the SE toolset to be selected by your organization.  Including these features will enable your organization to achieve the capabilities they need to effectively implement their SE processes from a data-centric perspective.

The order of the list does not in any way imply priority.  Priority of these features and functions and the “importance-weighing factor” for each is left up to the evaluating organization based on its unique product development and management needs.

We do not expect any one SE tool to include everything in this list as many vendors tailor their tool to a specific client base needs or a specific lifecycle stage and set of work products.   However, it would be preferable to minimize the number of different applications for the organization’s SE toolset tailored to their specific domain, product line, and processes consistent with the SCL they are moving toward.

Functionality (What capabilities do you need from your SE toolset?)

1. SE Best Practices: Does the SE toolset include the capability to support SE best practices. For example, does the SE toolset enforce requirement standards such as those defined in the INCOSE Guide to Writing Requirements?  This includes the ability of the SE toolset to help requirement authors to write properly formed requirements (spelling, grammar, ambiguous terms, requirement statement structure, consistency, etc.) and to assess the quality of a set of requirements based on the organization’s standards for writing requirements.
2. Allocation and Traceability: Does the SE toolset support the key concepts of allocation and traceability between not just requirements but all work products and their underlying data – no matter the level? For requirements, this includes allocating requirements from one level of the architecture to parts of the architecture at the next level.  Once children requirements are derived, traceability involves linking child requirements to their parent and linking requirements to their source.  If developing analytical models, this allows requirements to be linked to the applicable parts of the architecture in the model.
3. Interface management: Does the SE toolset support the documentation of interface definitions (e.g., ICDs) and the corresponding interface requirements that are linked to those definitions? Can the toolset be configured to link an interface requirement from one element of the system architecture to the corresponding interface requirement for another element with which the first element interacts with? Can the toolset be configured to notify owners of complementary interface requirements when a change is made to either of the interface requirements or their definition?  (This topic deals with the not only internal interfaces, but also interfaces between the system under development and external systems it is required to interact with.)
4. Dependencies between work products and their underlying data and information: Does the SE toolset allow users to link dependent requirements and other work products across all lifecycle stages and their underlying data and information to each other? (This is important when a change to one work product could necessitate a change in another work product and underlying data and information.) The dependent work product may be part of your system or another system. Does the SE toolset allow users to do consistency assessment between dependent requirements and work products?  Can the toolset be configured to notify owners of dependent work products when a change is made to one of the dependent work products?  Note: this feature is supported by the traceability feature.
5. Impact Assessment: Does the SE toolset allow the user to assess the impact of a change vertically among levels of the architecture as well as horizontally across all work products from all lifecycle stages helping the user to understand the impact of a change to other work products and the project’s delivery schedule, cost, quality? Does the SE toolset allow the user to do change impact assessment to other work products generated in other lifecycle processes whose underlying data resides in a separate database?  For example, what is the impact of a requirement change on design? A change in an analytical model on requirement linked to that model? On a dependent requirement?  A requirement change to system verification planning? System validation planning?
6. Ontology: Does the SE toolset include the capability to define an ontology for the organization and projects within the organization? An ontology includes the formal naming and definition of a set of terms, entities, data types, and properties as well as defining the relationships between these terms, entities, data types that are fundamental to a domain. Defining an ontology is critical to ensuring consistency and allowing sharing of data and information across lifecycle processes as well as reusability within the enterprise.
7. Schema: Does the project’s SE toolset include the capability to define a master project schema for its integrated dataset consistent with the project’s defined ontology? The schema is a description, in a formal language, of the database structure that defines the objects in the databases, shows how real-world entities are modelled in the database, and integrity constraints that ensure compatibility between parts of the schema.  Do the tools in your toolset conform to standards for development of a common schema (e.g., OSLC)?  SE tools in the projects SE toolset need to ensure their schemas are consistent with the project’s master schema defined for the project’s common, integrated dataset to ensure compatibility of the data, allowing the data to be shared among the various SE tools in the project’s SE toolset.  (See also interoperability and tool integration later in the list.)
8. Embedded Objects: Does the SE toolset allow the user to embed objects with various electronic formats (pictures, drawings, diagrams, RTF files, word processing documents, spreadsheet documents, test procedures, etc.) that can be linked to other work products?
9. Diagrams and Drawings: Does the SE toolset support the development and management of diagrams and drawings as electronic files independently from a analytical modeling tool that can be linked to other work products and their underlying data?
10. Modeling: Does the SE toolset support the development and use of architectural and analytical models? Does the SE toolset allow the development and documentation of use cases, functional flow block diagrams, states and modes diagrams, timing diagrams, and other types of models needed by the project and store the underlying data and information representing these activities and diagrams in a common database? Will the toolset allow the user to develop high fidelity models that support simulations – if that capability is needed by the organization? Does the toolset allow the creation of an extensible data model that can be easily constrained by a rule set; an extensible API to allow incorporation of custom data creation and manipulation utilities; a rich, natural language query engine?
11. Reusability: What features does the SE tool have that will enable re-use of work products and their underlying data and information for similar projects or projects involved in updating an existing product? Can the work products and underlying data for one version of a product be duplicated and used as the basis for the next version?
12. Product line management: In addition to reusability, what features does the SE toolset have that supports product line management? Does the tool allow branching of work products ,e.g., for a class of systems, the same root requirement can be branched to multiple versions of the root requirement?

Tool Attributes: (What features do you want the tools to have to allow the tool to be tailored to support your organization’s specific needs?)

13. Tailorablity: Can the SE toolset be tailored within the organization based on a project’s need e.g., complexity, team knowledge, development methodology, size, processes, timeframe, customer requirements, schema). While the enterprise may select a toolset consistent with SCL 4 or 5, can projects within the organization use a subset of the features that best meets their needs? It is not good practice to require the use of a large mallet when a small hammer is all that is needed.
14. Configuration/Customization: Does the SE toolset include the ability to configure and/or customize the toolset to the customer’s domain, culture, and organizational processes? Does the SE toolset allow individual users to configure their interface based on their unique role and use of the SE toolset? (With minimum help from the vendor.) Note: Configure refers to the ability to configure the tool to meet user needs without changing the code.  Customize involves changes to the tool code to provide new or tailored features needed by the customer. Having a tool that can be configured to meet your needs is cheaper than paying a vendor for the tool to be customized to meet your needs.
15. Learnability/usability: Does the SE tool have a user interface that is intuitive, user friendly, and easy to use with a small learning curve? How much training is necessary and available? Is online documentation and help functionality included? Does the SE toolset provide methods allowing the user to navigate between various work products and visualizations such as: requirements, documents, configuration management information, reports, design artifacts, models, etc.?
16. Accessibility: Does the SE toolset allow users to access data securely via desktops, laptops, portable devices (tablets, smart phones) both inside and outside the organization’s firewalls? Is the dataset created by your toolset accessible by another organization’s (vendor/supplier) toolset?
17. Online vs Offline Modes: Does the SE toolset require the user to be connected to the server continuously (online) to use the toolset or does the toolset allow offline work to be accomplished with synchronization after going back online?
18. Interoperability/tool integration: Does the SE tool allow the sharing of data with other SE tools (ReqIF compliant) as well as with other word processing and spreadsheet application supported formats? How easy is it for information to be transferred into and out of the SE tool to support the organization’s processes and people?  Does the SE toolset provide a standardized interface for importing or exporting data from/to other applications (e.g. Model-based tools), rather than requiring specialized scripting, etc. to achieve a transfer/interaction? Can the SE tool perform extraction, transformation, and loading (ETL) of data created by other SE toolsets external to the project’s integrated dataset so the data and information in the external tool’s database can be integrated into the project’s integrated dataset and used by the project? How well can the individual tools be integrated with each other, i.e., can one tool access and manipulate the data and information (single source truth) created by a different tool?  Do the tools in your toolset conform to a common data exchange standard (e.g. AP239, AP233 XML)?  Do the tools in your toolset allow data exchange between tools seamlessly, with minimal and straightforward data model mapping required on the part of the user?  How well does the degree of integration of the tools in your toolset meet the needs of the organization?
19. Sharing of Data: Does the SE toolset allow the project to identify and securely share specific datasets with external organizations, e.g., customer or vendors? Does the toolset include an industry standard import/export utility?
20. Storage Location: Does the SE toolset require the work products and their underlying data to be stored in the “cloud” provided by the SE tool vendor or stored in-house on the organization’s server(s)?
21. Security: Does the SE toolset provide security of the information in terms of data access (at multiple levels, within levels, and different user classes), protection of the data (from loss), and integrity of the data? Does the SE toolset support the security standards applicable to your domain and product type?
22. Scalability/Extendibility: Will the SE toolset be able to support the development and management of the volume of work products consistent with the size and complexity of the system your project develops? If the enterprise is procuring the SE toolset, will the SE toolset be able to support the number of projects within the enterprise given the size and complexity of the systems developed within the enterprise.
23. Performance: What is the maximum wait time between user actions? How does the SE toolset minimize performance impacts as the number of work products increase as well as the number of concurrent users increase?
24. Concurrent Access: How many concurrent users does the SE toolset allow to work within the same area? What happens when more than one user wants to edit the same work products and underlying data? For some complex systems, there may be over a hundred users modifying various work products in the integrated dataset simultaneously.
25. Collaboration: Does the SE toolset support collaboration among the users within the tool across all lifecycle phases? Does the SE toolset allow users to collaborate no matter where their workplace is located? Globally? Does the SE toolset allow external organizations (vendors/suppliers) to collaborate with your team?
26. Archive/Backup/Long Term Availability: Does the SE toolset provide the capability to archive and backup all the data and information in formats that provide long term availability as storage and retrieval technologies evolve or a specific tool changes or the user changes their toolset? (You want to avoid a backup/archive format that uses a proprietary format that is no longer accessable if the tool vendor goes out of business.)

Management and Reporting: (What features do you want to help to more effectively manage your project?)

27. Attributes: Does the SE toolset allow the user to define and manage attributes for work products. For example, for requirements, does the SE tool allow the user to define attributes needed to help manage their requirements? (A discussion on the use of attributes to manage your project and list of attributes is included in INCOSE-TP-2010-006-02, INCOSE Guide to Writing Requirements.)
28. Measures: Does the SE toolset allow the enterprise and project to define specific measures that will allow managers and systems engineers to monitor and assess progress, identify issues, and ensure the system being developed will meet stakeholder needs and expectations? Several key measures are commonly used that reflect overall customer/user satisfaction (e.g., performance, safety, reliability, availability, maintainability, and workload requirements): measures of suitability (MOSs), measures of performance (MOPs), and measures of effectiveness (MOEs), and leading indicators (LIs).
29. Reports: Does the SE toolset include a robust, well documented report feature that allows users to create unique reports (using the attributes and measures defined previously) as well as customize standard reports provided with the tool? Does the SE toolset allow reports to be exported in multiple formats (MS Word, Pages, RTF, spreadsheet, presentation, graphical, etc.)? At the beginning of your quest of the SE toolset, one of the first things you need to do is develop your overall process you want the tool to support. Include in this process description the specific reports you will need. That will drive the schema of the data, meta-data, measures, and attributes to be included in your database.
30. Metrics/dashboards: Closely related to a report feature, does the SE toolset provide the capability to do “data mining” and analytics of the measures and information in the attributes so you can display historical and trend data? It is often not enough to just know what percent of your requirements your system has been verified to have met. Often it would be useful to see if the trend to completion of system verification activities is on the right pace, is slowing down, or speeding up.  If slowing down, you may not be able to complete all your system verification and system validation activities in time for the customer acceptance review.
31. Notifications: Can the SE toolset send notifications via email or texting concerning changes to work products; design work products, system V&V work products? Can the SE toolset send notifications from one user to another user (or group of users) concerning actions, comments, and questions? Can the SE toolset send notifications to the appropriate users when a specific measure is predicted to or has exceeded a pre-specified threshold?
32. Project Management work products: Does the SE toolset allow various PM work products to be managed within the SE toolset? This includes budget, schedule, and risk management work products. Can these work products and underlying data and information be linked to parts of the product breakdown structure and other SE work products and their underlying data?
33. Lifecycle Support: Does the SE toolset support system development across all system development lifecycle processes: scope definition, requirement definition and management, gate reviews, design, system verification, system validation, and sustaining engineering? For example, system verification & validation: Does the SE toolset allow you to link your requirements to their system verification and system validation requirements, procedures, results of the procedures, and close out documentation of the system verification and validation activities?
34. Workflow: Does the SE toolset provide the ability to define and support the organization’s SE process workflow within the tool (e.g., for requirements does the SE toolset allow you to track their status: draft, review, approve, baseline; design, test, code/manufacture, system verification, and system validation)? Can the SE toolset allow the creation, management, and execution of SE processes, procedures, and work instructions within the toolset?
35. Configuration Management: Does the SE toolset provide robust configuration management of all lifecycle work products and underlying data and information including change, version, and baseline control? Does the SE Toolset allow the user to access the change history of any work product? If work products are developed and maintained within a database, does the SE toolset allow configuration control of the database (versus the various reports/visualizations representing the data and information in the database)?

Other:

36. Price: Is the SE toolset affordable in relation to the size of the project and number of requirements and requirement sets, design and verification and validation work products, number of concurrent users? Concerning affordability, is there a single upfront application fee, individual license fee (if a license fee, one-time or yearly)? Are the licenses fixed or floating? Does the price include initial setup, installation, configuration or customization or is that extra? Is ongoing technical support included or extra?  Is training included or extra?  Would it be more cost effective to spend more on a single tool that has most of the above features or multiple tools to give you all the features needed?
37. Cost of infrastructure to support the use of the SE toolset: What are the IT requirements to host and deploy the toolset? What specialty skills beyond engineering are required to operate, extend, and maintain the toolset?
38. Vendor/product maturity: How long has the SE tool been on the market? How long has the vendor been in business?
39. User feedback and satisfaction: In today’s social media driven world, you have access to actual user comments concerning the tool, the tool vendor, ease of use, reliability, tech support, etc. Don’t get overwhelmed by the hype and sales pitch from the vendor.  See what the actual users have to say about the SE tools being considered for inclusion in your SE toolset.

If we have left out an important item from the list, please let us know.

As always comments on this blog are welcome.