It is the end of your project and your tests are passing. Are you ready to confidently release your project?  Or, are you secretly crossing your fingers like a young child, hoping that your tests were adequate to confirm the product is of sufficient quality to release? Why not leverage the work that you put into your requirements when you create your test cases? Your graphical models, use cases, business requirements, functional requirements, business rules, non-functional requirements and tracing can all play a role in making better test cases. This approach is applicable to any category of testing, but for this blog I am going to focus on user acceptance testing.

User Acceptance Testing (UAT) versus Functional Testing

• UAT’s goal is to confirm coverage of the business requirements
• UAT is independent of testing done by the software testing team
• UAT focuses on the major functionality from a business viewpoint, often but not always end-to-end
• UAT does not cover full functional requirements testing, negative testing and boundary value testing  – This is covered by the software testing team
• UAT is usually done by business stakeholders or their advocates rather than software quality test staff
• UAT is often manual testing that is not cost-effective to automate

Using Requirements Tracing to Design User Acceptance

It worked well for finding requirements gaps during requirements definition so why not use it again to eliminate gaps in test case coverage?

• Do cross-tracing (bi-directional) to check from a higher level to the test case and the other way.
• Is every business requirement covered by one or more test cases?
• Are the main paths and alternate paths of each use case covered by one or more test cases?
• Are functional requirements adequately covered by the test cases? The key here is the word “adequately”. User acceptance tests do not need to cover all functional requirements (though the test cases used by the software test team must).  Are enough of the major functional requirements that map to any specific business requirement sufficient to indicate that the business requirement is adequately covered?
• Are the business rules adequately covered by the test cases?

Lastly, don’t forget to step back to a higher level view using the graphical models you developed for requirements design.

• How are the main process flows covered by the test cases?
• Based on the process flow, can you package groups of related test cases together?
• Based on the process flow, can you sequence test cases in a certain order to make execution time more efficient?
• How are the business data diagrams covered by the test cases?  Are the CRUD (create, read, update, delete) operations for the objects they represent adequately covered by the test cases?
• Are the state transitions in your transition diagram or transition table adequately covered?  Same for decision trees, data flow diagrams and any other graphical models you used during requirements definition.

Capturing Acceptance Testing Scenarios

For the recent project I was working on, we used a combination of PowerPoint for the high-level test requirements and Excel to capture the test cases themselves. Using a test tool is better and using a test tool integrated into a requirements management tool is the best way.  I have used both Excel and good tools at different times and good tools are essential for anything beyond simple projects.  Basic attributes to consider:

• One row for each test case (for Excel) or one test item in a tool
• ID
• Test Description
• Test Steps
• Expected Results
• Special Data Needed

I include the following so that coverage is explicit and can be referred to by the testers as needed:

• Business Requirements covered
• Use Case normal course and alternate courses covered
• Functional Requirements Covered
• Business Rules Covered

I prefer to put detailed steps to run the base tests that are the setup phase for many other tests, and then just refer to that base test in subsequent test case’s test steps.

Here is tiny snippet of a spreadsheet example:

Don’t Forget Data Needs

Data combinations are often required to fully cover the testing of key areas. In addition, setting up an adequate set of data elements that can be used over and over by testers is essential.

• Make sure that someone owns the data. This is rightfully the business stakeholder, but with the data complexity today in large software products, the business stakeholder needs support from the development team / architect.
• Does your project need special data combinations to test key areas?  For example, if you are working in a large multi-national company with large suites of products, you will most likely need combinations of country/business segment/product categories to test applications such as finance, sales, quoting, ordering, and manufacturing.
• Since data permutations can often become very large, use the prioritization (that you should have done) on the requirements that your test cases trace back to, to help you prioritize a minimally adequate set of test data combinations to achieve the desired test coverage.

UAT Execution Monitoring

When running the user acceptance tests, you need to record the following at a minimum, for each test run:

• Build Name/version
• Build Date
• Tester name
• Test Date
• Pass/Fail
• List defects logged
• Notes

Again, a tool to manage the execution and results of test runs beats doing it manually hands-down.

Leverage the work that you put into your requirements when you design and prioritize your test cases. When the user acceptance tests have passed, you will no longer need to cross your fingers and hope for the best. You can be confident that the software has been adequately tested.