Software Project Issue Metrics

ArgonDigital - enterprise automation experts

Share This Post

 

I love numbers, and so I’m always interested in what metrics we can pull from projects. Sometimes I pull them just because I’m curious, sometimes they actually help support opinions and gut feels, and most often they are useful to make decisions on your project.
On a recent project, I pulled metrics from our issue tracking system to look at the types of issues that were logged. I will first mention that we actually start tracking issues much earlier than most projects – we do it in the requirements phase.
For context, this snapshot was taken about 1 week before the initial deployment of the project.
The included:
  • Requirements questions – for any open questions about the requirements
  • Configuration – these are most relevant for out-of-the-box solutions that are heavily configured instead of developed
  • Test case – when QA test cases need to be updated based on misunderstanding of requirements, design, or changes to either
  • Functional – the functionality developed doesn’t work as expected
  • Data – migrated or created data has errors, fields missing, or the wrong set was extracted from an existing system
  • Training – when defects are logged because a user or tester did not understand how the system should work
  • Change requests – every time a new enhancement is requested it is logged
  • Development ignored requirements – this is an interesting category that came out of the fact that some developers did not read the requirements when they developed.
  • Test data – we had a lot of errors in test data that was created that caused functionality to look like it didn’t behave
A few interesting observations about the chart:
  1. You will never close all your issues so at deployment time, so it makes sense that there are still open issues (yellow).
  2. Good news is that there are fewer relative requirements questions, test case issues, and test data issues at deployment (yellow).
  3. Also good news in the closed issues (green), in that very few change requests were fixed and closed at deployment, relative to functional, data, and configuration issues.
  4. While a number of user experience issues are open at deployment (yellow), a large number were fixed as well (green). It is good to see this – it’s an indication (hopefully) that only the most critical ones were closed.
  5. Of the overall issues (blue), the high level of requirements questions is because we started tracking them early in the project.
  6. The number of overall of test case issues is actually concerning (blue). That’s a sign they were poorly written from the beginning or that the system changed significantly after they were developed – and notice that the change requests that were closed were low however here (green).
  7. Also in the overall issues, the number of change requests is extremely high if you look at it relative to functional issues (blue).
  8. In this project, the number of user experience improvements is high, but also not surprising, since we didn’t put a lot of the UI design in place until late in the project (blue).
  9. Finally, the number of data defects open at deployment is probably too high (yellow) while it’s not a huge percentage of the total defects (blue). The issue on that project was they didn’t do the data-code merge until very late in the project so the issues spiked just before deployment.

More To Explore