Over the spring, we took on a new project to identify a best of breed requirements management tool to use when our customers are not using sufficient tools. Since the thing we do is requirements….well, you won’t find it too surprising to hear that we defined requirements for a requirements management tool!

We successfully applied our vendor selection process to this project. The product management team identified actors, use cases, data types, functional and non-functional requirements. We ultimately used about 150 requirements in our selection process. To identify a vendor list, we used our knowledge of tools, our customer’s tool experiences and the INCOSE Requirements Management Tools Survey. A co-worker had previously evaluated the INCOSE survey list and eliminated many of the vendors. We did a slightly more in depth review including RequisitePro, Contour, Doors, Caliber, CaseComplete, FeaturePlan, CaseComplete and FeaturePlan. We narrowed it further to Doors, Caliber and RequisitePro based on feature richness.

I’ll jump to the punch line and tell you we selected Borland’s Caliber RM as our tool of choice and are currently piloting it on our first project! Borland has some other related tools we hope to make use of – DefineIT to help visually model use cases and Together for creating UML diagrams.

So, let me explain a little about how we came to that decision. I certainly don’t want to indicate the other tools on the market don’t have a lot of capabilities. In our analysis of features, I scored each of the vendors’ capabilities against our requirements on a 3 point scale. I prioritized each requirement on a 3 point scale. And I took the product of priority and capability scores and summed those across the requirements. Actually, the total scores of all 3 tools were very comparable! What I found was that where one tool was strong, another might be weak, but they made up for it in another set of requirements. So a couple highlights based on my opinion in looking at them:

• Caliber is best out of the box for collaboration
• Doors is better at imports
• Caliber does not handle working on requirements while disconnected
• Tables and images were handled better in Caliber and Doors than in ReqPro
• I preferred the traceability mechanism in Caliber
• You can map screenshots to workflows or use cases in DefineIT (links to Caliber)
• Bulk edit is easiest to do in Caliber, much easier than Doors

So in the end, the functionality alone did not give me a clear winner, so this is where my non-functional requirements became critical to the decision. To put it simply:

• ReqPro was by far the worst
• Doors felt archaic
• Caliber was just an inviting look and feel and obvious on how to use it

ReqPro was eliminated quickly at this stage. Some of this is obviously personal preference. But for example, in Caliber, I instinctively would right click to perform some actions and the menu I expected was there. It sounds simple, but I did not have that experience in Doors and it frustrated me. And actually one of my favorite things, the Requirements Grid in Caliber is slick! It was easy to do just about anything I could think to do in the grid as far as viewing, filtering and mass updating requirements.

The next thing I looked at is extensibility of the remaining contenders. Doors has an extension language, DXL, that allows you to do quite a bit. But Caliber has an API that allows you to do even more. The Doors solution means you have to learn a scripting language, but once you do it’s pretty easy to use. However, you are limited to the commands of the language. The Caliber solution means you have to have someone who can write code, however you can do far more. So I expect extending Caliber will be more challenging up front, but you have more possibilities.

I also headed out to talk to some of our customers about their experiences with the tools. Because we are in consulting, we have to pay attention to what our customers are using, to a degree. I have found a lot of people use Doors, so I couldn’t entirely ignore that. However, just because it’s popular does not mean it’s the best tool out there. I will say though I heard fantastic reviews of Doors from one of our customers. They loved the tool and they loved the team from Telelogic. Another customer of ours talked to me about RequistePro. Let’s just say I could not get him to get me excited about the tool. He himself was interested in what I found in this search process. And, we have no customers that are using Caliber, so I couldn’t compare feedback here. I did remember some data from a former trusted co-worker though, who’s preference was definitely for Caliber a few years ago after her own analysis.

Working with the teams proved to be enjoyable all around. Doors was very responsive. As we have gotten into the final selection with Borland, they too have been incredibly helpful. In both cases, I was able to work with technical people who understood the tool. They were all very patient with me as I asked them a lot of questions around how to execute scenarios in their respective tools.

So in the end, as I mentioned, we ended up selecting Caliber as our recommended tool. I would say the heaviest weighing factor for me was the look and feel and ease of use of the tool.

Now I will say there are a lot of newcomers to the market in requirements tools (or old timers coming back) and we’ll be keeping our eye on those to see how they fit into the mix. It does look like a couple of them are changing a lot faster than the big named tools are changing, but they have a long way to go on some of the critical features.

And for now, we are really excited to pilot Caliber on our first project!