“I am not a speed reader. I am a speed understander.”
Isaac Asimov, via Brainy Quote
As business analysts, it is critical that we understand the importance of speed and performance to users in the applications we are designing, especially when we are replacing legacy applications. And I have found that asking users direct questions on performance expectations will not always give me the answers I am looking for.
In many cases, the performance numbers or targets given are idealizations that will be fantastic if they can be hit but not necessarily realistic. But the two biggest dangers I have found when it comes to dealing with performance are:
- Having a proper context for the requested performance.
- Understanding the consequences of poor performance.
Context is critical to understand why performance is important.
No one is going to say they want their new system to perform slower than what they have now. But apart from understanding the user’s basic desire to finish their tasks and move on with their lives, it is very important to understand the underlying motivation that is driving their demands for speed.
Once you understand where the user is coming from, you need to understand what the real consequences of degraded performance are:
- Will it result in users being unable to perform their tasks?
- Will it mean that at certain times of the year when loads are very heavy, all the work cannot be completed in the allotted time?
- Will users lose money? Will the company lose money?
These are very important questions that must be answered in order to justify to yourself and eventually the developers the need for them to spend time and energy on improving performance.
In the course of my work, I have identified some ways of understanding and defining performance, the context of performance, and the impact of nonperformance on individuals and the organization as a whole. These are provided below.
- Define Performance Requirements for Maximum Load. This is a key point when it comes to defining Performance and Capacity requirements. These should always be defined based on the Maximum load or usage of the system. Far too often “averages” are used to determine acceptable Performance metrics and system throughput. Depending on how much higher the Maximum is than the Average value, the real world Performance of the application can go from “excellent” under normal circumstances to “totally unacceptable” at times of heavy usage if it optimized for the Average usage model.
- Observe Experts in Action. When writing requirements for Legacy application replacement, spend time observing how the experts use the current application, as much time as is practical and possible. There is a tremendous amount that can be learned from watching these users that simply cannot be gleaned from interviews or conversations.
It is the same difference between reading that a pitcher can send down a fastball at 98 mph and actually seeing one whiz past you. It is invaluable for an analyst to see how quickly and effortlessly expert users breeze through an application in the performance of their tasks. This is the same user experience that the analyst is attempting to recreate by defining the appropriate functional and non-functional (Performance) requirements.
- Identify Key Features and Optimize their Performance. All Features in an application while essential and necessary at some time (hopefully), are not used equally. Some Features are used repeatedly while others only rarely. Common sense dictates that the best improvements in Performance come from optimizing the Performance of these frequently used Features.
Find the 20% of the Features that are used 80% of the time and target them for optimization. Observation of experts in action is an excellent way of determining many of these high value Features for Performance improvement. Make a list of what you believe to be key Features from a Performance standpoint are and validate this with expert users. Keep iterating till you have a target list of Features that you want to make as responsive as possible.
- Understand the Entire Workflow or Process. It is important to understand the whole process and not just snippets of it. Requirements are typically defined for specific features and development is also done this way. It is critical that the analyst steps back from the weeds once the features are defined and observe the whole forest.
This gives a different perspective on how the entire application works from end to end. Besides the obvious benefit of gaining a holistic comprehension of the system, it provides extremely valuable insight into how best to optimize the performance of the entire system. It is practically impossible to match the time taken for each sub task on a legacy system with an identical outcome in the new system. Some tasks will take longer and others will be quicker.
The key is to keep the overall performance in check. Without knowing how the different pieces fit together, it becomes very difficult to optimize performance of the whole system.
- Understand How Users are Compensated. This might sound odd and you might wonder “what does compensation have to do with application performance?” It turns out that in certain situations compensation is the key. IF users are paid based on the number of transactions they can successfully complete in a given time period, then they will refuse to take delivery of an application that increases cycle time.
All else being equal, they will earn less money if it takes them longer to complete a transaction because the application is slower. This is not normally a question that an analyst would even think of asking but it is critical to understand if the performance of your planned system is going to hit your users in their pocketbook.
- Understand How Users Performance is Evaluated. This is closely related to compensation. In some situations, users may not be compensated based on volume but their efficiency may be measured based on their throughput. So any drop in performance of the application is going to start showing up in the user efficiency measurements. Simple rule of thumb – anything that makes the users look bad is going to be a huge problem for the application and its continued survival.
- Understand Key Metrics the Business Tracks. All businesses track metrics like sales per employee, margin, overall throughput, number of transactions, errors, defects and so on. You need to understand what the key metrics they track are and if any of them are dependent on performance.
This will give you an idea as to the importance business places on the performance of the application as a whole or on certain baseline assumptions about performance that are implicit in their measurements.
- Identify Opportunities for Automation. See if there are any manual steps in the current state that can be automated. The impact these kinds of improvements make to the overall throughput or performance of the entire application is huge. A lot of sins in the performance area can be washed away with a couple of key automation steps in the new system.
In the next and final installment of this series, I will provide ways of managing performance expectations of the new application that will replace an existing legacy system.
