This is a two part series on the impact of software requirements on technical debt. Part 1 summarizes technical debt and part 2 discusses the impact that software requirements can have on technical debt.

 What is Technical Debt?

Ward Cunningham coined the term Technical Debt in a 1992 OOPSLA experience report on his work on a portfolio management system:  “Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise.” (See Cunningham’s 2009 video for his more detailed explanation of the metaphor.)

Steve McConnell defines technical debt as:

“The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job. In some cases, this kind of debt can be incurred unknowingly, for example, your company might acquire a company that has accumulated significant technical debt that you don’t identify until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden platform and inadvertently creates more debt. We’ll call this general category of debt Type I. 

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. “If we don’t get this release done on time, there won’t be a next release” is a common refrain—and often a compelling one. This leads to decisions like, “We don’t have time to reconcile these two databases, so we’ll write some glue code that keeps them synchronized for now and reconcile them after we ship.” Or “We have some code written by a contractor that doesn’t follow our coding standards; we’ll clean that up later.” Or “We didn’t have time to write all the unit tests for the code we wrote the last 2 months of the project. We’ll right those tests after the release.” (We’ll call this Type II.)”

Why Should You Care?

Jim Highsmith  in his article The Financial Implications of Technical Debt, states: 

“One problem with technical debt is that the impact can be slow growing and somewhat hidden.  To the question “Fix the technical debt, or build new features” we know how it is usually answered. As it gets worse customers complain about slow delivery, increasing the pressure to take more short cuts, which increases the technical debt, which slows the delivery process, which increases customer dissatisfaction, in a rapidly spiraling vicious cycle. Unfortunately, by the time many organizations are paying attention, all the solutions are bad ones: 1) do nothing and it gets worse, 2) re-place/re-write the software (expensive, high risk, doesn’t address the root cause problem), or 3) systematically invest in incremental improvement.

 Israel Gat has been working to identify the financial cost of technical debt by examining existing software and calculating the cost of fixing bad code. Recent studies have indicated this overall “hidden” cost of technical debt in the $1 trillion range in the US. But this is only the tip of the iceberg when looking at the total financial impact.”

Highsmith illustrates in the above graph how technical debt increases the cost of change over time, until software becomes almost un-maintainable.

Symptoms of Technical Debt:

Ryan Shriver in his paper Seven Strategies for Technical Debt describes the following 6 symptoms of technical debt:

• “Poor Customer Responsiveness: Everyone’s generally frustrated about the elapsed time to deliver updates to customers. Because management is customer facing and making personal commitments, they in particular don’t understand why they can’t be more responsive to seemingly simple customer requests. Technical debt can often be the culprit.
• Long Delivery Times: Related to responsiveness, it seemingly takes forever to get new releases out the door. Six months to a year is not uncommon, and management in particular is perplexed as to how to shrink their time to market in order to be more competitive.
• Late Deliveries: Managers often complain how they just “relied on the estimates developers gave them.” What they don’t know is that it’s extremely hard for even the best developers to estimate when
  there’s a high level of technical debt. This is due to the risk and uncertainty with making any change, even minor ones (i.e. fixing one bug creates two more). Technical debt related to poor design and
  development practices is a tax on development, increasing the likelihood of late deliveries and frustrated teams.
• Lots of Defects: Managers and teams observe long periods of time spent in testing with many defects found very late in the process. This can cause late deliveries or require the creation of expensive hot
  fixes after release. In one recent company I visited, a variety of stakeholders from managers to team members estimated that 50 percent of their development capacity was spent fixing defects. In other
  organizations I’ve worked with, 30 percent is not uncommon. Not surprisingly, this leads to our next symptom…
• Rising Development Costs: This tends to get management’s attention, especially when it considers how rising costs negatively impact plans for business growth. Systems with high levels of technical
  debt don’t scale well to meet the needs of business, which inhibits revenue growth and results in shrinking profits due to ever-rising development costs. As Olivier Gaudin says, “Bankruptcy is the logical extension of technical debt uncontrolled…we call it a system rewrite.”
• Frustrated and Poor Performing Teams: When I ask the managers and teams about their morale, they often talk of being dedicated employees who are tired of working countless nights and weekends working overtime. Some are burned out and may quit (or already have), but most want to deliver real value to the customers. Teams are sandwiched between managers demanding dates and their own desire to “do the right thing” and pay down technical debt throughout the project. Morale deteriorates as technical debt continues to mount with each short cut taken without a view toward ending the vicious cycle.”

Strategies for Paying Down Technical Debt:

Shriver continues in his article to describe the following 7 common causes of technical debt:

1. Poor coding and testing practices
2. Poor system design
3. Poor communication and collaboration
4. Lack of standard development process
5. Poor requirements with no common definition of “done”
6. Lots of manual effort for a release
7. Lack of proactive investment in paying down deb

 Next time in part 2, I will focus on the impact that requirements have on technical debt. I will start with Shriver’s strategies for poor requirements (#5 in the list above) and expand upon it using business objectives, business value prioritization, models, traceability and business value driven requirements change management. For reference, Shriver’s repayment strategies for poor requirements are:

• Establish a common “definition of done” for each requirement, user story or use case and ensure it is validated with the business before development begins. A simple format such as “this story is done when: <list of criteria>” works well.
• The product owner presents “done” to the developers, user interface designers, testers and analysts and together they collaboratively work out the finer implementation details.
• Set expectations with developers that only stories meeting “done” (as validated by the testers) will be accepted and contribute toward velocity. Similarly, set expectations with management and analysts that only stories that are “ready” are scheduled for development to ensure poor requirements don’t cause further technical debt.