We have all been on challenged IT and product development projects — the ones that were over budget and behind schedule because we missed the mark and had to do a whole lot of rework just to get sign-off. And of course, after we finally delivered the product and caught our breath, we went through the usual ritual of project introspection – lessons learned – what we did well and where we needed to improve. But improve when, on the next project? Evidently not, that is if we are to believe all the horror stories we read of project nightmares.
George Santayana famously said “Those who cannot remember the past are condemned to repeat it.” We only have to look at the Vasa project of the 1600’s and compare it to the FBI Virtual Case File project of the 2000’s to conclude that George Santayana was a pretty smart guy.
Two Doomed Projects
The Vasa was a Swedish warship that was one of four ships that were to be built over a period of four years, 1625 – 1628. Tragically, it sank on its maiden voyage sailing all of 1300 meters before being toppled by a small gust of wind, killing 53 on board.
Then, almost 400 years later, we have the FBI’s Virtual Case File (VCF) Project. The VCF project, part of a larger project called Trilogy, was cancelled after four years and $170 million dollars. You are probably wondering what do ships and software have in common. Well…nothing except that the problems that plagued the Vasa project were the same problems that plagued the VCF project, namely scope creep and poorly defined and ever-changing requirements, to name just two of a myriad of project issues.
This article is the first of a series where we evaluate the Vasa and VCF projects, highlight the similarities and draw parallels to the idea that the root causes of challenges experienced on projects 400 years ago are not much different from the root causes of the challenges we face on projects today.
“Those who cannot remember the past….”
Let’s take a look at the two projects, focusing on a problem that is all too common…
A Good Beginning
IT and product development projects arise to address a problem or exploit an opportunity…what we refer to as the “Need”. It is from the Need that the other components of project scope can be defined: the goals and objectives, the interfaces, the drivers, constraints, assumptions, dependencies, the limitations and exclusions and the schedule and budget. By defining the Need and documenting the project scope, everyone involved with or who has a significant interest in the system or product will have a clear vision and common understanding of stakeholder expectations.
Now looking at the Vasa and the VCF, we find that both projects started with a pretty good vision and scope:
VASA – The Vasa was part of a larger shipbuilding project, the original need was to construct two large ships (135 foot keels) and two small ships (108 foot keels) and deliver over four years, starting in January 1625.
VCF – The precursor to the VCF project was the FBI IT Upgrade Project, approved by Congress in 2000. The vision was to upgrade and modernize the FBI’s aging IT infrastructure and systems. This upgrade project, known as Trilogy, was divided into the following three projects:
- Information Presentation Component (IPC): Upgrade the FBI’s hardware and software
- Transportation Network Component (TNC): Upgrade the communications networks (LANS and WANS)
- User Applications Component (UAC): Make the FBI’s most used investigative applications (which included the case management system known as the Automated Case Support system (ACS)) “accessible via a point-and-click Web Next…rebuild the FBI’s intranet. Finally…identify a way to replace the FBI’s 40-odd investigative software applications, including ACS.” (Goldstein, 2005)
- Due date: Mid-2004 (June 2001 start)
- Budget: $379.8 million
Scope Creep!
However, a few months after the start of both projects, tragedy struck. For the Vasa project, the king lost 10 ships in a storm. In the case of the VCF, 9/11 happened. Both events impacted these two projects in the same way…a change in direction and heightened urgency to complete.
VASA – September – November 1625 – New direction: The king ordered the following changes:
- The two smaller ships (one of which was the Vasa) were to be built first to replace two of the ships lost and delivered
- The length of the Vasa keel: Changed from 108 feet to 120 feet
- Two enclosed gun decks (as opposed to the original scope of one enclosed gun deck)
VCF – December 2011 – New direction: It was determined that the UAC project as originally scoped (build a web-interface onto the most used investigative applications) was not going to make the agents more effective in their access to and analysis/management of case information. Accordingly, the UAC portion of the Trilogy project was given a new name and a new direction: The name: Virtual Case File (VCF) project. The direction: Build a new case management system (the VCF) that would be comprised of “an entirely new database, graphical user interface, and applications, which would let agents search across various investigations to find relationships to their own cases. The new case management system would host millions of records containing information on everything from witnesses, suspects, and informants to evidence such as documents, photos, and audio recordings.”(Goldstein, 2005).
- New due date: Accelerated timeline with phased delivery
- Delivery 1 – Dec 2003: VCF (including data migrated from ACS)
- Delivery 2 – Dec 2004: VCF Enhancement
- Additional funding to support accelerated timeline: $78 million
Both projects had been working for a number of months to deliver to the original scope – the shipbuilding team worked for approximately nine months before the scope change to focus on the Vasa and the UAC/VCF team had worked for approximately six months on the web-based front-end before being directed to work on the VCF database and applications. With the change in direction, the time spent on the original deliverables was lost. Worse yet, the teams were expected to start over, deliver a product different from that originally scoped and do so on an accelerated schedule.
See the similarities? New direction, bigger scope, less time. As Frederick P. Brooks, Jr., observed in his book The Mythical Man-Month, “More software projects have gone awry for lack of calendar time than for all other causes combined”.
Breaking the Scope Bucket
We emphasize that for all projects, the scope bucket needs to be managed. This bucket looks like this:
Basically, what we are saying is for this amount of time, with this amount of money and with this technology, we are going to do all the things within these boundaries. However, if you want to add to this bucket, you will have to provide me with more time and/or more money and if that does not occur, something has to come out of the scope bucket. If boundaries do not increase or things do not come out of the bucket, you have just increased the risk of delivering late, being over-budget or not meeting quality and/or stakeholder expectations.
In both the Vasa and VCF cases, the original scope is assumed to have fit well within the boundaries of the bucket (schedule, costs and technology). However, with the change in direction, the scope bucket became unbalanced:
Vasa:
- Two smaller ships- delivered ASAP (schedule boundary goes down).
- Length of smaller ships changed from 108 feet to 120 feet (changes made to contents in bucket)
- Two enclosed gun decks (scope items added to bucket).
VCF:
- New case management system (VCF): database, graphical user interface, and applications (scope items added to bucket)
- Accelerated phased delivery ((schedule boundary goes down)
- Additional funding of $78 million (cost boundary goes up)
As you can see from the illustration, the scope of the Vasa and UAC/VCF projects were not managed. Things were added to the respective scope buckets but nothing came out. Timeline boundaries were reduced but nothing came out. Granted, in the case of the VCF, the cost boundary was increased, presumably to add more man-power. But, again to quote Fred Brooks: “Adding manpower to a late software project makes it later.”
Lessons Learned
It appears that both the original shipbuilding project and the original UAC/VCF project were scoped for work that was feasible within the time allotted, the budget appropriated, and the technologies available. However, unforeseen events that occurred in 1625 and 2001 caused the two projects to react and change direction to address new and more urgent needs. Did the change in direction, the change in scope lead to the sinking of the Vasa and the failure of the VCF? In and of itself, probably not…rather, change in direction/scope was just one of many factors (to be discussed in later articles) that led to the respective projects’ demise.
The lessons to be learned here are:
- Change in direction is one of the biggest factors behind cost/schedule overruns and the risk of not meeting quality and stakeholder
- If the Need changes due to a change in mission, re-evaluate your scope. It is often the case that when the need changes, you have an entirely different project that requires scoping and assessment of feasibility within the cost, time and technology
- Schedule-driven product development projects start out high risk – schedules should be set according to that needed to meet the scope.
Our next article in this series will explore another challenge the Vasa and VCF projects had in common and that contributed to overall failure – poorly defined and ever-changing requirements.
References:
- Frederick P. Brooks, Jr., The Mythical Man-Month by Addison Wesley Pub. Co., 1975, 25th Anniversary edition, 2000.
- Goldstein, H. (2005, September). Who killed the virtual case file? IEEE Spectrum.
- U.S. Department of Justice. (2005). The Federal Bureau of Investigation’s Management of the Trilogy Information Technology Modernization Project. Office of the Inspector General Audit Division. Audit Report: 05-07.
