Long Distance Agile – How Good Requirements Documentation Overcomes the Problems of Distance and Disconnected Agile Teams

In the last three months or so, I have been approached by a number of analysts and developers struggling with the problems of working on geographically dispersed Agile teams.  In a majority of the situations, the team members are scattered in countries on different sides of the globe with a minimum of 6 hours or more time difference separating them.

In case you are wondering why I am getting hit up by questions of this nature suddenly, the reasons are twofold.  First, a lot of companies have resolved to go away from traditional Waterfall methodology, that have largely failed them, for internal development and are attempting to adopt Agile methodologies.  Second, I have done a lot of teaching of late and have had the opportunity to meet with and talk to a large number of analysts from different organizations.

I have identified a common set of problems these teams are all struggling with.

1. Dispersed teams with key players having no realistic chance of working in the same country, let      alone same room.

2. Adhering to Agile principles that are heavily dependent on continuous, real-time, intense and rich conversation between team members in an environment where such communication is practically impossible due to geographic and time zone constraints.

3. Being required to deliver a “minimal” amount of documentation to the Development team so as to not bog them down and keep the Sprints moving forward.  In essence, the analysts have been told that they cannot create and deliver the kind of comprehensive and detailed documentation typically associated with Waterfall projects.  Distressingly, in many cases, they have      been told to supply the appropriate “User Stories” and nothing  more, by their project leadership.

4. Sprint planning and execution has been chaotic at best, and a total disaster in most cases.

Most of the analysts I spoke with are extremely discouraged and frustrated – finding themselves stuck between a rock and a hard place.  They are being forced to deliver substandard documentation to frustrated development teams with no chance of having real-time conversations to fill in the gaps in the documentation.  At the same time, due to logistical problems associated with distance and time, they are not able to interact in real time with the subject matter experts to truly understand their problem space.  It will not be an exaggeration to say that most, if not all, of these efforts will end in total failure.

So, what can one do in this situation?  One answer to the problem is to create Agile teams that are not separated by ridiculous time and distances so that constant communication is possible.  At the very least, the subject matter experts must be in the same location as the analysts, team leaders and a few developers, if not the entire development team.  This entails building new teams from ground up to support the Agile practices the company wishes to implement.  If this is not possible due to time or budgetary issues what is to be done then, short of abandoning Agile and reverting back to Waterfall or Iterative development practices?

I personally believe that the solution to the problem is obvious, but it requires that compromises be made in “classic” Agile techniques to achieve the overall goal of software that is developed iteratively and delivered in small chunks of functioning code.

The key to making “Long Distance Agile” work is thorough documentation of a depth and quality as one would find on a Waterfall project.  Waterfall projects typically do not fail because of insufficient documentation but because of problems inherent in the methodology that artificially (and unnecessarily) separate the user from the creation of the product in an inflexible, arcane process.  Documentation is the substitute for continuous communication between the users and the creators of the software in a Waterfall world. History has shown us that when the request is separated from a final product by months or even years, the best documentation in the world cannot overcome this communication vacuum.

However, in a methodology that emphasizes constant communication, delivery and feedback like Agile does, the METHOD by which the communication takes place WILL NOT and DOES NOT affect the final product.  Put differently, it does not matter whether the communication is done verbally in real time between the developer, analyst and user or via a comprehensive document created by the analyst in consultation with the user and provided to the Developer.  This is all that distributed Agile teams need to do to succeed.  It is that simple.  Or that difficult, if the team is blindly trying to apply classic Agile techniques, that assume colocation and constant real time conversation among members, to distributed teams.

What changes are needed to create “detailed documentation” in an Agile world?  There are surprisingly few and all are easily done if orthodoxy is set aside.

1. Do more planning BEFORE      starting the first Sprint.  If you have to go through an abbreviated version of a classic Waterfall Planning phase, then do it.  Some teams call  this phase Sprint 0.  If that is what it takes, then by all means do it.
2. Decide on a tentative Sprint 1 scope / backlog ahead of time. Ideally, if you can create a speculative scope for Sprints 2 and 3      ahead of time, that is even better.
   1. If multiple features are planned for Sprint 1, see if you can set some of priority in terms of what sequence these will be tackled by your Development teams.

3. Give your Analyst a week or two head start BEFORE Sprint 1 begins to start their documentation efforts.

4. Do NOT under any circumstances, place any kind of restrictions on the following:
   1. The types of  models they should (or should not) use in their documentation.
   2. The length of the documentation.
   3. The breadth of the documentation.
   4. The depth of the documentation.
   5. The Elicitation techniques       they should use in gathering the information needed to create their       models and requirements.

5. The only guideline they should be given is they deliver documentation necessary for an offsite      development team to create the targeted features.
6. The analyst creates the documentation for the features in Sprint 1, 2 and 3 ahead of time.
   1. They do as much as is practical given the amount of time available.
   2. No matter what, they will create documentation to support the Sprint schedules and not       based on some arbitrary sequence.
   3. The documentation will have any and all models needed to describe a feature clearly and       derive the appropriate requirements.

7. Once the documentation is complete, go through a review cycle to get feedback from the users, developers, testers and other key team members.
   1. All these things are just as if you are working in a Waterfall world.
   2. The only difference is that the scope of documentation is limited to the features targeted for a       given sprint or set of sprints.  It is not intended to be comprehensive.

8. Have your analysts to update documentation and be ready with a complete set when Sprint 1 begins.

Once the Sprint begins, any questions that come up from Developers or Users can easily be handled by the Analysts within the limitations imposed by time and distance without affecting the deliverable quality or timelines.  While the Development team is working on Sprint 1 deliverables, the Analyst will work on documentation from Sprint 2 and 3 in conjunction with the users.  In essence, the analyst gets a sprint or two ahead of the rest of the team and stays there, documenting in advance while also dealing with tactical issues of the current sprint.

There is some amount of inefficiency in this process that will not happen in a project where colocation of all team members is possible.  But they are minor compared to the real benefits of Agile in terms of delivering product iteratively.

P.S.  I have worked on several “Long Distance Agile” projects over the last couple of years where we used the guidelines provided in this post.  The results were very encouraging and actually resulted in good product being created.  So this is a field tested methodology that I believe will work well for teams struggling with the same issues identified in this post.