Agile is often spoken of a development methodology along the lines of “Agile is a development methodology; other common development methodologies are Waterfall, Iterative, etc.” This is akin to saying something like “Weightlifting is a means of keeping fit; other common techniques to stay fit are aerobics, swimming, running, walking, etc.”
There is nothing inherently incorrect with either statement. Agile and Waterfall are indeed development methodologies just as weight lifting or running are both means of keeping fit. The mistakes arise when these statements are misinterpreted to mean that you can just swap out one entry from the statement to another and arrive at the promised land – software that does what it was intended to, or six pack abs worthy of admiration.
Organizations often make the classic mistake of thinking that they can simply swap out their development methodologies from Waterfall to Agile since they are just different means of getting to the same end result. They think this is no more difficult to do than waking up one morning and saying, “I am going to stop lifting weights from here forward and just run instead. I mean how hard can that be?”
It turns out it can be extremely difficult to make the switch. As anyone who has even a passing interest and proficiency in different types of physical fitness activities can tell you, making a switch from weight lifting to a full blown running schedule is extremely non-trivial. Done blithely without a true understanding of what it takes to take up a serious running routine, you will be lucky if you tap out in a week or two without seriously injuring yourself. If you persist blindly, it is only a matter of time before you find yourself with a bunch of aches, pains and injuries that the unconsidered switch to running brings about. This new thing you switched to from your weight training practice to get in better shape quickly turns into a nightmare that puts you flat on your back.
The reason is not that there is anything inherently wrong with running as a means of keeping fit. It is just that it is very, very different from weight training. A whole new set of details and techniques need to be learned and executed properly to get the desired results. How to run (yes I know that sounds strange, but there really are ways of running that are safer and better for your knees, legs and back than other ways of pounding the pavement), when to run, how often, how long, when to rest, what to eat, what not to eat, how to warm up, how to cool down and a thousand other things. It turns out that proficient runners have mastered a ton of these small details that we have no clue about to keep them safe, strong, healthy and give them the results they want.
Organizations that make a switch from Waterfall to Agile without understanding the change they are undertaking are no different than someone who wakes up one morning and decides to ditch their weight lifting routine for a serious running practice. If they are lucky, they will realize their mistake and do one of two things before things get out of hand. Throw in the towel and go back to the old way of doing things. Or really learn how to do Agile properly.
To succeed in Agile, organizations need to do two things. First, they need to learn how to do it properly. This is the nuts and bolts of any Agile practice. Understanding how product backlogs, product owners, sprints, sprint planning and all the other things that make up the Agile methodology work. Second, they need to make the cultural shift. Agile is a completely different way of thinking and behaving. Agile is not Waterfall broken into sprints any more than running is weight lifting done moving your body around as fast as you possibly can for as long as you can.
Agile means working software at the end of each sprint. Agile means intense involvement from business partners for the duration of the effort. Agile means being ready to change as the world around us changes. Agile means really deciding what is important to be done and when. Agile means truly understanding the capabilities of the team and working within those limitations. Agile means holding the entire team accountable every day. Every single day. Not once at the end of the project when the “lessons learned” meeting is held before everyone goes off to onto the next great adventure. It is like nothing an organization with no prior Agile experience has ever done before. Ever.
Most Agile failures occur because of the lack of the cultural shift to accompany the skill shift. Almost every organization understands the need for retraining their teams and do a pretty decent job of this. They retrain their employees so that everyone has a working knowledge of what to do in an Agile team. However, the ones that fail in the transformation spend little to no time in considering the cultural changes that must take place simultaneously with the skill shift.
To me two things are key culturally – accountability and ownership. It is scary for team members to be held continuously accountable. Simple fact of the matter is that Agile is demanding. Very demanding. Mistakes are exposed immediately. Lack of progress cannot be buried in vague project reports. The project burndown tells the tale every day no matter how we choose to spin reality. The team is collectively exposed and held accountable in a manner they have likely never before experienced in their professional careers. Pathologically, it is not uncommon for Agile teams to look really disorganized and chaotic compared to their Waterfall counterparts in the same organization. The reason is not that they are inherently stupid or became incompetent once they switched methodologies. The fact is that the methodology is forcing issues to light to be fixed right away that are hidden in Waterfall until it is too late to do anything about it. Management needs to understand this fundamental difference and not come down hard on Agile teams because of the perception of problems. They need to support the teams without fear of repercussion so that the team holds itself accountable and fixes its own issues without outside interference.
The second cultural shift is ownership. On Waterfall projects, it is often very hard to find someone who will actually confess to “owning” a project or effort. The Project Manager is a specialist guiding along the project she has been given. Developers write code. Analysts define features. Testers test everything once it comes out of the process. And so on. Everyone owns a piece of it. No one owns the entirety. When all is said and done, autopsies of projects are also done similarly. Did Project Management do a good job? How about the analysts? And so on. Seldom is the team evaluated as a whole.
On Agile projects, there is nowhere to hide. Everyone is in the boat together. We either float together or sink together. If requirements are a problem, it will take a sprint or two for it to become obvious. Ditto for any other team member. If they don’t fix the problem, then everyone sinks. This feeling that the entire team wins or loses together fosters a true sense of ownership in both the process and outcome. No working software at the end of a sprint. Everyone loses. Working software at the end of the sprint. Everyone wins. It is as simple or as difficult as that.
It is essential for organizations to support this shift to ownership. Do not fire the whole team when they miss deliverables on a sprint. Let them work it out. Give them the tools and training to work through ownership issues. Things like conflict resolution, focusing on outcomes, team building techniques, understanding team dynamics and communication techniques go a long way to helping teams in a positive and meaningful way. For example, business users may have no idea how to communicate with developers. Or vice versa. Find these kinds of gaps and give meaningful assistance to teams to learn how to work together. The most important thing they must understand is this – they all own the entire outcome, for better or worse.
All of this takes time. I am not talking about years. But it is not going to happen in a month either. The company leaders need to understand that they have undertaken a significant cultural shift and plan accordingly. Be prepared for pain, a lot of pain during the transformation. Be patient while the process unfolds. At the end of the day, the reward will be a lean, nimble software development organization that is accountable for outcomes and takes true ownership in the product. It is worth the wait. Just don’t think it can be achieved with a fiat that directs teams to switch to Agile from Waterfall and expect miracles. They won’t be forthcoming.
