Evolution of Agile

Agile has been evolving as the most sought after methodology for Information Technology projects and software product development. With all buzz around agile, organizations that were reluctant earlier on its adoption are now seriously considering to try out some flavor of it. This article explores the evolution of Agile since its humble beginnings in the 1980s and the key differences between Scrum and Kanban, the two most popular Agile approaches.

1984 – Barry Boehm published an empirical study about projects that use prototyping.  He pointed out the increasing popularity in the iterative approach. At this point, Agile was meant to have two iterations in a project. The first iteration creates the prototype and the second iteration develops actual product based on feedback received from prototype validation.

1984 – In their paper ‘Knowledge-based communication processes in software engineering’, Gerhard Fischer and Matthias Schneider mentions about incremental design processes based on Rapid Prototyping approach as complete and stable specifications are not available at the beginning of development.

1985 – Tom Gilb published the paper ‘Evolutionary Delivery versus the “waterfall model”’ in which waterfall methodology was criticized as unrealistic and dangerous to the primary objectives of any software project. He introduced Evolutionary Delivery model as an incremental alternative to the traditional approach.

1986: In his paper ‘A Spiral model of software development and enhancement’, Barry Boehm presented an iterative model geared to identifying and reducing risks through prototyping, benchmarking, simulation, reference checking, analytic modeling, or combinations of these and other risk resolution techniques.

1988 – Scott Schultz and James Martin refined the ideas of prototyping and iterative development into a methodology called Rapid Iterative Production Prototyping (RIPP). It focused on the use of ‘timeboxed’ iterations for software development by small and motivated teams.

1991 – James Martin further expanded and formalized RIPP and published the book Rapid Application Development (RAD).  It focused on faster development of higher-quality products through more expedient processes such as early prototyping, reusing software components, and less formality in team communications.

1992 – The Crystal family of methodologies was initiated by Alistair Cockburn (it was officially named Crystal in 1997). It comprised of the agile methodologies such as Crystal Clear, Crystal Yellow, Crystal Orange, and others whose characteristics are driven by team size, system criticality, project priorities, etc. It addresses the fact that each project requires a customized set of policies, practices, and processes to meet the unique characteristics of a project.

1994 – Dynamic System Development Method (DSDM) was jointly developed and promoted by the members of DSDM Consortium. It is an Agile framework created out of the need for an industry-standard RAD framework.

1995 – Ken Schwaber and Jeff Sutherland published the paper ‘SCRUM Software Development Process’ that assumed systems development process as unpredictable and complicated and suggested cross-functional and self-organizing teams to develop product increments. Scrum uses the empirical inspect and adapt feedback loops to cope with complexity and risk.

1996 – Extreme Programming (XP) was formulated by Kent Beck. It is renowned for the practices such as pair programming and test driven development. XP empowers the developers to confidently respond to changing customer requirements, even late in the software development lifecycle thus achieving higher levels of customer satisfaction. It promotes high customer involvement, continuous planning, continuous testing, rapid feedback loops, and close teamwork to deliver working software at very frequent intervals.

1997 – Feature Driven Development (FDD) was introduced by Jeff De Luca and Peter Coad. It consists of two main stages: discovering the list of features and then the feature-by-feature implementation. FDD has more formal requirements and steps than XP.

1997 – Adaptive Software Development (ASD) was derived from FDD by Jim Highsmith and Sam Bayer. It consists of a repeating series of speculate, collaborate, and learn cycles that are feature-based, iterative, timeboxed, and risk-driven and facilitates continuous learning and adaptation.

2001 – Representatives from the non-waterfall light processes met together in Utah. They established the term ‘Agile’ for those light processes and developed Agile Manifesto that focuses on a common set of principles. Agile Alliance was incorporated.

2005 – David Anderson experimented with a visual board called the Virtual Kanban System for Software Engineering for a project at Microsoft. Kanban is a form of lean software development approach. The project team uses Work-In-Progress (WIP) limit to the number of items they work at any point in time and ‘pull’ the next work item accordingly.

2009 – Two entities for exploring the Kanban approach were formed, one addressing business concerns, the Lean Systems Society (LSS) and the other aimed at giving the community more visibility, the Limited WIP Society.

2012 – Dean Leffingwell published the first version of Scaled Agile Framework (SaFe) that combines Agile approaches with more enterprise-centric organizational practices to scale Agile to the enterprise-level.

2013 – Craig Larman and Bas Vodde introduced Large-scale Scrum (LeSS) framework for inspecting and adapting the product and process when there are many teams – at least two teams and up to groups of 500 or 1000 people.  It exposes weaknesses in organizational design with respect to structure, processes, rewards, people, and tasks and supports organizational change to facilitate an effective Agile transformation.

According to a survey (Fig 1) on product methodology adoption rates conducted by Actuation Consulting in November 2014, only 10% of organizations are using the traditional Waterfall methodology for software product development.  About 45% of the organizations are using some form of Agile methodology with Scrum (33%) as the leading choice followed by Kanban (3%).  The remaining 45% use a cautious and experimental, blended approach by choosing Agile for some of the projects and Waterfall for others. This survey indicates the acceptance of Agile methodology for software product development and the trend favoring Scrum and Kanban approaches.

Fig 1: Product Development Methodology Adoption Rates

Scrum has garnered increasing popularity in the agile software development community due to its simplicity, proven productivity, and ability to act as a wrapper for various engineering practices promoted by other agile methodologies (McLaughlin, 2015).  But, it may not be a one-stop solution towards agility. Due to the organizational culture or the nature of the project, there can be scenarios in which Scrum may not work effectively.  In those situations, teams have started opting for Kanban that is less prescriptive and more adaptive in nature. Kanban got only two constraints: Visualize Your Workflow and Limit Your WIP; everything else is up to the team to get adapted to. Following are the major differences between Scrum and Kanban.

Scrum

Kanban

Timeboxed iterations are prescribed Timeboxed iterations are optional. (Can be event-driven)
Team commits to a specific amount of work for each iteration Commitment is optional
Uses Velocity as default metric for planning and process improvement Uses Lead time as default metric for planning and process improvement
Cross-functional teams are prescribed Cross-functional teams are optional. Specialist teams are allowed
Items must be broken down so they can be completed within one sprint No particular item size is prescribed 
Burndown chart is prescribed No particular type of diagram is prescribed
WIP is limited indirectly (per sprint) WIP is limited directly (per workflow state)
Estimation is prescribed Estimation is optional
Cannot add items to ongoing iteration Can add new items whenever capacity is available
A sprint backlog is owned by one specific team A Kanban board may be shared by multiple teams or individuals
Prescribes three roles (Product Owner/Scrum Master/Development Team) Does not prescribe any roles
A Scrum board is reset between each sprint A Kanban board is persistent
Prescribes a prioritized product backlog Prioritization is optional

Source: Kanban and Scrum – Making the most of both (Henrik Kniberg & Mattias Skarin)

Organizations are realizing that adoption of Agile processes may not be as easy as it sounds and requires significant organizational culture and process changes and sustained management commitment to gain most out of it. Inspect and adapt nature of Agile methodology can be utilized by teams to experiment and choose an approach (XP, Scrum, Kanban, etc.) that works best for them.  Teams can do trial and error on the prescriptive and adaptive processes of an Agile approach and choose the ones that work best for them. The Agile adoption trends towards incorporating the processes from multiple Agile approaches that enable the teams and organizations to be more productive. It will be interesting to see the continuous evolution of Agile practices that facilitate more effective and sustainable enterprise-wide Agile transformation.

 References

Agile Alliance. (2013). Retrieved June 20, 2015, from http://guide.agilealliance.org/timeline.html

Garacie, G. (2014). Product Development Methods: Is Agile the Most Widely Used?. Retrieved July 1, 2015 from Actuation Consulting: http://www.actuationconsulting.com/product-development-methods-popular/

Kniberg, H., & Skarin, M. (2009). Kanban and Scrum – Making the most of both. InfoQ.

McLaughlin, M. (2015). Agile Methodologies for Software Development. Retrieved July 1, 2015 from VERSIONONE: http://www.versionone.com/agile-101/agile-development-methodologies-scrum-kanban-lean-xp/

This entry was posted in IT Operations. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *