“Agile is about making it safe to make changes.”
This was the conclusion blurted out by a client of mine after working with him for three days to design a development ecosystem in which his organization could bring their proof of concept tool to an enterprise capable application.
Incomplete though it may be, it is the most succinct explanation I have come across of what agile is really trying to accomplish. It forms a good starting point for an elevator speech, that 30 second sales pitch we should all be ready to present when advocating for ourselves, or something we care about. This post, when read, is more than a 30 second elevator speech, but as a Frequently Asked Questions style exploration of agile, it has been effective at supporting my agile elevator speech.
Making it Safe to Make Change
What did my client mean when he said agile is about making it safe to make change?
The chart to the right shows a typical cost of change over time curve. The black line on this chart has been around for some time, it was old information when I first saw it in college 35 years ago. The black line shows the general trend experienced by projects, wherein the cost of change increases exponentially as the project nears release.
My client was experiencing this phenomenon first hand. Customers were making frequent change requests. But implementing any change–that minor tweak in the data warehouse for example–was causing major headaches, resulting in unintended consequences, issues cascading down through libraries and web page scripts, breaking automated data loaders, some of which only ran weekly, meaning the effect could be distant in time as well as space. The result was that every change request was heavily padded with time to address cascading failures, annoying both the customers, and the leadership those customers went to, to complain.
In this environment, the prospect of flattening that line to more closely resemble the blue line in the above chart was very appealing to this client.
Note that the cost of change in this case will be higher early on as compared to a traditional waterfall approach because change is actually happening on real material, code development, or product clay modeling, tooling for test processes and staged release systems, while in the waterfall approach, these are still requirements on paper.
But Why so Many Changes?
Several reasons:
- Because the application didn’t meet the customer needs.
- Because the customers saw applicability for the application to do far more than it was originally conceived to do.
- Because the world changed. The businesses consolidated, the regulatory environment changed, the competitors drove the business in new directions.
- Because both the customers and the clients learned new things along the way. The process of development is in many respects a learning process about design elements, user interface aspects, applicability to business needs.
OK, but doesn’t this just indicate an inadequate job of planning in the first place?
Possibly. More likely:
- Especially as this was a proof of concept development, the customers didn’t really know what they wanted, or needed, or what could be accomplished at the beginning of the project. This is common for most developments.
- The developers learned along the way. There were a lot of assumptions which drove database design for example, which looked like good ideas early on, but later proved to not be so good, especially when it came to scaling the application to the enterprise level.
- Even if the customers had knows what they wanted, it was unlikely that information could have been fully assimilated by the developers who exist in a different knowledge domain that the customers.
- As the customers used the system, they learned more about what could be done, and so found other applications for this system, applications for which it was not designed at the beginning of the project.
This phenomenon has been around for a long time, how have projects managed this exponential cost of change in past?
The traditional response to this phenomenon of exponential cost of change growth over time has been the waterfall development method, whereby a project begins with requirements elicitation, then requirements commit, then system design, implementation, test, release, and then support. From the time of requirements commit until project delivery, all change is tightly controlled. This is in fact a perfectly reasonable means of controlling exponential cost growth over time. Except for one minor detail: Depending on who you read, more than 60% of all such projects fail.
These projects fail because, as indicated above, there are forces which are constantly driving change, and either the change is controlled (that is prevented) which likely means the project doesn’t deliver what is really needed, or the cost of those changes overwhelm the project budget.
How does Agile flatten the cost of change curve?
There are several aspects of agile which work to flatten the change curve, depending on the type of project and the specific tool set chosen. In software, this flattening occurs by using “test first” thinking, which means before you can code a solution, you need to code a test for the solution. Automated testing, combined with a rigorous incremental release strategy finds failures before they can do substantial damage. Short development iterations, combined with customers on the development team, validate the functionality of what is delivered in each iteration. There is more to agile software development than this, but this gives a flavor of things shared across specific implementations.
In lean product development efforts, the focus is on concurrent engineering, modeling and rapid prototyping to pursue multiple paths until the last responsible point in time. Input it taken and encouraged from all parts of the organization. Emphasis is placed on knowledge reuse, for example with the use of knowledge sheets and spaces such as Obeya rooms to enable rapid communication and knowledge sharing.
While most lean or agile advocates don’t delve into the underlying theory base, there is a substantial base of research in queuing, information, system and organizational theory to support these techniques.
So, is that all agile does, flatten the cost of change curve?
No, though this is probably where the most quantifiable benefit can be seen. There are advantages in terms of team learning, team function, and, as I will cover next, reducing Design In Process costs and risks.
Lowering Design In Process Costs
Work In Progress (WIP) is the term used in manufacturing for the amount of raw material inventory from the time of purchase, until the sale of the manufactured product. In manufacturing, WIP is visible from the raw material on pallets on the receiving dock to the packaged product loaded on outgoing truck. It is visible as material sitting in warehouses, on production lines, in queues, and in scrap. Sum up the cost of all that material, and you have WIP inventory, the economic cost of “owning” that inventory. Lean manufacturing focuses on minimizing queues in order to minimize WIP inventory. If you can free up $10M in inventory, you have just freed up $10M you are not paying financing costs on, or $10M which can be used for other value creating activity.
The product development world has a similar concept called Design In Process (DIP), which consists of the amount of money which has been spent on product development activity until that product begins selling, and thus producing some return on investment.
The chart to the right shows a typical Design In Process investment over time. Since development is an expenditure until such time as the product is released into the market, the chart shows a negative net present value during development and buildup to manufacturing. Note that the negative growth rate increases significantly when money has to be committed to factory development, production line construction, tooling, and inventory buildup. The slope of the curve doesn’t turn positive until product release, and hopefully bountiful sales which subsequently follow. Since the product sales must recoup the cost of development, there is a delay, sometimes a significant delay until the net present value of the product turns positive. This time lag, and the resulting cost of design in progress inventory, is the other area which Agile attempts to address.
Agile processes focus on rapid delivery of value to the customer. There are a couple of reasons for this, the first is related to cost of learning; it is better to get customer feedback early, which leads to more satisfied customers, which will likely lead to more sales (though there is also risk of lost sales due to limited capability). An incremental release strategy also minimized the kind of late point changes described above which can be expensive later. It is also useful to have real customer feedback so that the incremental planning can identify the most valuable deliverables.The other very important reason for incremental implementation is that it limits the economic cost of Design in Progress. There are two economic factors to consider here, the first is similar to the WIP inventory cost of manufacturing, that is the cost of capital invested, and the limit it has on investment in other economic activity.
Incremental deliver also limits the financial risk of product failure. If you find your customers aren’t finding value in that first iteration, maybe you are building the wrong product. You can change course, or even cancel, before you have huge sunk costs. The cost of failure is lower because you have lower sunk cost in the development.
This second value curve is most applicable for custom software development which can be delivered incrementally. This is most often found in custom developments but the concept is not limited to custom software development. Commercial software applications can be delivered as “applets,” which, if they gain a foothold, can be built upon into full blown applications, or abandoned if they flop. Products can be delivered as refreshes of prior products, limiting the total investment and risk of a full scale new product development.
These two objectives of Agile are important because each organization is slightly different, and the implementation of Agile tools should, like the product development they are intended to support, focus on providing value within the specific context. By being clear about your goals, flattening the cost of change curve, and minimizing design in progress costs and risks, you can be more intentional with what agile tools to implement, and when. It isn’t really about process, it is about the goal, and so long as you understand that, it really is OK to pick and choose the process which best fits your context.
Is that all there is to agile?
No, there are voluminous books on the subject. Many focus on a specific set of (sometimes trademarked) processes. A few go beyond the recipe level, to address the why behind the what. My favorite reads on the subject are James A. Highsmith, Adaptive Software Development, and while not focused on agile per se, my other favorite read, which I think gets much more to the theory base underlying effective product development is Donald Reinersten’s Managing the Design Factory. But discussion of those goes well beyond the goals of a 30 second elevator speech.