The ugly truth about predictability

As software systems grow more complex , more people become involved therefore managing software development becomes increasingly difficult. To keep up with this increased complexity the industry experts have been telling companies for year to add more metrics, more control, more rigor. Surely, if we could control everything up to a single keystroke we should be able to predict the future, right? To ensure such prediction people needed to be told what to do exactly, when and how otherwise chaos would be lurking around the corner waiting to jump out and take over. Therefore companies added more layers of control and even more resource were wasted on making sure process compliance was met.

So what did we learn, complexity leads to uncertainty. Uncertainty is evil so we added all these mechanism to be able to forecast and predict the future. The main problem with organizational cultures that focus on forecasting is that they significantly decrease any flexibility needed to deal with changing markets. On top of that we can see a significant decrease on innovation and learning.

As flexibility decreases, market responsiveness also significantly decreases. The system that has been put in place to eliminate chaos, to give us control over our projects, has now turned its back against and it causing companies not to be able to deal with market changes due the many layers of the chain of command and the lack of a culture of learning and innovation. Even worse, due to an increasing focus on predictability and control this has let to a major decrease in seeing and hearing what the customer really wanted. Companies aren't just being incapable to deal with market changes they stopped seeing them!

Last, let's see what's the impact of a culture of predictability is on people? Too be able to plan so much in advance managers need to treat people as resource with a certain performance rate. That way when deviating from a project plan you can calculate exactly which resource with which performance rate you need to be able to get back on track, much like machinery parts.

Unfortunately software development is anything like mass manufacturing. It's a creative role one for which creativity and problem solving skills are key. The tools we use are never the same, the software we built always differs from that what we made before, technology and market change on a daily basis, learning truly is king. What we need is way of thinking that stimulates organizational learning while focusing on delivering customer value first.

Lean Thinking beyond world class manufacturing...