Automation Makes You Rigid and Brittle (and Speeds You Up)

Automation is everywhere, without it we would struggle to do all sorts of things that we regard as part of our normal day-to-day lives.

The letters that we receive through our door are automatically sorted.

The containers in our fridges have been automatically filled.

The item I’ve just ordered online will be automatically picked and loaded into a logistics system (with some help from humans)

I’ve just received an automated notification on my phone asking for feedback about a recent interaction with someone from an insurance company.

As you can see I’m using quite a broad definition of automation here, but much of what I am about to say is particularly applicable to IT automation, an area that I work with every day.

Each of these automations is dependent upon a set of criteria to work. They have prerequisites. They need to have a steady “if this” before they can “then that.”

For automation to operate stably the “if this” needs to be known, predictable and repeatable. The “if this” can be complicated but the complexity needs to be constrained. For automatic letter sorting to work there needs to be an address that a machine can process. In the UK we use postcodes for the heavy lifting on that process, automation struggles with “Aunty Mable, Birmingham”. Automated container filling requires containers that fit a dimension profile.

The known, repeatable and predictable constraints that automation requires introduce rigidity into the end-to-end process. Ask a human to fill a completely random set of containers and they will use their flexible ingenuity to get the containers filled. You can’t ask the same of an automated system.

For automation to pay back the “if this” needs to stay the same for an extended period, automation needs volume. A container filling plant will run batches of work because switching between configurations takes time. You can’t do a one-off with a different size in the middle of the current batch.

Every good automation system has an exception process to deal with things that fall outside the parameters of the “if this”, but exception processing is the last thing you want to be doing. Exceptions are very expensive indeed. Imagine the parcel logistics situation where nearly all of the items are automatically handled, but a small number of them isn’t. What is going to handle those exceptions? A human, the pinnacle in flexibility. Not only is the human more expensive than the automation, but this human is also likely to be under-utilized driving in even more cost. Hopefully, the volume of exceptions is low, but if the “if this” is too brittle it’s easy for the exceptions process to become overrun and overwhelmed.

Flexibility comes with an inflated cost of production; automation comes with the cost of rigidity. We need to pick which is most important for the phase that we are in.

I see many projects where people are leaping to automation early in the activity without understanding that this drives a rigidity that isn’t helpful this early in the activity. I’ve worked on several projects where we’ve had to redevelop the automation to align to needs that have only become evident part way through the development cycle. Minor changes driving bigger changes. Development effort wasted on automation that hasn’t had the opportunity to get anywhere close to its required batch size. There’s also a risk that constantly changing automation carries unnecessary brittleness.

Some years ago, I adopted a three-phase thinking:

• Define the process
• Optimise the process
• Automate the process

I think this idea came from Lean but I struggled to find any reference to it in that documentation. It might even have been a Six Sigma thing. It doesn’t really matter where it came from, it makes sense to me.

Don’t get me wrong here, I’m not suggesting a massive waterfall activity where a whole project is defined, then optimised, and only then automated. Automation is best delivered throughout the lifecycle of development, but we should try to avoid taking on the rigidity too early in the cycle. We should automate what is known and predictable, avoiding automation of the unknown until we need to. We should also strive to make the cost of change in automation as low as possible so that the required batch size can stay low.

Automation is good, so long as you know exactly where to put the machine.

Eliyahu Goldratt

Header Image: We’ve seen a lot of this blue-sky stuff recently, and we aren’t really sure what to do with it. Taken in the Lake District from the shores of Derwentwater.