What gets measured gets manipulated | Working Principles

This post portrays a fiction, a caricature. I have never worked in this organisation, but I have seen parts of it in many organisations.

There’s a much-used management saying that often get mis-attributed to Peter Drucker: “What gets measured get managed” it also exists in the negative form “What doesn’t get measured doesn’t get managed.”

This is a mantra bouncing around inside the head of millions of people and today it will be regurgitated in hundreds of thousands of meetings.

Somewhere in the world, right now, there is a team of people meeting together who have been given a problem to solve and one of their first tasks will be to get the data together.

“How often has this happened?”

“Who has this problem?”

“What is the evidence for this problem?”

Somewhere else a team will be trying to work out how they complete a task, run an operation, define a service level, design a sales campaign, or any one of a thousand scenarios. In each of them somewhere conscious and subconscious will be the phrase “What doesn’t get measured doesn’t get managed.”

To many this is known as the understanding phase, or the problem definition phase, sometimes it’s known as the fact-finding phase. In this time of chaos people need information to help them to make decisions and take actions.

This is where it can all start to go a bit wonky. Somewhere in the mind of one of the meeting participants is another phrase “How are we going to measure success?” This question is followed by an equally wonky question “What are we going to report to the management?”

These two thoughts coalesce together into another thought “We need a metric.”

Not “some metrics”, not “a balanced scorecard”, not “metrics and observations”. That’s all too complicated for “the management”, they are too important and too busy to cope with anything fuzzy or squishy, they need a metric.

Once the team has a metric they can create dashboards, draw charts, create RAG (Red, Amber, Green) status charts. They can show that all of the effort is producing results because the metric says it is. The team is a success because you can’t argue with the metric.

The team looks around to see what the metric could be. Measuring overall business value is too difficult, too abstract, but there is a metric that can be used. This metric is easy to collect from the systems available and doesn’t require any complicated analysis. If the team focuses on this metric then it’s “guaranteed” to increase the value to the business, isn’t it?

And thus the task is set: the metric will be communicated, a dashboard will be built, the metric will be reported, the metric will be reviewed, the metric will be served. There will be consequences if the metric doesn’t go in the right direction.

Unfortunately, no-one in the team has given any consideration to the law of unintended consequences. The metric has been chosen without any consideration towards the people factors involved in the metric.

The communication of the metric begins “Reducing/increasing the metric is our highest priority.” The organisation has been set a new focus and the new focus will be served.

People’s focus switches to the metric and away from all other metrics. One unintended, but inevitable, consequence of an increase in focus on one area is a decrease in focus on all other areas.

Middle managers suspect that this will be another one of those short-lived initiatives so look at ways in which they can influence the metric without doing too much work. As they analyse the metric they realise that there are several factors involved in its composition.

These middle managers are wily operators who’ve seen this show before, they know that there are things that they can do to manipulate the metric.

They instruct their staff to enter details onto the system early/late so that the next collection of the metric is higher/lower.

They look through the list of things that are included in the metric and reclassify work into/out-of the metric.

They split/join records in the systems so that the metric is again higher/lower.

They start to record some records in an excel spreadsheet away from the corporate system to serve the metric.

They shift their focus away from the big things that only influence the metric a small amount onto the little things that influence it a lot.

They move staff away from work that doesn’t influence the metric and onto metric changing activities.

These middle managers are careful though, they don’t use all these measures from the beginning. They know that the metric and how it is shown on the dashboard will need to continue to change. It’s not enough for the metric to change once, it needs to change every week/month/quarter. The dashboard needs to move from red to amber to green. It can’t suddenly go green no-one will believe that. They sandbag some of their manipulations for the next iteration of the metric.

Steadily the metric begins to move. Everyone involved waves their hands in the air and cheers the success.

The team is seen as a huge success and is moved onto another project/problem/etc. where they again analyse the problem, define a new metric, and develop a new dashboard. No one visits the old dashboard anymore. The senior managers cancel the review meetings for the original metric, it’s still collected but it’s ignored. Everyone’s attention has moved to the new metric, the new dashboard and the new meetings.

Only a few people notice the irony of the attention given to the original metric being the cause of the new problem and the need for the new metric. The new metric, again, ignores the human aspects.

Beware of the metric – cave de metrico.

Header Image: This is a view of the Vatnajokull Glacier as it flows out toward the south of the island. (No, I don’t know how to say that word.)