A fellow OU alumni kindly pointed us at this article in the Guardian last Friday: Payment by results – a ‘dangerous idiocy’ that makes staff tell lies written by Toby Lowe. It’s an interesting article telling us about how targets are driving the wrong behaviours, something that is frequently debated by the Systems Thinking community. My problem is that we can all agree that it is wrong, but we can’t work out what is right (the  tell-tale signs of a wicked problem)

Pieces like this bring me round to thinking that there are two key aspects to target setting which are subtly linked:

• What problem are the targets actually trying to solve
• What is the cost of collecting the measures

There is a misquote often attributed to the management thinker WE Deming along the lines of “you can’t manage what you can’t measure” and I’d suggest that our target setting and measuring culture may have used this in error.

Deming actually said a lot of things, including that the most important things in your business could not accurately be measured.

To be fair there is some merit in measuring things and using them as metrics, Stafford Beer suggested a model of management through the Viable System Model back in the 1970s that worked on determining useful metrics to tell you whether a system was working, then setting automation around those metrics so that as long as everything stayed within that range, then you left the system to its own devices. This freed you to look at the things that you can’t measure, but that impact your operating successfully: changes around you, policy requirements etc

However I’d like to suggest that the cost of measuring, or collecting data for compliance purposes, becomes a determining factor of how targets are set and measured.

I come back to the two aspects of target use that I made reference to at the start. I’ll be generous and say that we may well set out having a good idea of what the problem is, however my experience is that at the very start we may well have already disregarded parts of the problem so that we can focus only on what we think we might be able to actually solve.

Then we come to the question of how do we collect information about it so that we can tell if we are making a difference or not? We look at the cost of measuring it, and where measures are too hard, we reduce them to something much easier to measure. As we do this, we may fall into the trap of believing that what we are measuring and solving is the problem itself, when actually we need to understand that we are now looking at an incomplete model of the problem

If I digress at this point into my Systems Practice, I’d say that defining the problem, and the defining the measurements schema are co-evolutionary, they emerge from interacting against each other, which is why at the start I said the problem definition and the cost of measuring were subtly linked. I would also suggest that the eventual perceived problem with the measures around it is a reification. People believe it is the real thing, yet it is a model and doesn’t reflect reality.

So what we do?

Firstly I think we have to look at the real costs of collecting data, and be honest enough with each other to say “In the real world we would do this, but we can’t because…. and that affects what we are seeing…”. In other words we are setting boundaries, and we should use a tool like Ulrich’s  Critical Systems Heuristics model here to communicate the impact of what we are, and are not, including in our measurement collection as it helps us to understand the original problem we set out to address.

Secondly we have to get out of the mind set of human data collection being cheap, Lowe refers to the vast amount of form-filling taking time of social workers away from doing what they really should do. The IT person in me suggests that looking at the possibility that the initial cost of automation will be paid back in the time it frees up would be useful.

Taking that a step further we could use newer technologies to collect data for us automatically, for example why not give a patient registering in an A & E department an RFID tag on their card then record the movement rather than hand time-stamping the card and later having someone else record the results.

Measure it once, and save time and money. But that brings about data protection and civil liberties issues, and suddenly our automated data capture has new, but not insurmountable, costs to contend with.

Finally, go back to Deming and the fact that we cannot measure some of the most important things. I think most people understand that knowing how quickly people get treated in hospital, or how many arrests were made on Friday night, is a useful measure, but it doesn’t tell the whole story and no amount of measurements will tell us that.

It still needs wisdom to look at the whole picture and decide, and I can’t see that we will ever be able to automate that.