Why K.P.I.’s are a double edged sword

Quick disclaimer; I’m writing here about something that is an amalgamation of personal experience, observation, and information I’ve absorbed from obsessively reading books and listening to interviews/podcasts/discussions about productivity and effective management over the past fifteen to twenty years. Perhaps in time I’ll revisit this and do a more properly researched and sourced article, for now consider this an opinion piece. Something to encourage you to think and start a conversation and look into the data yourself. It’s a concept that, honestly, the cure is just being aware of the concept.


When I was a kid, my family ran a printing shop. Originally opened by Opa (dutch for grandfather) in the 1970’s I believe, when he retired it was passed to my aunt and then, when she decided to sell the place, my mum and dad said “we’ll have that then.”

Ultimately keeping afloat a business built around the kind of small scale printing work that most modern home computers can now achieve was always going to be an uphill battle, and the shop was already doomed before my parents took control for several other reasons that wouldn’t be out of place in a cheesy soap opera. But my parents did manage to keep the place going for several years, long enough for me to have clear vivid memories of lurking around after school and day dreaming of how one day I’d be taking over the shop myself when I was old enough.

I learned two things from that shop. One, I really liked the idea of being self-employed and running my own business. Or multiple. I had the plan as a kid that once I was old enough, I’d take over and then hire someone I trust to run the place for me, then put all the profit into buying some other small business and doing the same, until I wound up with dozens and dozens of small places doing different things all self-sufficient.

“First the print shop, then the city, then ve conquer ze verld!”

Secondly, watching some of my aunt’s methods for handling business (particularly the one she setup after selling the print shop) I became very conscious of the fact that it’s very possible to keep up the appearance of being successful and productive while actually being the exact opposite. On the surface she looked to be the most well off of anyone in the family, and it wound up causing a lot of drama that we were struggling with the print shop so much when she’d been showing a happy profit up until the day she she sold, because she’d kept quiet how much debt was building up unpaid on credit cards to cover the difference that should have been cutting into those profits.

I don’t want to rip too much into a relative that is no longer with us, and I don’t know the exact specifics, so I won’t say more except it made me aware. Indicators are not the equivalent of proof, they’re just clues. Someone who always has cash to spare for drinks on the weekend and an expensive house might be earning good money and be well off, or they might have dozens of overdue bills and are about to have their car repossessed. Many people in high paying jobs are barely covering their living costs while the cashier at mcdonalds has a few grand saved up for emergencies.

To the point!

Which gets me to the point of the title. Key Performance Indicators. If you aren’t familiar with the term it’s a common thing in business to setup kpi targets for staff. When I worked pizza delivery for example, one of our kpi’s was average delivery time, measured (in theory at least, but I’ll get to that) from when we leave the store with the order to when we get back ready for the next one. It’d be assumed that half that time was spent getting to the customer and the other half getting back, so if you had a delivery time of thirty minutes it meant you were generally getting to the customers house within 15 minutes of their food coming out of the oven.

Simple yeah? Deceptively simple. Because what was really being measured was “What is the average amount of time each delivery is assigned to this driver”. And one of the other kpi’s the managers were tracked on was how long between an order being placed before it was assigned to a driver. Again, in theory, that was meant to give a sense of how long each order was waiting on the racks for a driver to pick it up. And a few of the managers I worked under had the idea to start assigning orders to drivers while those drivers weren’t yet back from their previous run. I’d get back from a delivery, go to mark it as complete, and find I was already listed as being ten minutes into the next delivery.

The managers brushed it off by pointing out they were also marking me as back from deliveries earlier, so it “shouldn’t” be effecting my kpi’s in the job, but once it got quieter there would always be at least one delivery that showed me as taking twice as long as I actually had because I’d been signed out early and then left until I signed myself back in.

All this manipulation of data had three impacts.

It made the data inaccurate as an indicator, since the manager was hitting the button as soon as possible rather than when the driver actually got there. The data implied there was always a driver ready to go as soon as an order came out of the oven, instead of correctly showing how long orders had to wait for someone to be available. The on road times were also now largely inaccurate, making it look like some drivers worked well during that short rush but then immediately slacked off afterwards instead of accurately showing the times were roughly the same.

Second, it made the handful of staff who only worked during those rush times and were happy to game the system look like they were outperforming everyone despite making less effort themselves. They started being rostered on more often in place of other staff, which leads to a feedback loop - the more time spent with staff cheesing the data like that, the less time spent with it tracking correctly, so the more skewed the data gets.

More number more good?

And third, the store suffered. High staff turnover because the workplace became toxic with unrealistic expectations (if half the time is showing drivers completing a certain delivery in fifteen minutes, they aren’t going to accept that the address is actually a twenty minute drive both ways). All the experienced staff who knew how to be effective without cutting corners left, all the people remaining where either burnt out, or just didn’t care about anything beyond gaming the metrics, so the product being provided (the pizza’s, and the service) all got worse. Customers start going elsewhere, so the pressure goes up to improve, and another feedback loop happens.

Sadly, I haven’t yet had a job where this hasn’t happened sooner or later. Some managers have been able to push back and mitigate the damage, but the short term boost in stats usually makes it seem good, and by the time the inevitibale drop happens it’s been just long enough nobody seems to be able to connect the dots. (At least, when I’ve tried discussing this with managers, I haven’t been able to articulate it in a way they’re willing to hear).

It’s like having someone who is sick, you take their temp and confirm they have a fever. But then because your goal is to get a lower temp on the thermometer you stick a piece of cloth between it and the patients skin, get a cooler result, and call it success. The patient is now getting heatstroke because that extra fabric is trapping more heat in their body but mission accomplished the indicator is giving a better measurement.

That example is a bit extreme but you get the idea. I hope. If anyone’s read that last paragraph and thought that’s a genius way to handle lowering someones temperature you are officially forbidden from working in medicine.

Seriously, get to the point!

So is there a point to all this or am I just rambling again? Yes, and yes.

It’s important to keep in mind what the indicators are an meant to be indicating. If a thermometer is meant to say if the patient has a fever or not, you don’t care about changing the what the thermometer reads, you care about getting that persons fever under control. And if you successfully get the fever under control but the thermometer still gives a bad number, you get another thermometer. You also wouldn’t just take the thermometer reading in isolation, you’d be looking at the patient. Do they look unwell or uncomfortable? If they look perfectly healthy, have no discomfort, but just came in from a walk on a hot sunny day then maybe the high reading is fine. If they look half dead, can barely speak, are flushed red and their temp says normal, you’re going to know something is wrong because in that situation they SHOULD have a fever.

Kpi’s in a business are meant to indicate how well the staff are doing their job. But it’s an “at a glance” thing. It's one piece of the information. If you aren’t conscious of exactly what the indicator is measuring, you’re going to take the wrong information.

Data doesn’t lie. Manipulating data so it’s misundersood and points to the wrong conclusion is painfully easy. Remember that.

It’s also important to keep in mind what the contributing factors are to an indicator. In my current day job, one of the metrics we’re tracked on is the average time we spend with customers. The intention is to track efficiency, if someone is on average taking much longer than expected then you know it’s worth looking into why. And it’s done as an average over time to try and account for the variation in customers - some interactions are going to take an hour, most take a few minutes, and it’s more or less at random which it will be.

But it often gets treated as something the staff have direct control over, which isn’t really true unless they’re in a position to say “Sorry, appointment time is over, come back next week”.

If there’s a shift in the ratio between short easy concerns and those longer hour long complicated cases, the average is going to go up as a direct concequence.

Someone looking at just the immediate data with a short term view might see that, and push for staff to be faster. Start closely monitoring their times during the day and reaching out to them anytime an interaction is taking longer than the targeted time to ask if they need help. That will drop the overall times short term, definitely - but that kind of push to be faster means people will be encouraged to start cutting corners where they can to save time. Rushing staff make mistakes. Mistakes lead to problems which are more likely to be unclear in cause, if everyone is assuming the previous person did things correctly. Those problems lead to more complicated cases. Which tips the ratio further, now instead of one in ten calls needing that hour it’s one in six. So the average time goes up again. And the person who pushed for speed thinks “No problem, I just need to remind everyone to double down again”. And before long anyone who cares about actually fixing the problem and not just gaming the statistics packs up and leaves because the targets just aren’t realistic anymore.

Eventually the only staff left are newbies who don’t have any experience, so are being encouraged to not worry about times and just focus on doing the job properly, and the balance of customer issues shifts back towards mostly simple things with an occasional complicated one - but most places I’ve worked that didn’t happen until years after I’d left, either because the place went bankrupt and was forced to restart or in one case because of a huge class action suit that lasted several years and drew attention to working conditions.

A conclusion at last!

This is why I say kpi’s are a double edged sword. In principle they’re good. I work better when I have some means of knowing how I’m doing, if I’m improving overall or if I’m slipping and need to re-evaluate my workflow. Kpi’s give incredibly useful information.

At the same time, it’s easy to stop looking at the thing being measured and just focus on the measurement itself, at which point the data quickly becomes useless at best, actively detrimental at worst.

One last example. When I’ve tried finding freelance writing work in the past, the work is almost always measured and paid by wordcount. X cents per 100 words, stuff like that. But a shorter piece generally takes more time and effort to produce. The two posts on this site so far that have taken me the most actual work are “Simplicity” and the home page. Guess which ones raw kpi data would favour though? :P