Posted by Mike on 4th April 2006
This is not a new problem to establishing meaningful metrics.Ã‚Â I call it “Gaming the System.”Ã‚Â This behavior has been around since people started measuring the performance of other people.Ã‚Â Those being measured will always find a way to make their numbers look good.
I know someone who used to be a cashier for Kmart. They had a metric for average scan time.Ã‚Â Her time was the best in the store; because she figured out how to make her numbers look good.Ã‚Â She would arrange all the customer’s items with the bar codes ready.Ã‚Â Then, she would quickly scan each item and hit the subtotal button.Ã‚Â At first, the other cashiers were angry with her, saying that she made them look bad.Ã‚Â However, management caught on to what she was doing to get great numbers.Ã‚Â Guess what?Ã‚Â They trained all the other cashiers to do the same thing and the store’s numbers became the best in the district!
Yesterday, I was at Sears buying a dehumidifier. After making my purchase, I went to merchandise pickup.Ã‚Â I scanned my receipt and my name was added to the queue.Ã‚Â I looked at the screen and to my delight; the average wait time was 2.20 minutes.Ã‚Â Great!Ã‚Â I will be out of here in no time!Ã‚Â WRONG. I watched the worker and saw quickly that he was “Gaming the System.”Ã‚Â Once orders were added to the queue, he would take his hand-held device and mark all the orders complete.Ã‚Â Then, everyone would wait.Ã‚Â The metric looked great, but didn’t reflect reality.
Sears probably spent some serious money to develop this system, including lovely computerized voice prompts.Ã‚Â They probably use this average service time metric to determine staffing levels.Ã‚Â I can tell from personal experience that this store is probably under-staffed.Ã‚Â I didn’t have my watch on, so I don’t have the exact time of my wait, but it was much longer than the 1.40 minutes that was recorded. (I would guess that I waited 20 to 25 minutes.)
Sears could have the customer indicate that they received their item as the event that clocks the actual service time.Ã‚Â (Perhaps by scanning their receipt again.)Ã‚Â That would eliminate the “game.”Ã‚Â In this scenario, the customer starts and stops the clock.Ã‚Â (Also, the worker didn’t check my receipt. They probably have a shrink problem too.)
“Gaming the system” is easy to avoid.Ã‚Â You just have to observe the system in action and make adjustments.Ã‚Â Don’t set your metrics program to reward speed, unless you have an independent way to ensure accuracy.Ã‚Â Set it up to reward accuracy.Ã‚Â I can tell you one thing.Ã‚Â The workers at this particular Sears Merchandise Pickup area were not motivated to hustle.