The public sector has invested great hope and massive fortunes in the promises of performance management. From dashboards and scorecards to STAT systems and big data platforms, we’ve bought into the idea that “what gets measured gets done” and if we want better results we need to hold people accountable for the achievement of measurable goals. But after all this time and investment, what have we really achieved? Has all of this measurement produced great insights and innovations? Has performance management increased engagement and overhauled performance? What good has it done your agency?
Wells Fargo is the canary in the mine for performance management. Rather than an aberration or a group of bad actors, they are the poster child for following the performance management playbook. Top management had crystal clear priorities (increasing the number of accounts customers had with the bank), SMART measures with a stretch target (8 accounts per customer because, as the CEO testified, “8 rhymes with great”) and then cascaded those measures from the top all the way to the front line. Each individual employee had a performance goal (the number of new accounts opened) with direct line of site to top management’s priority. Employees who met their goals were rewarded — those who did not were coached up or coached out. With such a robust performance management system, what could go wrong? Well, ask the millions of customers who had unauthorized accounts opened in their name, the CEO and much of upper management that had to resign and the millions Wells Fargo is spending to apologize and rebrand itself. More than a century of trust was wiped out. Why did the employees do it? For the exact same reason the VA health centers fudged patient wait time data — hiding thousands of patients off-book with scores of patients dying without ever being seen. The exact same reason teachers and principals cheated on standardized tests in DC, Chicago, Atlanta, and countless other places. And it’s the same reason public assistance agencies gamed the payment accuracy performance standard. Fear. Performance measurement, rather than being a flashlight that illuminates insights and improvements, was used as a hammer to hold people accountable for systems that were beyond their control.
Gaming the system occurs any time someone is held accountable for a broken system without the power or resources to improve that system. Quite simply, how can we make the numbers if we can’t make improvements? And this is precisely where performance management has led us astray. Performance management believes that the only variable that matters is effort or motivation. Therefore, data and measurement are used as tools to incentivize and motivate; for control and accountability.
Rarely is performance the sole domain of individuals. Granted in areas of personal development (preparing for a 10K, losing weight, stopping smoking) the tools of performance management can be helpful. Setting a goal, measuring your progress, and incentivizing your behavior can all help you get what you want. But in each of these cases the motivation is intrinsic and the variables are under your control. Imagine instead if the weight loss goal was mandated to you by your significant other, and you were to report your progress daily to them. Now imagine that you have no control over what foods you eat, the gym is 45 minutes away and you work two jobs. How would you feel? For most people in organizational life this is exactly how it feels. Measures and targets are imposed upon them. Accountability is geared upward in the organization to someone they are likely to fear. Performance is dependent on count- less variables and constraints, over which they have little control. When we are held accountable for a broken system without the power or resources to improve the system, often our only recourse is to game the system.
Improvement in our organizations comes from improving the design and operation of our systems — our methods — how we do what we do (sometimes even starting over and coming up with a new method). Improvement in method comes from insight — those “aha” moments that come when we see something new or see something old in a new light. This is the purpose of measurement — to provide us with feedback on how our vital systems are performing so we can convene, understand the data, gain new insights, and develop new methods. The purpose of measurement is not to hold those systems accountable. Systems are finely tuned to give you the exact results you are getting. Systems don’t respond to stretch targets, incentives, or exhortations. If you want better results, you have to fix the system. Doing that doesn’t require accountability, it requires knowledge. Insight will not appear where fear persists. One of the enduring legacies of the quality movement was W. Edwards Deming’s exhortation to drive out fear. Fear is toxic. Fear corrodes. Fear gums up our systems and processes. Fear distorts our data and makes it hard to find the truth. (Two examples: (1) Ford’s CEO’s question to his leadership team — How can the company be losing billions of dollars if all their dashboard metrics are green; (2) How is it possible that the crime rate went down in a major U.S. city every quarter for 13 years? It’s likely and applaudable that the trend would be down over that period, but over 50 consecutive data points with no variance, all showing improvement strains credulity and is statistically impossible.) So what are we doing as leaders to create fear and make insight so illusive?
Figure 1: Avoiding the ‘Fear Hole’
Falling Down the Fear Hole
Wells Fargo had every right to want to sell more products to its customers and was wise to measure how many products each customer currently used. VA leadership was right to want to know patient wait times, just as USDA/FNS was right to want to know payment accuracy. Wanting to know something is the purpose of measurement. It is the first step toward insight. However, the next step you take makes all the difference in whether you get insights and improvement or fear and gaming the system.
he diagram above shows the two paths we can take with performance management. Both start at a neutral place with measurement. From there, the two paths go in completely opposite directions, depending upon whether your intent for measurement is learning or accountability.
Think about how our industry went down the fear hole on payment accuracy. We started wanting to know how accurate we were. But from there we quickly chose the accountability path over the learning path. Rather than getting useful feedback that could drive change in methods throughout the system data was used for judgment — to compare states in an attempt to shame poorer performers into excellence. Consequences were created — financial and scarce — therefore ensuring competition and withholding of insights. The exact opposite of what we wanted. And this is precisely what happens when we start down the accountability path. We don’t want fear. We don’t want people gaming the system. We want truth and insight and sharing. It starts innocently enough: How are we doing? How does that compare to everyone else? But it’s that next step that trips you up and sends you tumbling down the fear hole. It’s a quick descent the moment we move from wanting to know something to wanting to hold people accountable for achieving something. The moment we attach judgment and consequence, fear is multiplied by how little control people have over the variables and constraints of their system.
Gaming the system occurs any time someone is held accountable for a broken system without the power or resources to improve that system.
What You Can Do
Let me be absolutely clear: measurement is one of the most vital tools we have at our disposal. Our performance tomorrow depends on our learning today. And we can’t learn from our data if it is tainted with fear. Here’s how you can ensure learning happens:
- Go to the light. Choose the path of enlightenment. Use data and measures for clarity not judgment. You set the tone for your agency. How you use performance management will be how everyone uses it. Please use it to understand systems. Please choose the flashlight over the hammer.
- Ask why, not who. Insight comes to the curious. Be curious. Create a thirst for knowledge in the agency. But make sure the questions being asked are “why,” not “who.” Performance management is a puzzle not a “whodunit” mystery. Great leaders stoke the curiosity of their people by asking great questions and framing great puzzles. People love to solve puzzles. Puzzles allow us all to work together and contribute a piece.
- Encourage horizontal account ability. True accountability is rarely vertical. It is horizontal — shoulder to shoulder. (Please see my article in Policy and Practice [February 2016] on how to create horizontal accountability.) Top down, vertical accountability is based on fear and, at best, motivates people to do just enough to stay out of trouble. At worst (when coupled with financial consequences and limited control), it entices the rampant gaming of the system described before. We want our people using their ingenuity outward — toward their customer needs — not upward to their boss’ dashboard.
- Become data literate. While the explosion in the use of performance measures and data in the public sector has been a welcome sight, unfortunately our lack of understanding of how to use the data is also contributing to the fear-making phenomena. Not only have we weaponized performance measures by using them for judgment and consequence, few of us have been licensed on how to safely use these firearms. My colleagues and I joke that what government needs is a performance measurement safety class. Here’s a small sample of things people would have to be certified on before they were ever allowed to create a dashboard or scorecard:
- I recognize that 94 percent of problems are system problems.
- I understand that all performance varies around a mean—that what goes up must come down.
- I understand variation, common, and special cause, therefore I do not seek heroes or villains where there are none.
- I know two data points do not equal a trend.
- I understand that systems are finely tuned to give the exact result that we are getting. If I don’t like the results, then I will work to study and improve the system.
- I recognize that all targets are arbitrary.
- I will constantly seek profound knowledge.
I recognize this topic can be controversial and my viewpoint easily misunderstood. I welcome any feedback or pushback at ken@changeagents.info where I hope we can have a lengthier dialogue on how measurement and accountability can be used for good.
[fusion_separator style_type=”single solid” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” sep_color=”#606060″ top_margin=”25px” bottom_margin=”1px” border_size=”” icon=”” icon_circle=”” icon_circle_color=”” width=”45%” alignment=”left” /][fusion_text]
*Originally published as Gaming the System: How the Misuse of Data Impedes Innovation, (The American Public Human Services Association, August 2018)
