The other night, my 11-year-old asked if we could move to Chicago. Apparently news of the teacher strike had hit his sixth-grade class and he liked the idea of getting a couple days off from school.
I asked him if he knew why the teachers were on strike and if he agreed with them. He assumed they wanted more money, which is what we all assume when we hear of a strike. But he was shocked to learn it was partly about student testing and that some people wanted the teachers to be graded based on the grades the students received. In no time, my son reached the same conclusion that Ken had reached last year: “Why wouldn’t they give all the kids A’s and everyone would think they were great teachers?” Bingo. (He has his mom’s smarts.)
I don’t want to reiterate the dangers of tying success measures to a person’s individual performance, but I do want to touch on the overwhelming fear that, as an employee, we are going to be held “accountable” for something over which we have very little influence.
For teachers, we can assume they carry the larger load on educating our kids. However, the sixth-grade teacher is dealing with students who have already had at least five other teachers that laid the foundation for their studies. If that foundation isn’t solid, why should they be held accountable? Of course, while teachers have our kids for seven hours a day, parents have them a lifetime, and they play a vital role in teaching their children how to learn and to assure they are actively engaged in their learning experience. Teachers have absolutely no influence over the parents, so why should they be held accountable for failures at home? And then there’s the individual student herself. If she’s not willing to learn, there’s only so much a teacher can do. A obstinate child’s choosing not to learn should not be blamed on the teacher alone.
Those are three huge factors that influence learning, all out of the individual teacher’s control. Let’s add a few more: larger class sizes, lack of technology, reduction in special need programs, the new math, and Beiber fever. All impacting education, and all outside the teachers’ control.
I’d be scared too. Enough to strike? Maybe. It’s pretty easy to see why the Chicago teachers felt they were being set up for failure. Luckily, that pushback is only happening in education. So far.
Several years ago, while the agency I worked for was putting together an organizational strategic plan, we began discussing success measures and specifically what we could count to tell us how we were doing. Every path led to the same thing: “We don’t control that, so we shouldn’t measure it.”
It was frustrating. I remember explaining that we don’t control the stock market, or the weather, or the tide of the oceans, but we measure and track these things so we can make predictions and learn about patterns. My pleas felt on deaf ears. Our leadership team refused to measure things that they could not control, at least to some degree of certainty.
So we ended up counting the same stuff that everybody else counts: performance-based metrics, and specifically, those performance-based measures they were comfortable that we could report positive numbers.
In part, they feared the same thing the Chicago teachers feared: that they would be held “accountable” to any negative reports. Why carve the paddle that’s going to be used to spank you?
While fear was part of their motivation, fear is only the tip of the iceberg when it comes to the challenges with success measures in government. There are four problems we face whenever we try to measure performance:
First, each employee is only one of 10,000 factors that influence success. Just as teachers cannot be expected to influence every aspect of a child’s learning, government employees in, say, an environmental agency cannot be expected to control every factor that influences clean air and streams. Rising pollution does not always correlate to the performance of your department of natural resources.
Second, many of the programs and policy changes we implement will not have an immediate impact on our measures. In fact, it could be decades before we see if these ideas accomplished what we hope. Educational reform is a perfect example: How do we know if it worked until an entire generation of children graduates and moves on to careers and college?
The third factor is that we cannot measure what does not happen. How do we measure how many newborns were not born malnourished thanks to our programs? Or how many crimes were not committed? Or how much pollution was avoided? Most of our attempts to develop these types of measures take us on a short trolly ride to King Friday’s castle: They’re make-believe. You just can’t count what doesn’t happen.
The final factor ties back to the fear. It’s the thought that because individual performance is certainly the main indicator of program success, the inverse must also be true. If a program is not successful, it must be because a lack of individual performance. I would argue that individual performance has almost zero influence on program success. A really bad egg can kill a great idea, but the bad eggs are few and far between. When programs fail from internal influences, it’s almost always because of bad pipes, not bad people. After all, we can only work as well as the pipes will let us.
So where does that leave us?
Well, in Chicago, it led to a strike. But even in more cool-headed discussions, it mostly just leaves us avoiding the discussion about how to really measure the “profit” of government.
To me, it’s a matter of the chicken and the egg. What is going to come first, a universal understanding that outcome measures do not tell the whole story, but the story they tell is important to learn from? Or, the universal agreement that measures should never be used for “accountability and transparency” in government?
Since neither seems on the horizon, you can try what we did for years back at my agency: Secretly, in the old storage closet in a hidden corner of the office, meet to review and discuss your outcome measures. Then destroy all the evidence. Share the safe and politically washed strategic plan with whoever asks for it, and safeguard the good stuff. After all, test scores are an indicator of student success, and while we might not want to hold the individual teacher accountable, we may still want to look at how we’re teaching kids and talk about why it might not be working anymore.
[fusion_separator style_type=”single solid” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” sep_color=”#606060″ top_margin=”25px” bottom_margin=”1px” border_size=”” icon=”” icon_circle=”” icon_circle_color=”” width=”45%” alignment=”left” /][fusion_text]
*Originally published as Fear and Loathing in Chicago (Public Great @ Governing.com, September 20, 2012)
