For a while there, I thought Joe Gerstandt was full of crap.
Somebody tweeted a line from one of Joe’s blog posts: “We are not accountants. We are Jedi. We play on a completely different field.”
“Jedi” alone is often enough to make me go find something else to do, but instead I read the full post, The False Tyranny of Metrics. And for a while, I continued to think Gerstandt was full of crap. Or maybe just way out there, because my initial skimming said that he was saying metrics don’t matter.
That wasn’t the case. It took me longer than I like to admit to realize that the Talent Anarchy blog is (at least in part) a dialog between Gerstandt and his business partner, Jason Lauritsen. So this post was part of their thinking out loud about what matters.
The heart of what Gerstandt is talking about emerges in a follow-up post (and at the end of my post, I’ve put links to several posts from Talent Anarchy):
And maybe I do not think that measurement is evil…measurement is a tool after all, so it boils down to how you use it. But this is what I do believe:
- one: We over-prioritize things that come with metrics.
- two: We have told ourselves some great lies about what we can measure.
- three: The outcome of our use of metrics is often evil.
The conversation really struck me because of several themes or issues running through my life right now. One of them is a client I’ll call Hephaestus. I’ll say they make household fans and heaters. As a manufacturer, Hephaestus has some serious metrics having to do with production–rate, quality, reject rate, cost, all the sorts of things you’d expect. And the sorts of things that make sense there.
Does Hephaestus have other ways of knowing how they’re doing? I’m pretty sure they do, though the project I’m dealing with doesn’t extend that far. I haven’t been called in by the CEO or the VP of manufacturing. Even so, I see potential wisdom for me and for my client in the Talent Anarchy discussion.
Our project is about how to bring new manufacturing workers to competency. If you’ve worked in a plant, you have some idea what these jobs can be like. At a GE appliance factory, I observed workers in charge of powder-paint application, wire-harness installation, and similar jobs.
How do you help a new person do that safely and accurately–and with acceptable progress to the necessary speed?
It’s not all feeds and speeds, either with regard to turning out those appliances, or with regard to how people learn. I think there’s a lot of value in questioning assumptions, especially those we don’t even recognize as assumptions.
Here are links to the posts in the discussion at Talent Anarchy, along with a quote pulled from each. Worth the time to go through. You’re likely to find value in the comments as well:
The Measurement Imperative (Jason)
(the post in which Jason starts the discussion)
I know that measurement and metrics aren’t your favorite thing to talk about, but what do you think? Where does measurement fit into the work we do?
The False Tyranny of Metrics (Joe)
(from a comment on this post)
I was talking with my boss about the situation [of half the staff at a health facility frequently arriving late] when he asked me if my team cared about the people we served and if they were dedicated to helping those folks achieve outcomes. I answered yes – they excelled at achieving outcomes. He then challenged me by pointing out that the only reason I was on my time-clock tirade was because I could hit a button on the computer and spit out the metrics related to the situation. Punch reports were the metrics I had available so that was what I managed to.
Despite What You May Have Heard, Measurement Isn’t Evil (Jason)
What I heard you say is that putting metrics and measurement before the actual work, or worse, substituting it as the work is really damaging and counter-productive. And I would agree with that. When the metric becomes what you are trying to accomplish, you have lost.
More Metrics Madness (Joe)
I do understand the importance of profit. I am a business owner myself…I get it. But the purpose of my business is not profit. I work, at least partly, because I need to make a living, but I do the particular work that I do for reasons that have nothing to do with profit. Profit is mandatory, I am not in any way confused about that, but saying that an organizations exists for the purpose of profit is kind of like saying that the purpose of a persons life is breathing (which also can be measured quite well by the way).
A Defense (of a sort) of Metrics
(a guest post by Mark D. Hirschfeld and F. Leigh Branham
We may not be able to measure honesty, compassion, and courage, but we can measure the results that those traits produce–lower voluntary turnover, lower quit rates, fewer grievances filed, more internal job progressions allowed, more customers returning more frequently and referring their friends, more managers coaching (often confronting), recognizing (more often) and giving constructive feedback, more new employees being hired through referrals from happier, more engaged employees–all measures of not just more, but of better places to work that do indeed serve as measures of progress toward becoming a remarkable workplace.
CC-licensed production parts image by iamphejom.
The exchange makes me think of Goodhart’s law.
HES, I apologize for slowness in approving your comment. Mostly I wanted to understand Goodhart’s law well enough to briefly summarize it here (for people who don’t click your link to the Wikipedia article).
I found this piece by Tim Worstall at AdamSmith.org. As he says, Goodhart’s law began in economics, but you can generalize it. The sense is that whenever you use one thing as a “proxy” to measure something you want, people inevitably start gaming the system. Worstall uses the example of auto traffic in a store parking lot as an indicator of customer traffic. Not a bad idea, unless somebody starts driving cars in and out all the time to pump up the numbers.
I’m not persuaded by Worstall’s concluding that “setting targets doesn’t work.” In an enterprise, if you don’t have targets, what do you use to determine if you’re on track or not? It’s more the case, I think, that you have to re-examine your targets and what they’re telling you. It’s not easy, but that doesn’t mean you shouldn’t do it.
Not A problem. I will be honest and say I have no idea about business. I just found the parallels interesting. :)
In what I think is the sense you referred to Goodhart’s law, I often refer to Gresham’s. “Bad money drives out good” is the quick version. My understanding is that he was speaking of debased currency (cheap metal substituted for silver or gold), and by extension inflated currency (people will try to get rid of their Weimar Republic marks and hang onto dollars or Swiss francs).
As an analogy, I’ll say Gresham applies to training. Take a given format or delivery system (like, say, videotape in an earlier time, or streaming video now) that seems to be less expensive than its alternative (having training sessions with actual instructors). While you can learn from video, there’s no guarantee–but the low cost of streaming a videotaped talking head will often shut out the possibility of more effective but more expensive training.