Most Metrics Are RELATIVE, Not Absolute
May 5, 2014
What are metrics for? When properly used, most are nothing more than a relative tool for comparing one time period – or company, or country, or person – to another, or actual results to planned or expected results. They are not an absolute score, meaningful on its own. It’s important to remember this, whether the metric involves business, government, sports, or national economic performance.
I saw yet another article about a change in the U.S. unemployment rate (downward this time) with the same tired old qualifying observation that the change doesn’t factor in people who have given up and chosen to stop looking for work, thereby understating the true unemployment rate by reducing the number of verifiably, measurably unemployed.
To this news, my reaction is, “So what?” Let’s call this phenomenon “the flaw,” and ask ourselves:
Is the “flaw” a truly material number? (A home exercise: compute how many unemployed have to leave the labor force to move the unemployment rate from 7% to 6%.) Is the “flaw” any more or less pronounced than it was the last time the rate went up (or down) by a similar amount? Are the causes of the “flaw” uniformly negative – i.e., people leaving the workforce specifically because they’ve given up – or are there also positive explanations? For example, has someone else in the household who was unemployed now found a job? Have some people found “off the books” employment? And most important, is the “flaw” so severe that that you can’t really tell whether the problem of unemployment is getting better or worse? If so, then the metric should be thrown out and replaced with a better one. If not, the hand-wringing is pointless.
Many, many metrics have similar flaws, but that doesn’t mean that they aren’t immensely useful even so.
“Painting with Numbers” is my effort to get people to focus on making numbers understandable. I welcome your feedback and your favorite examples. Follow me on twitter at @RandallBolten.
I saw yet another article about a change in the U.S. unemployment rate (downward this time) with the same tired old qualifying observation that the change doesn’t factor in people who have given up and chosen to stop looking for work, thereby understating the true unemployment rate by reducing the number of verifiably, measurably unemployed.
To this news, my reaction is, “So what?” Let’s call this phenomenon “the flaw,” and ask ourselves:
Is the “flaw” a truly material number? (A home exercise: compute how many unemployed have to leave the labor force to move the unemployment rate from 7% to 6%.) Is the “flaw” any more or less pronounced than it was the last time the rate went up (or down) by a similar amount? Are the causes of the “flaw” uniformly negative – i.e., people leaving the workforce specifically because they’ve given up – or are there also positive explanations? For example, has someone else in the household who was unemployed now found a job? Have some people found “off the books” employment? And most important, is the “flaw” so severe that that you can’t really tell whether the problem of unemployment is getting better or worse? If so, then the metric should be thrown out and replaced with a better one. If not, the hand-wringing is pointless.
Many, many metrics have similar flaws, but that doesn’t mean that they aren’t immensely useful even so.
“Painting with Numbers” is my effort to get people to focus on making numbers understandable. I welcome your feedback and your favorite examples. Follow me on twitter at @RandallBolten.
Related Blogs
Other Topics
Other Topics