Stupid Metrics Drive Out No Metrics At All

Developing scorekeeping metrics is a critically important yet undervalued role of the chief financial officer. CFOs ignore this role at their and their organizations’ peril, because if they don’t set the scorekeeping metrics, others will, and will make a mess of it.

All organizations crave metrics. We’re human, and that means we need to compare how we’re doing against our competitors, our peers, our commitments, or simply what we were hoping for. There are lots of possible metrics – business examples include revenues, profits, and dollar compensation. Metrics based on ratios make scoring less dependent on size and more memorable to the audience – e.g., margins, EPS, growth rates, percentage variances, market share, percent of new business, percentage raise. The art of all this lies in coming up with a small but meaningful number of the metrics that make the most sense for each organization.

Two recent stories in The Washington Post bring all this to mind. One reports that JPMorgan Chase’s 2013 proxy statement shows that CEO Jamie Dimon got either a huge pay cut in 2013 or a huge raise, depending on whether you look at the SEC-mandated disclosures (pay cut – see p. 47 of the proxy) or JPMC’s voluntary detailed disclosure (raise – p. 34). The other reports that the American Statistical Association has harshly criticized the “value-added method” (VAM), a highly quantitative method for evaluating teachers that depends heavily on standardized test score results. While VAM is gaining increasing traction in the U.S., it is extremely controversial.

In my opinion, both the SEC-mandated compensation disclosures and the VAM methodology are not only deeply flawed, but so complex that it’s hard to understand either the presentation of the results or the underlying methodology. Now, it’s easy to blame the regulators and politicians for both messes, but they were simply filling a void.  That void was created by the decades-long reluctance – if not downright refusal – of U.S. public companies and the teaching profession, respectively, to comply with the public’s request for fuller self-evaluation. In other words, “If you won’t tell us how you’d like to be scored, we’ll decide for you.”

For any enterprise, scorekeeping takes on many dimensions – investor valuation, general performance assessment, sales compensation, employee performance reviews, just to name a few. But regardless of the reason for the scorekeeping, the CFO is the right person to take responsibility for the task, because he or she is the one most likely to:

  • Be a neutral third party to all of the line functions in the enterprise,
  • Know what data is available how to retrieve it, and
  • Have the necessary mathematical and analytical skills.

And if the CFO doesn’t determine the scorekeeping practices, someone else will serve your enterprise a fried telephone book.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in Compensation, GAAP & Reporting, In the Media | Tagged , , , , | Leave a comment

The Death of “Whom,” and the Labor Market for Words

Enough with numbers and March Madness and Flight 370; let’s talk about words instead. John Merrow of Learning Matters, the Education Correspondent for PBS NewsHour, has written a nice obituary for the word “Whom.” He described Whom’s declining years, while its distant cousin, Who, survives. I take the whole sad story as an allegory for how labor markets work.

As a passing acquaintance of Whom’s, I, like John, mourn Whom’s passing. Unfortunately for Whom, he (she?) has met the same fate as railroad engine firemen, when steam locomotives got replaced by diesel and electric locomotives. The firemen fought valiantly, or at least violently, to keep their jobs, but struggling against irrelevance is a losing cause. It turned out that Who could do Whom’s job as well, so who needs Whom?

In contrast, As has more successfully navigated its way through the labor market of words by holding down several jobs. You may be old enough to remember when Like, that ambitious sycophant, sucked up to the American people with, “Winston tastes good LIKE a cigarette should.” Well, As survived by falling back on its many other jobs in the English language. Perhaps Whom is a candidate for a job retraining program.

Words that serve a purpose should stay.  For example, even though “hopefully” as it is now commonly used is (was?) incorrect English, that new usage filled a need that no other single word addressed.  On the other hand, the increasingly common use of the verb “impact” to mean “affect” sets my teeth on edge.  Not only does that add nothing to the English language, lost in the shuffle is the former principal use of the verb “impact” as a useful and colorful word, as in, “When Abigail Witherspoon saw her husband enjoying an intimate lunch at Per Se with Beatrice Coleridge, the excrement certainly impacted the ventilating device.”

I hope that Whom is now in a better place. RIP.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in In the Media, Ordinary Life | Tagged | Leave a comment

Flight 370, and Why We Produce Information

Finance professionals produce information. It’s what we do, so let’s think carefully about why and how we produce that information. In that respect, CNN’s coverage of Flight 370 is instructive.

I’ve recently written some critical, sarcastic things about CNN’s coverage. As one interested in probability and statistics, I’ve marveled at the fact that, in checking in with CNN at random a few times each day, 100% of the time – yes, 100%! – they were covering Flight 370. You could infer that CNN is not covering anything else; that’s disgraceful, considering what else is going on in the world today. A friend has suggested that it’s cruel for me to be so flip about a real tragedy to the people who have loved ones on that plane. Au contraire, it’s CNN that being cruel, with endlessly repetitive, uninformative coverage of a situation that appears increasingly hopeless.

Now, we finance professionals are deluged with pundits’ advice and with software products – business intelligence, big data, and increasingly elaborate dashboards are examples – suggesting that in this increasingly fast-paced world it’s essential that we deliver more and more information, faster and faster.

Yes, delivering information sooner is better than later, but not at the expense of accuracy. And more information is better than less, but not at the expense of obscuring the important messages. Lastly, the value of any information – with “value” for financial information measured by more efficient operations and better decisions – should be greater than its cost to generate. By comparison, CNN’s coverage of Flight 370 has always chosen “more” and faster,” regardless of accuracy and meaningfulness. They’re sure spending a lot of money on it, but then again, their “value” depends on eyeballs.

So be careful out there – you don’t want to become the CNN of enterprise management.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in In the Media, Quantation Philosophy | Tagged | Leave a comment

L’Affaire Madoff: A Black Mark on the Auditing Profession

[This piece appeared originally on, an excellent website for senior finance professionals. It’s a little technical, and pretty long as my posts go, but very spicy – I take a novel perspective on what really went wrong in the Madoff fiasco. If it’s not your cup of tea, feel free to scroll down to the next post. Thanks for your interest.]

There is a theory of tort law that liability should be apportioned based on who could most efficiently have assured that a tortious event did not take place. If that standard were applied to the Bernie Madoff fiasco, the auditing profession would owe a whole lot of money to a whole lot of people. It’s a shame that the thoroughness, doggedness, and precision the profession is known for wasn’t applied to its own practitioners.

My nostalgia for this whole kerfuffle arises because after a six-month trial, a jury has just returned guilty verdicts against five of Madoff’s former employees. To understand why this event has raised unpleasant memories and unresolved concerns, let’s have a short Socratic dialogue:

Q: What was the Madoff fiasco?

A: Bernie Madoff ran a Ponzi scheme, for 20 years or more, in which investors lost billions when it finally unraveled.

Q: Why did people invest with him?

A: Because he reported astounding returns.

Q: Didn’t they know that if something seemed too good to be true, it probably was?

A: Normally yes, but he had audited financial statements proving his results.

Q: Really?

A: OK, actually it was just an annual letter saying that an audit had been performed.
(We know this because David Friehling of Friehling & Horowitz, the accounting firm that signed the audit opinion letters from 1991 on, testified in 2009 that his firm never actually performed any audits.)

Q: But public accounting firms have a peer review process. Wouldn’t that process have uncovered the fact that audits were never performed, and that the whole thing was a fraud?

A: Yes. The process of confirming broker statements with the brokerage firms would have brought the whole house of cards down instantly.

Q: OK, so what happened with the peer reviews?

A: They were never performed – although Friehling & Horowitz nominally participated in the peer review process, they stated annually in writing that they were not subject to peer review because they had performed no audits.
(In retrospect, I suppose those statements were factually correct. J )

Q: But, but, but, loads of people – including the SEC and many journalists – saw Madoff’s “audited” financial statements. Didn’t the auditing profession get any sense that there was an inconsistency here?

A: The auditing profession did not appear to have any controls in place whereby it could compare audit opinion letters against its database of firms that acknowledged their obligation to have peer reviews done.
(Mind you, I am not a CPA and I have not made finding this out my life’s work. Perhaps my Googling skills need to be improved.)

Q: In the wake of all this, have there been reforms?

A: Perhaps. My search has not uncovered anything from the AICPA, but a number of state CPA societies, including California and – stop the presses! – New York, have taken further steps toward “mandatory” peer reviews. That is a good thing, but what I have read does not make it clear whether these steps will ensure that any financial statements accompanied by an auditor’s opinion may become the subject of a peer review.

Q: How has the judicial system punished David Friehling?

A: No action so far. In 2009, he pleaded guilty to nine counts of fraud and other crimes, but his sentence has continually been deferred pending the resolution of other prosecutions he is helping with, including the trial of the five former Madoff employees that just ended.
(I might add that the same uncertain fate awaits Frank DiPascali, Madoff’s self-characterized CFO. DiPascali was the star witness in the trial just ended.)

As a final aside, I find it absolutely shocking that this glaring and gigantic hole in how effectively stakeholders are protected from fraudulent financial statements has received so little coverage from the mainstream press. Criticism has been directed at many constituencies – Wall Street, the SEC, a greed-driven culture, careless investors who should have known better, George Bush, and global warming, just to name a few – but the whole sorry mess wouldn’t have lasted through more than one or two audit cycles if the audit work had been reviewed.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in GAAP & Reporting, In the Media | Tagged , , | Leave a comment

Are You an Ethical Presenter? Oh, Really? (#3)

Again, we look at how the way numbers are presented can affect how the presenter’s integrity and ethics are perceived. But this time, we look at what’s omitted. Our subject is again Constant Contact’s (Nasdaq: CTCT) investor presentation (click here for the original document).

In several places in the presentation, CTCT alludes to its past and potential profitability. Here’s an example:

CTCT presents “Adjusted EBITDA” and “Free Cash Flow” in several places in the presentation, but nowhere do they present Operating Income, Net Income, EPS or any other GAAP-standard profitability metrics. They do include the following footnote:

“Adjusted EBITDA margin is a non-GAAP financial measure; a reconciliation to the nearest GAAP financial measure can be found on”

Put another way, “If you want to see the corresponding GAAP numbers, you’ll have to work for it.”

Both EBITDA and Free Cash Flow are metrics that look “better” than their closest GAAP equivalents because they are calculated by adding back certain expenses and expenditures. For that reason, even if CTCT asserts that these are the most appropriate and valid metrics – a highly arguable assertion, but let’s leave that aside for now – presenting those metrics to the complete exclusion of their GAAP equivalents is certain to raise questions in the minds of some in the audience – questions like, “Why don’t they want us to look at the GAAP numbers?” “Are they hiding something?”

Again I ask: Is the potential benefit of spinning your information worth the potential impact on how you are perceived as a presenter?  Only this time, the spinning doesn’t lie in what you do present, but in what you don’t present.

As we leave this subject, let’s last consider the frequency of these perhaps unfortunate presentation choices. One such choice is probably innocent or at least not worth focusing on, but three (click here and here for my other posts about CTCT’s presentation) is more likely to make us question the presenter’s intentions.

[We’ll also leave aside for now the facts that “Adjusted” EBITDA (%) isn’t defined and that they don’t state what it’s a percentage of, as well as the question of whether showing these two metrics on the same graph, with two vertical axes, is a valid and appropriate way to present this information. Perhaps some other time. . .]

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in GAAP & Reporting, Quantation Philosophy | Tagged , | Leave a comment

Flight 370, CNN, and My Limits of Endurance

I’m in the habit of tuning in to CNN fairly regularly for a few minutes just to see what’s going on in the world, especially since CNN is on the satellite radio that’s in my car. But within a few days after Malaysia Airlines flight 370 disappeared, I got tired of CNN newscasters breathlessly announcing, “And this just in: Flight 370 is still missing!!!!”

So I eased up on checking in with CNN for my news, and I’m down to once or twice a day. And yet, even though it’s now two weeks since MH370’s disappearance, in the last few days not once has CNN’s subject been anything but MH370.  What are the odds of that, if CNN is actually covering other stories as well? Is CNN cutting way back on its coverage of everything else? Are they trying to save money?

Come on, CNN. Yes, the disappearance of MH370 is a tragedy and a sensational mystery, but life goes on.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in In the Media | Tagged , , | Leave a comment

Are You an Ethical Presenter? Oh, Really? (#2)

Yes, we’re still looking at Constant Contact’s (Nasdaq: CTCT) investor presentation (click here for the original document), and the topic is still how tiny changes in the way we present numbers can have a significant impact on how our integrity and our ethics may be perceived.

The example for this post is the following pair of graphs, presented side-by-side on the same slide, showing paying customer additions. Note the main takeaways CTCT wants you to have, in the titles at the top of each graph. The color coding in the left-hand graph appears to be intended to enable you to make year-over-year comparisons of the four quarters of 2013 versus 2012:

Now look more closely. The left-hand graph does not show customer additions in each quarter – every value in that graph is 35,000 or higher, and every quarter-to-quarter change in total number of paying customers shown in the right-hand graph appears to be less than 25,000. Perhaps the left-hand graph is some measure of year-over-year change, but even if that’s the case, the left-hand and right-hand graphs do not tie to each other in any way that I have been able to figure out. (Perhaps labeling the axes might have helped.)

Also, note the comment below the left-hand graph: “Customer counts rounded to the nearest 5,000.” I can’t think of a valid, honest reason why CTCT would choose to round a bunch of numbers – none higher than about 50,000 – to the nearest 5,000, when it’s obvious that they have the exact numbers of new customers at their fingertips.

Lastly, note the heading above the left-hand graph – “Four consecutive quarters of year-over-year growth. . .” This statement could be the equivalent of saying, “One consecutive year of year-over-year growth.” That just doesn’t seem impressive. Why are they bothering?

Here’s the overall problem: Rather than simply giving investors the raw data – and letting the investors calculate their own derivative metrics – CTCT has chosen to cherry-pick its metrics, present the information imprecisely (i.e., rounded on the left and hard to read accurately on the right) and in a way that the reader can’t get back to the raw numbers, and put a hard-to-validate spin on the results. Even if CTCT’s reasons are completely innocent, their approach is guaranteed to provoke suspicion in the minds of some.

Again I ask: Is the potential benefit of spinning your information worth the potential impact on how you are perceived as a presenter?

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.

Posted in GAAP & Reporting, Quantation Philosophy | Tagged , , | Leave a comment

Happy Pi Day 2014!

Today – March 14, or 3.14 – is Pi Day, a day celebrated by mathematics aficionados the world over. I shudder to think of the partying that will break loose in two years, when the date is 3.14.16. Hide the women and children!

No blog about presenting numbers would be complete without a brief Pi Day observance and, at least in my case, taking a few shots at that nemesis of sound quantation practitioners everywhere: the pie chart. Rather than going on at length about the evils of pie charts, I will simply refer you to a terrific post I just read, by Xan Gregg.  Not only does he remind us of what’s wrong with pie charts, he actually makes suggestions about how to fix the problem at sites like Wikipedia. Read it. And if you’re really interested, follow the #OneLessPie hashtag on Twitter.

From my perspective, using a pie chart is the sort of presentation mistake that will cause some in your audience to question your intelligence or your competence. Sort of like not knowing when you should put an apostrophe in the word “its” (uhh, “it’s”), or consistently mispronouncing the names of people you’re speaking with.

Lastly, in the interest of fairness to pie charts, I’ll point you to a couple of my posts on the subject:

  1. The one situation where a pie chart actually is cognitively appropriate, titled “The Big Kahuna of Fiscal Issues
  2. The most accurate pie chart in history.  Honest.

Happy Pi Day!

“Painting with Numbers” is my effort to get people to focus on making numbers understandable.  I welcome your feedback and your favorite examples.  Follow me on twitter at @RandallBolten.


Posted in Policy & Politics, Quantation Philosophy | Tagged , , , | Leave a comment

Are You an Ethical Presenter? Oh, Really? (#1)

In recent posts we’ve seen how tiny changes in the way we present numbers can have a huge impact on how well the information is understood. In this post, we look instead at how those little things can affect how your integrity or your ethics might be perceived.

For our example, consider a slide from an investor presentation I found online (click here for the original document). Just for the record, the company (Constant Contact – Nasdaq: CTCT) looks at first glance like a perfectly fine company; I’m only commenting on how they present some of their information. Here’s slide #13, showing the trend in Average Revenue Per User (ARPU):

A quick glance at this graph suggests impressive growth – the height of the rightmost bar appears to be nearly triple that of the leftmost bar. But that appearance was created by truncating the vertical axis scale, so that $34.00 is at the bottom. Here’s what the graph would look like if the vertical axis scale began at $0.00:

Not as visually impressive, is it? Raising the baseline of a graph is one of the oldest quantation tricks in the book; it features prominently in How to Lie with Statistics, the 1954 classic by Darrell Huff. So the question becomes: Was CTCT just making a naïve attempt to produce a more visually pleasing graph, or were they trying to make a trend look better than it really is? As far the audience is concerned:

  • The most sophisticated consumers of quantitative information – that is, the ones whose good opinion is probably most critical – are most likely to draw the harshest conclusions about presenter intentions, and
  • Those forming harsh conclusions are unlikely to share them with the presenter.

The sad thing is that the $5.10, or 14%, improvement in ARPU over the time span shown really does seem impressive to me. CTCT asserts that ARPU is only one of several metrics affecting lifetime customer value, and the effect of each metric is multiplicative with the others. So if they all improved by 14%, that’s a big bump in total lifetime value. Moreover, the graph shows quarterly changes; the apparently modest growth rate of 1.1% per time period shown translates into 4.4% per year.

The tactic of truncating a graph’s vertical axis is one we see all the time. But as you consider it, ask yourself: Is the potential benefit to how the content is interpreted worth the potential impact on how you are perceived as a presenter?

Stay tuned – we’re not quite finished with the CTCT investor presentation.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable. I welcome your feedback and your favorite examples. Follow me on twitter at @RandallBolten.

Posted in GAAP & Reporting | Tagged , , | Leave a comment

And the Oscar for Clarity Goes to. . . the FDA!

We’ve recently run posts about bad quantation in the media, about Olympic medal counts and Oscar betting odds. This time we discuss good quantation – the Food & Drug Administration’s proposed changes to the “Nutrition Facts” label. This is a great example of how small changes in the way we present numbers can have a significant impact. Though small, the changes are sensible and meaningful, and therefore likely to be highly controversial in some quarters.

Current Label ……………………..Proposed Label

The red circles identify the changes I consider most significant with respect to quantation, roughly in priority order:

  1. Servings per container is shown in larger, boldface type, to make it absolutely clear what is meant by a “serving.” (Although not shown here, the FDA is also proposing to get much more realistic about what a single serving really is. No longer will a  muffin be considered two servings just because it’s huge, nor a 20-ounce soda be considered two-and-a-half servings – after all, who has that kind of willpower?
  2. Calorie count gets similar emphasis, in recognition that obesity is the largest diet problem we have, and that caloric intake is the most significant contributor to that problem.
  3. Calories from fat is no longer presented. Calories is calories is calories – regardless of the source. And anyway, current nutrition theory says there are “good” fats.
  4. % of DV (Daily Value) now has a more prominent position, to the left of the item in question. To almost all of us, how much of the Daily Value we’re consuming is much more relevant and meaningful than the total number of grams or milligrams – quantities that only a scientist can appreciate.

What’s interesting about all this is how small, seemingly unimportant changes to the way numbers are presented – font size, position on the page, adding ratios to the raw numbers – can make such a big difference in what the information means to the audience. Consider that, the next time you ponder how effective and meaningful your company’s financial statements are.

(For the FDA’s own press release on this subject, click here.)

“Painting with Numbers” is my effort to get people talking about financial statements and other numbers in ways that we can all understand.  I welcome your interest and your feedback.



Posted in Ordinary Life, Policy & Politics | Tagged , , | Leave a comment