Lies, Damn Lies, And Financial Statistics

It’s hard to beat the market, but there’s always a new product that tries anyway

- Peter Coy

Early in January in a Chicago hotel, Campbell Harvey gave a rip-snorting presidential address to the American Finance Association, the world’s leading society for research on financial economics. To get published in journals, he said, there’s a powerful temptation to torture the data until it confesses—that is, to conduct round after round of tests in search of a finding that can be claimed to be statistically significant. Said Harvey, a professor at Duke University’s Fuqua School of Business: “Unfortunately, our standard testing methods are often ill-equipped to answer the questions that we pose.” He exhorted the group: “We are not salespeople. We are scientists!”

The problems Harvey identified in academia are as bad or worse in the investing world. Mass-market products such as exchange-traded funds are being concocted using the same flawed statistical techniques you find in scholarly journals. Most of the empirical research in finance is likely false, Harvey wrote in a paper with a Duke colleague, Yan Liu, in 2014. “This implies that half the financial products (promising outperformance) that companies are selling to clients are false.”

Most of us have a vague sense that we’re being ripped off by investment firms that charge hefty fees while producing results that are no better than you’d get throwing darts at a page of stock listings. It’s troubling nonetheless to find out we’re correct. And it’s important to understand the mechanics of what has gone wrong.