In his 2008 annual letter for Berkshire Hathaway, Warren Buffett writes about the dangers of back-tested, historical results and formula-based investing:
“The type of fallacy involved in projecting loss experience from a universe of non-insured bonds onto a deceptively-similar universe in which many bonds are insured pops up in other areas of finance. ‘Back-tested’ models of many kinds are susceptible to this sort of error. Nevertheless, they are frequently touted in financial markets as guides to future action. (If merely looking up past financial data would tell you what the future holds, the Forbes 400 would consist of librarians.)
Indeed, the stupefying losses in mortgage-related securities came in large part because of flawed, history-based models used by salesmen, rating agencies and investors. These parties looked at loss experience over periods when home prices rose only moderately and speculation in houses was negligible. They then made this experience a yardstick for evaluating future losses. They blissfully ignored the fact that house prices had recently skyrocketed, loan practices had deteriorated and many buyers had opted for houses they couldn’t afford. In short, universe ‘past’ and universe ‘current’ had very different characteristics. But lenders, government and media largely failed to recognize this all-important fact.
Investors should be skeptical of history-based models. Constructed by a nerdy-sounding priesthood using esoteric terms such as beta, gamma, sigma and the like, these models tend to look impressive. Too often, though, investors forget to examine the assumptions behind the symbols. Our advice: Beware of geeks bearing formulas.”
The larger point is that the past universe for a particular investment, whether it’s a bond, stock, or some other type of financial instrument, may not be the current universe. The present situation may be fundamentally different than what’s generally prevailed in the past. If you invest in a way that doesn’t recognize or factor in this type of fundamental change, then your results may suffer.
If, for example, you don’t feel comfortable about a particular retail company’s ability to survive in today’s competitive landscape, then it probably shouldn’t be in your portfolio regardless of whether it looks quantitatively cheap. Prospects like these should probably go in what Buffett calls the “too hard” pile.
It seems like many value investors are buying groups of stocks that appear quantitatively undervalued but really should be avoided because the past universe is much different than the current universe. This may explain — at least in part — why these same investors have often underperformed the FAANG companies disrupting so many traditional businesses and industries. If the competitive landscape of a traditional business/industry has fundamentally changed, then past financial results may not matter much, making an apparent quantitative bargain illusory. In this situation value metrics drawn from past data may be misleading and unreliable, leaving the investor unable to intelligently predict or estimate whether: (1) an investment prospect will survive, and (2) what the prospect’s normalized earning power will be going forward. Funds invested in the face of these problems probably amount to speculation and guesswork rather than businesslike, Graham-style investment.
The author is not an investment advisor or CFA and readers should consult an investment advisor before buying or selling any publicly traded stock. The views expressed in this article are the author's personal opinions and should not be construed as investment advice.