It's no secret that I think most investors should index.  To be more precise, I'd call most “investors” savers.  And if you're treating your portfolio as if it's your savings then your financial goals are pretty simple: 1) outpace inflation and 2) reduce the risk of permanent loss.  You don't need to “beat the market” or just maximize returns.  The best way to achieve these two goals is to implement a diversified, low fee and tax efficient portfolio.

Given all that, indexing is the obvious way to achieve this given its inherent diversification, low fees and tax efficiencies.  Of course, there are lots of ways to index and I personally prefer a countercyclical indexing approach (as opposed to a more traditional procyclical indexing approach), but that's not what this is about.  This post is just highlighting a nice new paper that was released yesterday further discussing why indexing works:

“We develop a simple stock selection model to explain why active equity managers tend to underperform a benchmark index. We motivate our model with the empirical observation that the best performing stocks in a broad market index perform much better than the other stocks in the index. While randomly selecting a subset of securities from the index increases the chance of outperforming the index, it also increases the chance of underperforming the index, with the frequency of underperformance being larger than the frequency of overperformance. The relative likelihood of underperformance by investors choosing active management likely is much more important than the loss to those same investors of the higher fees for active management relative to passive index investing. Thus, the stakes for finding the best active managers may be larger than previously assumed.” [a nice new paper ]

This is consistent with something I posted not too long ago.  One of the problems with stock picking is that the gains tend to be highly skewed.  Your top performers produce most of the returns.  The distribution is very uneven.  So, it's not like you're just trying to pick the stocks that outperform the average.  In order to create consistent market beating returns you basically have to know which stocks will be in the 20% of the outliers.  Add on taxes and fees and you're climbing a huge uphill battle.