Index investing or passive investing seeks to track the return of a portion of the market. The opposite is active management, which seeks to beat the return of the markets by using market timing and individual stock selection.

Ironically, active managers do worse than the market on average.

Active management costs more than passive management. While index investors trade infrequently, active management requires more persistent buying and selling stocks in an attempt to time the markets. These additional trades add costs to the fund.

Furthermore, there are a lot of employees involved who get paid. Researchers review company financials, managers make decisions, traders implement these decisions, marketers push the products into the hands of a sales force, and the commission-based sales force takes their promised share.

Every buy or sell experiences a spread between the bid and ask price. Buying a stock pushes the share price up as you buy. Selling a stock pushes the share price down as you sell shares. The larger the fund, the more the fund’s own buys and sells pushes the market in the wrong direction.

To beat the index, fund managers need to pick the best time to buy and the best time to sell. If fund managers are not pushing the stock in the wrong direction then for every active manager who is selling a particular stock there is another active manager who must be buying that stock.

With active managers on each side of the trade and their higher than normal fees and expenses and they cannot as a group do better than index investing.

In 1991 Nobel Prize winning economist William F. Sharpe wrote “The Arithmetic of Active Management .” In that article, he demonstrated that after costs, the return of the average actively-managed dollar will be less than the return of the average passively-managed dollar.

His reasoning is simple mathematics. Actively-managed funds need to return more on average than they cost extra in fees in order to beat the return of passive management. However, on average, there are as many actively-managed funds underperforming the index as outperforming the index. As a result, on average, actively-managed funds have a lower return than passive index funds.

The idea of active management is that you should be able to anticipate movements in the market before or at least while they are moving. The idea sounds good in theory, but when put into practice, all you can say with certainty is the movements which have already happened. Our minds want to use the present tense and say “the markets are going” in a certain direction when in fact the markets have gone in a certain direction in the past and we have little or no idea of where they are heading from here.

Our own studies have shown that actively trading stocks adds to the fees and expenses without actually producing a better return and that increasing the number of holdings generally increases returns probably because of the additional smaller companies known to have both higher risk and higher return.

This principle, that the average active manager underperforms the average passive manager, is true for every possible index not just the S&P 500. Managers expending time and money trying to opportunistically pick the best Far East funds will underperform a lower cost Far East index fund. Managers trying to pick the best small cap stocks fare no better on average than a monkey throwing darts.

Active fund managers do seek to beat their respective benchmarks and there is an enormous marketing value to having a fund which has beaten its index for 3 or 5 years even if it did so by luck. When managers are not able to beat their benchmark through stock selection and market timing, they use other techniques.

Many purposefully pick an index which they can beat. Fund managers for example often have more small and mid-cap stocks than the index would suggest they should have. Or they will include more value stocks in order to do better in down markets. You can achieve the same effect simply by adding a small cap value index fund to your asset allocation.

Active managers will often have a significant cash position in the fund. This cash position allows them to do better when markets go down giving them another edge in advertising during volatile times. With index funds, your fund is fully invested, and you could intentionally keep a separate cash position in your portfolio for the same effect and avoid the higher fees and expenses.

If all else fails, fund companies simply close the funds which have underperformed the market and open new funds with a blank track record to take their place. Between 2001 and 2012, around 7 percent of mutual funds were closed each year . During that same time the number of mutual funds grew as new funds were launched to replace them.

Fund companies know that investors feel the loss from a prior high water mark much more acutely than they do the gain from a prior trough. Investors regret not being entirely in the best performing asset class more than they appreciate not being entirely in the worst performing asset class.

This emphasis on short term returns rather than long term process is one we seek to avoid. There will always be funds which have done better than even the most brilliant asset allocation over any finite time period.

Instead of active management, we recommend smart portfolio construction.

Perhaps the best way to explain the difference between active management and our methodology of portfolio constructions is that our investment philosophy is not dependent on finding the lucky fund manager who can beat the S&P 500 for the next decade.

We look at the characteristics of each sector of the markets over long periods of time and we invest in that track record. Only after deciding how much to invest in these categories do we search for a low cost method of purchasing the index fund. At no point are we entrusting reaching our goals to the ability of an up and coming fund manager to pick stocks and time the markets.

Given a dozen stellar financial planning firms, only one will have the best returns over any five or ten year period. Yet given the right methodology, every firm could help clients ensure that they have the best chance to meet their goals and secure a safe and prosperous retirement.

Only when we look backward can we see that fund managers rarely outwit bear markets and that mutual fund investors underperform the very mutual funds they are invested in because they chase returns moving out of funds after they have gone down and moving into funds after they have gone up. It is the advisor who recommends sticking to a long-term plan who, in the end, will provide the most value.

The wisdom from this analysis is to have a healthy skepticism about any claims of being able to beat the market. The Wall Street Journal had an interesting article a couple of years ago in which they asked a number of experts, “When would you recommend using active money managers over index funds? .”

My favorite answer was from Scott Adams, the creator of the “Dilbert” comic strip:

I can think of many cases in which I would recommend active money managers over index funds. For example, I might be giving the advice to someone I hate or—and this happens a lot—someone I expect to hate later. I would also recommend active money managers if I were accepting bribes to do so, if I were an active money manager myself, or if it were April Fools’ Day. And let’s also consider the possibility that I might be drunk, stupid or forced to say things at gunpoint. I’ve also heard good things about a German emotion called schadenfreude, so that could be a factor too.

No matter the marketing hype, chasing the returns of supposedly lucky active managers is not a good long term strategy. Instead, there are many index and passive funds with very low fees and expenses which can be used to craft a diversified asset allocation with appropriate risk for your situation, especially your future withdrawal rates.

Photo used here under Flickr Creative Commons.