For example, in the study Luck versus Skill in the Cross-Section of Mutual Fund Returns, published in the October 2010 issue of The Journal of Finance, Eugene Fama and Kenneth French found that only mutual fund managers in the 98th and 99th percentiles showed evidence of statistically significant skill. A related study by Laurent Barras, Olivier Scaillet and Russ Wermers, False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas, which appeared in the February 2010 issue of The Journal of Finance, found that only 0.6% of funds have a true positive risk-adjusted net return, while 24% have a true negative risk-adjusted net return. And these very low figures would be even lower for taxable investors, as they are all based on pre-tax returns.

Bradford D. Jordan and Timothy B. Riley, authors of the March 2016 paper Skill and Persistence in Mutual Fund Returns: Evidence from a Six-Factor Model, further contribute to the literature by extending the factor analysis to include the relatively new profitability (RMW) and investment (CMA) factors.

Jordan and Riley use the CRSP Survivor-Bias-Free U.S. Mutual Fund Database and specifically subset it down to traditional long-only U.S. equity strategies that invest at least 80% of their assets in common equity. They also remove young and small funds so that their average fund has about $1.3 billion in assets, is 15.5 years old, and has an average expense ratio of 1.24%. Their data covers the period from September 1998 through December 2014. The following is a summary of their findings:

• Consistent with prior literature, using the four-factor model, there’s no evidence of after-fee alpha added from active management and no strong evidence of persistence in performance.

• The addition of the RMW and CMA factors to the standard four-factor model reveals persistent positive alpha after fees for mutual funds.

• Over the period from 2000 through 2014, about 15% of managers overcame transaction costs and fees, 50% of fund managers could overcome transaction costs but not fees, and 35% of managers overcame neither transaction costs nor fees.

• The top 5% of funds produced alphas of 3.7% (t-stat of 3.4). In contrast, the bottom 5% produced an alpha of -3% (t-stat of 3.3). The difference of 6.7% was highly significant with a t-stat of 4.7 (a measure of statistical significance).

• The best-performing funds (those with positive alphas) had significant negative exposure to both the RMW and CMA factors, while the worst-performing funds (those with negative alphas) had significant positive exposure.

Jordan and Riley noted that the “failure to account for these factors masks a difference in alpha between the best and worst performers of almost 7 percent per year.” They added that the differences “cannot be attributed to luck” and concluded that “skilled managers exist, and they do not capture the entire value of their skill through fees.”

These findings raise some questions. Jordan and Riley found that the top funds in their system have big and statistically significant negative loadings on the RMW and CMA factors. This is puzzling because these factors have been shown to have large positive historical returns. Why would skilled managers seek negative exposure to factors with premiums? The annual average premium for RMW has been 3% and the premium for CMA has been 3.9%. That leads to the question: Do funds with high annual returns have high alpha? To answer it, we can use the information from Table 2 of their paper.

There are some puzzling results here. First, the portfolio of all funds, the best 5% and the worst 5% all have similar annual average returns. Another puzzling result is that both the portfolios with best 5% of funds and the worst 5% of funds have slightly lower average returns than the portfolio of all funds.

The fact that all three portfolios have higher returns than the S&P 500 can be explained by the fact that from 2000 through 2014, small and value stocks outperformed the S&P 500 by fairly wide margins. For example, using MSCI indexes, while the S&P 500 returned 4.4%, small caps returned 9.4%, small value stocks returned 11.2% and large value stocks returned 6.6%. Since the study equally weights funds, these returns do not represent the experience of the average investor.

A third puzzling result is that the average annual return difference between the best 5% and the worst 5% of funds is only 10 basis points, even though the corresponding difference in annualized alphas is 6.7%. Ranking funds on alpha from their daily return regressions produced virtually no dispersion in average return. If an investor’s objective is to identify funds with high expected returns, the methodology employed here doesn’t seem to accomplish that objective.

NASCAR racing machines are sophisticated, complex automobiles. In the hands of a Kyle Busch, they are capable of great feats. The same machine, however, in the hands of a drunk driver is a very dangerous vehicle.

Factor models are valuable tools. They are so important because they allow investors to identify the sources of returns of a portfolio/fund and also to determine if there is residual alpha after adjusting for a fund’s exposure to common factors. However, factor models can be abused in the same way that a racing car can be abused.

The results of this study provide an excellent example of why it’s important to not misuse factor models. In making any portfolio construction decisions, investors should consider not just whether a fund is generating alpha, but also the fund’s exposure to various factors. In the case of the study by Jordan and Riley, we saw that while the stock-picking skills of the top 5% of performers allowed them to generate significant alphas, investors didn’t benefit in the form of higher returns. This is because the benefits derived from their skill were offset by the negative loadings on factors with positive returns.

Investors seeking higher returns would have been better served by investing in low-cost, passively managed funds with positive exposure to the desired factors. In other words, it’s not only about alpha, but also about beta (loadings on a factor).