Abstract: A new paper argues that annualized hedge fund returns in the Lipper TASS database are 50% lower after adjusting for backfill and survivorship bias. We note that hedge fund index providers already adjust for these biases, and find that the long-term rate of return obtained using their indexes is a better estimate of actual realized returns than the authors’ lowball value. We support our claim using historical fund of funds and endowment returns, both of which are less affected by biases.
In case you missed it, a small, but vocal portion of the Twittersphere blew up on Monday when Bloomberg linked to a new research paper1 claiming that roughly half of hedge funds’ historical annualized return is an illusion caused by reporting biases.
The paper’s authors, Mila Getmansky, Peter Lee, and Andew Lo (hereafter, GLL), used data from the Lipper TASS hedge fund database to compare fund returns before and after adjusting for two well-known biases: backfill and survivorship. They found that hedge funds produced unadjusted returns of 12.6% annually between 1996 and 2014. However, they also calculated that after accounting for the biases, the actual hedge fund return realized by investors was only 6.3%.
There’s a lot of shock value in the disparity between the two calculations. So much so that Bloomberg ran its article with the headline “Hedge Funds Do Half as Well as You Think” which understandably generated a lot of buzz. If long-term hedge fund returns are truly 50% lower than previously reported, hedge fund investors should feel duped and absolutely incensed. But is this really the case?
GLL make two significant claims: firstly, that hedge funds returned 12.6% unadjusted and secondly, that the actual return was 6.3%. We’ll consider these claims individually.
Investigating Claim #1: Did Hedge Funds Return 12.6% Unadjusted Between 1996 and 2014?
I’ll first confess that Eqira doesn’t subscribe to the TASS database, so there’s no way for us to independently verify GLL’s computations. What we do have, however, are hedge fund indexes from several providers covering the full period, none of which come close to estimating a 12.6% return.
Below are the compound annual growth rates (CAGRs) from four leading index providers, CISDM, Credit Suisse, Greenwich Associates, and Hedge Fund Research, as well as from our own composite index of indexes and a second composite index comprised of averaged index returns.
The index CAGRs range from 8.1% to 9.6%, well below GLL’s estimate. In fact, GLL’s estimate is a full 5.7 standard deviations greater than the mean index CAGR, which is much too high to attribute to random error. So what’s happening here?
What the paper’s authors fail to mention (in print that is, to her credit Getmansky did mention it when interviewed on air) is that the major hedge fund index providers adjust for survivorship and backfill biases. So in the almost certain case that you’ve been evaluating hedge funds using an index instead of calculating the monthly average of thousands of individual hedge fund returns yourself, you’re probably fine.
We can safely refute claim #1 because hedge fund indexes have never suggested a 12.6% annualized return. It follows then that there’s simply not a 50% difference between actual hedge fund returns and what we’ve been led to believe.
Nevertheless, there’s still a meaningful difference between that 8.9% mean CAGR and GLL’s 6.3% estimate, so we need to refute claim #2 as well before you can feel completely comfortable with any analyses you may have conducted in the past.
A Brief Aside: Understanding the Backfill and Survivorship Biases
Before we dive into the second claim, let’s take a brief detour to understand the two biases under discussion. Both are prevalent in hedge fund databases because the databases depend entirely upon managers to self-report returns. Managers have complete discretion over when, how, and to which, if any, databases they report.
Backfill bias occurs when new managers join a database and retroactively add their historical performance. Since presumably only managers with good historical track records will join the database, their doing so biases average fund returns upwards. Survivorship bias occurs when managers stop reporting returns to a database. This often happens when managers stumble on hard times. Their attempts to hide poor performance again bias the average return of the remaining funds upwards. Both biases effectively overinflate mean returns.
There has never been any debate that hedge fund databases suffer from these biases. Fung and Hsieh2 first estimated the degree of backfill bias in the Lipper TASS database in 2000. Ackermann, McEnally and Ravenscraft3 did the same with survivorship bias in 1999.
Investigating Claim #2: Did Hedge Funds Actually Return 6.3% between 1996 and 2014?
To measure the impact of the two biases, GLL first estimated unadjusted hedge fund returns by chaining together a series of monthly average returns for all funds in the TASS database. They then estimated unbiased returns by repeating the calculation after inserting returns from TASS’s dead funds database, which includes returns for funds that have since dropped out of the main database, and stripping out returns that they deemed to have been added retroactively. This second method produced the 6.3% return under discussion.
There are a couple of ways that we can approximate bias-free hedge fund returns without resorting to the subjective methods GLL used. The first is using Fund of Funds returns.
Funds of Funds are much less likely to suffer from backfill and survivorship biases because they are essentially portfolios of hedge funds. They tend to be diversified and usually outlive their worst investments. So while they may still experience periods of low return and go out of business, they generally do so much less frequently than individual funds. In a 2002 paper, Fung and Hsieh4 advocate using Fund of Funds returns as bias-free estimates of hedge fund returns precisely for this reason.
Below we provide CAGRs for fund of funds indexes from CISDM and Hedge Fund Research, as well as for Eqira’s Fund of Funds Composite Index and an index of monthly mean index returns.
These range from 5.5% to 6.1%, all of which are below GLL’s 6.3% estimate. Does this validate GLL’s claim?
Not exactly. Funds of funds add an additional layer of fees (typically a 1% management fee and 10% performance fee) on top of the underlying hedge fund fees. To approximate hedge fund returns using fund of funds returns, we need to strip out that second layer. We do so by adjusting the return for any given year with the following formula:
Ra = Ru + Max((Rh * Fp) / (1- Fp), 0) + Fm
Where Ru is the unadjusted return, Rh is the return in excess of the index’s high water mark, Fm is the management fee, Fp is the performance fee, and Ra is the adjusted return. We use 10% for the performance fee and 1% for the management fee.
We now get returns ranging from 7.1% to 7.7%, which is considerably greater than GLL’s estimate and much closer to the historical hedge fund index CAGRs. They’re still about 1.3% short though, so it may help to use one more approach.
Estimating Hedge Fund Returns Using Endowment Returns
Each year, the National Association of College and University Business Officers (NACUBO) partners with Commonfund to produce a study of endowment performance. The full studies are only available for a fee, but fortunately NACUBO releases partial findings and summaries on their website5.
The website provides the mean return collectively earned by endowments in marketable alternative strategies (i.e., hedge funds) for each fiscal year between 2005 and 2014 except 2008, which is unfortunately missing. Luckily, the fiscal year for endowments runs from June-to-June instead of December-to-December, so losses from any funds that imploded during the 2008 financial crisis should still be included in the 2009 returns.
In its most recent iteration, the NACUBO study included 832 total institutions, a sample size large enough to reflect the broad performance of the endowment universe. We believe that a sample that large should also reflect the broad performance of hedge fund investors in general, even though the top endowments do tend to exhibit higher skill.
Below, we provide NACUBO’s year-by-year returns, as well as returns for Eqira’s composite index and seven leading hedge fund index providers (the four used earlier plus BarclayHedge, Eurekahedge, and Hedge Fund Intelligence). We can see that the index provider returns fairly accurately mirror NACUBO’s. When we calculate CAGRs using only data for the years NACUBO data is available, we see that, although the index returns do tend to be a bit higher, they are only marginally so, and considerably less than the GLL estimate would have led us to believe. The mean CAGR and the Eqira CAGR are only 50 basis points higher than NACUBOs.
NACUBO’s returns are well below average in 2009 which does hint at some possible survivorship bias, particularly in the Eurekahedge and HFI indexes. However, it’s not good to infer too much from a single data point. Investors would have needed to realize a shortfall of that size nearly every year on average to substantiate GLL’s lower value. Instead, we see that there are actually more years when endowments outperform the indexes than vice versa.
For confirmation, we can repeat the exercise using fund of funds returns. We use pre-free returns from the two providers we looked at earlier plus BarclayHedge, Eurekahedge, and InvestHedge.
Here we see that endowments meaningfully outperformed funds of funds: in excess of 1.3% annually. It’s interesting to note that this outperformance is roughly the same as the shortfall between the pre-fee fund of fund indexes and the hedge fund indexes we observed earlier. Admittedly, we are dealing with a relative small sample that may not be representative of other periods, but if we were to assume that endowments were able to outperform funds of funds by 1.3% annually for the entire 1996-2014 sample, then their long-term returns would be nearly identical to that of the hedge fund indexes.
There are a lot of reasons to criticize hedge funds. We identified many of them in our first Strategy Spotlight last week: high fees, decreasing alpha, rising correlations with traditional assets, high betas, and an overreliance on equity-based risk factors. Fortunately, it does not appear that misrepresented returns are one of them. Hedge fund returns are not 50% less than previously thought, both because no one ever claimed them to be as high as 12.6% and because they are likely not as low as 6.3%.
Without access to GLL’s data set it’s impossible to replicate their results. Their return differences could be the result of computational errors, a la Reinhart and Rogoff, but these are smart, accomplished professors and we’re willing to give their calculations the benefit of the doubt. It’s more likely that the discrepancies are caused by a perfectly valid reason, such as a high number of very small funds in the database. Since GLL are taking simple averages instead of value-weighting returns, their calculations will be heavily biased towards small firms instead of the major players that comprise most investor portfolios. The important thing to note is that, whatever the flaw in hedge fund indexes may be, it is not as significant as GLL would lead you to believe.
The blame here probably falls on Bloomberg for sensationalizing what was a very small piece of a very long and wide-ranging 135 page paper, the point of which was to review academic literature on hedge funds, not to bash hedge fund databases.
The paper is well worth a read. Just don’t read too much into it.
- Getmansky, Mila, Peter A. Lee and Andrew W. Lo, 2015, “Hedge Funds: A Dynamic Industry in Transition”, Available at SSRN 2637007.
- Fung, W. and D. A. Hsieh, 2000, “Performance Characteristics of Hedge Funds and Commodity Funds: Natural Versus Spurious Biases”, Journal of Financial and Quantitative Analysis 35, 291–307.
- Ackermann, C., McEnally, R., and D. Ravenscraft, 1999, “The Performance of Hedge Funds: Risk, Return, and Incentives”, Journal of Finance 54, 833–874.
- Fung, William, and David A. Hsieh, 2002, “Hedge-fund benchmarks: Information Content and Biases”, Financial Analysts Journal 58.1: 22-34.