Please rotate your device to landscape mode to view the charts.

Thousands of Alpha Tests

Journal: Review of Financial Studies

Date: 20210701

Author: Giglio, Stefano; Liao, Yuan; Xiu, Dacheng

Abstract:
Data snooping is a major concern in empirical asset pricing. We develop a new framework to rigorously perform multiple hypothesis testing in linear asset pricing models, while limiting the occurrence of false positive results typically associated with data snooping. By exploiting a variety of machine learning techniques, our multiple-testing procedure is robust to omitted factors and missing data. We also prove its asymptotic validity when the number of tests is large relative to the sample size, as in many finance applications. To improve the finite sample performance, we also provide a wild-bootstrap procedure for inference and prove its validity in this setting. Finally, we illustrate the empirical relevance in the context of hedge fund performance evaluation.

Link: Google Scholar

Background and Context

The Challenge

Data snooping is a major concern in empirical asset pricing - when testing multiple hypotheses, many positive results may appear significant purely by chance.

The Solution

The authors develop a rigorous framework for multiple hypothesis testing in linear asset pricing models that controls the false discovery rate while handling missing data and latent factors.

The Method

The approach combines false discovery rate control with matrix completion techniques and principal component analysis to handle missing data and unobservable factors.

Performance of Different Fund Selection Methods Shows Superiority of FDR Control

  • Shows in-sample performance comparison of different fund selection approaches
  • Mixed FDR with latent factors achieves highest average alpha of 70.4 bp/month
  • Demonstrates clear advantage over methods that ignore latent factors or multiple testing

Fund Selection Methods: Balancing Fraction Selected vs Performance

  • Shows the fraction of funds selected by each methodology
  • Mixed FDR selects 25% of funds while maintaining higher performance
  • Demonstrates efficient balance between selectivity and performance

Out-of-Sample Performance Shows Robustness of FDR Approach

  • Shows out-of-sample performance of different selection methods
  • Mixed FDR maintains superior performance of 22.3 bp/month
  • Demonstrates robustness of the approach in real-world application

Impact of Latent Factors on Return Variation

  • Shows eigenvalues of return covariance matrix
  • Demonstrates presence of strong latent factors in hedge fund returns
  • Justifies inclusion of latent factors in the model

Monthly Rebalancing Performance Across Methods

  • Shows performance under monthly portfolio rebalancing
  • Mixed FDR maintains superior performance even with frequent rebalancing
  • Demonstrates robustness to different implementation approaches

Contribution and Implications

  • Provides a robust framework for controlling false discoveries in asset pricing tests while handling real-world complications of missing data and latent factors
  • Demonstrates superior performance in hedge fund selection, both in-sample and out-of-sample
  • Offers practical implementation guidance for institutional investors and researchers

Data Sources

  • Performance comparison charts based on Table 2 (In-sample results)
  • Fund selection statistics based on Table 4 (Out-of-sample results: Selection)
  • Out-of-sample performance based on Table 3 (Out-of-sample results: Performance)
  • Factor analysis based on Figure 2 (Properties of hedge fund excess returns)
  • Monthly rebalancing results based on Table 3 rows with monthly rebalancing specification