Please rotate your device to landscape mode to view the charts.

The Virtue of Complexity in Return Prediction

Journal: Journal of Finance

Date: 20240201

Author: Kelly, Bryan; Malamud, Semyon; Zhou, Kangying

Abstract:
Much of the extant literature predicts market returns with "simple" models that use only a few parameters. Contrary to conventional wisdom, we theoretically prove that simple models severely understate return predictability compared to "complex" models in which the number of parameters exceeds the number of observations. We empirically document the virtue of complexity in U.S. equity market return prediction. Our findings establish the rationale for modeling expected returns through machine learning.

Link: Google Scholar

Background and Context

Research Focus

The study examines how complex machine learning models with many parameters can improve market return predictions compared to simpler models with fewer parameters.

Methodology

The researchers analyze market timing strategies using random Fourier features (RFF) to create models of varying complexity, testing them on U.S. equity market returns from 1926-2020.

Data Sources

The study uses 15 standard predictor variables from the finance literature, analyzing their effectiveness in predicting market returns across different model complexities.

Higher Model Complexity Improves Market Timing Performance

  • As model complexity increases, the Sharpe ratio (risk-adjusted return) improves
  • Complex models achieve Sharpe ratios around 0.4, significantly outperforming simpler models
  • Performance remains robust even with very high levels of complexity

Expected Returns Rise with Model Complexity

  • Out-of-sample expected returns increase monotonically with model complexity
  • Complex models capture more subtle predictive relationships in the data
  • Benefits persist even at very high levels of complexity

Strategy Performance Across Training Windows

  • Strategy performs well across different training window lengths
  • 12-month window achieves highest information ratio of 0.31
  • Performance remains robust even with longer training periods

Model Performance During Market Stress

  • Model successfully reduces market exposure before 14 out of 15 recessions
  • Strategy naturally learns to be defensive during market stress
  • Demonstrates ability to identify changing market conditions

Variable Importance in Prediction

  • Lagged market return is the most important predictor
  • Model effectively leverages short-term fluctuations in predictors
  • Complex models capture subtle interactions between variables

Contribution and Implications

  • The study challenges conventional wisdom by showing that more complex models can improve market timing performance
  • Results demonstrate that machine learning models can successfully predict market returns even with negative R² values
  • Findings suggest investors should consider using more sophisticated models for market timing strategies

Data Sources

  • Sharpe Ratio chart based on Figure 8 in the paper
  • Expected Returns visualization based on Figure 5
  • Training window performance based on Table I
  • Recession performance based on Figure 10
  • Variable importance visualization based on Figure 11