Please rotate your device to landscape mode to view the charts.

The Virtue of Complexity in Return Prediction

Journal: Journal of Finance

Date: 20240201

Author: Kelly, Bryan; Malamud, Semyon; Zhou, Kangying

Abstract:
Much of the extant literature predicts market returns with "simple" models that use only a few parameters. Contrary to conventional wisdom, we theoretically prove that simple models severely understate return predictability compared to "complex" models in which the number of parameters exceeds the number of observations. We empirically document the virtue of complexity in U.S. equity market return prediction. Our findings establish the rationale for modeling expected returns through machine learning.

Link: Google Scholar

Key Findings

Complex Models Outperform Simple Models

Contrary to conventional wisdom, highly complex prediction models with more parameters than observations can achieve better out-of-sample performance than simple models when appropriate shrinkage is applied.

Positive Economic Returns Despite Negative R²

Machine learning timing strategies can generate significant positive returns and Sharpe ratios even when the out-of-sample R² is negative, challenging traditional evaluation metrics.

Market Timing Success

High-complexity models successfully predict market movements, achieving information ratios around 0.3 versus the market and correctly reducing positions before 14 out of 15 recessions.

Out-of-Sample Performance Across Model Complexity

  • Shows Sharpe ratios increasing with model complexity
  • Performance peaks at highest complexity levels (c=1000)
  • Ridge shrinkage (z=10³) helps stabilize performance

Strategy Performance Metrics

  • Information ratio of 0.31 vs market (t-stat = 2.9)
  • Positive alpha persists across different training windows
  • Performance robust to varying degrees of shrinkage

Market Timing Positions

  • Strategy learns to reduce market exposure before recessions
  • Predominantly maintains long positions during normal periods
  • Successfully predicted 14 out of 15 recessions in sample

Contribution and Implications

  • Challenges conventional wisdom about model parsimony in finance
  • Demonstrates that complex machine learning models can deliver robust out-of-sample performance
  • Provides theoretical justification for using high-dimensional models in asset pricing
  • Shows that traditional R² metrics may be misleading for evaluating trading strategies

Data Sources

  • Performance Chart: Based on Figures 7-8 showing out-of-sample performance metrics across model complexity
  • Metrics Chart: Constructed from Table I showing information ratios and statistical significance across different training windows
  • Timing Chart: Derived from Figure 10 showing market timing positions and recession prediction accuracy