Please rotate your device to landscape mode to view the charts.

A Picture Is Worth a Thousand Words: Measuring Investor Sentiment by Combining Machine Learning and Photos from News

Journal: Journal of Financial Economics

Date: 20220401

Author: Obaid, Khaled; Pukthuanthong, Kuntara

Abstract:
By applying machine learning to the accurate and cost-effective classification of photos based on sentiment, we introduce a daily market-level investor sentiment index (Photo Pessimism) obtained from a large sample of news photos. Consistent with behavioral models, Photo Pessimism predicts market return reversals and trading volume. The relation is strongest among stocks with high limits to arbitrage and during periods of elevated fear. We examine whether Photo Pessimism and pessimism embedded in news text act as complements or substitutes for each other in predicting stock returns and find evidence that the two are substitutes.

Link: Google Scholar

Key Findings

Photo Sentiment Predicts Market Returns

Using machine learning to analyze news photos, the study finds that high photo pessimism predicts lower next-day market returns followed by reversals over the next trading week, suggesting sentiment-driven price pressure.

Photos and Text are Substitutes

Photo and text pessimism act as substitutes rather than complements. Photos dominate during periods of high photo salience, while text pessimism matters more when photos have mixed sentiment.

Stronger Effects During Fear Periods

The impact of photo pessimism on returns is 2.8 times larger during periods of elevated market fear compared to normal periods, suggesting photos are particularly effective at conveying negative sentiment.

Return Predictability Across Market Indices

  • One standard deviation increase in PhotoPes predicts 4.2-4.7 basis point decline in next-day returns
  • Effect reverses over next 4 trading days, suggesting temporary price pressure
  • Impact is consistent across major market indices (VWRETD, SPX, SPY, INDU, DIA)

Trading Volume Response to Photo Sentiment

  • High absolute photo sentiment predicts increased trading volume
  • One standard deviation change in |PhotoPes| associated with 0.08 standard deviation increase in next-day volume
  • Volume effect persists for multiple trading days

Fear Period Return Impact

  • PhotoPes effect is 2.8x stronger during high fear periods
  • 10.3 basis point impact during fear periods vs 3.7 basis points normally
  • Photos appear especially effective at conveying negative sentiment

Contribution and Implications

  • First study to use machine learning to extract market-relevant sentiment from news photos
  • Demonstrates photos and text act as substitutes in conveying market-relevant information
  • Shows visual content is especially important during periods of market fear
  • Provides practical trading strategies that generate significant risk-adjusted returns

Data Sources

  • Return impact chart based on Table 2 Panel A showing coefficients for PhotoPest-1 across different market indices
  • Volume response visualization uses coefficients from Table 10 showing the relationship between |PhotoPest| and trading volume
  • Fear period comparison based on analysis presented in Table 6 comparing coefficients during high vs low fear periods