Please rotate your device to landscape mode to view the charts.

Does Machine Learning Automate Moral Hazard and Error?

Journal: American Economic Review

Date: 20170501

Author: Mullainathan, Sendhil; Obermeyer, Ziad

Abstract:
Machine learning tools are beginning to be deployed en masse in health care. While the statistical underpinnings of these techniques have been questioned with regard to causality and stability, we highlight a different concern here, relating to measurement issues. A characteristic feature of health data, unlike other applications of machine learning, is that neither y nor x is measured perfectly. Far from a minor nuance, this can undermine the power of machine learning algorithms to drive change in the health care system--and indeed, can cause them to reproduce and even magnify existing errors in human judgment.

Link: Google Scholar

Key Findings

Measurement Error in Medical Data

Machine learning algorithms trained on medical records data can inadvertently amplify existing biases and measurement errors due to how medical data is collected and recorded.

Predictive Bias

Algorithms may predict healthcare utilization patterns rather than true medical risk, potentially directing resources to already high-utilizing patients rather than those with greatest medical need.

Automation of Disparities

Without accounting for measurement issues, predictive algorithms risk automating and magnifying existing healthcare disparities and clinical errors.

Stroke Risk Predictors

  • Prior stroke has the strongest predictive association with future stroke
  • Seemingly unrelated conditions like accidental injury show surprisingly strong associations
  • These relationships may reflect healthcare utilization patterns rather than true stroke risk

Stroke vs Mortality Prediction

  • Coefficients decrease significantly when predicting mortality instead of stroke
  • Some strong stroke predictors show no association with mortality
  • Suggests original predictions may capture healthcare utilization rather than true medical risk

Medical Data Measurement Issues

Subjective Assessment Selective Testing Event-Based Recording
  • Medical data collection involves multiple layers of potential measurement error
  • Each stage introduces potential biases and selective recording
  • Resulting data reflects both medical conditions and healthcare seeking behavior

Contribution and Implications

  • Highlights critical measurement challenges in applying machine learning to healthcare that could inadvertently amplify existing biases
  • Demonstrates need for careful validation of prediction algorithms against gold-standard measurements
  • Suggests combining machine learning with randomized trials and high-quality follow-up data for optimal implementation

Data Sources

  • Stroke Risk Predictors chart based on Table 1 coefficient values from logistic regression analysis
  • Mortality Comparison chart combines stroke and 30-day mortality coefficients from Table 1
  • Measurement Error Framework diagram synthesizes conceptual framework described in Section II.B of the paper