Please rotate your device to landscape mode to view the charts.

Regulatory Arbitrage or Random Errors? Implications of Race Prediction Algorithms in Fair Lending Analysis

Journal: Journal of Financial Economics

Date: 20240701

Author: Greenwald, Daniel L.; Howell, Sabrina T.; Li, Cangyuan; Yimfor, Emmanuel

Abstract:
When race is not directly observed, regulators and analysts commonly predict it using algorithms based on last name and address. In small business lending--where regulators assess fair lending law compliance using the Bayesian Improved Surname Geocoding (BISG) algorithm--we document large prediction errors among Black Americans. The errors bias measured racial disparities in loan approval rates downward by 43%, with greater bias for traditional vs. fintech lenders. Regulation using self-identified race would increase lending to Black borrowers, but also shift lending toward affluent areas because errors correlate with socioeconomics. Overall, using race proxies in policymaking and research presents challenges.

Link: Google Scholar

Background and Context

Research Problem

Regulators commonly use algorithms based on surname and location to predict race when assessing fair lending compliance, but the accuracy and implications of these predictions are not well understood.

Study Setting

The research examines small business lending where regulators use the Bayesian Improved Surname Geocoding (BISG) algorithm to evaluate fair lending law compliance.

Methodology

The study analyzes lending data from Lendio marketplace (2017-2019) and Paycheck Protection Program loans, comparing BISG predictions to image-based and self-identified race measures.

High Error Rates in BISG Race Prediction Algorithm

  • BISG algorithm produces more errors than correct classifications when identifying Black borrowers
  • Only 27.2% of predictions are correct (true positives)
  • 44.2% are incorrectly classified as Black (false positives) and 28.6% are incorrectly classified as non-Black (false negatives)

Disparity in Loan Approval Rates Underestimated by BISG

  • True racial disparity in approval rates is 2.3 percentage points using image-based race
  • BISG measures only 1.3 percentage point gap, underestimating disparity by 43%
  • This underestimation could lead regulators to miss significant discrimination

Approval Rates by BISG Classification Type

  • Correctly identified Black borrowers have lowest approval rate at 5.6%
  • Misclassified Black borrowers (false negatives) have slightly higher approval rate at 6.7%
  • Non-Black borrowers misclassified as Black have highest approval rates at 8.7%

Socioeconomic Correlation with BISG Errors

  • BISG errors correlate strongly with socioeconomic characteristics
  • Higher false negative rates in areas with higher income and education
  • Lower false negative rates in areas with higher Black population share

Lender Type Impact on Race Classification Accuracy

  • Fintech lenders show 64 percentage points higher accuracy in serving Black borrowers compared to traditional banks
  • Traditional banks more likely to benefit from BISG-based evaluation
  • Suggests systematic differences in how different lender types serve Black borrowers

Contribution and Implications

  • The study reveals significant limitations in using BISG for fair lending compliance, with implications for regulatory policy and lending practices
  • Findings suggest that moving to self-identified race collection could better identify discrimination but may shift lending toward more affluent areas
  • Results highlight the need for more accurate methods of measuring racial disparities in lending practices

Data Sources

  • Error rates visualization based on Figure 5 in the paper
  • Approval rate disparities based on Table 4
  • Classification type approval rates based on Figure 7
  • Socioeconomic correlations based on Table 9
  • Lender type analysis based on Table 8