Please rotate your device to landscape mode to view the charts.

Regulatory Arbitrage or Random Errors? Implications of Race Prediction Algorithms in Fair Lending Analysis

Journal: Journal of Financial Economics

Date: 20240701

Author: Greenwald, Daniel L.; Howell, Sabrina T.; Li, Cangyuan; Yimfor, Emmanuel

Abstract:
When race is not directly observed, regulators and analysts commonly predict it using algorithms based on last name and address. In small business lending--where regulators assess fair lending law compliance using the Bayesian Improved Surname Geocoding (BISG) algorithm--we document large prediction errors among Black Americans. The errors bias measured racial disparities in loan approval rates downward by 43%, with greater bias for traditional vs. fintech lenders. Regulation using self-identified race would increase lending to Black borrowers, but also shift lending toward affluent areas because errors correlate with socioeconomics. Overall, using race proxies in policymaking and research presents challenges.

Link: Google Scholar

Key Findings

BISG Has High Error Rate for Black Americans

The Bayesian Improved Surname Geocoding (BISG) algorithm produces more false positives and false negatives than true positives when identifying Black borrowers. The correlation between BISG and self-identified race is only 0.54, compared to 0.87 for image-based classification.

BISG Errors Bias Approval Rate Gaps

Using BISG understates racial disparities in loan approval rates by 43% compared to image-based race measures. The approval gap between Black and non-Black applicants is 2.3pp using image-based race but only 1.3pp using BISG.

Different Effects Across Lender Types

Traditional banks appear to benefit more from BISG-based evaluation compared to fintech lenders. Fintechs are 64pp more likely to have positive differences between image-based and BISG-based lending rates to Black borrowers.

BISG Classification Accuracy

  • Only 27.2% of Black borrowers are correctly identified by BISG (True Positive)
  • 44.2% of non-Black borrowers are incorrectly classified as Black (False Positive)
  • 28.6% of Black borrowers are misclassified as non-Black (False Negative)

Loan Approval Rate Disparities

  • Image-based measure shows 2.3pp gap in approval rates between Black and non-Black borrowers
  • BISG-based measure shows only 1.3pp gap, understating disparity by 43%
  • Both measures show lower approval rates for Black borrowers

Lender Type Analysis

  • Fintech lenders are 64pp more likely to show higher Black lending rates using image vs BISG measures
  • Traditional banks generally show smaller or negative differences between measures
  • Credit unions and CDFIs show moderate positive differences

Contribution and Implications

  • First comprehensive documentation of race prediction algorithm errors in non-mortgage lending context
  • Demonstrates how BISG errors can lead to systematic underestimation of racial disparities in lending
  • Highlights need for careful consideration in choice of race measures for regulatory compliance
  • Suggests potential benefits of collecting self-identified race data in small business lending

Data Sources

  • Error rate chart constructed using data from Figure 5 Panel D showing BISG classification rates
  • Approval rate chart based on data from Table 4 showing approval rates by race measure
  • Lender analysis chart created using coefficients from Table 8 columns 4-6 showing lender type relationships