SHARE:
FacebookTwitterFacebookPrintShare

Impact of Healthcare Algorithms on Racial Disparities in Health and Healthcare

Key Questions Nov 1, 2021
Impact of Healthcare Algorithms on Racial Disparities in Health and Healthcare

Background

Healthcare algorithms are frequently used to guide clinical decision making both at the point of care and for resource allocation and healthcare management. For the purposes of this review algorithms are defined as mathematical formulas and models that combine different variables or factors to inform a calculation or an estimate – frequently an estimate of risk. Algorithms are often incorporated into healthcare decision tools such as clinical guidelines, pathways, clinical decision support programs in electronic health records, and operational systems used by health systems and payers. End-users such as healthcare providers, integrated delivery systems, payers, and consumers use algorithms for at least six broad purposes: screening; risk prediction; diagnosis; prognosis; treatment planning; and allocation of resources. While algorithms have long been derived from traditional statistical techniques such as regression analysis, they are increasingly based on predictive analytic approaches fueled by artificial intelligence techniques including machine learning.

Healthcare algorithms and algorithm-informed decision tools (e.g., clinical guidelines, pathways, clinical decision support programs in electronic health records, operational systems used by health systems and payers) commonly include clinical and sociodemographic variables and measures of healthcare utilization. Race and ethnicity are often used as input variables and influence clinical decision-making and patient outcomes.1-3 Because race and ethnicity are socially constructed, their inclusion as variables may lead to unknown or unwanted effects, including the potential for exacerbation and/or perpetuation of health and healthcare disparities.4,5 Disparities are differences in measures of health and healthcare such as burden of disease, health outcomes, and quality of care that negatively affect disadvantaged populations. When disparities are caused in significant part by societal factors such as structural bias, socioeconomic conditions, governmental policies, discrimination, or racism specifically, they are described as inequities.6,7

Race/ethnicity is a poor proxy for genetic predisposition, as there is typically greater genetic variation within groups classified as the same race or ethnicity than between them.8-10 Numerous purported racial/ethnic genetically predisposed differences regarding muscle mass, pain sensitivity, lung function, and similar mechanisms have been debunked. Mounting research seeks to detail non-biological root causes of observed differences in health, including structural racism, chronic discrimination more generally, and social determinants of health (SDOH).11-14 Furthermore, racial/ethnic categories lack specificity and sensitivity even when self-identified, and exclusive categories are inaccurate for multi-racial and multiethnic individuals. Additionally, standard definitions of these categories are not uniformly used.15,16

Other variables used in algorithms and decision tools may also contribute to health disparities and exacerbate inequities. For example, an algorithm used to allocate access to disease management support programs was found to include a variable based on prior use of healthcare services as a marker of severity of illness. This led to stark racial disparities in program use because previous use of resources – i.e., healthcare utilization – was, itself, indicative of barriers to care and existing healthcare inequity and thus did not accurately represent need for services.5

Developers of healthcare algorithms and algorithm-informed decision tools often justify the inclusion of racial/ethnic variables by citing observational studies or post-hoc analyses of trial data that demonstrate differences in characteristics or outcomes among different racial/ethnic groups. These studies may be small and unrepresentative, and serve to reinforce misconceptions and/or assign race/ethnicity as a contributing cause when other factors may be causative, confounding or modifying effects.17,18 The most robust example in the published literature examines a “race-correction” used to estimate glomerular filtration rate (eGFR) for Black patients, a key indicator in diagnosing and treating kidney disease. Recent studies have modeled the effect of removing the race-based coefficient,19-21 and demonstrated that Black patients would be more likely to receive needed kidney transplants without use of the race-correction. However, controversy around this issue remains,22-24 as the evidence base lacks prospective trials comparing differing approaches to assessing kidney disease and subsequent need for treatments including transplant. Accordingly, a task force was convened by the National Kidney Foundation and the American Society of Nephrology to address this topic. In September 2021, the task force released its final report recommending the discontinuation of the race variable25 in calculating eGFR.

Similar evidence gaps are likely for other healthcare algorithms and algorithm-informed decision tools that include race/ethnicity, with few studies comparing the effects of alternative strategies. Moreover, little is currently known about how healthcare algorithms and algorithm-informed decision tools that do not explicitly include variables based on race/ethnicity may nevertheless exacerbate or perpetuate racial/ethnic health and healthcare disparities.

This report will examine how healthcare algorithms and algorithm-informed decision tools can introduce racial or ethnic bias into clinical care explicitly or implicitly, and examine how they affect racial/ethnic disparities in access to care, the quality of care, and health outcomes. Specific goals include:

  • Describe the ecosystem of healthcare algorithms that incorporate race/ethnicity into screening, risk prediction, diagnosis, prognosis, treatment, and resource allocation, and describe their dissemination and uptake in healthcare decision tools.
  • Synthesize the evidence base that has examined effects of healthcare algorithms and algorithm-informed decision tools on racial/ethnic disparities in healthcare and health outcomes. This includes algorithms and decision tools that explicitly use race/ethnicity, as well as those that use other variables that can lead to bias.
  • Identify healthcare algorithms and algorithm-informed decision tools incorporating race/ethnicity or other variables that could lead to disparities and are currently in development or in use, but have not yet been studied sufficiently to assess their effects on health and healthcare disparities.
  • Examine strategies to mitigate bias in the development and use of algorithms and algorithm-informed decision tools, including: elimination of variables based on race/ethnicity; use of different variables to address effects of structural racism and SDOH; increasing the representativeness of data sets used to develop algorithms; and approaches used during validation and/or implementation to identify and address bias.
  • Explore contextual concerns including: the roles of healthcare algorithm developers and end-users; available or emerging guidance on preventing bias during the development of algorithms; stakeholder awareness of and perspectives on potentially biased algorithms and decision tools; and incentives and barriers that affect how stakeholders use, evaluate, implement and/or de-implement algorithm-informed decision tools.

Draft Systematic Review Key Questions

Key Question 1. What is the nature of the evidence that has shown that healthcare algorithms and algorithm-informed decision tools in common use contribute to racial/ethnic disparities in access to care, quality of care, and health outcomes?

Key Question 2. What approaches have been used to mitigate bias in the development, validation, assessment, and implementation of healthcare algorithms and algorithm-informed decision tools?

  1. Datasets: What is the evidence for different approaches to mitigate racial/ethnic bias in datasets used for development and validation of algorithms?
  2. Algorithms/Tools: What is the evidence for different approaches to mitigate racial/ethnic bias in algorithms and algorithm-informed decision tools?

Draft Contextual Questions

Contextual Question 1: How widespread is the inclusion of variables based on race/ethnicity in healthcare algorithm-informed decision tools?

  1. What are commonly used types of algorithm-informed decision tools used in healthcare that include variables based on race/ethnicity?
  2. Who develops algorithms and algorithm-informed decision tools used in healthcare that might include variables based on race/ethnicity?
  3. Who are the end-users of algorithm-informed decision tools used in healthcare? What incentives and barriers are there to implementing or de-implementing these tools?
  4. What patient populations are included?
  5. What clinical conditions, processes of care, and healthcare settings are included?

Contextual Question 2: What are existing and emerging national or international standards or guidance for how algorithms and algorithm-informed decision tools should be developed, validated, and updated to avoid introducing bias that could lead to health and healthcare disparities?

  1. Within these standards or guidance, what are the recommendations about the use of variables or datasets that include race/ethnicity to develop and/or validate algorithms?
  2. What are the recommendations about variables used or sought in place of race/ethnicity (e.g., genetic markers and biomarkers, SDOH, the experience of individual and structural racism), including standards or guidance for how to define and collect data on these variables, and their impact on exacerbating or mitigating bias?
  3. What are the recommendations for identifying and addressing other types of variables that could introduce bias leading to disparities, such as measures of healthcare use or SDOH?

Contextual Question 3: To what extent are patients, providers (e.g., clinicians, hospitals, health systems), payers (e.g., insurers, employers), and policymakers (e.g., healthcare and insurance regulators, state Medicaid directors) aware of the inclusion of variables based on race/ethnicity in healthcare algorithms and algorithm-informed decision tools?

  1. Is there evidence of how these types of algorithms and tools might contribute to biases in provider and payer perceptions of affected populations and their clinical care?

Contextual Question 4: Select a sample of approximately 5-10 healthcare algorithms and algorithm-informed decision tools that have the potential to impact racial/ethnic disparities in access to care, quality of care, or health outcomes but have not yet been identified as such (e.g., not included in Key Questions 1 or 2). For each tool, describe the type of tool, its purpose (e.g., screening, risk prediction, diagnosis, etc.), its developer and intended end-users, affected patient population, clinical condition or process or care, healthcare setting, and information on outcomes, if available. The intent of this question is to consider the use of healthcare algorithms and algorithm-informed decision tools that may be perpetuating racial/ethnic bias but have not been previously linked to disparities in health or healthcare.

  1. If race/ethnicity is included as a variable, how is it defined? Are definitions consistent with available standards, guidance, or important considerations as identified in Contextual Question 2?
  2. For healthcare algorithms and algorithm-informed decision tools that include other variables in place of or associated with race/ethnicity, how were these other variables defined? Are these definitions consistent with available standards, guidance, or important considerations as identified in Contextual Question 2?
  3. For each healthcare algorithm and algorithm-informed decision tool, what methods were used for development and validation? What evidence, evidence quality, data sources, and study populations were used for development and validation?
  4. Are development and validation methods consistent with available standards, guidance, and strategies to mitigate algorithmic bias and reduce the potential of healthcare algorithms or algorithm-informed decision tools to contribute to health inequities?
  5. What approaches and practices are there to implement, adapt, or update each healthcare algorithm or algorithm-informed decision tool?

Analytical Approach

Key Question 1

Key Question (KQ) 1 will be addressed by a systematic review of published studies and grey literature. Figure 1 presents a draft analytical framework that displays the interaction between the major components of the evidence base, organized according to the PICOTS model (Population, Interventions, Comparator, Outcomes, Timing, and Setting). Table 1 summarizes the draft PICOTS criteria that will guide study inclusion and the assessment of findings.

Draft Analytic Framework

This figure depicts Key Question 1 within the context of the PICOTS described below. The box on the far left indicates the patient population: those whose healthcare could be affected by  algorithms and algorithm-informed decision tools that have been shown to contribute to racial/ethnic disparities in access to care,  quality of care, or health outcomes. Moving to the right, the next two boxes indicate the interventions of interest (algorithms and algorith-informed decision tools) and four comparators of interest. Finally on the far right, the figure lists the outcomes of interest in three categories: access to care (e.g., allocation of resources, costs to patients), quality of care (e.g., appropriateness and timeliness of treatment), and health outcomes (e.g., survival, quality of life). Along the bottom of the figure is a box indicating the settings of interest, which include hospital settings, ambulatory settings, and non-clinical sites.

 

Table 1. Draft PICOTS for Key Question 1

Category Definition

Population

Patients whose healthcare could be affected by algorithms and algorithm-informed decision tools (e.g., clinical guidelines, pathways, clinical decision support programs in electronic health records, operational systems used by health systems and payers) that have been shown to contribute to racial/ethnic disparities in access to care, quality of care, or health outcomes

Interventions/ Exposures

Algorithms and algorithm-informed decision tools that are commonly used for screening, risk prediction, diagnosis, prognosis, treatment, or resource allocation. They do not have to explicitly use race/ethnicity variables as inputs.

Comparators

Appropriate comparators include:

  • No algorithm or algorithm-informed decision tool
  • Same algorithm/tool with or without variable(s) based on race/ethnicity
  • Same algorithm/tool using race/ethnicity in a different way
  • Different algorithm/tool designed for the same clinical purpose, with or without variable(s) based on race/ethnicity

Outcomes

 

Access to care

  • Allocation and use of resources
  • Direct costs to patients

Quality of care

  • Appropriateness of treatment
  • Timeliness of care
  • Patient experience/satisfaction

Health outcomes

  • Mortality / Survival
  • Morbidity
  • Quality of life
  • Functional status
  • Harms

Timing

No minimum follow-up

Setting

Hospital care

  • Inpatient
  • Emergency department
  • Observation unit

Non-hospital care

  • Post-acute care, primary, specialty, rehabilitation care sites
  • Long term care (e.g., assisted living facilities, nursing home)

Non-clinical sites

  • Home care (e.g., telemedicine, self-care)

Studies conducted only in the United States

 

Key Question 2

KQ2 focuses on strategies to mitigate racial/ethnic bias in either 1) the datasets used to produce algorithms or algorithm-informed decision tools, and/or 2) the algorithms or algorithm-informed decision tools themselves. We will identify and describe strategies to address potential racial/ethnic bias, and review any evidence for their effectiveness in mitigating disparities.

References

  1. Vyas DA, Eisenstein LG, Jones DS. Hidden in plain sight - reconsidering the use of race correction in clinical algorithms. N Engl J Med. 2020 Aug 27;383(9):874-82. PMID: 32853499.
  2. Cerdeña JP, Plaisime MV, Tsai J. From race-based to race-conscious medicine: how anti-racist uprisings call us to act. Lancet. 2020 Oct 10;396(10257):1125-8. PMID: 33038972.
  3. Schmidt IM, Waikar SS. Separate and unequal: race-based algorithms and implications for nephrology. J Am Soc Nephrol. 2021 Mar;32(3):529-33. PMID: 33510038.
  4. Eneanya ND, Yang W, Reese PP. Reconsidering the consequences of using race to estimate kidney function. JAMA. 2019 Jul 9;322(2):113-4. PMID: 31169890.
  5. Obermeyer Z, Powers B, Vogeli C, et al. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019 Oct 25;366(6464):447-53. PMID: 31649194.
  6. Braveman P. Health disparities and health equity: concepts and measurement. Annu Rev Public Health. 2006;27:167-94. PMID: 16533114.
  7. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Smedley BD, Stith AY, Nelson AR, editor(s). Unequal treatment: confronting racial and ethnic disparities in health care. Washington (DC): National Academy of Sciences; 2003. 781 p.
  8. Hu AC, Chapman LW, Mesinkovska NA. The efficacy and use of finasteride in women: a systematic review. Int J Dermatol. 2019 Jul;58(7):759-76. PMID: 30604525.
  9. Roberts D. Fatal invention: how science, politics, and big business re-create race in the twenty-first century. New York (NY): The New Press; 2011. 400 p.
  10. Yudell M, Roberts D, DeSalle R, et al. Science and society. Taking race out of human genetics. Science. 2016 Feb 5;351(6273):564-5. PMID: 26912690.
  11. Smedley BD. The lived experience of race and its health consequences. Am J Public Health. 2012 May;102(5):933-5. PMID: 22420805.
  12. Williams DR, Lawrence JA, Davis BA. Racism and health: evidence and needed research. Annu Rev Public Health. 2019 Apr 1;40:105-25. PMID: 30601726.
  13. Bailey ZD, Krieger N, Agénor M, et al. Structural racism and health inequities in the USA: evidence and interventions. Lancet. 2017 Apr 8;389(10077):1453-63. PMID: 28402827.
  14. Paradies Y, Ben J, Denson N, et al. Racism as a determinant of health: a systematic review and meta-analysis. PLoS ONE. 2015;10(9):e0138511. PMID: 26398658.
  15. Race, ethnicity, and language data: standardization for health care quality improvement. AHRQ publication no. 10-0058-EF. [internet]. Rockville (MD): Agency for Healthcare Research and Quality; 2010 Mar [updated 2018 Apr 1]; [accessed 2011 May 16].
  16. Klinger EV, Carlini SV, Gonzalez I, et al. Accuracy of race, ethnicity, and language preference in an electronic health record. J Gen Intern Med. 2015 Jun;30(6):719-23. PMID: 25527336.
  17. Kowalsky RH, Rondini AC, Platt SL. The case for removing race from the American Academy of Pediatrics clinical practice guideline for urinary tract infection in infants and young children with fever. JAMA Pediatr. 2020 Mar 1;174(3):229-30. PMID: 31930353.
  18. Vyas DA, Jones DS, Meadows AR, et al. Challenging the use of race in the vaginal birth after cesarean section calculator. Womens Health Issues. 2019 May-Jun;29(3):201-4. PMID: 31072754.
  19. Ahmed S, Nutt CT, Eneanya ND, et al. Examining the potential impact of race multiplier utilization in estimated glomerular filtration rate calculation on African-American care outcomes. J Gen Intern Med. 2021 Feb;36(2):464-71. PMID: 33063202.
  20. Diao JA, Wu GJ, Taylor HA, et al. Clinical implications of removing race from estimates of kidney function. JAMA. 2021 Jan 12;325(2):184-6. PMID: 33263721.
  21. Inker LA, Couture SJ, Tighiouart H, et al. A new panel-estimated GFR, including ß2-microglobulin and ß-trace protein and not including race, developed in a diverse population. Am J Kidney Dis. 2020 Dec 7;S0272-6386(20):31126-4. PMID: 33301877.
  22. Norris KC, Eneanya ND, Boulware LE. Removal of race from estimates of kidney function: first, do no harm. JAMA. 2021 Jan 12;325(2):135-7. PMID: 33263722.
  23. Levey AS, Titan SM, Powe NR, et al. Kidney disease, race, and GFR estimation. Clin J Am Soc Nephrol. 2020 Aug 7;15(8):1203-12. PMID: 32393465.
  24. Powe NR. Black kidney function matters: use or misuse of race? JAMA. 2020 Aug 25;324(8):737-8. PMID: 32761164.
  25. Delgado C, Baweja M, Crews D, et al. A unifying approach for GFR estimation: recommendations of the NKF-ASN task force on reassessing the inclusion of race in diagnosing kidney disease. J Am Soc Nephrol. 2021 Sep 23;ASN.2021070988. Online ahead of print. doi.org/10.1681/ASN.2021070988. PMID: 34556489.

Project Timeline

Impact of Healthcare Algorithms on Racial Disparities in Health and Healthcare

Nov 1, 2021
Topic Initiated
Nov 1, 2021
Key Questions
Page last reviewed November 2021
Page originally created November 2021

Internet Citation: Key Questions: Impact of Healthcare Algorithms on Racial Disparities in Health and Healthcare. Content last reviewed November 2021. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.
https://effectivehealthcare.ahrq.gov/products/racial-disparities-health-healthcare/draft-comments

Select to copy citation