Artificial intelligence (AI) is rapidly entering health care including safety-net, rural, and underserved settings, where access, capacity, and workforce constraints are most acute.
Across the literature, AI is described as a tool that can support clinical decision-making, improve outcomes, and expand access. But those same systems also raise concerns about bias, data limitations, and uneven impacts on underserved populations.
This Digging Deeper series draws from a systematic review of peer-reviewed research to explore both sides of that tension: what AI could make possible, and what risks must be addressed to ensure it does not reinforce existing inequities.
Article Authored by Briana Martin | EHF
The risk isn’t abstract; it shows up in real ways
When the literature talks about the risks of AI, it’s not just pointing to rare errors or technical glitches. It describes patterns and ways these tools can fall short, especially when they’re built or used without enough attention to equity.
In safety-net, rural, and underserved settings, those risks can be harder to catch. Systems are already stretched, and there may be less time or capacity to monitor how tools are performing.
Across the research, a common thread emerges: challenges tend to show up at very predictable points.
What happens when the data doesn’t reflect the population?
AI systems depend on data. When that data doesn’t reflect the populations being served, the results can be uneven.
The literature highlights that rural and underserved communities are often underrepresented in the datasets used to train AI tools. On the surface, a model might perform well overall, but still miss the mark for specific groups.
There are also limits to how social needs are captured. Even when health systems try to include non-medical drivers of health, some factors like financial strain or social isolation aren’t consistently measured. That shapes what AI systems can recognize, and what they leave out.
Over time, those gaps can translate into differences in care that may not always be obvious but can still add up.
What happens when decisions aren’t easy to explain?
Another issue the literature raises is transparency.
Many AI tools operate as “black boxes,” meaning it’s not always clear how they arrive at a recommendation. Clinicians may see an output, but not the full reasoning behind it.
That lack of clarity can make it harder to trust the system or know when to question it. It also makes it more difficult to identify when something has gone wrong.
For safety-net, rural, and underserved settings, this matters in a very practical way. If it’s harder to see how decisions are being made, it’s also harder to catch and correct errors, especially in already stretched systems.
What happens when “neutral” choices aren’t actually neutral?
One of the clearest lessons in the literature is that bias doesn’t have to be intentional to have real consequences. Sometimes, it shows up through decisions that seem reasonable on the surface but reflect deeper inequities in the system.
Example: When cost becomes a stand-in for need
In one widely-cited case, an algorithm used to identify patients with complex health needs relied on health care costs as a key factor. At first glance, that might seem like a reasonable proxy. But because health care spending already reflects disparities in access and treatment, the algorithm systematically underestimated the needs of Black patients.
This meant fewer resources were directed to patients who may have needed more care simply because the model was built on a measure shaped by existing inequities. The issue wasn’t the intent of the tool. It was the assumption built into it.
This example shows how decisions that appear neutral can still produce unequal outcomes, depending on how they’re constructed.
What happens when errors go unnoticed?
AI tools can produce outputs that feel precise and authoritative, even when they’re incomplete or incorrect. That creates a practical challenge: when something looks reliable, it can be harder to recognize when it isn’t.
Example: When missing context leads to the wrong recommendation
The literature describes a case where an AI tool designed to predict pneumonia-related complications suggested that patients with asthma could be discharged. The issue wasn’t with the tool’s intention; it was that it failed to account for key clinical risks associated with asthma.
Because the system didn’t fully capture that context, its recommendation could have led to decisions that put patients at risk. This kind of error highlights a broader challenge: when systems don’t incorporate the full picture, their outputs can appear confident while still being incomplete.
These situations point to an important takeaway: if errors aren’t easy to detect, they can persist and their impact may not be evenly distributed across populations.
What connects these risks?
Across these scenarios, the pattern is consistent.
The challenges aren’t random; they stem from how AI tools are built, what data they rely on, and how they’re used in practice. When those pieces aren’t aligned with the realities of safety-net, rural, and underserved care, gaps can widen instead of close.
At the same time, the literature doesn’t suggest that these outcomes are inevitable. What it shows is that they are predictable and that addressing them requires intentional choices at every stage.
Looking ahead: what needs to be in place
While this post focused on where risks show up, our final post will look at what helps prevent them.
The literature points to a set of conditions like stronger oversight, clearer accountability, and better preparation for the workforce that can support more responsible use of AI in practice.
Next up in this series: What it Takes to Use AI Responsibly: Guardrails for Safety-Net, Rural, and Underserved Care
ATTRIBUTION & SOURCES
This post draws from Literature Review: AI in Safety-Net Health Care, prepared by the IC² Institute at The University of Texas at Austin.
More from Digging Deeper
Where AI Could Help: Filling the Gaps that Safety-Net, Rural, and Underserved Systems Face Every Day
Artificial intelligence (AI) is rapidly entering health care including safety-net, rural, and underserved settings, where access, capacity, and workforce constraints are most acute. Across the literature, AI is described as a tool that can support clinical decision-making, improve outcomes, and expand access. But those same systems also raise concerns about
The opportunity to advance diabetes prevention through Texas Medicaid and the Rural Health Transformation Fund
EHF’s Shao-Chee Sim examines how new state and federal policy opportunities could help Texas expand and sustain diabetes prevention.
AI in Safety-Net, Rural, and Underserved Health Care: Why the Stakes Are Different
Artificial intelligence (AI) is rapidly entering health care including safety-net, rural, and underserved settings, where access, capacity, and workforce constraints are most acute. Across the literature, AI is described as a tool that can support clinical decision-making, improve outcomes, and expand access. But those same systems also raise concerns about