"When we need medical care, we expect decisions about our treatment to be made by our doctors, not a tech company's algorithm," writes the EFF newsletter EFFector.
But in January America's federal government launched a program called WISeR that uses AI to evaluate authorization requests for certain procedures from America's federal health insurance program for seniors, Medicare. EFF sued for records and discovered "additional issues" with the program that they say "could jeopardize Medicare beneficiaries' access to care."
The records we received revealed that WISeR has resulted in widespread delays and denials of care, operational chaos, and reports of patient harm. Further documents obtained by EFF show a rushed and rocky rollout, highlighting the dangers that arise when governments treat AI like a silver bullet for complex problems.
EFF has now published hundreds of pages of documents obtained through our lawsuit, available here... The documents also highlight the need for more transparency about safeguards against inappropriate, AI-driven denials. EFF has long argued that AI has no place in consequential government decisions about peoples' rights or access to benefits, such as immigration enforcement and policing decisions. That's because using automated decision-making in these contexts risks decisions that violate people's rights or deprive them of benefits they're entitled to based on biased or inappropriate considerations and without proper due process or accountability if there is an error. By outsourcing decision-making to private AI companies without sufficient transparency, testing, or legal limits, WISeR jeopardizes seniors' access to medical care. Congress should step in to place greater legal limits on the use of automated decision-making in Medicare.
The new documents "show why we need stronger legal limits around government use of automated decision-making..." according to the newsletter. And another EFF page argues that the American public "still lacks the information needed to assess whether clinician review is providing meaningful oversight of AI systems and whether appropriate data is being used to evaluate prior authorization requests."
WISeR's payment structure gives vendors, as the government put it, "an incentive to deny as many claims as possible...." [One company] denied more requests than it approved in the first few months of the program, and CMS inquired about "denials that appear inaccurate or inconsistent with applicable criteria." Ultimately, the records obtained by EFF raise more questions than they answer about the safeguards used by WISeR vendors...
A growing body of scholarship points to the substantial limitations of "human in the loop" policies as guardrails for AI systems. AI-generated recommendations can unduly influence human decisions, leading reviewers to over-rely on automated judgements and rubber stamp them as correct. Even worse, at least one health insurance company actively pressured clinical reviewers to accept its AI systems' recommendations. Another company built a system that allowed clinical reviewers to reject claims without opening patient files; clinical reviewers using this system spent an average of just 1.2 seconds on each case..
As the government expands its use of automated decision-making, the potential for consequential mistakes increases rapidly. The records we've obtained so far raise questions about the rushed rollout and lack of guardrails for automated decision-making by the U.S. government. EFF urges transparency, oversight, and legal limits to protect people from the harms of automated decision-making in high risk systems.
Read more of this story at Slashdot.
16 minutes ago
1
