The Department for Work and Pensions (DWP) has been using AI to determine the eligibility of applications for disability benefits. These algorithmic programmes have already demonstrated bias on individual factors like age, disability and nationality, with errors leading to thousands of wrongful investigations for benefit fraud. The first version of this AI tool, used by DWP from 2020 to 2024, only achieved a 35% correct match rate, with 65% of cases having to be corrected by a human agent.
Big Brother Watch have warned about the use of AI in the benefits system, legal and policy officer, Jasleen Chaggar, commenting:
“[The DWP] acknowledges the risk of its agents blindly following automated decisions on welfare eligibility.
“Without a human backstop, thousands of claimants could have been wrongfully declared ineligible due to algorithmic error – and the secrecy of the system leaves them powerless to challenge it.
“Covert use of this highly inaccurate automated tool between 2020 and 2024 highlights the disquieting feeling that members of the public are being used as guinea pigs to test unreliable algorithms with life-altering consequences.
“At the same time as the government is rolling out automated systems across public services, it is stripping away the protections that safeguard us from the errors, bias and discriminatory outcomes that invariably accompany them.
“The government should reconsider its plans to water down protections on automated decision-making and fulfil its promise to publish information about the algorithms it is already using.”
Big Issue – Warning after DWP reveals how it uses AI to make decisions on benefits