Politics

DWP should never use AI to suspend benefit claims, experts insist: 'People are still struggling'

Artificial Intelligence should never have be used to automatically suspend welfare payments, experts have warned

A miniature figurine businessman looking down on an information booklet for department for work and pensions

The DWP faces criticism from campaigners. Image: Alamy

Artificial Intelligence (AI) should never have be used to automatically suspend universal credit payments, experts have warned, welcoming a “sensible” government decision to roll back the policy.

With the postmaster scandal dominating the news, public concern over the faulty automation of public services is mounting.

Meanwhile, the Department of Work and Pensions has quietly rolled back the influence of its AI-powered fraud detector. Previously, the DWP automatically suspended payments to accounts the bot flagged as ‘suspicious’. This policy has now been dropped, a senior civil servant has revealed.

“We actually changed our approach in the light of feedback from claimants and elected representatives,” DWP official Neil Couling told a select committee of MPs on Monday (22 January). “We used to suspend all the cases and now we don’t suspend.”

The decision has been welcomed by experts and campaigners who nonetheless urged the DWP to do more to protect vulnerable people.

The use of AI must not come at the expense of “accessibility, transparency, and accountability” in the benefits system, said Tom Pollard, head of social policy at the New Economics Foundation.

“While it’s clearly important to try to detect and prevent genuine fraud, this needs to be balanced against the risk of people being unfairly denied vital financial support,” he said.

“It’s therefore positive to hear that the DWP is no longer routinely suspending claims flagged by their AI-powered fraud detection. However too many people are still struggling to access the support they need, while feeling under constant suspicion by the DWP.”

There are already mistakes in the welfare system. People are sometimes wrongly refused benefits, and some have been incorrectly sanctioned. The Big Issue reported some of these cases, including a blind woman who had her disability benefits wrongly slashed and a woman who was being falsely accused of owing the DWP more than £12,000.

Adding AI into the mix could compound these risks, warned Steve Kuncewicz, a lawyer specialising in data and privacy.

“[This decision] seems to be a significant recognition of the risk that can come about from AI being rolled out without very careful diligence,” he said.

The technology could ultimately be useful for detecting fraud, he said – but only after careful consideration of the risks.

A lack of human contact can also create problems for people attempting to access financial help, warned Michael Clarke, head of information programmes at Turn2us.

The decision to stop automatic suspension is a “positive step”, he said, but added that “concerns remain about the pressure on claimants to quickly engage with investigators.”

“We urge the DWP to make this change permanent and to continue improving the system with transparent, accountable practices and essential human oversight,” he concluded.

How does the DWP use AI?

The department has not been clear on just how much it uses AI. It has previously declined to publish the results of an equalities assessment around using machine learning technology.

The roll-back of the automatic suspension policy doesn’t mean the end of AI in the DWP. In December, the department purchased an automated ‘social media listening’ tool that will be able to search claimants’ profiles for specified terms and flag up thousands of individuals’ posts each day.

But the Horizon postmaster’s scandal has brought increasing attention to the risks of automation. Between 2000 and 2014, more than 700 post office branch managers across the UK were prosecuted on the basis of a faulty IT system called Horizon, which made it look like innocent postmasters had stolen money.

While not specifically involving AI, the scandal is a testament to the importance of human oversight, said Kuncewicz.

“Horizon led to very negative results – and that was a system with a fair amount of human intervention. It sounds like this [the AI system] may have had even less,” he said.

“So it’s good they’re pausing it, going away, and hopefully properly assessing what the risks might be.”

When asked if the technology could lead to a situation similar to the Horizon scandal, the DWP official Neil Couling said: “I really hope not.”

A DWP spokesperson said that humans have – and always have had – the final say in claims.

“The department continues to explore the potential of new technologies in combatting fraud but we have always been consistent that a member of staff will always make the final decision to determine fraud or error,” they said.

The rollback of AI is welcome, said Jamie Thunder, policy and public affairs officer at Z2K. But all suspensions can be devastating, regardless of whether they are issued by a human or an AI bot.

“We see people who have a universal credit claims suspended by the review team. And that suspension can go on for months, even while the claimant is providing the evidence that apparently they need to be providing,” he said.

“Because it’s a suspension, not a decision to close the claim, you can’t challenge that formally. Which really puts someone in a in a catch-22 position. If AI is part of that decision, we certainly have concerns. But even if it isn’t, this is a this is a really unfair way to run a system.”

Trust in the DWP is subsequently “quite low” – and the use of AI could further erode it, Thunder warned.

“The decisions that the department makes have a huge impact on every aspect of people’s lives. It affects their money, it can affect their housing, it can affect their relationships, ultimately, it can affect their health,” he added.

“Rolling out AI without transparency on how it works, adds to the fear that many people already have of the department. There’s already this sense that someone, in some office somewhere in Caxton House, is going to make a decision, and it could ruin my life.”

“AI really shouldn’t be used for something that has a direct negative effect on someone’s claim.”

Support the Big Issue

For over 30 years, the Big Issue has been committed to ending poverty in the UK. In 2024, our work is needed more than ever. Find out how you can support the Big Issue today.
Vendor martin Hawes

Recommended for you

View all
Mark Menzies scandal: How Tory sleaze may shape next UK general election as Sunak's party plummets
Mark Menzies
Politics

Mark Menzies scandal: How Tory sleaze may shape next UK general election as Sunak's party plummets

How Rishi Sunak's smoking ban became a fight between individual freedom and public health
Smoking ban

How Rishi Sunak's smoking ban became a fight between individual freedom and public health

Rishi Sunak is celebrating UK's inflation fall – there's just several problems
Inflation

Rishi Sunak is celebrating UK's inflation fall – there's just several problems

Liz Truss blames the 'deep state' and the 'establishment' for her downfall. Here's why she's wrong
liz truss
Politics

Liz Truss blames the 'deep state' and the 'establishment' for her downfall. Here's why she's wrong

Most Popular

Read All
Renters pay their landlords' buy-to-let mortgages, so they should get a share of the profits
Renters: A mortgage lender's window advertising buy-to-let products
1.

Renters pay their landlords' buy-to-let mortgages, so they should get a share of the profits

Exclusive: Disabled people are 'set up to fail' by the DWP in target-driven disability benefits system, whistleblowers reveal
Pound coins on a piece of paper with disability living allowancve
2.

Exclusive: Disabled people are 'set up to fail' by the DWP in target-driven disability benefits system, whistleblowers reveal

Cost of living payment 2024: Where to get help now the scheme is over
next dwp cost of living payment 2023
3.

Cost of living payment 2024: Where to get help now the scheme is over

Strike dates 2023: From train drivers to NHS doctors, here are the dates to know
4.

Strike dates 2023: From train drivers to NHS doctors, here are the dates to know