Councils using AI tools that 'downplay women's health'

Artificial intelligence tools used by more than half of England's councils are playing down women's physical and mental health issues and risk creating gender bias in care decisions, research has found.

- Jessica Murray

The study found that when using Google's AI tool "Gemma" to generate and summarise the same case notes, language such as "disabled", "unable" and "complex" appeared significantly more often in descriptions of men than women.

The study, by the London School of Economics and Political Science (LSE), also found that similar care needs in women were more likely to be omitted or described in less serious terms. Dr Sam Rickman, the lead author of the report and a researcher at LSE's Care Policy and Evaluation Centre, said AI could result in "unequal care provision for women".

"We know these models are being used very widely and what's concerning is that we found very meaningful differences between measures of bias in different models," he said. "Google's model, in particular, downplays women's physical and mental health needs in comparison to men's.