S'pore study flags racial, cultural, gender biases in AI models

Landmark exercise in Asia-Pacific also finds culturally offensive replies from chatbots

- Osmond Chia

When asked which gender is most likely to be scammed online and which enclave is likely to have the most crime in Singapore, many artificial intelligence (AI) chatbots singled out women and large immigrant groups, respectively, in their answers.

Such inaccurate claims were just the tip of the iceberg, a landmark study of cultural biases in AI-powered large language models (LLMs) found.

These LLMs also spewed racially and culturally offensive answers when queried in English and eight Asian languages, including Hindi, Chinese and Malay.

The study is the first AI safety exercise in the Asia-Pacific that tests LLM biases related to culture, language, socio-economic status, gender, age and race.

It was conducted in late 2024 by the Infocomm Media Development Authority (IMDA), in partnership with international AI auditing firm Humane Intelligence.

The results were published on Feb 11 in the Singapore AI Safety Red Teaming Challenge Evaluation Report, meant to flag blind spots to urge developers to fix their models amid growing concerns about AI bias in hiring or credit approval systems, among other things.

In the report, IMDA said most AI testing today is Western-centric, focusing on vulnerabilities and biases relevant to the regions of North America and Western Europe.

"As AI is increasingly adopted by the rest of the world, it is essential that models reflect regional concerns with sensitivity and accuracy," said IMDA.

Four LLMs were tested in the study after the companies behind them responded to an open call. They are: Meta's Llama 3, Amazon Web Services-backed Anthropic's Claude 3.5, Aya - a model by research lab Cohere for AI - and AI Singapore's regionally tailored Sea-Lion.

Notably, OpenAI's ChatGPT and Google's Gemini were not among the list.

Of the 5,313 answers generated by the four AI models, more than half were verified as biased, according to the report.