Intentar ORO - Gratis
Benchmarks in medicine: the promise and pitfalls of evaluating AI tools with mismatched yardsticks
Indian Chronicle
|June 21, 2025
In May 2024, OpenAl released HealthBench, a new benchmarking system to test the clinical capabilities of large language models (LLMs) such as ChatGPT. On the surface, this may sound like yet another technical update.
-

But for the medical world, it marked an important moment—a quiet acknowledgement that our current ways of evaluating medical Al are fundamentally wrong.Headlines in the recent past have trumpeted that Al “outperforms doctors” or “aces medical exams.” The impression that’s coming through is these models are smarter, faster, and perhaps even safer. But this hype masks a deeper truth. To put it plainly, the benchmarks used to arrive at these claims are based on exams built for evaluating human memory retention from classroom teachings. They reward fact recall, not clinical judgment.
Acalculator can multiply two six-digit numbers within seconds. Impressive, no doubt. But does this mean calculators are better than, and understand maths more than mathematics experts ? Or better even than an ordinary person who takes a few minutes to do the calculation with a pen and paper?Language models are celebrated because they can churn out textbook-style answers to MCQs and fillin the blanks for medical facts and questions faster than medical professors. But the practice of medicine is not a quiz. Real doctors deal with ambiguity, emotion, and decision-making under uncertainty. They listen, observe, and adapt.The irony is that while Al beats doctors in answering questions, it still struggles to generate the very case vignettes that form the basis of those questions. Writing a good clinical scenario from real patients in clinical practice requires understanding human suffering, filtering irrelevant details, and framing the diagnostic dilemma with context. So far, that remains a deeply human ability.
Esta historia es de la edición June 21, 2025 de Indian Chronicle.
Suscríbete a Magzter GOLD para acceder a miles de historias premium seleccionadas y a más de 9000 revistas y periódicos.
¿Ya eres suscriptor? Iniciar sesión
MÁS HISTORIAS DE Indian Chronicle
Indian Chronicle
Jubilee Hills bypoll: Voters can cast their vote using 12 valid photo Id documents
In the Jubilee Hills Assembly bye-election to be held on November 11, voters can exercise their franchise using any of the 12 valid photo identity proof.
1 min
October 11, 2025

Indian Chronicle
Mamata vs BJP: Suvendu seeks Central security for Bengal electoral official, cites CM's 'threat' over SIR
Suvendu, Mamata vs BJP over SIRThe BJP and Mamata Banerjee are locked in a war of words over the conduct of SIR exercise in Bengal (Photos: FB pages of leaders).
1 mins
October 11, 2025

Indian Chronicle
Major setback to Congress Govt as Telangana HC stays local body polls
In a major setback to the Congress government, the Telangana High Court on Thursday temporarily stayed the process of local body elections across the State after taking strong note of petitions challenging the enhancement of Backward Classes (BC) reservation from 23 per cent to 42 per cent through Government Order (GO) No. 9, issued on September 26.The State government had released a notification on September 29 announcing the schedule for conducting local body elections in five phases across the State.
1 mins
October 10, 2025

Indian Chronicle
US pharma giant Eli Lilly to invest over $1 billion in Telangana
Telangana secured a massive investment of $1 Billion by US Pharma major Eli Lilly, which will expand its manufacturing and global medicine supply capacity in Hyderabad.
2 mins
October 07, 2025

Indian Chronicle
PM Modi greets citizens on occasion of 'Sharad Purnima'
Prime Minister Narendra Modi on Monday conveyed wishes to citizens across the country on the auspicious occasion of ‘Sharad Purnima’.In a post in Hindi on X, PM Modi said, “Heartfelt wishes of Sharad Purnima to all my family members across the country.
1 mins
October 07, 2025

Indian Chronicle
'India stands with people of Nepal': PM condoles loss of lives due to heavy rainfall, offers support
Prime Minister Narendra Modi on Sunday expressed grief over the loss of lives and damage caused by heavy rainfall in Nepal. He stated that India stands with the people and Government of Nepal in this difficult time.\"The loss of lives and damage caused by heavy rains in Nepal is distressing. We stand with the people and the Government of Nepal in this difficult time. Asa friendly neighbour and first responder, India remains committed to providing any assistance that may be required,\" PM Modi posted on X.
2 mins
October 07, 2025
Indian Chronicle
PM to launch Rs 62,000 crore youth and education push with focus on Bihar tomorrow
New Delhi: Prime Minister Narendra Modi will unveil various youth-focused initiatives worth more than Rs 62,000 crore on Saturday, his office said, billing the exercise as a landmark initiative for youth development which will give a decisive push to education, skilling, and entrepreneurship. The PMO said Modi will launch PM-SETU (Pradhan Mantri Skilling and Employability Transformation through Upgraded ITIs), a centrally sponsored scheme with an investment of Rs 60,000 crore.
1 mins
October 04, 2025
Indian Chronicle
Banni Festival Clash in Devaragattu Leaves Two Dead and Over 100 Injured
In a tragic turn of events, the Banni festival in Devaragattu, Kurnool district, turned deadly on Thursday night as devotees clashed over the transportation of deity idols.
1 min
October 04, 2025
Indian Chronicle
India stands in solidarity with Philippines after deadly earthquake: PM Modi
Prime Minister Narendra Modi on Wednesday expressed sadness over the loss of lives and damage caused by the earthquake in the Philippines.
1 min
October 02, 2025
Indian Chronicle
Establishment of 57 new Kendriya Vidyalayas is ‘landmark step': PM Modi
New Delhi: Prime Minister Narender Modi on Wednesday termed the establishment of 57 new Kendriya Vidyalayas “a landmark step in expanding access to quality education!”
1 mins
October 02, 2025
Listen
Translate
Change font size