AI on the Front Lines

AI progress can stall when end users resist adoption. Developers must think beyond a project's business benefits and ensure that end users' workflow concerns are addressed.

- By Katherine C. Kellogg, Mark Sendak, and Suresh Balu

It’s 10 a.m. on a Monday, and Aman, one of the developers of a new artificial intelligence tool, is excited about the technology launching that day. Leaders of Duke University Hospital’s intensive care unit had asked Aman and his colleagues to develop an AI tool to help prevent overcrowding in their unit. Research had shown that patients coming to the hospital with a particular type of heart attack did not require hospitalization in the c and its leaders hoped that an AI tool would help emergency room clinicians identify these patients and refer them to noncritical care. This would both improve quality of care for patients and reduce unnecessary costs.

A man and his team of cardiologists, data scientists, computer scientists, and project managers had developed an AI tool that made it easy for clinicians to identify these patients. It also inserted language into the patients’ electronic medical records to explain why they did not need to be transferred to the ICU. Finally, after a year of work, the tool was ready for action.

Fast-forward three weeks. The launch of the tool had failed. One ER doctor’s comment that “we don’t need a tool to tell us how to do our job” is typical of front-line employees’ reactions to the introduction of AI decision support tools. Busy clinicians in the fast-paced ER environment objected to the extra work of inputting data into a system outside of their regular workflow — and they resented the intrusion on their domain of expertise by outsiders who they felt had little understanding of ER operations.