Uncertainty Made Measurable

Rob Selzer sizes up a human confidence interval.

- Rob Selzer

You, like most people, would probably prefer a doctor who’s competent. Perhaps one, I would imagine, who’s even good. One who has the skills to take a blood pressure measurement, diagnose a tumour, smile reassuringly when your self-diagnosed angina turns out to be simple indigestion. And like most people, you instantly know a good doctor when you see one. Measuring those doctoring skills, however, is harder than you think. It requires swarms of experts to be called into service. These experts assess future doctors to within an inch of their lives as the students journey from school uniform to white coat (although quite frankly, the last time I saw someone wearing a white coat, they were applying make-up in a department store).

I am one of those examiners. But something about the whole examination process has always bothered me. It’s not the measurement of the students’ knowledge; about that, I’m confident. Written examinations such as multiple choice questions give a fairly accurate gauge of a memory for facts – for details such as which antibiotics to prescribe for pneumonia, or the ECG changes in a heart attack. Likewise, discrete skills such as listening to heart sounds or tapping out a knee reflex are easily measured because they are objective, circumscribed, and have clear outcomes.

What bugs me, is the way that we experts measure complex skills used in messier real life situations, like taking a report of an illness from a distraught parent, or doing a physical exam on a frail pensioner. Such contexts are teeming with multiple, minute, but important two-way interactions, at the end of which an examiner is expected to give a grade.