Who is responsible, fairness and bias, privacy, and how to talk with patients about AI
A tool can be accurate, tested and worth the money, and still raise hard questions. Who is at fault when it is wrong? Does it work equally well for every patient? Should patients be told it was used? This lesson covers the four questions you are most likely to meet. About 15-20 minutes.
By the end of this lesson you can:
Back to Mr. Diaz. Suppose the AI that drafts impressions had left out the nodule follow-up, the radiologist did not notice and signed the report, and the follow-up CT never happened. Who is responsible?
Today the short answer is: the clinician who signs. AI tools are treated as tools, and the radiologist who signs a report is responsible for everything in it, whoever drafted it. Others share responsibility for their own parts:
The law here is still developing, and it may change as AI tools do more on their own. For now, "the AI said so" is not a defense.
A model learns from its training data (Lesson 1.2). If some groups of patients are missing from that data, or if the data carry an unfairness from the past, the model learns that too. This is called algorithmic bias: a tool that works worse, or makes worse recommendations, for some groups of people.
A risk score that used cost as a stand-in for illness. A program used by US health systems to decide who got extra care predicted future health care costs. Because less money had historically been spent on Black patients with the same illnesses, the program rated Black patients as healthier than equally sick white patients, and fewer were offered extra care (Obermeyer et al., 2019).
Chest X-ray AI that missed more disease in some groups. Models trained on large chest X-ray datasets more often labeled sick patients as "no finding" if they were female, Black, Hispanic, younger, or on Medicaid insurance (Seyyed-Kalantari et al., 2021). Those patients would be the most likely to have a diagnosis delayed.
Questions to ask about any tool: Who was in the training data? Was performance reported separately for different groups, such as by sex, age and race? Has it been tested on patients like ours?
AI is built from patient data, and using it often means sending data somewhere. A few basics:
A classmate is about to paste this into a free public chatbot. Click every piece of information that could identify Mr. Diaz. There are five.
The findings on their own are safe to ask about. Even so, the safer habit is to put the whole report into a tool your hospital has approved, because a rare finding plus a date can sometimes be enough to identify someone.
Should patients be told when AI is used in their care? Views differ. Few patients are told about every piece of software involved in a scan. But most surveys find patients want to know when AI plays a real part in their diagnosis or in messages they receive, and they want a person to stay responsible. Some laws now require it: in California, since 2025, a message to a patient written by generative AI must say so, unless a licensed clinician has reviewed it.
At his clinic visit, Mr. Diaz asks: "I read that computers read scans now. Did a computer read mine? Should I trust it?" Write two or three sentences you could say, then compare with a sample answer.
Sample answer: "Our hospital uses a computer program that helps flag scans that may be urgent, so yours was looked at quickly. But a radiologist, a doctor who specializes in reading scans, looked at every image and wrote your report, and they are responsible for it. The program is a helper, like a second set of eyes. I'm happy to go over what the report found."
A good answer is honest, short and free of jargon, makes clear that a person is responsible, and brings the conversation back to what matters to him: his results and what happens next.
Each situation mainly raises one of the four issues from this lesson. Pick the best fit.
Answer each question to continue. Any answer is fine; read the explanation.
An AI drafts a radiology impression with an error, and the radiologist signs it. Today, who is mainly responsible for the report?
The clinician who signs is responsible for the content, whoever drafted it. The hospital and developer have their own duties, but they do not replace the signer's.
Why did the health-care risk score in the Obermeyer study treat Black patients unfairly?
The target the model learned, cost, carried a past unfairness. Bias can enter through what a model is asked to predict, not only through who is in the data.
A patient asks if AI read their scan. Which reply is best?
Be honest, keep it simple, and make clear that a person is responsible. Dismissing the question or overstating the AI both damage trust.