Treat clinical AI like a medical student, not a staff physician

It’s close to 5 p.m., and I’m reviewing the last ambient AI note of my clinic day. This morning, a patient told me her mother had been treated for breast cancer. The draft says the patient herself has a history of breast cancer. She doesn’t. The AI caught the diagnosis but attached it to the wrong person, and the rest of the note is accurate enough that it looks ready to sign.

I see the error because I read every line, which is slow work after a full day of patients. Signed as written, the note would record her as a breast cancer survivor, and every clinician who opens her chart after me would see it, along with her health plan.

Ambient scribes and other AI tools are moving into exam rooms quickly, and health systems are deciding how much to trust them whether they mean to or not. Each note a physician signs without a close read gives the tool a little more independence than anyone formally granted it.

Medicine knows how to supervise a trainee

Medical education has a defined system for deciding how much responsibility a new clinician gets. Educators break clinical work into specific tasks and rate trainees on each one using a five-level supervision scale. A trainee starts by observing. Next comes work with a supervisor in the room, then with one close by, then alone, and eventually supervising others.

I use the same progression when I introduce AI into a health system. A new model starts as a medical student, and a physician checks all its work. With feedback and better data, it advances to resident level and drafts notes and plans for a physician to review. On narrow, well-defined tasks, some tools perform like fellows and need only occasional checks. But at every stage, a physician remains accountable for the result.

The medical student stage

Medical students are thorough. They present every lab value and every symptom the patient mentioned, and the attending physician decides what matters. A new AI tool works the same way, and the physician has to read the draft note to find what belongs in the record.

The errors are common enough to require a full read. In one pilot at UC Davis Health, physicians who rated their own ambient AI notes found omissions in 18% and hallucinations in 11.5%. They also left 14.9% of AI-drafted notes entirely unedited. Few attendings would sign a medical student’s note that way.

When I evaluate a tool, I look at how much scrolling and clicking it adds, and whether a physician’s decisions at 3 p.m. are as sound as at 8 a.m. Decision quality already drops over the day. In one study, breast cancer screening orders for eligible patients fell from 63.7% at 8 a.m. to 47.8% at 5 p.m. A tool that adds reading time makes those late-afternoon visits riskier.

Promotion happens one task at a time

The Association of American Medical Colleges defines residency readiness as 13 specific tasks, and educators clear trainees for each one separately. A resident can run a routine clinic visit alone long before anyone lets her manage an ICU patient without supervision. AI should earn independence the same way. One tool can work at fellow level on discharge summaries and at medical student level on prior authorization, with closer review on the second.

I start with low-risk tasks. Say a tool flags 100 patients it predicts will have a high hemoglobin A1C, and their physicians order a test. If 98 come back high, the tool is performing well. If only 75 do, the other 25 patients simply had a routine blood test. Physicians see how AI performs in their own workflow before any tool handles work that could delay care.

An error in an automated prior authorization can delay a patient’s treatment, so a physician reviews every request until the model’s accuracy on that task has been measured.

Setting matters as well. A resident cleared for complex deliveries at a general hospital may need supervision again after moving to a tertiary center with sicker patients. A tool validated elsewhere should prove itself again on this system’s patients and workflows.

Every trainee needs an attending

Reviewing a draft is easier than writing one, which is part of the risk. Radiation oncologists who edited AI-drafted replies to simulated patient messages produced final answers that stayed closer to the AI’s drafts than to the replies they wrote themselves. About 7% of the unedited drafts, if sent, risked severe harm to the patient.

An attending physician who reviews a resident’s notes corrects them and expects the next set to be better. AI needs the same attention. A model improves as physicians fix its errors and those corrections go back into the system. That happens only when a health system names the physicians who will do the review and protects their time for it.

In my experience, a deployment takes 90 to 120 days of studying how a department works before the tool goes live. Leaders under pressure to show results may find that slow. On the other hand, teams that skip that work often build a tool their physicians stop using.

A physician still signs the note

Some of my colleagues expect AI to replace physicians. A tool can move through these stages much faster than a human trainee, and on narrow tasks it may eventually work with very little review. A model can pull a family history from a transcript. Deciding what that history means for a patient’s next screening still takes a physician who knows her.

Every trainee in medicine works under someone who answers for the outcome. Clinical AI should, too.

Niki Panich, M.D., is a practicing family physician and health care innovator who bridges clinical medicine, data science and business strategy to transform how care is delivered. As chief medical officer of Penguin Ai, she leads clinical governance, ethical AI design and real-world validation, ensuring the company’s AI Workers reflect the realities of modern health care.

A Stanford-trained physician and executive, Panich holds master’s degrees in data science and in management (MSx) from Stanford University. She previously served as chief medical officer of XO Health, where she led the development of AI-enabled care models and value-based payment strategies, and as corporate medical director for the Cleveland Clinic. She also advises innovation networks including The 51, AltaML and Creative Destruction Lab.

Panich is passionate about shaping a future where AI augments clinical judgment rather than replacing it, turning complex data into compassionate, intelligent care.