AMA adopts new policies to keep AI under physician oversight

Physicians and medical students at the American Medical Association House of Delegates annual meeting in Chicago adopted new policies designed to ensure artificial intelligence strengthens patient care and supports evidence-based medicine while remaining under physician oversight rather than replacing physician judgment.

"AI has enormous potential in health care, but it cannot replace physician judgment," John Whyte, MD, MPH, CEO of the AMA, said in a statement. "Patients deserve care decisions that are informed by the latest medical evidence and guided by a physician who understands their individual needs. Whether AI is helping a physician make a clinical decision or assisting with an insurance review, there must always be transparency, accountability and meaningful physician oversight. Technology should support better care, not stand between patients and the care they need."

The policies address the growing use of AI in both clinical decision support and health insurance coverage determinations, establishing the principle that AI should serve as an assistive tool rather than an autonomous decision-maker.

The action comes as physician adoption of AI accelerates. Physician use of AI tools has more than doubled in three years, according to survey data from the AMA Center for Digital Health and AI, with clinicians increasingly leaning on the technology for documentation and clinical decision-making even as practice policies lag behind.

New guardrails for payer AI

Recognizing the increasing use of AI by health plans and third-party payers, delegates adopted policy calling for safeguards to ensure coverage decisions are based on accurate, up-to-date, evidence-based medical information and are reviewed by physicians with appropriate expertise.

The new policy opposes the use of autonomous or semiautonomous AI systems as substitutes for physician review in coverage determinations and calls for regulations requiring AI-enabled technologies to be integrated into physician-led processes. The AMA will also advocate for greater transparency when AI is used in prior authorization and other utilization management decisions, including disclosure of the clinical logic, data sources and guidelines used to reach adverse determinations.

Physician frustration with payer AI is well documented. A recent AMA survey found 61% of physicians are concerned that AI-driven prior authorization decisions are leading to more denials, and 75% reported that prior authorization denials have increased over the past five years.

"When health plans use AI-driven tools to deny or delay care without explaining how those decisions were reached, physicians and patients are left in the dark," Whyte said. "AI should never function as an unaccountable black box. Health plans must be transparent about how these tools work, what evidence and data sources they rely on, and whether a qualified physician reviewed the decision."

To strengthen accountability, the AMA will advocate for regular audits of AI-driven clinical review tools, including audits triggered by significant changes to AI models, training data or clinical guidelines, as well as comprehensive annual reviews to ensure continued alignment with current standards of care.

Standards for clinical decision support

Delegates also adopted policy aimed at ensuring AI clinical decision support tools reflect the principles of evidence-based medicine and give physicians information they can understand, evaluate and trust. Under the policy, the AMA will work with medical specialty societies, regulators, AI developers and other stakeholders to establish standards for evidence attribution, evaluation, validation, transparency and explainability in AI-enabled clinical decision support systems.

While AI-enabled tools may improve efficiency and help synthesize large amounts of information, the AMA notes that important concerns remain around transparency, bias, explainability and the long-term impact on physician practice and patient outcomes.

The AMA will continue its work through the AMA Center for Digital Health and AI to ensure physicians help shape how AI is developed, implemented and regulated across the health care system.