AI malfunction highlights need for health care safeguards

© ipuwadol - stock.adobe.com

For those who have read what I write or attended one of my presentations on generative artificial intelligence (GenAI), you are familiar with the White House issued Blueprint for an AI Bill of Rights, which sets forth five (5) items to ensure that the GenAI application is legal, ethical, and safe. Those five principles are: (1) safe and effective systems; (2) algorithmic discrimination protections; (3) data privacy; (4) notice and explanation; and (5) human alternatives, consideration, and fallback. These should be somewhat intuitive and the last principle should translate into “trust but verify.”

A.M.C., by her next friend, C.D.C., et al. v. Stephen Smith, in his official capacity as Deputy Commissioner of Finance and Administration and Director of the Division of TennCare, Case No. 3:20-cv-00240 (M.D. Tenn. 2020) is illustrative of what can happen when the principles set forth in the AI Bill of Rights are not followed. A class action alleges that thousands of Tennesseans lost their healthcare coverage under TennCare – Tennessee’s state-administered Medicaid program because of the 2019 GenAI implemented Eligibility Determination System (TEDS) and the related inadequate policies and procedures.

The bottom of page 1/top of page 2 of the Judge’s Memorandum Opinion is illustrative. “After years of litigation, Plaintiffs have proven TennCare violated their rights under the Medicaid Act, the Due Process Clause of the Fourteenth Amendment, and the Americans with Disabilities Act.” (emphasis added). In addition to TennCare recipients, a subclass “Disability Subclass” separately allege that TennCare “violated and is continuing to violate the Americans with Disabilities Act, (“ADA”) 42 U.S.C. § 12132, through policies and processes for TennCare’s administration of the redetermination of members’ eligibility and the termination of their coverage.”

Built by Deloitte and other contractors, the TennCare Connect system which includes TEDS, cost more than $400 million. Its purpose – to analyze income and health information to automatically determine eligibility for benefits program applicants. But in practice, the system often doesn’t load the appropriate data, assigns beneficiaries to the wrong households, and makes incorrect eligibility determinations. Said another way, it violates the Health Insurance Portability and Accountability Act of 1996 (HIPAA), has an skewed algorithm that generates results, which have a discriminatory impact, and did not have an adequate “check” done by human beings.

The TennCare Connect system was implemented to adhere to new eligibility criteria and streamlined enrollment prescribed by the Affordable Care Act to provide a single application process that would collect information and determine the programs for which individuals were eligible. Deliotte was arguably the biggest financial beneficiary, who won contracts in 20 states to build systems similar to TennCare Connect. One of those states was Texas. In the Matter of Deloitte Consulting LLP, a limited liability partnership, also d/b/a Deloitte, is a Complaint and request for injunctive and other equitable relief sought by the National Health Law Program, Electronic Privacy Information Center (EPIC) and Upturn, Inc. for Texans.

In sum, these matters underscore the importance of having the right safeguards in place and continuing to monitor the five principles both when adopting and continuing to use GenAI.

Rachel V. Rose, JD, MBA, advises clients on compliance, transactions, government administrative actions, and litigation involving healthcare, cybersecurity, corporate and securities law, as well as False Claims Act and Dodd-Frank whistleblower cases.