On May 21, 2025, a federal judge in Florida refused to dismiss a wrongful death case against an AI chatbot company and let product liability claims go forward. The case, Garcia v. Character Technologies (M.D. Fla., No. 6:24-cv-01903), is the clearest US ruling so far on a question that Medical Affairs, Pharmacovigilance, Regulatory, and Brand teams now face: if an AI system contributes to patient harm, which party is legally responsible? [1]
No court has issued a final, appellate-level answer. The Garcia case settled in principle in January 2026, so the ruling stands as a trial-court decision without appeal [4][5]. Other cases are still open. This article reconstructs the record, calculates the timelines, sorts the parties into an original taxonomy, and identifies where the law is still undecided as of November 3, 2026.
Scope note: this article reviews public court records, regulatory documents, and published studies. It is not legal advice. It does not report new AI testing. Where it makes a calculation, it labels the figure as analysis.
The Short Answer: Who Is Responsible When AI Harms a Patient?
Responsibility depends on who built the system, who deployed it, who relied on it, and what that party said about its reliability. Current law places liability on different parties in different settings:
Developers face product liability and negligence claims when a chatbot output is treated as a product [1][2].
Clinicians are likely to bear liability for care decisions they make with AI tools, according to a Stanford researcher who reviewed malpractice cases involving software [34].
Health plans face contract and good-faith claims when an algorithm replaces clinical judgment in coverage decisions [10].
Vendors face consumer-protection enforcement when they overstate accuracy [27].
In the EU, the revised Product Liability Directive adds strict liability for software, including AI, for products placed on the market after December 9, 2026 [13].
The Five Findings That Matter
These findings are written so that each stands alone. Each has a source.
A US federal court allowed product liability claims against an AI chatbot maker to proceed on May 21, 2025, and declined at the motion-to-dismiss stage to treat the chatbot’s output as protected speech [1][2].
Character.AI and Google agreed in January 2026 to settle five lawsuits in Florida, Colorado, New York, and Texas. Terms were not disclosed, so the settlements created no appellate precedent [4][5].
A federal court in Minnesota let breach-of-contract and good-faith claims proceed in February 2025 against UnitedHealth over its alleged use of the nH Predict algorithm in post-acute coverage decisions [10][11].
The FDA’s revised clinical decision support guidance of January 2026 addresses software for health care professionals. Reviewers noted that it announced no new policy for consumer-facing health chatbots [16].
The EU Directive 2024/2853 classifies software, including AI, as a product under strict liability rules, and it applies to products placed on the market after December 9, 2026 [13][15].
A sixth finding concerns the evidence base for clinical AI. An external validation of the Epic Sepsis Model at Michigan Medicine found that the model missed 67% of patients with sepsis while generating alerts on 18% of all hospitalized patients [24]. That study did not address liability. It matters here because it shows how a widely deployed tool can perform below its developer’s reported figures, which is the fact pattern that liability disputes depend on.
What Counts as AI Patient Harm? Definitions Used in This Article
What is AI patient harm?
In this article, AI patient harm means physical, psychological, or financial injury to a patient in which an AI system’s output is alleged to be a contributing cause. The definition does not assume that the AI caused the injury. Causation is the contested question in every case reviewed below.
What is a general-purpose chatbot versus a clinical AI tool?
A general-purpose chatbot answers questions from the public on any subject. A clinical AI tool is built for a care setting, such as a sepsis alert inside an electronic health record, a documentation summarizer, or a coverage-review algorithm. The legal analysis differs because the clinical tool has a deployer (a hospital or insurer), a professional user, and a procurement contract. The chatbot often has none of these.
An original taxonomy: five responsibility pathways
The table below is an original framework from this analysis. It is not an established legal or industry classification. It sorts the cases and rules reviewed in this article by the party that carries the claimed responsibility.
Pathway
Responsible party
Typical legal theory
Example in the record
Source
A. Design and warning pathway
AI developer
Product defect, failure to warn, negligence
Garcia v. Character Technologies; Raine v. OpenAI
[1][3][6]
B. Deployment pathway
Hospital, health system, or payer that put the tool into use
Negligence, breach of contract, bad faith
Estate of Lokken v. UnitedHealth Group
[10][12]
C. Reliance pathway
Clinician who acts on, or ignores, the output
Medical malpractice
Analysis by Price, Gerke, and Cohen; juror study by Tobia and colleagues
[22][23]
D. Representation pathway
Vendor or platform that made accuracy or credential claims
Consumer protection, deceptive trade practices, licensing law
Texas AG and Pieces Technologies; California AB 489
[27][30]
E. Gap pathway
No clearly responsible party under current rules
None established
Consumer-facing health chatbots outside FDA’s CDS guidance
[16][18]
What Have Courts Decided? The Case Record Through November 2026
Garcia v. Character Technologies: is a chatbot a product?
Megan Garcia sued Character Technologies, its two founders, and Google after the death of her 14-year-old son, Sewell Setzer III. Her complaint pleaded wrongful death, strict product liability on defect and failure-to-warn theories, negligence, unjust enrichment, and other claims [3].
What the court held in May 2025
Judge Anne Conway denied the motion to dismiss on most claims on May 21, 2025 [1]. She did not accept at that stage that the chatbot’s output was speech, and commentary on the ruling reads it as treating the output as a product for product liability purposes [2]. The court allowed Character Technologies to assert the First Amendment rights of its users, but that did not end the case [1].
The holding is narrow. A motion to dismiss tests whether the complaint states a claim. It does not decide whether the claims are true.
Why Google stayed in the case
The court allowed claims against Google to proceed on a theory that Google could be liable as a component part manufacturer, based on the alleged level of its involvement in the language model. The court did not allow a direct claim against Alphabet, Google’s parent [1]. For deployers and platforms, the point is that a financial and technical relationship with an AI developer can bring a larger company into the case.
What the January 2026 settlement shows and does not show
Character Technologies and Google agreed in January 2026 to settle the Florida case and four others filed in Colorado, New York, and Texas. The settlement documents did not disclose terms, and judges still had to approve them [4][5]. A settlement shows that the defendants preferred to resolve the claims. It does not establish that the claims would have succeeded at trial. Character.AI had also barred users under 18 from open-ended chatbot conversations in late 2025 [5].
Raine v. OpenAI: what is alleged and what OpenAI says
The allegations
Matthew and Maria Raine sued OpenAI and its chief executive in San Francisco County Superior Court in August 2025 (Case No. CGC-25-628528) [8]. Reporting at the time described it as the first known wrongful death lawsuit against OpenAI. The complaint alleges that OpenAI rushed GPT-4o to market despite known safety issues [6]. The family’s claims include wrongful death, product liability, and negligence.
OpenAI’s response
OpenAI denied responsibility. Its November 2025 filing argued, among other points, that the teenager had violated the service’s terms of use [7]. That argument raises a question the sources I reviewed do not answer: whether a terms-of-use violation by a minor can limit a developer’s liability. No court decision on that point appears in the material reviewed here.
Coordination and status
A petition to coordinate several California cases against OpenAI was heard on January 30, 2026, and a coordination order dated February 3, 2026 lists Raine among the cases [8]. As of reporting in October 2026, Raine remained unresolved in San Francisco Superior Court [9]. I found no reported verdict or settlement. Readers should check the docket before citing the status.
Estate of Lokken v. UnitedHealth Group: when the algorithm is the coverage decision
Plaintiffs in a putative class action allege that UnitedHealth and its naviHealth unit used an AI tool, nH Predict, in place of physicians to make post-acute care coverage decisions for Medicare Advantage members [10][11].
Which claims survived
On February 13, 2025, the District of Minnesota allowed breach-of-contract and good-faith claims to proceed [10]. The plaintiffs’ theory is that plan materials promised clinician-made decisions. The plaintiffs also allege that UnitedHealth knew the tool’s output was unreliable because most appealed denials were reversed; the judge recited that allegation in the ruling, and it remains an allegation [12]. UnitedHealth disputes the plaintiffs’ account, and the first discovery fight centered on whether nH Predict was actually used in place of physicians for the plaintiffs’ claims [11].
What discovery covers
A magistrate judge later ordered broad discovery into the tool’s role in coverage decisions [11]. For payers, the lesson is procedural: a dispute about AI-driven denials can open internal documents on how the tool was designed, validated, and used.
Timeline and calculated intervals
The table lists documented events. The elapsed-day figures in the last column are calculations from the listed dates (analysis). Garcia was filed in October 2024; the exact filing day is not stated in the sources I reviewed, so I do not calculate from it.
Date
Event
Elapsed days from prior listed event (analysis)
Source
Nov 14, 2023
Lokken putative class action filed, D. Minn.
Not applicable
[32]
Feb 13, 2025
Lokken: court allows contract and good-faith claims to proceed
457 days after filing
[10]
May 21, 2025
Garcia: motion to dismiss denied in most part
Not applicable
[1]
Aug 26, 2025
Raine v. OpenAI filed, San Francisco Superior Court
Not applicable
[6][8]
Jan 6-7, 2026
Garcia and four related cases: settlement filings
231 days after the Garcia ruling (using Jan 7)
[4][5]
Feb 3, 2026
Coordination order lists Raine among OpenAI cases
Not applicable
[8]
Dec 9, 2026
EU Directive 2024/2853 applies to newly marketed products
36 days after November 3, 2026
[13]
Analysis: the Lokken court needed 457 days to reach its first major ruling on the pleadings. Garcia’s motion-to-dismiss ruling was followed by settlement filings in 231 days. Both intervals show that these cases move slowly relative to the release cycle of the AI systems involved.
Can a Physician Be Liable for Following or Ignoring an AI Tool?
Yes. A physician remains the accountable decision-maker under medical malpractice law, and the literature reviewed here does not identify a rule that moves that accountability to the software. The open question is how courts will treat a physician who departs from an AI recommendation.
What Price, Gerke, and Cohen concluded in 2019
W. Nicholson Price II, Sara Gerke, and I. Glenn Cohen published a JAMA viewpoint in November 2019 on physician liability for medical AI [22]. Their 2021 follow-up commentary restates the 2019 conclusion: under existing law, the safest course for physicians was to use AI as a confirmatory tool and to stay with the standard of care [36]. The 2019 piece also suggested the law could change so that rejecting a correct AI recommendation might create exposure [23].
What potential jurors said
A survey experiment by Tobia and colleagues, discussed in the Journal of Nuclear Medicine, tested how lay adults judge physician decisions. Physicians who followed standard advice faced considerably less liability than those who rejected it. Physicians who followed or rejected nonstandard AI advice were almost equally likely to be held liable when something went wrong [23]. The authors noted this differed from the more cautious doctrinal reading in the 2019 JAMA piece [23]. This is a jury-perception study, not a court outcome.
What Mello and Guha found in 2024
Michelle Mello and Neel Guha published “Understanding Liability Risk from Using Health Care Artificial Intelligence Tools” in the New England Journal of Medicine in January 2024 [20]. Their analysis covered more than 800 tort cases involving AI and conventional software in health care and other settings [21]. Guha, describing their review of malpractice cases involving software, said courts were treating AI similarly to traditional software, and that clinicians are likely to bear liability for patient care decisions regardless of what the tool recommends [34]. Mello advised health care providers to push back on software liability disclaimers and to bargain for terms of use that limit the purchaser’s risk [35].
Can a Hospital Be Liable for Deploying AI That Underperforms? The Epic Sepsis Model Evidence
I found no judgment against a hospital for deploying the Epic Sepsis Model. The study below is evidence about performance, and it is the type of evidence that would matter in a deployment-liability claim.
What the validation study measured
Wong and colleagues externally validated the Epic Sepsis Model across 38,455 hospitalizations at Michigan Medicine between December 2018 and October 2019 [24]. At the alert threshold used, the model had a sensitivity of 33%, a specificity of 83%, a positive predictive value of 12%, a negative predictive value of 95%, and an area under the curve of 0.63 [24]. The accompanying editorial notes that the developer and a partner health system had reported an area under the curve of 0.76 to 0.83 [24].
What the numbers mean in practice
The model did not detect sepsis in 67% of patients who had it, and it generated alerts on 18% of all hospitalized patients [24]. A positive predictive value of 12% means that about 12 of every 100 alerts were followed by sepsis. By calculation (analysis), roughly 88 of every 100 alerts were not.
Why the gap between reported and validated performance matters
The Michigan authors explained part of the gap by how sepsis onset was defined. The developer’s definition used the time a clinician intervened, so the model learned to predict sepsis that clinicians had already recognized [25]. A court assessing a deployment claim would ask what the hospital knew about this limitation and what it did about it. The study authors also warned that proprietary models can come with performance documents that are confidential and not peer reviewed [33].
Where hospital responsibility could arise
Mello and Guha recommend that organizations assess the liability risk of each AI tool separately rather than treat all AI alike, and that they negotiate the contract terms that allocate risk [20][35]. The Epic data show why local validation is a practical safeguard and a record that can be produced in a dispute.
What Does the FDA Regulate, and What Falls Outside Its Reach?
The FDA regulates software that meets the statutory definition of a medical device. Software that is clinical decision support for health care professionals and meets four statutory criteria is excluded from that definition [16]. Consumer-facing chatbots sit in a less defined space.
What did the FDA’s January 2026 clinical decision support guidance change?
The FDA issued a revised final guidance on clinical decision support software on January 6, 2026. It supersedes the 2022 guidance [16]. The revision extends enforcement discretion to some software that gives a health care professional a single recommendation, which the 2022 guidance had treated as device functionality [16]. A later version dated January 29, 2026 states that it supersedes the January 6 document, so readers should cite the current FDA posting and not a secondary summary [17].
The guidance covers software for health care professionals. Commentators noted that it announced no new or clarified policy for consumer-facing tools such as patient decision support, health chatbots, or symptom checkers, and that it does not address AI directly [16]. The guidance also does not address labeling requirements for decision support software disseminated by or on behalf of a drug or biological product sponsor [17]. That gap matters to pharmaceutical companies that offer AI-enabled tools to prescribers or patients.
Does the FDA regulate health chatbots that talk to patients?
The sources reviewed show no FDA policy that authorizes or clears a general-purpose chatbot for medical advice. The FDA’s Digital Health Advisory Committee met on November 6, 2025 on generative AI-enabled digital mental health devices. In the FDA’s summary of the meeting, an agency speaker said that while the FDA has authorized more than 1,200 AI-enabled medical devices, none yet involve generative AI for mental health conditions [18][19].
FDA summary of the November 6, 2025 Digital Health Advisory Committee meeting: more than 1,200 AI-enabled medical devices have been authorized, and none yet use generative AI for mental health conditions. [18]
Why regulatory status affects liability
Analysis: a product that the FDA authorizes as a device comes with premarket evidence, labeling, and postmarket monitoring duties. A chatbot outside that framework is left mainly to state tort law, consumer-protection law, and new state statutes. This is the logic behind pathway E in the taxonomy above, and it explains why the Garcia and Raine plaintiffs rely on product liability and negligence theories and not on a violation of an FDA requirement [3][6].
How Does the EU Product Liability Directive Change the Answer After December 9, 2026?
It adds a strict-liability route for injured people in the EU. Directive (EU) 2024/2853 was adopted on October 23, 2024, published in the Official Journal on November 18, 2024, and entered into force on December 8, 2024 [13][15]. Member States must transpose it by December 9, 2026, and it applies to products placed on the market or put into service after that date [13].
Is AI software a product under the new directive?
Yes. The directive redefines “product” to cover software, including AI systems, and it widens the range of liable economic operators [13]. Under the old regime, a plaintiff arguing that software was a product faced a classification fight. The new text removes that fight for products first placed on the market after the application date.
Which products stay under the old rules?
Products placed on the market before December 9, 2026 remain under the earlier Directive 85/374/EEC. A substantial modification or update after that date may bring a product within the new directive [14]. For a company that updates an AI model frequently, the practical question is which update counts as a new placing on the market. The sources reviewed do not settle that question.
How much time is left before the directive applies?
Analysis: from November 3, 2026, the directive’s application date of December 9, 2026 is 36 days away. A company with an AI-enabled drug-information tool or patient-support chatbot in the EU has little time to review documentation, contracts, and update procedures.
What Do State Laws Add? Illinois, California, and Texas
What does the Illinois WOPR Act prohibit?
Illinois enacted the Wellness and Oversight for Psychological Resources Act in August 2025. It prohibits using AI to provide mental health and therapeutic decision-making, while allowing AI for administrative and supplementary support to licensed behavioral health professionals. The Illinois Department of Financial and Professional Regulation can investigate violations and levy fines of up to $10,000 per violation [29]. The act does not apply to religious counseling, peer support, or self-help materials that do not claim to offer therapy [29].
What do California AB 489 and SB 243 require?
California AB 489, effective January 1, 2026, bans developers and deployers of AI tools from indicating or implying that the tool holds a license or certificate to practice a health care profession [30]. Enforcement sits with the relevant licensing board or enforcement agency [31]. California SB 243 sets requirements for companion chatbots. A plaintiff who succeeds under SB 243 can recover injunctive relief and damages equal to the greater of actual damages or $1,000 per violation, plus attorney’s fees [31]. Both laws are state-level; the sources reviewed do not describe a federal statute that displaces them.
What did the Texas attorney general settlement with Pieces Technologies establish?
In September 2024, the Texas attorney general announced a settlement with Pieces Technologies, a Dallas company whose generative AI drafts clinical summaries for hospital staff. The attorney general alleged that Pieces had advertised a “severe hallucination rate” of less than 1 per 100,000 and that the metric was likely inaccurate [27]. Pieces denied wrongdoing and disputed the attorney general’s description of the agreement [28]. The agreement runs five years and carried no monetary payment [37]. It requires Pieces to disclose how it defines and measures accuracy metrics, and to help hospital staff understand how far they should rely on the product [27][37].
What the Pieces settlement means for any vendor that publishes accuracy figures
An accuracy statistic is a representation. If a regulator later judges it unsupported, the vendor faces enforcement even if no patient was harmed. This is pathway D in the taxonomy. It applies to a hospital documentation tool and, by extension, to any company that markets an AI answer-accuracy figure.
AI Drug Information Cases: The Sodium Bromide Report and the Label Gap
This section addresses the pharmaceutical question directly. A drug-information error from a chatbot raises the same responsibility questions as the other cases, but the harm path runs through a patient who acts on a statement about a substance.
What happened in the Annals of Internal Medicine bromide case?
Three University of Washington physicians published a case report in Annals of Internal Medicine: Clinical Cases in August 2025. A 60-year-old man replaced table salt with sodium bromide for three months after consulting ChatGPT about alternatives to chloride. He developed paranoia and auditory and visual hallucinations and was placed on an involuntary psychiatric hold [26]. The physicians did not have his conversation logs. They asked ChatGPT 3.5 the same type of question themselves, and the response they received included bromide [26].
Who would be responsible in a case like the bromide report?
I found no lawsuit arising from this case. The following is analysis, not a legal conclusion. A claim would face an evidence problem first: without the patient’s own chat logs, a physician’s later test of a different model version does not show what the patient saw [26]. If logs existed, the analysis would turn on the theories already tested in Garcia: whether the chatbot is a product, whether a missing warning is a design or warning defect, and whether the developer owed a duty to the user [1][2]. A claim by an adult who also bought the compound online would also have to deal with the patient’s own conduct. No court in the record reviewed here has ruled on a drug-information claim against a general-purpose chatbot.
How can pharmaceutical teams document what AI systems say about their drugs?
Monitoring programs create a dated record. DrugChatter, for example, monitors how AI systems answer questions about specific drugs and compares the answers with prescribing information. A record of this kind is only useful if another person can reproduce it. The fields below are the minimum I would record for each tested answer. They follow the reproducibility requirements for AI monitoring: platform, model, prompt, date and time, whether web search was enabled, the label version used as the reference, and how discrepancies were classified.
An original classification of AI drug-information discrepancies
The table is an original framework from this analysis. It is not an industry standard. It gives teams a shared vocabulary for comparing an AI answer with a current label.
Category
Definition
Authoritative reference to check
Outdated information
The answer reflects a label, indication, or warning that has since changed
Current FDA prescribing information and approval history
Missing safety information
The answer omits a boxed warning, contraindication, or major warning that applies to the question
Prescribing information, safety communications
Dosing discrepancy
The stated dose, schedule, or adjustment differs from the label
Dosage and Administration section
Indication discrepancy
The answer states an indication that is not approved or omits one that is
Indications and Usage section
Unsupported claim
The answer makes a clinical claim that no cited or findable source supports
ClinicalTrials.gov, PubMed, label
Citation failure
The cited source does not exist or does not support the claim
The cited document itself
This article does not report counts for these categories, because it did not run an AI test. Any team that publishes counts should state the sample size, the date range, and how duplicate responses were treated, so that the figures can be checked.
Scenario Analysis: Who Is Most Likely to Be Named in a Claim?
The table is analysis. It maps scenarios to the parties and theories that the record reviewed above makes plausible. It does not predict outcomes and gives no probabilities.
Scenario
Parties likely to be named
Plausible theory
Evidence that would matter
Record support
Consumer acts on chatbot drug advice
AI developer
Product defect, failure to warn
Full chat logs, model version, warnings shown
[1][26]
Minor harmed after extended companion-chatbot use
Developer, platform operator, corporate partner
Product liability, negligence, state chatbot statute
Safety design, age controls, internal testing
[1][4][31]
Hospital alert model underperforms
Hospital, vendor, treating clinician
Negligence, malpractice
Local validation, alert thresholds, vendor contract
[24][34]
Payer algorithm drives coverage denial
Payer, algorithm vendor
Breach of contract, bad faith
Plan language, denial and reversal rates, staff targets
[10][11][12]
Vendor overstates accuracy
Vendor
Consumer-protection enforcement
Metric definitions, testing method
[27][28]
Clinician rejects a correct AI recommendation
Clinician, hospital
Malpractice
Standard of care, how the tool was presented to the clinician
[23][36]
What This Means for Medical Affairs, Pharmacovigilance, Regulatory Affairs, and Brand Teams
The sections below are analysis and interpretation. They draw on the record above and do not state legal requirements.
What this means for Medical Affairs and Medical Information
Medical Information teams answer questions from prescribers and patients, and AI systems now answer many of the same questions. A dated record of what AI systems say about a company’s products, compared with the current label, gives the team a factual basis to correct errors with the platform, to update its own content, and to brief colleagues. The Pieces settlement shows what happens when a company states accuracy without a documented method [27].
What this means for Pharmacovigilance
None of the sources reviewed sets a reporting rule for patient harm linked to a general-purpose chatbot. Treat this as an open question for each organization’s safety governance. In the bromide report, the AI link surfaced only because physicians took a history and asked about information sources [26]. Safety teams may wish to add an AI-source field to case intake forms so the question is asked consistently.
What this means for Regulatory Affairs
The FDA’s clinical decision support guidance does not address labeling for decision support software disseminated by or on behalf of a drug or biological product sponsor [17]. A sponsor that offers an AI-enabled assistant to prescribers or patients should confirm its regulatory status instead of relying on the absence of a rule. In the EU, the Product Liability Directive adds a December 9, 2026 date to the planning calendar [13].
What this means for Brand, Legal, and Corporate Communications teams
Brand teams that report AI visibility or AI accuracy figures are making representations. The Texas action shows that a regulator will test the definition and method behind a headline metric [27][37]. Legal teams should review any published accuracy figure, and any claim that an AI answer is label-aligned, against the underlying test records.
Methodology and Limitations
How was this analysis built?
I searched public sources on November 3, 2026 for court rulings, regulatory documents, statutes, and peer-reviewed or journal-published analyses on AI patient harm and liability. Where I could reach a primary document, I used it: the Lokken opinion, the California coordination order, the FDA advisory committee summary, and the Texas attorney general’s release [8][10][18][27]. For several items, including the Garcia ruling and the text of the EU directive, I relied on law-firm and news summaries plus the official citation, because I did not retrieve the full primary text [1][2][13][15]. The elapsed-day figures were calculated from the dates in the sources, counting calendar days. The Epic percentages are as reported in the cited editorial; the 88% figure is the complement of the reported 12% positive predictive value (analysis).
What are the limits of this analysis?
It ran no AI testing. It reports no count of AI answer errors.
Case status can change daily. Readers should check dockets before citing a status.
Settlement terms are confidential, so the Character.AI settlements show no outcome on the merits [4][5].
It covers the United States and the European Union. Other jurisdictions are outside scope.
It is not legal advice, and the scenario table does not predict how a court would rule.
Key Takeaways
A federal court in Florida let product liability claims against an AI chatbot developer proceed on May 21, 2025, and did not accept at the pleading stage that chatbot output is protected speech [1][2].
The Character.AI and Google settlements in January 2026 produced no appellate ruling and no public terms [4][5].
Raine v. OpenAI was unresolved in October 2026 in San Francisco Superior Court [9].
A Minnesota federal court let contract and good-faith claims proceed against UnitedHealth over an AI coverage tool [10].
The FDA’s 2026 clinical decision support guidance covers software for health care professionals and announced no new policy for consumer health chatbots [16].
The EU Product Liability Directive applies to software, including AI, placed on the market after December 9, 2026, which is 36 days from November 3, 2026 (analysis) [13].
The Epic Sepsis Model missed 67% of sepsis cases and alerted on 18% of hospitalized patients in one external validation [24].
No source reviewed shows a court decision on a drug-information claim against a general-purpose chatbot.
FAQ
Can you sue an AI company if a chatbot gives harmful medical advice?
You can file a claim, and one federal court has allowed product liability claims against a chatbot developer to proceed past a motion to dismiss [1]. That ruling did not decide the merits, and the case settled [4]. Outcomes on medical-advice claims against general-purpose chatbots remain untested in the record I reviewed.
Is a chatbot legally a product or a service?
In Garcia, the court allowed product liability claims to proceed, and commentary reads the ruling as treating the chatbot as a product for those claims [1][2]. The ruling is a trial-court decision and does not bind other courts. The EU directive removes the question for software in its scope by defining software as a product [13].
Does the First Amendment protect chatbot output?
The Garcia court declined at the motion-to-dismiss stage to treat the chatbot’s output as protected speech, although it allowed the developer to assert users’ rights to receive speech [1]. No appellate court has resolved the question in the sources reviewed.
Can a doctor be held liable for following or ignoring an AI recommendation?
Yes. A Stanford researcher who reviewed malpractice cases involving software said clinicians are likely to bear liability for care decisions regardless of what the tool recommends [34]. In a juror experiment, physicians who followed or rejected nonstandard AI advice were almost equally likely to be held liable if something went wrong [23].
Does the FDA regulate ChatGPT-style health advice?
The sources reviewed show no FDA authorization for a generative AI chatbot for mental health conditions, and the 2026 clinical decision support guidance announced no new policy for consumer-facing chatbots [16][18].
What changes in the EU on December 9, 2026?
Directive (EU) 2024/2853 applies to products placed on the market or put into service after that date, and it treats software, including AI, as a product [13]. Products placed on the market earlier stay under the older directive, with an exception for substantial modifications [14].
Are insurers liable when an algorithm drives a coverage denial?
A federal court in Minnesota allowed breach-of-contract and good-faith claims against UnitedHealth to proceed in February 2025 over the alleged use of nH Predict [10]. UnitedHealth disputes the allegations, and the case remains in discovery [11].
Can an AI system tell users it is a licensed professional?
In California, no. AB 489, effective January 1, 2026, bans developers and deployers from indicating or implying that an AI tool holds a health care license [30]. Nevada and Oregon have passed similar laws [30].
Does a disclaimer protect an AI developer from liability?
The record reviewed shows no court decision on that point. OpenAI argued in the Raine case that the teenager violated its terms of use [7]. A Stanford researcher advised providers to push back on software liability disclaimers in their own contracts [35].
How should a pharmaceutical company track what AI systems say about its drugs?
Record the platform, model, prompt, date and time, whether web search was on, and the label version used as the reference, and classify each discrepancy with a stable scheme such as the one in this article. DrugChatter provides AI answer monitoring of this type. A company that publishes any accuracy figure should define the metric and the method, because the Texas Pieces settlement turned on those definitions [27][37].
References
[1] Bloomberg Government. (2025, May 21). Google, chatbot maker to face bulk of suit over teen suicide (Garcia v. Character Technologies, Inc., No. 6:24-cv-01903, M.D. Fla.). https://news.bgov.com/ip-law/chatbot-maker-to-face-bulk-of-mothers-suit-after-teens-suicide
[2] Goldberg Segalla. (n.d.). Peter Gregory authors article on ramifications of major federal AI ruling. https://www.goldbergsegalla.com/news-and-knowledge/news/peter-gregory-authors-article-on-ramifications-of-major-federal-ai-ruling/
[3] Tech Justice Law Project. (n.d.). Garcia v. Character Technologies, Google, and Character AI co-founders. https://techjusticelaw.org/cases/garcia-v-character-technologies-google-and-character-ai-co-founders-daniel-de-frietas-and-noam-shazeer/
[4] Associated Press. (2026, January 7). Google and chatbot maker Character to settle lawsuit alleging chatbot pushed teen to suicide. The Washington Times. https://m.washingtontimes.com/news/2026/jan/7/google-chatbot-maker-character-settle-lawsuit-alleging-chatbot-pushed/
[5] Bloomberg Law. (2026, January 8). Character.AI, Google agree to settle teen chatbot harm lawsuits. https://news.bloomberglaw.com/insurance/character-ai-google-agree-to-settle-teen-chatbot-harm-lawsuits
[6] Courthouse News Service. (2025). Suicide coach: Parents sue OpenAI for ChatGPT’s role in son’s death. https://www.courthousenews.com/suicide-coach-parents-sue-openai-for-chatgpts-role-in-sons-death/
[7] Ars Technica. (2025, November). OpenAI says dead teen violated TOS when he used ChatGPT to plan suicide. https://arstechnica.com/tech-policy/2025/11/openai-says-dead-teen-violated-tos-when-he-used-chatgpt-to-plan-suicide
[8] Superior Court of California, County of San Francisco. (2026, February 3). Order on petition for coordination of ChatGPT product liability cases (including Raine v. OpenAI, No. CGC-25-628528). https://reason.com/wp-content/uploads/2026/06/chatgpt-product-liability-cases-coordination.pdf
[9] Let’s Data Science. (2026, October 20). Maria Raine presses OpenAI for answers in ChatGPT wrongful death case. https://letsdatascience.com/news/maria-raine-presses-openai-for-answers-in-chatgpt-wrongful-d-183807ba
[10] United States District Court, District of Minnesota. (2025, February 13). Memorandum opinion and order, Estate of Lokken v. UnitedHealth Group, Inc., Civil No. 23-3514. Georgetown Law Litigation Tracker. https://litigationtracker.law.georgetown.edu/wp-content/uploads/2023/11/Lokken_2025.02.13_MEMORANDUM-OPINION-AND-ORDER.pdf
[11] JD Supra. (n.d.). Federal court orders broad discovery against UHC in AI coverage denial lawsuit. https://www.jdsupra.com/legalnews/federal-court-orders-broad-discovery-7958780/
[12] Information Security Media Group. (n.d.). Court: UnitedHealth must answer for AI-based claim denials. BankInfoSecurity. https://www.bankinfosecurity.com/court-unitedhealth-must-answer-for-ai-based-claim-denials-a-27534
[14] Gibson Dunn. (2026, March). EU Product Liability Directive: Responding to software, AI and complex supply chains. https://www.gibsondunn.com/wp-content/uploads/2026/03/eu-product-liability-directive-responding-to-software-ai-and-complex-supply-chains.pdf
[15] European Parliament and Council of the European Union. (2024, October 23). Directive (EU) 2024/2853 on liability for defective products and repealing Council Directive 85/374/EEC. EUR-Lex. https://eur-lex.europa.eu/eli/dir/2024/2853/oj
[16] Covington & Burling LLP. (2026, January). 5 key takeaways from FDA’s revised clinical decision support (CDS) software guidance. https://www.cov.com/en/news-and-insights/insights/2026/01/5-key-takeaways-from-fdas-revised-clinical-decision-support-cds-software-guidance
[17] Innolitics. (2026, January 29). 2026 FDA guidance: Clinical decision support software. https://innolitics.com/articles/fda-guidance-clinical-decision-support-software-2026/
[18] U.S. Food and Drug Administration, Center for Devices and Radiological Health. (2025, November 6). Digital Health Advisory Committee meeting brief summary: Generative artificial intelligence-enabled digital mental health medical devices. https://www.fda.gov/media/190450/download
[19] U.S. Food and Drug Administration. (n.d.). FDA Digital Health Advisory Committee. https://www.fda.gov/medical-devices/digital-health-center-excellence/fda-digital-health-advisory-committee
[20] Mello, M. M., & Guha, N. (2024). Understanding liability risk from using health care artificial intelligence tools. New England Journal of Medicine, 390(3), 271-278. https://doi.org/10.1056/nejmhle2308901
[21] Stanford Freeman Spogli Institute, REAP. (2024). Exploring liability risks of using AI tools in patient care. https://reap.fsi.stanford.edu/node/35778
[22] Price, W. N., II, Gerke, S., & Cohen, I. G. (2019). Potential liability for physicians using artificial intelligence. JAMA, 322(18), 1765-1766. https://doi.org/10.1001/jama.2019.15064
[23] Journal of Nuclear Medicine. (2021). Interview with Tobia and Stremitzer on juror perceptions of liability for medical AI. https://jnm.snmjournals.org/node/99233
[24] Habib, A. R., Lin, A. L., & Grant, R. W. (2021). The Epic Sepsis Model falls short: The importance of external validation. JAMA Internal Medicine. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2781313
[25] Michigan Medicine. (2021). Popular sepsis prediction tool less accurate than claimed. https://michiganmedicine.org/health-lab/popular-sepsis-prediction-tool-less-accurate-claimed
[26] NBC News. (2025, August). Man who asked ChatGPT about cutting out salt from his diet was hospitalized with hallucinations. https://www.nbcnews.com/tech/tech-news/man-asked-chatgpt-cutting-salt-diet-was-hospitalized-hallucinations-rcna225055
[27] Office of the Texas Attorney General. (2024, September). Attorney General Ken Paxton reaches settlement in first-of-its-kind healthcare generative AI investigation. https://www.texasattorneygeneral.gov/news/releases/attorney-general-ken-paxton-reaches-settlement-first-its-kind-healthcare-generative-ai-investigation
[28] Fierce Healthcare. (2024, September). Texas attorney general, Pieces Technologies settle allegations of inaccurate generative AI advertising. https://www.fiercehealthcare.com/ai-and-machine-learning/texas-ag-pieces-technologies-settle-allegations-inaccurate-generative-ai
[29] Data Privacy & Security Insider. (2025, August). Illinois enacts act prohibiting AI therapy. https://www.dataprivacyandsecurityinsider.com/2025/08/illinois-enacts-act-prohibiting-ai-therapy/
[30] Manatt Health. (2025). Health AI policy tracker. Mondaq. https://www.mondaq.com/unitedstates/new-technology/1699938/manatt-health-health-ai-policy-tracker
[31] Epstein Becker Green. (2025, October 28). Novel AI laws target companion AI and mental health. National Law Review. https://natlawreview.com/article/novel-ai-laws-target-companion-ai-and-mental-health
[32] Gadget Review. (n.d.). Lawsuit says UnitedHealth used AI with a 90% error rate to deny care. https://www.gadgetreview.com/?p=412793
[33] HealthDay. (2021, June 22). Clinical benefit of Epic Sepsis Model in question. https://www.healthday.com/healthpro-news/infectious-disease/epic-sepsis-model-has-poor-discrimination-of-sepsis-2653467079.html
[34] Clinician.com. (n.d.). Malpractice claims involving AI tools are emerging: EDs can mitigate legal exposure. https://www.clinician.com/articles/malpractice-claims-involving-ai-tools-are-emerging-eds-can-mitigate-legal-exposure
[35] Stanford Medicine. (2024, January 25). Research explores liability risk of using AI tools in patient care. https://med.stanford.edu/news/insights/2024/01/liability-risk-ai-tools-patient-care.html
[36] Price, W. N., II, Gerke, S., & Cohen, I. G. (2021). How much can potential jurors tell us about liability for medical artificial intelligence? Journal of Nuclear Medicine, 62(1), 15-16. https://doi.org/10.2967/jnumed.120.257196
[37] Orrick. (2024, September 24). Texas Attorney General reaches novel generative AI settlement: What health tech companies need to know. https://www.orrick.com/en/Insights/2024/09/Texas-Attorney-General-Reaches-Novel-Generative-AI-Settlement
