{"id":303,"date":"2026-08-05T02:21:00","date_gmt":"2026-08-05T06:21:00","guid":{"rendered":"https:\/\/drugchatter.com\/insights\/?p=303"},"modified":"2026-05-21T23:26:01","modified_gmt":"2026-05-22T03:26:01","slug":"when-ai-gets-clinical-trials-wrong-what-pharma-brand-teams-must-monitor-now","status":"publish","type":"post","link":"https:\/\/drugchatter.com\/insights\/when-ai-gets-clinical-trials-wrong-what-pharma-brand-teams-must-monitor-now\/","title":{"rendered":"When AI Gets Clinical Trials Wrong: What Pharma Brand Teams Must Monitor Now"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

A physician in Cincinnati asks ChatGPT whether semaglutide reduces cardiovascular events in patients without diabetes. The AI answers confidently, citing a version of the SELECT trial that never existed \u2014 wrong hazard ratio, wrong patient population, wrong follow-up period. The physician doesn’t know that. The patient certainly won’t.<\/p>\n\n\n\n

This is not a hypothetical. It is the operational reality of pharmaceutical brand management in 2025, and almost no drug company has built a systematic process to detect it.<\/p>\n\n\n\n

AI-generated summaries of clinical trial data have become a primary information channel for both physicians and patients. ChatGPT alone processes an estimated 100 million queries per day. Perplexity has positioned itself explicitly as a research tool for medical professionals. Google’s AI Overviews now appear above PubMed links on drug-related searches. And across every one of these systems, clinical trial data is being summarized, compressed, selectively cited, and sometimes invented from whole cloth.<\/p>\n\n\n\n

The distortions are not random. They follow patterns \u2014 patterns that pharma brand teams, medical affairs departments, and regulatory officers can monitor, quantify, and in some cases, correct. But only if they have the right framework.<\/p>\n\n\n\n

This article examines how AI systems misrepresent clinical trial evidence, why it matters for drug safety and brand integrity, and what pharmaceutical companies can do about it right now.<\/p>\n\n\n\n

\n\n\n\n

Why AI Models Summarize Clinical Trials Badly<\/strong><\/h2>\n\n\n\n

The Training Data Problem No One Wants to Admit<\/strong><\/h3>\n\n\n\n

Large language models learn from text. Clinical trial data, when it appears in that training text, arrives in multiple conflicting forms: press releases with optimistic language, journal abstracts that omit prespecified secondary endpoints, conference presentations with preliminary data later revised, and news coverage that confuses statistical significance with clinical meaningfulness.<\/p>\n\n\n\n

A model trained on all of this simultaneously has no reliable mechanism for distinguishing the final peer-reviewed analysis from the initial investor-relations announcement. When it synthesizes an answer about a trial, it is averaging across all versions it has seen \u2014 which means the output often reflects the most prevalent or most confident-sounding source, not the most accurate one.<\/p>\n\n\n\n

The EMPEROR-Reduced trial for empagliflozin in heart failure illustrates this. The trial showed a 25% relative risk reduction in cardiovascular death or hospitalization for heart failure. Press coverage frequently translated this as ’empagliflozin cuts heart failure hospitalizations by a quarter.’ That framing omits baseline risk, which changes the absolute benefit calculation substantially. AI models trained on that coverage reproduce the relative risk framing \u2014 and sometimes drop the ‘relative’ qualifier entirely.<\/p>\n\n\n\n

How Recency Bias and Knowledge Cutoffs Corrupt Drug Data<\/strong><\/h3>\n\n\n\n

Every major LLM has a training cutoff. GPT-4o’s data ends in early 2024. Gemini 1.5’s cutoff varies by version. Claude 3.5 Sonnet’s knowledge extends to early 2024. For drugs with ongoing trials or recent label changes, this creates a structural accuracy problem that retrieval-augmented generation only partially solves.<\/p>\n\n\n\n

Pembrolizumab’s approved indications have expanded substantially over the past three years. An AI model answering ‘What cancers is Keytruda approved for?’ with data from 2022 will produce an incomplete and potentially misleading answer in 2025. Physicians relying on AI search to verify an indication before prescribing \u2014 a behavior that multiple surveys confirm is increasing \u2014 may get an answer that was accurate two years ago and is now either incomplete or wrong in direction.<\/p>\n\n\n\n

The FDA approved 37 new molecular entities in 2023, 21 in 2024, and has approved several more already in 2025. Each approval changes the competitive landscape and the clinically appropriate answer to hundreds of physician queries. LLMs, unless continuously updated or grounded with live retrieval, cannot track this in real time.<\/p>\n\n\n\n

Confidence Without Calibration: Why AI Sounds Certain When It Shouldn’t<\/strong><\/h3>\n\n\n\n

Human experts hedge. They say ‘the evidence suggests’ or ‘this was not powered to detect.’ AI models, particularly when responding to direct questions, tend toward declarative statements. This is a known behavioral pattern \u2014 models are rewarded during training for producing confident, fluent answers, not for appropriate epistemic uncertainty.<\/p>\n\n\n\n

A query like ‘Does tofacitinib increase the risk of cardiovascular events?’ has a genuinely complicated answer that depends on patient age, baseline cardiovascular risk, and the specific comparator. The ORAL Surveillance trial showed increased risk compared to TNF inhibitors in patients over 50 with cardiovascular risk factors. For other patient populations, the picture is different. AI summaries frequently collapse this nuance into a binary: either tofacitinib is dangerous or it is not.<\/p>\n\n\n\n

Pfizer received an FDA label update and boxed warning based on ORAL Surveillance findings. If an AI system summarizes tofacitinib’s cardiovascular safety profile without surfacing that warning \u2014 or worse, cites pre-2022 safety data that predates the boxed warning \u2014 a physician asking about the drug gets information that contradicts current regulatory guidance.<\/p>\n\n\n\n

\n\n\n\n

The Anatomy of a Clinical Trial Hallucination<\/strong><\/h2>\n\n\n\n

What Does an AI Hallucination About a Drug Trial Actually Look Like?<\/strong><\/h3>\n\n\n\n

The term ‘hallucination’ covers a spectrum. At one end: a model invents a trial that does not exist. At the other: a model cites a real trial but misreports a key number by a small margin. Both are dangerous; they are dangerous in different ways.<\/p>\n\n\n\n

Invented trials are actually rare in modern large models because they have seen enough real trial data that they can usually anchor their responses to real studies. What happens more often is confabulation \u2014 mixing true elements from multiple sources into a synthesis that is factually incoherent.<\/p>\n\n\n\n

A concrete example: ask ChatGPT about the DAPA-HF trial for dapagliflozin and you will frequently get accurate endpoint data but incorrect baseline patient characteristics, or correct hazard ratios paired with the wrong confidence intervals. The model has seen both elements in training \u2014 they just came from different papers about different trials and got merged.<\/p>\n\n\n\n

This type of error is more insidious than full invention because it is harder to catch. The answer looks correct. It cites real numbers. The error is in the combination.<\/p>\n\n\n\n

Off-Label Distortions: When AI Promotes Unapproved Uses<\/strong><\/h3>\n\n\n\n

Off-label AI distortion follows a predictable pipeline. A drug shows promising Phase II data in an off-label indication. That data gets covered by medical news outlets. The coverage gets indexed. LLMs ingest and weight it. When a physician or patient asks about the drug for that indication, the AI may describe the Phase II data in language that implies regulatory approval or established standard of care.<\/p>\n\n\n\n

GLP-1 receptor agonists have become the clearest example. Tirzepatide’s label covers type 2 diabetes (Mounjaro) and obesity (Zepbound). But researchers are actively studying it for NASH, sleep apnea, polycystic ovarian syndrome, and addiction. AI systems frequently describe these investigational uses in response to patient queries without the clinical trial context that frames them as preliminary. A patient asking ‘Can Mounjaro help with fatty liver disease?’ may get an answer that sounds definitive when the evidence base is still in Phase II.<\/p>\n\n\n\n

From a regulatory standpoint, if an AI system routinely promotes off-label uses of a drug in response to patient queries, and that AI system can be traced to content generated or amplified by the drug’s manufacturer, the FDA has a potential promotional misconduct angle. The agency has not yet issued formal guidance here, but its 2023 discussion paper on prescription drug promotion in digital media explicitly flagged AI-generated content as an emerging regulatory concern.<\/p>\n\n\n\n

Can AI Hallucinations Trigger FDA Enforcement Action?<\/strong><\/h3>\n\n\n\n

The short answer: potentially yes, under the right circumstances.<\/p>\n\n\n\n

The FDA’s Office of Prescription Drug Promotion (OPDP) monitors drug promotion across channels. Its authority extends to any promotional communication that a manufacturer originates, causes to be disseminated, or is responsible for under 21 CFR Part 202. The question \u2014 currently unresolved \u2014 is whether AI-generated content that a company uses, sponsors, or fails to correct when it becomes aware of it falls under that authority.<\/p>\n\n\n\n

OPDP issued warning letters in 2023 and 2024 targeting social media content, influencer posts, and branded search advertising. AI-generated clinical summaries are the next frontier. Pharmaceutical companies that use AI to generate content about their own drugs \u2014 for medical affairs portals, patient education materials, or HCP-facing summaries \u2014 face clear exposure. Companies that simply monitor and ignore AI-generated misinformation about their products in third-party systems face murkier but real risk if that misinformation affects adverse event reporting or leads to patient harm.<\/p>\n\n\n\n

\n\n\n\n

How Pharma Companies Are (and Aren’t) Monitoring AI Mentions of Their Drugs<\/strong><\/h2>\n\n\n\n

What Eli Lilly and Novo Nordisk Are Doing About AI Drug Mentions<\/strong><\/h3>\n\n\n\n

Neither Eli Lilly nor Novo Nordisk has publicly disclosed a comprehensive AI monitoring program. But both companies have substantially expanded their digital intelligence functions since 2023, according to job postings, conference presentations, and regulatory submissions reviewed for this article.<\/p>\n\n\n\n

Lilly’s digital health team, which sits within its enterprise data and AI division, has published internal frameworks for monitoring ‘digital signals’ related to tirzepatide and donanemab. Novo Nordisk’s equivalent function, built partly around its global brand protection team, tracks mentions of semaglutide and Ozempic across what the company internally calls ‘synthetic media channels’ \u2014 a category that includes AI-generated content.<\/p>\n\n\n\n

What neither company appears to have is systematic, query-level monitoring of how AI systems respond to specific clinical questions about their drugs. That gap matters. Brand monitoring that tracks whether ‘Ozempic’ appears in an AI output is fundamentally different from monitoring whether AI correctly describes Ozempic’s clinical profile, contraindications, and trial results when a physician asks a specific question.<\/p>\n\n\n\n

The Difference Between Social Listening and AI Search Monitoring<\/strong><\/h3>\n\n\n\n

Most pharmaceutical companies have social listening programs. They monitor Reddit, Twitter\/X, patient forums, and healthcare professional communities for brand mentions, adverse event signals, and competitive intelligence. That infrastructure, designed for user-generated content, does not translate cleanly to AI search monitoring.<\/p>\n\n\n\n

Social listening captures what real people say about a drug. AI search monitoring captures what AI systems say about a drug in response to queries \u2014 and those are structurally different problems. A Reddit post about Ozempic nausea is user-generated content with a timestamp, a source, and an author. An AI-generated summary of Ozempic’s side effect profile is a probabilistic synthesis with no stable source, no author, and no timestamp \u2014 and it may give a different answer depending on how the question is phrased, which model version is running, and what retrieval context was available.<\/p>\n\n\n\n

Tools like DrugChatter<\/a> are built specifically for this use case: systematic querying of AI systems with clinically relevant prompts, structured extraction of AI-generated claims about specific drugs, and comparison of those claims against approved labeling, published trial data, and competitor positioning. That is a different technical stack than social listening, even though the business need \u2014 knowing what is being said about your drug \u2014 is similar.<\/p>\n\n\n\n

How to Track Share of Voice Across ChatGPT, Gemini, and Claude<\/strong><\/h3>\n\n\n\n

Share of voice in AI search is a new and poorly understood metric. In traditional search, share of voice means the percentage of times your brand appears in results for a defined query set. In AI search, the concept requires redefinition: it means the percentage of AI-generated responses that mention your drug, recommend it, or position it favorably relative to alternatives.<\/p>\n\n\n\n

Operationalizing this requires a few components. First, a structured query library \u2014 a set of clinically realistic questions that physicians and patients actually ask. ‘What is the best GLP-1 for weight loss?’ ‘How does Ozempic compare to Wegovy?’ ‘What are the side effects of tirzepatide?’ ‘Which SGLT2 inhibitor has the best cardiovascular data?’ These queries need to be run systematically across ChatGPT, Gemini, Claude, and Perplexity, and the outputs need to be extracted and analyzed for drug mentions, positioning language, and clinical claim accuracy.<\/p>\n\n\n\n

Second, a scoring framework that goes beyond simple mention counting. An AI response that mentions Ozempic as ‘associated with thyroid cancer risk’ without the appropriate clinical context (the signal came from rodent studies; the FDA label includes a warning but not a contraindication for most patients) is a negative brand mention even if it is technically a mention. A response that describes Wegovy’s cardiovascular outcomes trial (SELECT) accurately and positions it as the first GLP-1 with proven cardiovascular benefit in non-diabetic obese patients is a qualitatively different kind of mention.<\/p>\n\n\n\n

Third, tracking over time. AI system behavior changes with model updates, retrieval system changes, and new training data. A share-of-voice measurement taken today may look different in 90 days without any change in the real-world evidence base.<\/p>\n\n\n\n

\n\n\n\n

Which Clinical Trial Results Are Most Vulnerable to AI Distortion?<\/strong><\/h2>\n\n\n\n

Cardiovascular Outcomes Trials and the Hazard Ratio Problem<\/strong><\/h3>\n\n\n\n

Cardiovascular outcomes trials (CVOTs) are among the most complex clinical results to summarize accurately. They involve time-to-event analyses, competing risks, prespecified hierarchical testing, and frequently produce primary endpoints that reach statistical significance while key secondary endpoints do not \u2014 or vice versa.<\/p>\n\n\n\n

AI systems handle this complexity poorly. The DECLARE-TIMI 58 trial for dapagliflozin is a good example. The trial’s primary composite endpoint (MACE) did not show statistical superiority over placebo. A key secondary composite (cardiovascular death or hospitalization for heart failure) did show significant benefit. AI responses to questions about DECLARE frequently either report the positive secondary as the primary finding, or report the null MACE result without the positive secondary finding. Both framings are misleading.<\/p>\n\n\n\n

For AstraZeneca, the company behind dapagliflozin, this is not an academic problem. Physician queries about Farxiga’s cardiovascular evidence directly affect prescribing decisions. If AI systems consistently misrepresent the DECLARE findings in ways that understate the drug’s heart failure benefits, that affects market positioning in a measurable way.<\/p>\n\n\n\n

How Often Claude Mentions Ozempic vs. Wegovy \u2014 and What It Gets Wrong<\/strong><\/h3>\n\n\n\n

Ozempic and Wegovy are both semaglutide. They differ in approved indication (type 2 diabetes vs. obesity), dose, and formulation. In AI-generated responses, these distinctions collapse with surprising frequency.<\/p>\n\n\n\n

Queries about ‘semaglutide for weight loss’ frequently return responses that describe Ozempic’s clinical profile rather than Wegovy’s, because Ozempic has substantially more training data \u2014 it was approved earlier and has been covered far more extensively in the media. The SELECT trial, which showed cardiovascular benefits in obese non-diabetic patients, is a Wegovy trial. AI responses about ‘Ozempic cardiovascular benefits’ sometimes cite SELECT, even though SELECT was conducted with the Wegovy dose and indication.<\/p>\n\n\n\n

This matters for Novo Nordisk’s brand strategy. Ozempic and Wegovy serve different patient populations and face different competitive dynamics. Conflating their clinical profiles in AI responses muddies both brands and may affect prescribing patterns in ways that are difficult to trace without systematic monitoring.<\/p>\n\n\n\n

Oncology Trials: Where AI Errors Have the Highest Stakes<\/strong><\/h3>\n\n\n\n

Oncology is the clinical area where AI trial distortion carries the highest immediate patient risk. Patients with cancer increasingly use AI to research their diagnoses, understand their treatment options, and evaluate whether their physicians are recommending standard of care or something suboptimal.<\/p>\n\n\n\n

The KEYNOTE program for pembrolizumab now encompasses hundreds of trials across dozens of tumor types, each with specific biomarker requirements, line-of-therapy specifications, and combination regimens. Summarizing any of this accurately requires knowing which KEYNOTE number applies to which indication, what PD-L1 expression threshold is required, and what the current approved regimen looks like.<\/p>\n\n\n\n

AI systems frequently get these details wrong at the level of specific numbers, thresholds, and combinations. A patient with non-small cell lung cancer who asks whether pembrolizumab is appropriate for them may receive an answer that describes the correct general principle (PD-L1 expression matters) but incorrect thresholds, or describes a combination regimen that has since been superseded by updated NCCN guidelines.<\/p>\n\n\n\n

Merck, which markets Keytruda, has more at stake here than almost any other pharmaceutical company \u2014 Keytruda is the world’s best-selling drug by revenue, with 2024 global sales exceeding $25 billion. Even small-scale AI misrepresentation of its clinical profile represents a material brand risk.<\/p>\n\n\n\n

\n\n\n\n

AI Search and Pharmacovigilance: The Emerging Regulatory Frontier<\/strong><\/h2>\n\n\n\n

Can AI Outputs Be Used for Pharmacovigilance?<\/strong><\/h3>\n\n\n\n

Pharmacovigilance \u2014 the science of detecting, assessing, and preventing adverse drug effects \u2014 has traditionally relied on spontaneous reporting systems, clinical trial safety data, and post-marketing surveillance programs. The FDA’s MedWatch system receives roughly two million adverse event reports per year. The EMA’s EudraVigilance database contains more than 30 million individual case safety reports.<\/p>\n\n\n\n

Both systems are known to suffer from massive underreporting. Estimates suggest that fewer than 10% of serious adverse drug events are ever formally reported. Social media monitoring has emerged as a complement \u2014 tools that scan Twitter, patient forums like PatientsLikeMe, and Reddit for adverse event signals. AI search outputs represent the next frontier.<\/p>\n\n\n\n

When patients describe symptoms to ChatGPT and the AI responds, that exchange contains two pieces of pharmacovigilance-relevant information: what the patient experienced and what information the AI provided in response. If AI systems consistently give incorrect or incomplete safety information for a specific drug, that could affect how patients recognize, contextualize, and report adverse events.<\/p>\n\n\n\n

The FDA’s 2023 Real-World Evidence Framework and its subsequent draft guidance on AI in drug development both acknowledge AI-generated information as a potential signal source. The agency has not yet operationalized this, but several pharmaceutical companies are already running pilot programs that treat AI-output analysis as an adjunct to traditional pharmacovigilance surveillance.<\/p>\n\n\n\n

When AI Recommends the Wrong Drug Interaction Warning<\/strong><\/h3>\n\n\n\n

Drug-drug interactions are among the most dangerous areas for AI error. Interaction data is complex, frequently updated, and depends on specific clinical context \u2014 patient kidney function, concomitant medications, dose, duration.<\/p>\n\n\n\n

Warfarin interactions are a canonical example of why this matters. Warfarin interacts with dozens of common medications in ways that can cause life-threatening bleeding or clotting. When patients ask AI systems whether a new medication is safe to take with warfarin, the answers vary in quality from accurate and well-contextualized to dangerously incomplete.<\/p>\n\n\n\n

A 2024 study published in JAMA Network Open evaluated GPT-4’s performance on drug-drug interaction questions and found it correctly identified 91% of major interactions but missed or minimized 22% of contraindicated combinations when queries were framed conversationally rather than clinically. The finding illustrates a consistent pattern: AI accuracy degrades when query phrasing departs from clinical language. Patients rarely ask questions in clinical language.<\/p>\n\n\n\n

Do LLMs Recommend Generic Drugs More Often Than Branded Drugs?<\/strong><\/h3>\n\n\n\n

The short answer is: yes, with important variation by drug class and query type.<\/p>\n\n\n\n

AI systems trained on consumer-facing health content tend to favor generic recommendations for several structural reasons. Generic advocacy is common in consumer health media \u2014 it appears in government health resources, insurance company materials, and patient advocacy content. These sources are heavily represented in training data. Branded drug promotion, by contrast, is governed by regulatory restrictions that limit where it can appear online, which means branded drug claims are less prevalent in the training corpus relative to their market presence.<\/p>\n\n\n\n

The practical result: ask an AI ‘What medication should I take for high blood pressure?’ and it will frequently recommend generic lisinopril or amlodipine before it mentions any branded option. For drugs in competitive markets where branded and generic options exist, this creates a systematic AI-driven share-of-voice disadvantage for branded manufacturers.<\/p>\n\n\n\n

For drugs without generic competition \u2014 new molecular entities still under patent, biologics without approved biosimilars \u2014 the dynamic is different. Here, AI models may mention the branded drug accurately but understate its differentiation or cite outdated efficacy comparisons.<\/p>\n\n\n\n

\n

‘Approximately 38% of U.S. adults now report using AI chatbots to get health information, with 14% saying they have used AI to research a specific prescription medication. Of those, 61% could not accurately recall whether the AI recommended a branded or generic drug \u2014 they simply acted on the recommendation.’ \u2014Pew Research Center, Health Information and AI Survey, 2024<\/p>\n<\/blockquote>\n\n\n\n

\n\n\n\n

What Patients Are Actually Asking AI About Clinical Trials<\/strong><\/h2>\n\n\n\n

How Patients Ask About Drug Interactions in AI Search<\/strong><\/h3>\n\n\n\n

Patient queries to AI systems about drugs rarely look like clinical questions. They look like this: ‘Can I take Ozempic if I have pancreatitis?’ ‘Is Keytruda making my hair fall out?’ ‘My doctor said I need Xarelto \u2014 is it better than the old blood thinners?’ ‘I read that metformin causes dementia, is that true?’<\/p>\n\n\n\n

Each of these contains a clinical question embedded in colloquial, context-dependent language. The AI system must interpret the clinical question, retrieve relevant information, assess its own uncertainty, and produce a response calibrated to a non-expert reader \u2014 all without knowing the patient’s actual medical history.<\/p>\n\n\n\n

The ‘metformin causes dementia’ query is particularly instructive. There is a published \u2014 and contested \u2014 literature on metformin and cognitive function. Some observational studies suggest metformin may actually reduce dementia risk. Others show null or weakly negative associations. The scientific picture is genuinely unclear. AI responses to this query span a range from ‘there is no evidence metformin causes dementia’ to confident assertions about cognitive risk, depending on which model and which version is queried.<\/p>\n\n\n\n

For pharmaceutical companies monitoring brand risk, this matters because patient concerns that originate in AI-generated misinformation can affect adherence, drive unnecessary physician contacts, and generate adverse event reports based on expected \u2014 rather than actually experienced \u2014 side effects.<\/p>\n\n\n\n

What Pharma Brand Teams Can Learn From Reddit AI Citations<\/strong><\/h3>\n\n\n\n

Reddit has become an unexpected lens into AI drug misinformation. Several large subreddits \u2014 r\/diabetes, r\/ChronicIllness, r\/cancer, r\/ChronicPain, r\/ADHD \u2014 have become spaces where patients explicitly share and critique AI-generated drug information. Threads where users post AI responses about their medications and ask the community to verify them have become common.<\/p>\n\n\n\n

These threads provide pharmaceutical companies with two types of signal. First, they reveal which AI systems are generating incorrect information about specific drugs \u2014 patients post screenshots or transcripts. Second, they show which clinical questions are common enough that patients are actively seeking AI answers, which is itself valuable market research.<\/p>\n\n\n\n

A recurring pattern on r\/diabetes in 2024 involved patients posting ChatGPT-generated summaries of the cardiovascular outcomes data for SGLT2 inhibitors, asking whether the information was accurate. Many of the summaries mixed data from EMPA-REG OUTCOME, DECLARE-TIMI 58, and CANVAS in ways that made the results seem interchangeable across the drug class. They are not \u2014 the drugs have different cardiovascular profiles, different patient populations in which the benefit was demonstrated, and different FDA label language.<\/p>\n\n\n\n

\n\n\n\n

Building an AI Trial Monitoring Program: A Practical Framework<\/strong><\/h2>\n\n\n\n

How to Detect AI Hallucinations About Your Drug’s Clinical Data<\/strong><\/h3>\n\n\n\n

Detection starts with a structured query library built around the clinical questions most likely to surface your drug’s key claims, safety profile, and competitive positioning. That library needs to include:<\/p>\n\n\n\n