{"id":1169,"date":"2026-10-18T05:17:00","date_gmt":"2026-10-18T09:17:00","guid":{"rendered":"https:\/\/drugchatter.com\/insights\/?p=1169"},"modified":"2026-09-19T16:17:22","modified_gmt":"2026-09-19T20:17:22","slug":"what-is-ai-telling-patients-about-your-drug-a-practical-guide-to-monitoring-chatgpt-claude-and-gemini","status":"publish","type":"post","link":"https:\/\/drugchatter.com\/insights\/what-is-ai-telling-patients-about-your-drug-a-practical-guide-to-monitoring-chatgpt-claude-and-gemini\/","title":{"rendered":"What Is AI Telling Patients About Your Drug? A Practical Guide to Monitoring ChatGPT, Claude, and Gemini"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

A working guide for Medical Affairs, Regulatory Affairs, Pharmacovigilance, and brand teams<\/p>\n\n\n\n

The short answer:<\/strong> Patients and physicians are already asking ChatGPT, Claude, and Gemini about indications, dosing, interactions, and alternatives, and the published evidence shows a meaningful share of those answers are wrong or incomplete. Building a monitoring program means five operational layers: a maintained prompt library, scheduled sampling across models, structured response capture, claim review against the approved label, and a defined escalation path across Medical Affairs, Pharmacovigilance, Regulatory Affairs, Legal, and Brand. None of this converts into a guarantee of regulatory approval or protection from enforcement. It converts into evidence a company can act on.<\/p>\n\n\n\n

Why AI Answers Are Now a Monitoring Problem<\/h2>\n\n\n\n

Physician use of AI has moved from experimental to routine in about two years. The American Medical Association’s 2026 Physician Survey on Augmented Intelligence, fielded among 1,692 physicians between January 15 and February 2, 2026, found that 81% of physicians now use AI in their practice, up from 38% in 2023 [1]. Doximity’s 2026 State of AI in Medicine Report, drawing on 3,151 physicians surveyed across two periods, found AI use climbing from 47% in an April 2025 cohort to 63% by January 2026 [2]. A 2024 Fierce Healthcare and Sermo survey, limited to physicians who already use general-purpose large language models, found 76% of that group reported using tools like ChatGPT in clinical decision-making, and more than 60% specifically to check drug interactions [3].<\/p>\n\n\n\n

None of these tools were built as clinical references. ChatGPT, Claude, and Gemini answer drug questions the same way they answer any other question: by synthesizing language patterns from training data and, increasingly, live web retrieval, without a built-in mechanism to flag when an answer conflicts with the approved prescribing information. That gap is measurable.<\/p>\n\n\n\n

What the Evidence Shows About Accuracy<\/h3>\n\n\n\n

A 2024 cross-sectional study in BMJ Quality & Safety<\/em>, led by Wahram Andrikyan at Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg, tested an AI chatbot embedded in a search engine against real patient drug questions. Overall completeness and accuracy scored high on median, but a subset of 20 lower-quality answers was reviewed by seven medication-safety experts [4].<\/p>\n\n\n\n

\n

“Of the subset of 20 chatbot answers, experts found 66% to be potentially harmful… 22% [were assessed] to cause severe harm or even death if patients follow the chatbot’s advice.” \u2014 BMJ Quality & Safety<\/em>, 2024 [4]<\/p>\n<\/blockquote>\n\n\n\n

A broader 2026 audit, covered by CIDRAP and published in BMJ Open<\/em>, tested five chatbots, ChatGPT, Gemini, Grok, Meta AI, and DeepSeek, against 250 health questions spanning cancer, vaccines, stem cells, nutrition, and athletic performance. Just under half of all responses, 49.6%, were rated problematic, and 19.6% were rated highly problematic or potentially harmful [5].<\/p>\n\n\n\n

Drug-specific accuracy testing tells a similar story. Two posters presented at the American Society of Health-System Pharmacists’ 2023 Midyear meeting are among the most directly comparable data points available. Researchers at Long Island University’s College of Pharmacy took 39 real questions submitted to a drug information service and checked ChatGPT’s answers against the professional literature: only 10 of 39, about 26%, were judged satisfactory, and the chatbot fabricated citations, including URLs that led to nonexistent studies, in several cases. In a separate poster, researchers from Iwate Medical University and Torrance Memorial Medical Center compared ChatGPT’s answers on the most common side effects of 30 drugs against Lexicomp, a clinical reference: only 2 of 30 answers were fully accurate [6].<\/p>\n\n\n\n

Source<\/th>Method<\/th>Key finding<\/th><\/tr>
Andrikyan et al., BMJ Quality & Safety<\/em>, 2024 [4]<\/td>Search-engine chatbot tested on real patient drug questions; 20 lower-quality answers reviewed by 7 medication-safety experts<\/td>66% of the reviewed subset judged potentially harmful; 22% rated capable of causing severe harm or death if followed<\/td><\/tr>
BMJ Open<\/em> audit via CIDRAP, 2026 [5]<\/td>5 chatbots (ChatGPT, Gemini, Grok, Meta AI, DeepSeek), 250 health questions, February 2025<\/td>49.6% of responses rated problematic; 19.6% highly problematic or potentially harmful<\/td><\/tr>
Long Island Univ. College of Pharmacy, ASHP poster, 2023 [6]<\/td>39 real drug-information-service questions checked against professional literature<\/td>Only 10 of 39 (26%) answered satisfactorily; several fabricated citations<\/td><\/tr>
Iwate Medical Univ. \/ Torrance Memorial, ASHP poster, 2023 [6]<\/td>30 drugs, “most common side effects” question, compared to Lexicomp<\/td>Only 2 of 30 (7%) fully accurate; 26 of 30 inaccurate<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

These are not the same chatbot, the same drug set, or the same question type, and none of them should be read as a fixed error rate for any specific model today; models are updated on a rolling basis and none of these studies claim to be current benchmarks. What they establish, consistently, across different research teams and different years, is that AI-generated drug information contains a non-trivial rate of clinically material errors, and that the errors are not evenly distributed. That is the baseline reason to monitor rather than assume.<\/p>\n\n\n\n

The Regulatory Backdrop: What Agencies Are, and Are Not, Saying<\/h2>\n\n\n\n

On January 14, 2026, the FDA’s Center for Drug Evaluation and Research and Center for Biologics Evaluation and Research, working with the European Medicines Agency, published ten guiding principles for good AI practice across the drug product life cycle: nonclinical, clinical, manufacturing, and postmarketing [7]. The principles include human-centric design, a risk-based approach, data governance and documentation, and scheduled monitoring with periodic re-evaluation of AI system performance. They are explicitly non-binding: a framework intended to inform future guidance rather than an enforceable rule, and they are addressed to AI used in evidence generation for drug development, not to how ChatGPT or Gemini answer a patient’s question about an approved product. No monitoring program built on these principles converts into evidence of, or a pathway toward, regulatory approval; they describe good practice, not a compliance checklist with a pass mark.<\/p>\n\n\n\n

A separate April 2026 warning letter shows how FDA treats AI reliance once it touches regulated documentation. On April 2, 2026, FDA cited Purolea Cosmetics Lab, a small manufacturer in Livonia, Michigan, primarily for insanitary manufacturing conditions and for marketing two products, one for shingles and one for genital herpes relief, as unapproved new drugs. Within that letter, FDA also flagged that the firm had used AI agents to generate manufacturing procedures and specifications without review by its Quality Unit, a violation of 21 CFR 211.22(c); when investigators asked why a required process validation step was skipped, the company said its AI agent had never told them it was necessary [8]. It is the first FDA warning letter to carve out AI misuse as its own citable failure. The letter concerns manufacturing records, not chatbot answers to patients, but the underlying standard travels well beyond manufacturing: an AI-generated output is not self-validating, and a named, accountable person has to review it before anyone relies on it. That is exactly the standard a drug-information monitoring program should hold itself to.<\/p>\n\n\n\n

Building the Program: Five Operational Layers<\/h2>\n\n\n\n

A monitoring program that can actually support Medical Affairs, Regulatory Affairs, Pharmacovigilance, and Brand teams needs to do five things repeatably: decide what to ask, decide which models to ask it of, capture the answers in a structured and auditable way, grade those answers against the approved label, and route the ones that matter to the right team. Each layer has its own failure modes.<\/p>\n\n\n\n

1. Prompt Libraries: What to Ask, and in Whose Voice<\/h3>\n\n\n\n

A prompt library is the fixed, repeatable set of questions run against each model on each monitoring cycle. Ad hoc querying, one analyst typing a question when they think of it, cannot be compared over time or across models, and it is the single most common shortfall in early-stage internal monitoring efforts. A working library typically spans several query types for each monitored brand:<\/p>\n\n\n\n