
More than 40 million people ask ChatGPT healthcare questions every day, according to an OpenAI report published in January 2026. [1] In a 2009 industry survey, medical information (MI) departments that received more than 1,000 inquiries a month tended to log them with manual capture methods. [2] Both groups hold a record of what people want to know about drugs. One record grows at the scale of a consumer internet product. The pharmaceutical company cannot see it.
This article compares the two records. It sets out what an MI database holds, what it misses, and where real questions to large language models (LLMs) are stored today. It also explains where the DrugChatter chatmon database fits. DrugChatter’s position is that chatmon is the only drug-specific database that stores real questions people put to LLMs next to the answers each model returned, re-run every week. The comparison table in the section on where real questions live shows how that claim differs from other question sources, so you can test it.
The Short Answer: Who Holds the Larger Record of Drug Questions?
AI platforms hold the larger record of drug questions. OpenAI reports more than 40 million daily users asking ChatGPT health questions. [1] An MI team holds a smaller record with better structure: each inquiry has a requester, a product, a topic, an approved reply, and a log. [2][3][4]
The gap matters because the two records do not overlap. A person must contact a company before an MI team sees the question. A person who asks a chatbot never contacts the company at all.
The Five Findings That Matter
- OpenAI reports that more than 40 million people ask ChatGPT healthcare questions every day, and more than 230 million people ask health and wellness questions every week. [1][5]
- KFF found that 32% of US adults used an AI chatbot for health information in the past year. Of those who asked about physical health, 42% did not follow up with a doctor or other health professional. [6]
- In a 2009 survey of pharmaceutical MI departments, 96% of respondents used inquiry volume to find trends, but 54% used verbatim content. That is a 42-percentage-point gap between counting questions and reading them. [2] (The gap is our calculation from the survey figures.)
- Public research datasets hold about one million real chatbot conversations each, but they cover all topics and are not organized by drug, indication, or label. [7][8]
- Published accuracy studies of AI drug answers report results from 6.7% to 98% on different measures. The question set, model version, and rating method differ in every study, so the results cannot be ranked against each other. [9][10][11][12] (The percentages are our calculations.)
32% of US adults used AI chatbots for health information in the past year, about as many as used social media for health information. Source: KFF Tracking Poll on Health Information and Trust, conducted February 24 to March 2, 2026, with 1,343 US adults. [6]
What Is a Medical Information Question-Answer Database?
A medical information question-answer database is the record a pharmaceutical company keeps of external inquiries about its products and the replies it sent. It includes the inquiry, the requester type, the product, the topic, the approved response, and any follow-up. FDA’s 2011 draft guidance on unsolicited requests describes the same core elements as records a firm should keep. [4]
What Does a Medical Information Inquiry Log Contain?
A 2009 survey by Ortho-McNeil Janssen Scientific Affairs asked 26 respondents at US pharmaceutical and biotechnology companies which data elements they used to find trends in MI inquiries. [2] The survey was sent in December 2008. It is old and small, and the authors say so. It is also one of the few published descriptions of what MI databases capture.
| Data element used to identify trends | Share of respondents | Source |
|---|---|---|
| Product inquiry volume | 96% | [2] |
| Product inquired about | 81% | [2] |
| Clinical reply topic | 73% | [2] |
| Verbatim content | 54% | [2] |
| Special events (scientific meetings, media activity) | 54% | [2] |
| Customer demographics | 50% | [2] |
The same survey found that 92% of departments (24 of 26) did trend and signal analysis, but 48% (12 of 25) did not validate the trends they found. [2] Reports went mainly to Medical Affairs (72%), medical liaisons (56%), and Marketing (52%). Reports were most often monthly (38%). [2]
How Do Standard Response Documents Turn Answers Into Reusable Assets?
A standard response document (SRD) is a pre-approved reply to a question the company receives often. The workflow described in a CLS white paper hosted by the Medical Affairs Professional Society has four steps: staff check for a pre-approved response, search databases if none exists, draft a new response, and obtain approval. [3]
The same paper reports that MI requests arrive by phone, email, live chat, online chatbots, discussion forums, and company or third-party websites. [3] It also reports that emails are often lengthy 40-page to 50-page Word or PDF documents, which can bury the answer. [3]
What Does Each Inquiry Cost to Handle?
The CLS white paper puts handling costs at $25 to $60 per query, covering answering, handling, and logging, with costs that can exceed $100 and rise with each repeat call. [3] The figure comes from a vendor white paper, so we treat it as an illustrative range and not as an industry benchmark.
Our calculation: a department at the 1,000-inquiries-per-month level from the 2009 survey would spend $25,000 to $60,000 a month at that range, or $300,000 to $720,000 a year. [2][3] This is an illustration built on two sources with different dates. It is not a reported cost for any company.
What Can a Medical Information Database Not See?
An MI database cannot see a question that was never sent to the company. That is its structural limit, and it applies to every channel in the survey: toll-free number, meeting booth, email, and CRM tools. [2]
Why Does the Asker Have to Contact the Company First?
Every record in an MI database begins with a person choosing to contact the manufacturer. The 2009 survey lists the intake channels as toll-free numbers (96%), meeting booths (96%), email (81%), and CRM tools (73%). [2] A person who searches the web, asks a colleague, or asks a chatbot appears in none of them.
Why Is Verbatim Wording the Least Captured Element?
Volume and topic are easy to code. Wording is harder. In the 2009 survey, 54% of respondents used verbatim content to find trends, against 96% for volume. [2] The words a person uses to ask about a drug show what they believe about it. A topic code such as “dosing” removes that.
McKinsey made a related point in 2018. Its Medical Information practice called inquiry data a resource that “remains largely untapped,” and it suggested uses such as reporting off-label inquiries to R&D and building heat maps of trending issues by product, geography, or side effect. [13]
How Many People Ask AI About Health, and How Many Ask a Medical Information Team?
No public source reports both numbers on the same basis. We can report the AI side from OpenAI and KFF. We cannot report a matching figure for MI contacts, because companies do not publish inquiry counts, and the sources we reviewed give none. [3][13]
What Does OpenAI Report About Health Questions on ChatGPT?
OpenAI’s January 2026 report, as summarized by Healthcare Dive, states four facts. More than 40 million people ask ChatGPT healthcare questions every day. More than 5% of all ChatGPT messages worldwide are about healthcare. About 1.5 million to 2 million questions a week concern health insurance. About 7 in 10 health conversations happen outside typical clinical hours. [1]
Separately, Fortune reported on January 7, 2026 that OpenAI said more than 230 million people globally ask health and wellness questions on ChatGPT every week, based on an analysis of deidentified conversations. [5]
These are company-reported figures. They count people and messages across all health topics. They do not say how many questions concern a specific drug, and no source we reviewed gives that number.
What Did KFF Find About US Adults and AI Health Information?
KFF’s poll, published March 25, 2026, found that 32% of US adults used AI chatbots for health information in the past year. Of all adults, 29% sought physical health information from AI and 16% sought mental health information. [6] The poll surveyed 1,343 US adults from February 24 to March 2, 2026. The margin of sampling error is plus or minus 3 percentage points for the full sample. [6]
Why Does “Look It Up Before Seeing a Provider” Matter to Medical Information?
KFF reports that 41% of AI health users wanted to look up information before deciding whether to see a provider. [6] That is the moment an MI team used to reach: a person has a question about a drug and has not yet talked to a clinician. The chatbot now sits in that moment.
What Does the 42% Who Did Not Follow Up Mean?
KFF found that 42% of people who asked AI about physical health, and 58% of those who asked about mental health, did not follow up with a doctor or other health professional. [6] For a drug question, a chatbot answer can be the only answer the person receives. That raises the cost of a wrong or outdated answer.
How Big Is the Gap Between One MI Desk and One Chatbot?
Our calculation: 40 million daily users is 14.6 billion person-days a year (40,000,000 x 365). [1] A department at 1,000 inquiries a month logs 12,000 inquiries a year. [2] The ratio is about 1.2 million to 1.
Read this ratio with care. The numerator counts people asking about every health topic. The denominator counts one department’s inquiries about one company’s products. The ratio shows an order of magnitude. It does not show how many people ask about any single drug.
Where Do Real Questions to LLMs Live Today?
Real questions to LLMs live in five places that we can document: public research datasets, a consented HCP prompt pilot at IQVIA, analyst-written prompt batteries, the platforms’ own logs, and company-maintained libraries such as DrugChatter’s chatmon database. The platforms hold their own logs, and the sources we reviewed describe no route for pharmaceutical companies to access them. The other four differ in who wrote the question, what is stored beside it, and whether the record is about drugs.
What Do the Public LLM Conversation Datasets Contain?
Two public datasets hold real user conversations with chatbots. LMSYS-Chat-1M contains one million conversations with 25 LLMs, collected from the Vicuna demo and the Chatbot Arena website. [7] WildChat contains 1,039,785 conversations with ChatGPT and GPT-4, collected from users who received free access in exchange for opt-in consent to log their transcripts. Its collection ran from April 9, 2023 to May 1, 2024. [8]
How Many Conversations Does Each Dataset Hold Per User?
LMSYS-Chat-1M reports 210,000 unique IP addresses. WildChat reports 204,736. Both papers use IP addresses as a proxy for users, not as a confirmed user count. [7][8] Our calculation: that is about 4.8 conversations per IP address in LMSYS-Chat-1M (1,000,000 / 210,000) and about 5.1 in WildChat (1,039,785 / 204,736).
What Do the Public Datasets Lack for Drug Questions?
Both datasets cover all topics. Neither is organized by drug, brand, indication, or label section. [7][8] Both were collected through research sites, so their users are people who chose to use a research demo. Neither tests the same question against current prescribing information over time. A medical affairs analyst can mine them for drug questions, but the analyst must build the drug layer first.
What Is IQVIA’s HCP Prompt Pilot?
IQVIA’s EMEA thought-leadership team described an ongoing analysis of more than 2,000 real-world prompts from 107 healthcare professionals in the UK, France, and Germany, mapped to 971 clinical topics, with data current as of August 31, 2026. [14] The prompts came from an IQVIA generative AI pilot and depend on participating medical platforms providing consented, governed data. [14]
IQVIA reports that guidelines were the largest single topic in the prompts, followed by treatment selection and diagnosis. Almost a third of clinical topics were patient-specific. [14] This is a useful finding for Medical Affairs, because it shows HCPs asking how evidence applies to a particular patient.
IQVIA also separates two intelligence sources. One is what HCPs ask. The other is how AI answers, tested with structured, persona-based prompts that pharma teams write. [14] In IQVIA’s description, the question source and the answer test are two separate things.
What Are Synthetic Prompt Batteries and What Can They Show?
A synthetic prompt battery is a fixed set of prompts that an analyst writes and runs against one or more models. IQVIA describes this approach as testing citations, framing, share of answer, and gaps versus competitors. [14]
A battery answers a clear question: how does a model respond to this prompt today? It cannot answer a different question: is this the prompt a patient or clinician would type? The analyst’s wording reflects the analyst’s knowledge of the drug, and that knowledge is exactly what a real user may lack.
Original Classification: Six Types of Drug-Question Records
The table below is an original classification built for this article. It is not an established industry standard. We sorted every question source we found by who writes the question, what is stored beside it, and what the record cannot show.
| Record type | Who writes the question | What is stored beside it | Drug-specific? | What it cannot show | Source |
|---|---|---|---|---|---|
| 1. MI inquiry log | A person who contacts the company | Approved reply, topic, product, requester type | Yes | People who never contact the company | [2][3][13] |
| 2. Standard response document | The MI team, anticipating demand | Reviewed and approved answer with references | Yes | The wording real people use | [3] |
| 3. Public LLM conversation dataset | Users of a research chatbot site | The model’s reply in that conversation | No (all topics) | Drug-level coverage; comparison with current labels | [7][8] |
| 4. HCP prompt pilot | 107 consented HCPs in three countries | Clinical topic mapping (971 topics); answer tests are run separately | Partly (clinical topics) | Patient questions; answer quality in the same record | [14] |
| 5. Synthetic prompt battery | An analyst at a company, agency, or vendor | One model’s answer per run | Yes | Whether real people ask the question | [14] |
| 6. Real-question library re-run across models (chatmon) | People who ask LLMs about drugs, per DrugChatter | Each model’s answer, tagged by brand and query type, repeated weekly | Yes | Coverage depends on the question set; no counts are reported in this article | DrugChatter description; not independently verified |
Where Does the DrugChatter Chatmon Database Fit?
Type 6 is the DrugChatter chatmon database. It holds questions that real people ask LLMs about drugs. It stores each question beside the answer that each model returned, tagged by brand and query type. DrugChatter re-runs the questions every week, so each question has a history across models and dates.
DrugChatter’s position is that no other source combines those four properties for drugs: real questions, drug-level tagging, answers from several models stored in the same record, and repeated runs over time. The comparison above supports a narrower statement that readers can check. Each other record type we found lacks at least one of the four. MI logs have the real questions but not the models’ answers. Public datasets have real questions and answers but no drug layer. IQVIA’s pilot has real HCP prompts but keeps answer testing in a separate step. Synthetic batteries have answers but not real questions. [2][7][8][14]
What Does the Chatmon Database Make Possible?
Because question and answer sit in one record, a team can ask three things of one data set. Which questions about our drug do models answer differently from one another? Which answers changed since last week? Which answers disagree with the current label? DrugChatter uses the same data for weekly briefings written for Medical Affairs, Pharmacovigilance, Regulatory Affairs, Medical Information, Marketing, and executive readers.
Why Do Weekly Re-Runs Matter?
An LLM answer is a snapshot. The model, its retrieval sources, and the label can each change between two dates. A single audit cannot separate those three causes. A weekly series lets a team see when an answer changed and check what else changed that week. This is an analytical argument, and this article reports no weekly series.
What Should You Ask Any Vendor That Claims Real User Questions?
Ask five things. Who wrote the questions, and how were they collected? Was consent obtained, and how is personal information removed? What is the size and date range of the set? Are answers stored beside each question, from which models, and how often? Can the vendor show a question and its answer history for one named drug? The answers decide whether a database is a question record or a prompt list with a different name.
Why Does the Source of the Question Change the Accuracy Result?
The source of the question changes the result because accuracy depends on what is asked, of which model, and who rates the answer. Studies of AI drug answers report success rates from 6.7% to 98%, but they test different models, use different question sets, and measure different things. [9][10][11][12]
| Study | Question set | Model tested | Reported result | Source |
|---|---|---|---|---|
| Long Island University (Grossman, Zerilli, Nathan), ASHP Midyear 2023, Poster 8-021 | 45 questions collected January 2022 to April 2023; 39 evaluated | ChatGPT | 10 of 39 satisfactory (25.6%, our calculation) | [9] |
| Iwate Medical University (Matsuura et al.), ASHP Midyear 2023, Poster 8-023 | Side effects of 30 FDA-approved drugs, queried April to June 2023, compared with Lexicomp | ChatGPT | 2 accurate, 2 partly accurate, 26 inaccurate (accurate: 6.7%, our calculation) | [9] |
| Andrikyan et al., BMJ Quality & Safety, 2025 | 10 patient questions for each of 50 frequently prescribed drugs (500 answers) | Microsoft Bing Copilot | Mean accuracy 88.7%; mean completeness 76.7%; 66% of 20 expert-reviewed answers judged potentially harmful | [10] |
| Helgestad et al., Danish Medical Journal, 2025 | 49 real inquiries to a Danish regional pharmaceutical counselling service, rewritten with minimal context | ChatGPT 3.5 and 4.0 | 48 of 49 (98%) rated at least satisfactory for GPT-4.0; 80% for GPT-3.5; none rated dangerous | [11] |
| BMJ Open audit, 2026 (volume 16, issue 4) | 50 adversarial prompts across five health topics (not drug-specific); 250 responses | Gemini 2.0, DeepSeek V3, Meta AI (Llama 3.3), ChatGPT 3.5, Grok 2; free versions, February 2025 | 49.6% of responses problematic; 19.6% highly problematic | [12] |
What Did the 2023 ASHP Studies Find About ChatGPT and Drug Questions?
The Long Island University study evaluated 39 questions. ChatGPT gave 10 satisfactory answers and 29 unsatisfactory ones. For 11 of the 29, ChatGPT gave no direct answer and offered general background. It supplied references for only 8 of the 28 questions that received a direct response, and every listed reference did not exist. [9] The authors wrote that ChatGPT “was unable to provide an accurate and complete response to most questions.” [9]
The Iwate study compared ChatGPT’s side-effect information for 30 FDA-approved drugs with Lexicomp. It found 26 inaccurate answers, 2 partly accurate, and 2 accurate. Only one pharmacist evaluated the side effects, which the authors list as a limitation. [9]
What Did Andrikyan and Colleagues Find About Bing Copilot and Patient Drug Questions?
The German cross-sectional study asked Bing Copilot 10 patient questions about each of 50 drugs among the most frequently prescribed in the US outpatient market in 2020. The 500 answers had a mean accuracy of 88.7% and a mean completeness of 76.7%. [10] The mean Flesch Reading Ease Score was 37.2, which indicates difficult, college-level text. [10]
Experts reviewed 20 of the answers. They judged 66% potentially harmful and 22% likely to lead to death or severe harm if followed. [10] The reviewed subset is small, and the study reports wide confidence intervals for those figures.
What Happened When Researchers Used Real Inquiries From a Drug Counselling Service?
The Danish study took consecutive real inquiries to the joint pharmaceutical counselling service of the Central and North Denmark regions from autumn 2023. Physicians’ answers served as the gold standard. Of 49 analyzed inquiries, GPT-4.0 answered 48 at a level rated at least satisfactory, and GPT-3.5 answered about 80% (the abstract reports 39 and the results section reports 38, an inconsistency in the source). [11]
The same study reports that neither model gave references or prioritized what was most clinically relevant when it offered several explanations. [11] It is the only study in the table that starts from real inquiries to a drug information service. It also reports the best results. We cannot say the first fact causes the second, because the model versions, raters, and rating scales also differ.
What Did the 2026 BMJ Open Audit Find About Five Chatbots?
The audit used a red-teaming design. Its authors wrote 50 prompts built to strain models toward misinformation, 10 in each of five categories: cancer, vaccines, stem cells, nutrition, and athletic performance. [12] Of 250 responses, 49.6% were problematic (30% somewhat, 19.6% highly). The differences among chatbots were not statistically significant overall, though Grok produced more highly problematic responses than expected. [12]
The audit also asked for references. Chatbots returned 1,013 of the 1,250 requested references (about 81%). Median reference completeness was 40%, and no chatbot produced a fully complete and accurate reference list for any prompt. [12]
What Does the Spread of Results Tell a Medical Affairs Team?
The reported rates in the table run from 6.7% to 98%. They cannot be ranked, because the metrics differ: accurate, at least satisfactory, and not problematic are three different tests. The table supports four narrower statements.
- No single number describes how accurate AI is on drug questions.
- The studies tested ChatGPT 3.5, GPT-4.0, Bing Copilot, Gemini 2.0, DeepSeek V3, Meta AI, and Grok 2. [9][10][11][12] None of them tests the models people use in November 2026.
- Only one study began with real inquiries from a drug information service. [11]
- Three of the five studies report problems with references. [9][11][12]
The practical conclusion is about method. A team that wants to know how models answer questions about its own drug needs its own question set, built from real wording, re-run as models and labels change.
Does a Chatbot Question Count as an Unsolicited Request Under FDA Guidance?
No. FDA’s draft guidance addresses requests made to a firm and the firm’s response. A person who asks a chatbot has not made a request to the firm. The guidance predates consumer chatbots, and FDA’s page still lists it as a draft with the statement “Not for implementation. Contains non-binding recommendations.” [4] The points below are our interpretation. Legal and Regulatory Affairs should review them before any operating decision.
What Does the FDA Draft Guidance Say About Unsolicited Requests?
The December 2011 draft guidance defines the term. “Unsolicited requests are those initiated by persons or entities that are completely independent of the relevant firm.” [4] It adds: “Requests that are prompted in any way by a manufacturer or its representatives are not unsolicited requests.” [4]
For non-public responses, the draft recommends that firms provide the response only to the individual making the request directly to the firm. It also recommends including a copy of the FDA-required labeling, a prominent notice that FDA has not approved or cleared the product for the requested use, and a complete list of references. [4]
What Records Does the Draft Guidance Say a Firm Should Keep?
The draft says a firm should maintain records of the nature of the request, including the name, address, and affiliation of the requestor. It also lists records of the information provided and any follow-up inquiries from the requestor. [4] This is the regulatory basis for the MI database described earlier. [4]
Why Is an AI Chat Question Different From an MI Inquiry?
An MI inquiry is addressed to the company, answered by the company, and logged by the company. A chatbot question is addressed to a platform. The company does not receive it, answer it, or log it. The guidance’s record-keeping elements describe a relationship that does not exist in a chatbot session. [4]
Why Should Legal Review Any Use of Question Data?
A company that writes its own prompts and runs them against models is not receiving unsolicited requests. A company that licenses or builds a database of real user questions faces a different set of questions: consent, privacy, and how the data may inform content. IQVIA’s own description conditions access on consented, governed platform data. [14] Any team that uses such data should confirm its terms with Legal first.
What Does a Real-Question Database Change for Each Function?
A real-question database gives each function a different starting point. The functions below read the same question record for different reasons. The recommendations in this section are our analysis, not findings from the cited studies.
What This Means for Medical Information
MI teams own the answers, so they can test them first. Take the ten most frequent inquiries for a product, find the closest real-world question in an LLM question set, and compare the model’s answer with the approved SRD. [3] Where the SRD says one thing and the model says another, the MI team has a content gap or a monitoring finding.
MI teams can also correct the 2009 pattern in which 54% of surveyed departments used verbatim content for trend analysis. [2] Coding the exact wording of each inquiry gives an MI team material it can compare with the wording in an LLM question set.
What This Means for Medical Affairs
IQVIA reports that guidelines were the largest topic in HCP prompts and that almost a third of clinical topics were patient-specific. [14] A Medical Affairs team can use this to ask whether its publications and congress materials answer patient-specific questions in plain language that a model can retrieve. McKinsey suggested that inquiry data could inform off-label questions sent to R&D and heat maps by product and geography. [13] A question database adds the AI channel to the same heat map.
What This Means for Pharmacovigilance
An inaccurate AI answer about a labeled risk is a communications finding. It is not an adverse event report, and it does not need to enter a safety database by itself. A contact that describes an adverse event is different. McKinsey notes that adverse events must be reported to the European Medicines Agency within 14 days and that many are first reported through Medical Information. [13] If a caller says a chatbot told them to take a drug in a way that caused harm, the MI intake process should capture the adverse event first and the chatbot detail second.
The Andrikyan study gives the reason for care. Expert reviewers judged 22% of 20 reviewed Bing Copilot answers likely to lead to death or severe harm if followed. [10] The sample is small. It is enough to show that wrong answers on dosing and interactions can matter clinically.
What This Means for Regulatory Affairs
Regulatory Affairs owns the label, so it can state what the reference answer is. A model answer can only be called outdated if someone has fixed the label version and date it is compared against. Record both for each comparison. The FDA draft guidance recommends that firms provide current FDA-required labeling in responses to unsolicited requests. [4] The same principle applies to a monitoring file: attach the label.
What This Means for Brand and Competitive Intelligence Teams
Brand teams write prompts using brand language. Patients and many clinicians use other words. A real-question library shows the words people use for an indication, a side effect, or a competitor. IQVIA lists gaps versus competitors among the items tested in structured prompts. [14] A real-question set lets a team check whether those prompts match what people ask.
How Can a Team Connect an MI Database to AI Question Monitoring?
A team connects the two in four steps: export real wording from MI, pair each question with its label section and approved reply, re-run the question across models on a schedule, and classify each difference from the reference answer. The steps below are a working method, not a published standard.
Step 1: Export the Verbatim Wording of MI Inquiries
Pull the exact text of inquiries for one product over a fixed period. Remove names and any personal data according to company policy. The draft FDA guidance lists the requestor’s name, address, and affiliation among the records a firm keeps for the request itself, so the monitoring copy should not carry those fields. [4]
Step 2: Pair Each Question With a Label Section and an SRD
For each question, record the section of the current prescribing information that answers it, the label version date, and the SRD that covers it. [3][4] This pair is the reference answer. Without it, a reviewer judges the model’s answer by memory.
Step 3: Re-Run the Question Across Models on a Schedule
Run each question against each model you want to track and store the answer, the model name, the model version if shown, the date, and whether web search was on. Weekly is the interval DrugChatter uses for chatmon. A team can choose another interval. The Danish and BMJ Open studies show why a schedule matters: each tested one snapshot of specific model versions. [11][12]
Step 4: Classify Each Difference and Route It
The table below shows the working categories we use for routing. They are an original classification for this article.
| Difference from reference answer | What the reviewer checks | Usual first owner |
|---|---|---|
| Outdated statement | Does the answer match a prior label version but not the current one? | Regulatory Affairs |
| Missing safety information | Does the label list a warning, contraindication, or interaction that the answer omits? | Pharmacovigilance |
| Unsupported claim | Does any source support the statement? | Medical Affairs |
| Citation failure | Does the cited source exist, and does it say what the answer claims? | Medical Information |
| Off-label content | Does the answer describe a use outside the approved indication? | Legal and Regulatory Affairs |
The citation check has support in the literature. The Long Island University study found that every reference ChatGPT listed did not exist. [9] The BMJ Open audit found that no chatbot produced a fully complete and accurate reference list for any prompt. [12]
Methodology and Limits of This Analysis
Every statistic in this article comes from a cited source or is labeled as our calculation. This article reports no new counts from the chatmon database and no new tests of any model.
Which Figures Are Calculations?
- The 42-percentage-point gap between volume (96%) and verbatim content (54%) is 96 minus 54, from the 2009 survey. [2]
- The cost range of $25,000 to $60,000 a month, and $300,000 to $720,000 a year, multiplies 1,000 inquiries by the $25 to $60 range in the CLS white paper. [2][3]
- The 14.6 billion person-days figure is 40 million x 365. The 1.2 million to 1 ratio divides that by 12,000 inquiries a year. [1][2]
- The 4.8 and 5.1 conversations per IP address divide the dataset sizes by the reported IP counts. [7][8]
- The 25.6% and 6.7% rates divide 10 by 39 and 2 by 30. [9]
What Are the Limits?
- The MI survey is from December 2008, has 26 respondents, and the authors note possible multiple respondents from the same company. [2]
- The cost range comes from a vendor white paper. [3]
- The OpenAI figures reach us through Healthcare Dive and Fortune, not through a direct reading of the OpenAI report. [1][5]
- The IQVIA pilot covers 107 HCPs in three countries and is a company blog description. [14]
- The accuracy studies test older models and different measures. [9][10][11][12]
- The description of the chatmon database is DrugChatter’s own and has not been independently verified.
What Happens Next for Medical Information Teams?
This section is a forecast. It is our judgment, not a documented event.
We expect MI leaders to be asked a new question: what are people asking AI about our drugs, and how do the answers compare with our SRDs? OpenAI’s own reported usage figures make the question hard to avoid. [1][5] The MI function already owns the vocabulary of inquiries, SRDs, and records. It is well placed to own the AI-question equivalent if it adds one habit: keeping the exact wording of every question.
Key Takeaways
- OpenAI reports more than 40 million daily and more than 230 million weekly users asking ChatGPT health questions. [1][5]
- KFF found that 32% of US adults used AI chatbots for health information in the past year, and 42% of those asking about physical health did not follow up with a health professional. [6]
- An MI database records only people who contact the company, so it cannot see questions asked of chatbots. [2]
- In a 2009 MI survey, 96% of respondents used inquiry volume for trend finding and 54% used verbatim content. [2]
- Public datasets of real chatbot conversations exist but cover all topics. [7][8]
- IQVIA’s pilot analyzed more than 2,000 real HCP prompts from 107 HCPs, and keeps answer testing as a separate step. [14]
- Published accuracy results for AI drug answers range from 6.7% to 98% on different measures and cannot be ranked. [9][10][11][12]
- DrugChatter’s position is that chatmon is the only drug-specific database that stores real questions people put to LLMs beside each model’s answer, re-run weekly. Readers can test that claim with the five vendor questions above.
FAQ
What is the difference between an MI inquiry and a question typed into a chatbot?
An MI inquiry goes to the company, which answers and logs it. A chatbot question goes to a platform. The company does not receive it. [2][4]
Can a pharmaceutical company see what people ask chatbots about its drug?
Not directly. OpenAI reports aggregated figures from deidentified conversations. [5] IQVIA says access to HCP prompt data depends on participating platforms providing consented, governed data. [14]
Does a chatbot answer count as a company response to an unsolicited request?
No. FDA’s draft guidance addresses requests made to a firm and the firm’s response to them. [4] A chatbot answer is not a firm’s response. Legal should review any plan that uses chatbot content.
What should an MI team log when a caller says a chatbot gave them an answer?
Log the exact wording of the question and the answer if the caller can supply them, the model name, and the date. Capture any adverse event first, because many adverse events are first reported through Medical Information. [13]
How often should a team re-run its questions across models?
We know of no regulatory standard for the interval. The 2011 FDA draft guidance predates consumer chatbots. [4] DrugChatter re-runs its chatmon questions weekly. A team should choose an interval that matches how often its label and the major models change.
Who should own AI answer monitoring?
Medical Information owns the approved replies, Regulatory Affairs owns the label, Pharmacovigilance owns safety follow-up, and Legal owns off-label review. Most teams will need one coordinator and a routing table such as the one in this article.
Does a question database replace an MI database?
No. The MI database records requests made to the company and the approved replies. [2][4] A question database records what people ask models. The two answer different questions and work best together.
How can a team test a vendor’s claim that it holds real user questions?
Ask for the collection method, consent and privacy handling, size and date range, which models answered and how often, and a question-and-answer history for one named drug. Public datasets such as WildChat document their collection method in a published paper, and a vendor can be held to the same standard. [8]
How much does it cost to handle one MI inquiry?
A CLS white paper gives a range of $25 to $60 per query for answering, handling, and logging, with some costs above $100. [3] The figure is a vendor estimate, not a benchmark.
Do the 2023 to 2025 accuracy studies describe the chatbots people use today?
No. The studies tested ChatGPT 3.5, GPT-4.0, Bing Copilot, Gemini 2.0, DeepSeek V3, Meta AI, and Grok 2. [9][10][11][12] Models change, so a result from one version does not carry to a later one. That is the case for continuous testing.
References
- Healthcare Dive. (2026, January). [Dive Brief reporting OpenAI’s AI as a Healthcare Ally report]. https://www.healthcaredive.com/news/40-million-use-chatgpt-health-questions-openai/808861/
- Chow, C., Burks, E. J., Patel, S., Walsh, L., Kern Sliwa, J., & Danyluk, A. P. (2009). Identification, analysis, and communication of trends and signals from medical information inquiries in the pharmaceutical industry. Ortho-McNeil Janssen Scientific Affairs, LLC. https://pharmafellows.rutgers.edu/wp-content/uploads/resources/2009-identificationanalysisandcommunicationoftrendsandsignalsfrommedicalinformationinquiriesinthepharmaceuticalindustry.pdf
- Fortune. (2026, January 7). OpenAI launches ChatGPT Health in a push to become a hub for personal health data. https://fortune.com/2026/01/07/openai-launches-chatgpt-health-in-a-push-to-become-a-hub-for-personal-health-data/
- KFF. (2026, March 25). KFF Tracking Poll on Health Information and Trust: Use of AI for health information and advice. https://www.kff.org/health-information-trust/poll-1-in-3-adults-are-turning-to-ai-chatbots-for-health-information-equaling-the-share-who-use-social-media-for-health/
- Evers, M., Hartmann, J., Pradel, C., Suresh, B., & Westra, A. (2018). Medical information for the future. McKinsey & Company, Pharmaceuticals and Medical Products Practice. https://mckinsey.de/~/media/McKinsey/Industries/Pharmaceuticals%20and%20Medical%20Products/Our%20Insights/How%20pharma%20manufacturers%20can%20enhance%20their%20medical%20information%20teams/Medical-information-for-the-future.pdf
- CLS. (2022). Medical information in the digital age [White paper]. Medical Affairs Professional Society. https://medicalaffairs.org/wp-content/uploads/2022/09/CLS-whitepaper-Medical-Information-in-the-Digital-Age.pdf
- U.S. Food and Drug Administration. (2011, December). Responding to unsolicited requests for off-label information about prescription drugs and medical devices [Draft guidance]. Docket FDA-2011-D-0868. https://fda.gov/media/82660/download
- Zheng, L., Chiang, W.-L., Sheng, Y., Li, T., Zhuang, S., Wu, Z., Zhuang, Y., Li, Z., Lin, Z., Xing, E. P., Gonzalez, J. E., Stoica, I., & Zhang, H. (2023). LMSYS-Chat-1M: A large-scale real-world LLM conversation dataset. arXiv:2309.11998. https://arxiv.org/abs/2309.11998
- Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., & Deng, Y. (2024). WildChat: 1M ChatGPT interaction logs in the wild. arXiv:2405.01470. https://arxiv.org/abs/2405.01470
- Bailey, O., Bennet, K., Lutzmayer, S., & Rodriguez, L. (2026, October 16). Beyond GEO: Turning AI conversations into strategic intelligence. IQVIA EMEA International Thought Leadership. https://www.iqvia.com/de-de/locations/emea/blogs/2026/10/beyond-geo-turning-ai-conversations-into-strategic-intelligence
- Drug Topics. (2023, December). ChatGPT provides inaccurate responses to questions about drug information [Reporting on Grossman, Zerilli, & Nathan, Poster 8-021, and Matsuura, Nagasawa, Nguyen, et al., Poster 8-023, ASHP Midyear Clinical Meeting, Anaheim, CA]. https://www.drugtopics.com/view/chatgpt-provides-inaccurate-responses-to-questions-about-drug-information
- Andrikyan, W., Sametinger, S. M., Kosfeld, F., Jung-Poppe, L., Fromm, M. F., Maas, R., & Nicolaus, H. F. (2025). Artificial intelligence-powered chatbots and search engines: A cross-sectional study on the quality and risks of drug information for patients. BMJ Quality & Safety, 34(2), 100. https://qualitysafety.bmj.com/content/34/2/100
- Helgestad, O. K. L., Hjelholt, A. J., Vestergaard, S. V., Azuz, S., Sædder, E. A., & Overvad, T. F. (2025). ChatGPT versus physician-derived answers to drug-related questions. Danish Medical Journal, 72(1), A05240360. https://ugeskriftet.dk/dmj/chatgpt-versus-physician-derived-answers-drug-related-questions
- Generative artificial intelligence-driven chatbots and medical misinformation: An accuracy, referencing and readability audit. (2026). BMJ Open, 16(4), e112695. https://bmjopen.bmj.com/content/16/4/e112695






