{"id":1286,"date":"2026-11-11T07:23:00","date_gmt":"2026-11-11T12:23:00","guid":{"rendered":"https:\/\/drugchatter.com\/insights\/?p=1286"},"modified":"2026-11-04T09:12:04","modified_gmt":"2026-11-04T14:12:04","slug":"pharma-never-had-a-dataset-of-what-people-ask-ai-about-drugs-now-it-does","status":"publish","type":"post","link":"https:\/\/drugchatter.com\/insights\/pharma-never-had-a-dataset-of-what-people-ask-ai-about-drugs-now-it-does\/","title":{"rendered":"Pharma Never Had a Dataset of What People Ask AI About Drugs. Now It Does."},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

More than 5% of all ChatGPT messages worldwide are about health care, and roughly one in four of OpenAI’s more than 800 million regular users sends at least one health-related prompt each week, according to OpenAI’s January 2026 report ‘AI as a Healthcare Ally’ as relayed by trade press. [1][2] No pharmaceutical company, regulator, or medical-information department sees those questions. OpenAI holds them. OpenAI published only aggregates.<\/p>\n\n\n\n

That gap defines a new problem for Medical Affairs, Pharmacovigilance, Regulatory, and Brand teams. The questions people ask about a drug now shape what they believe about it. The answers an AI model returns now shape what they do. Until recently, nobody outside the AI companies could read both sides of that exchange at the drug level, by model, over time.<\/p>\n\n\n\n

This article does four things. It reviews what pharma knew about patient questions before AI. It sets out what the public evidence says about AI drug answers. It proposes a four-type classification of LLM drug-question data. It then explains what DrugChatter’s question-and-answer library contains, and how DrugChatter positions it against the other three types. DrugChatter’s position is that its library gives pharma something no platform report, panel vendor, or analyst-written prompt list provides: real questions about named drugs, paired with the answers each model gave, on a schedule.<\/p>\n\n\n\n

The Short Answer: What Is the New Dataset and Why Does Pharma Lack It?<\/strong><\/h2>\n\n\n\n

The new dataset is the set of questions people put to large language models about drugs, matched to the answers the models return. Pharma lacks it because the conversations sit inside AI platforms, and the platforms release only aggregate statistics. [2][3]<\/p>\n\n\n\n

Five findings frame the rest of the article. Findings 1 to 3 come from published sources. Findings 4 and 5 are original analysis and are labeled as such.<\/p>\n\n\n\n

    \n
  1. OpenAI reports that more than 5% of ChatGPT messages globally are about health care. [2][3]<\/li>\n\n\n\n
  2. OpenAI reports that more than 40 million people submit health-related prompts every day. [2][3]<\/li>\n\n\n\n
  3. In a 2010 search-log study of 6 million web searchers, about 10% of users who searched for both paroxetine and pravastatin also searched for hyperglycemia-related terms. The rate for users who searched for neither drug was about 0.3%. [4]<\/li>\n\n\n\n
  4. Analysis: the platforms that hold LLM question logs release them as aggregates or modeled estimates. The only public question-level view of LLM drug questions is what researchers and monitoring vendors build themselves.<\/li>\n\n\n\n
  5. Analysis: a question has little value to a drug-safety or medical-information team unless it is stored with the answer, the model that gave it, and the date.<\/li>\n<\/ol>\n\n\n\n

    What Did Pharma Know About Patient Questions Before AI?<\/strong><\/h2>\n\n\n\n

    Before LLMs, pharma learned what people asked from three places: its own inbound channels, public search data, and academic studies of search logs. Each had limits. None showed the answer the person received.<\/p>\n\n\n\n

    What Do Medical Information Inquiry Logs Capture?<\/strong><\/h3>\n\n\n\n

    Medical Information departments log inbound requests from health care professionals and patients who contact the company. These logs show what people ask the company. They do not show what people ask elsewhere. A person who has a question about a drug and never calls the manufacturer does not appear.<\/p>\n\n\n\n

    FDA’s draft guidance on unsolicited requests recognizes this channel. It tells firms to keep records of each unsolicited request, the information provided, and any follow-up. [6] The guidance therefore assumes a company-held log of questions. It does not assume the company sees questions asked of third parties.<\/p>\n\n\n\n

    What Can Web Search Logs Show About Drug Questions?<\/strong><\/h3>\n\n\n\n

    Search logs showed more, and researchers used them. In 2013, White, Tatonetti, Shah, Altman, and Horvitz published a pharmacovigilance study built on anonymized search logs from consenting users of a browser add-on. [4] The study covered 12 months of 2010, 6 million web searchers, and 82 million drug, symptom, and condition queries. [4]<\/p>\n\n\n\n

    Search logs gave a list of queries. They did not give an answer, because a search engine returns a ranked list of pages, not a single statement. A chatbot returns one statement. That statement is the thing a safety or regulatory reviewer can check against a label.<\/p>\n\n\n\n

    How Did FDA Treat Unsolicited Questions Before AI?<\/strong><\/h3>\n\n\n\n

    FDA’s draft guidance ‘Responding to Unsolicited Requests for Off-Label Information About Prescription Drugs and Medical Devices’ appeared in the Federal Register on December 30, 2011 (Docket No. FDA-2011-D-0872). [6] It covers requests made through electronic media. [6] It says a response to a private request should be tailored to the question, balanced, and non-promotional, and should include the approved labeling and a statement that FDA has not approved the product for the use in question. [6]<\/p>\n\n\n\n

    The guidance rests on one idea: a real, unprompted question is different from a question the company wrote. Section ‘Where Is the Compliance Line?’ below returns to this point, because it matters for how a company may use LLM question data.<\/p>\n\n\n\n

    Gap 1: The Company Sees Only Questions Sent to the Company<\/strong><\/h4>\n\n\n\n

    An inquiry log is a sample of people who chose to contact the manufacturer. It excludes everyone who asked a search engine, a pharmacist, or now a chatbot.<\/p>\n\n\n\n

    Gap 2: Search Logs Do Not Contain the Answer<\/strong><\/h4>\n\n\n\n

    A search query shows interest. It does not show what the person read or believed afterward.<\/p>\n\n\n\n

    Gap 3: Neither Source Is Model-Specific<\/strong><\/h4>\n\n\n\n

    An inquiry log has no ‘model’ field. Two chatbots can answer the same question differently. A team that monitors one model does not know what a second model says.<\/p>\n\n\n\n

    How Many People Ask ChatGPT About Health and Drugs?<\/strong><\/h2>\n\n\n\n

    OpenAI reports more than 40 million daily health-related users and a health share above 5% of messages. [2][3] These are the largest published figures for any single LLM and they come from the platform itself.<\/p>\n\n\n\n

    \n

    More than 5% of all ChatGPT messages globally are about health care, according to OpenAI’s January 2026 report ‘AI as a Healthcare Ally.’ [2][3]<\/p>\n<\/blockquote>\n\n\n\n

    What Does the OpenAI Health Report Say?<\/strong><\/h3>\n\n\n\n

    The report is titled ‘AI as a Healthcare Ally,’ dated January 2026, and runs 18 pages. [3] It states that more than 5% of ChatGPT messages globally concern health care. [3] It states that, of more than 800 million regular users, roughly one in four submits at least one health-related prompt per week. [3] It states that more than 40 million people submit health-related prompts daily. [3]<\/p>\n\n\n\n

    Axios, which received the report first, adds three figures: 1.6 to 1.9 million health insurance questions per week, nearly 600,000 health care messages per week from underserved rural communities, and 7 in 10 health care conversations outside normal clinic hours. [2] Axios describes the underlying data as anonymized interactions. [2] Neither article explains how OpenAI classified a message as health-related. [2][3] The full PDF is the place to check that.<\/p>\n\n\n\n

    Caveat: These Are Platform Self-Reports<\/strong><\/h4>\n\n\n\n

    I have not independently verified the report. I cite it through two trade-press accounts. A researcher who needs to quote these figures should read the primary PDF.<\/p>\n\n\n\n

    How Many People Is One in Four of 800 Million?<\/strong><\/h3>\n\n\n\n

    Analysis: one in four of 800 million is about 200 million people per week. This is my arithmetic on OpenAI’s rounded figures, not a number OpenAI reported. [3] The 40 million daily figure and the roughly 200 million weekly figure are consistent with each other, since a weekly audience should be larger than a daily one.<\/p>\n\n\n\n

    The figures describe health in general. They do not say how many of those prompts name a prescription drug. No public source I found gives a drug-specific share. That missing share is the number a pharma team most wants.<\/p>\n\n\n\n

    Why Do Questions Outside Clinic Hours Matter?<\/strong><\/h3>\n\n\n\n

    Axios reports that 7 in 10 health care conversations happen outside normal clinic hours. [2] A medical-information line with business hours cannot see most of this demand, and a nurse or pharmacist is not present when it happens. Analysis: this makes an AI answer the only drug information source available to many people at the moment they ask.<\/p>\n\n\n\n

    What Else Do We Know About Who Uses ChatGPT?<\/strong><\/h4>\n\n\n\n

    A 2025 NBER working paper by Chatterji and colleagues tracked ChatGPT’s consumer product from its November 2022 launch through July 2025, when about 10% of the world’s adult population had used it. [1] It found that non-work messages rose from 53% to over 70% of usage, and that Practical Guidance, Seeking Information, and Writing made up nearly 80% of conversations. [1] The paper uses a privacy-preserving automated pipeline on a representative sample of conversations. [1] It is a primary source for the broad pattern. It is not a drug-level dataset.<\/p>\n\n\n\n

    Who Owns LLM Question Data, and Who Can Buy It?<\/strong><\/h2>\n\n\n\n

    The platforms own it. A small number of vendors license portions of it, convert them to modeled volumes, and resell the result. Pharma teams can buy modeled keyword volumes. They cannot buy the raw log.<\/p>\n\n\n\n

    What Is a Licensed Prompt-Volume Dataset?<\/strong><\/h3>\n\n\n\n

    Profound, a general AI-visibility vendor, documents a product called Prompt Volumes. Its help page describes it as a dataset of prompts users submit to AI answer engines, currently ChatGPT, Gemini, and Perplexity. [7] The page says Profound builds it by licensing conversations, cleaning them, and applying probabilistic modeling. [7] It says the data is anonymized, aggregated, and scrubbed of personal information. [7]<\/p>\n\n\n\n

    The same page lists limits. Coverage is strongest in the US, UK, and select European markets. US history starts in January 2025, and other regions start in January 2026. Figures are modeled estimates, not raw counts. The page does not state a panel size. [7]<\/p>\n\n\n\n

    What This Means for a Drug Team<\/strong><\/h4>\n\n\n\n

    A modeled volume tells a team that a topic is asked about, and how often relative to other topics. It does not show the answer a model returned. It does not give a pharmacovigilance reviewer a statement to check against a boxed warning. It is a demand measure, not an accuracy measure. Profound’s product is general-purpose, and its documentation does not describe pharma-specific handling. [7]<\/p>\n\n\n\n

    Can a Pharma Company Get OpenAI’s Raw Health Logs?<\/strong><\/h3>\n\n\n\n

    I found no public mechanism for a pharmaceutical company to obtain them. OpenAI described its figures as aggregate, anonymized findings. [2] The NBER study’s authors state that they classify a representative sample using a privacy-preserving pipeline. [1] That design keeps individual conversations away from outside readers, which is appropriate for privacy and also means pharma cannot read them.<\/p>\n\n\n\n

    What Is Left for Teams That Need Question-Level Data?<\/strong><\/h3>\n\n\n\n

    Two options remain. A team can write its own prompts and run them. Or a team can maintain a library of questions phrased the way people ask, run it across models on a schedule, and store every answer. DrugChatter’s chatmon system uses the second approach. The next sections explain why the difference matters.<\/p>\n\n\n\n

    How Accurate Are AI Answers to Real Patient Drug Questions?<\/strong><\/h2>\n\n\n\n

    The published evidence shows mixed results. A model can be accurate on most statements and still produce a minority of answers that experts judge potentially harmful. The best-documented example is a study of Bing Copilot in BMJ Quality and Safety. [5]<\/p>\n\n\n\n

    What Did the BMJ Quality and Safety Chatbot Study Find?<\/strong><\/h3>\n\n\n\n

    Andrikyan and colleagues asked Bing Copilot, in ‘balanced’ mode, 10 patient questions about each of 50 drugs, which gave 500 answers. They generated the answers in April 2023. [5] The study is published in BMJ Quality and Safety, volume 34, issue 2. [5]<\/p>\n\n\n\n

    The study reports these figures: [5]<\/p>\n\n\n\n

    Measure<\/th>Result<\/th>Source<\/th><\/tr><\/thead>
    Flesch Reading Ease, overall mean (SD)<\/td>37.2 (17.7), college-level difficulty<\/td>[5]<\/td><\/tr>
    Completeness, mean (SD)<\/td>76.7% (32.0%)<\/td>[5]<\/td><\/tr>
    Accuracy, mean (SD)<\/td>88.7% (22.3%)<\/td>[5]<\/td><\/tr>
    Answers with accuracy below 100%<\/td>126 of 484 (26.0%)<\/td>[5]<\/td><\/tr>
    Expert-reviewed answers judged potentially harmful (20 answers)<\/td>66% (95% CI 50% to 85%)<\/td>[5]<\/td><\/tr>
    Of those: moderate to mild harm<\/td>42% (95% CI 25% to 60%)<\/td>[5]<\/td><\/tr>
    Of those: severe harm or death<\/td>22% (95% CI 10% to 40%)<\/td>[5]<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    The authors conclude that the chatbot generally gave complete and accurate drug information, but that experts judged many answers incorrect or potentially harmful, and that answer complexity may limit patient understanding. [5] They advise health care professionals to be cautious about recommending AI-powered search engines until more reliable alternatives exist. [5]<\/p>\n\n\n\n

    Limitation: The Harm Figures Come From 20 Answers<\/strong><\/h4>\n\n\n\n

    The harm percentages rest on expert review of 20 answers, and the confidence intervals are wide. [5] A reader should not treat 66% as the share of all Copilot answers that were potentially harmful. The 126 of 484 accuracy figure rests on a larger set.<\/p>\n\n\n\n

    Limitation: One Model, One Month<\/strong><\/h4>\n\n\n\n

    The study tested one product in April 2023. [5] Model versions change. Label text changes. A result from 2023 does not describe any system in November 2026.<\/p>\n\n\n\n

    What Does the Study Not Tell You?<\/strong><\/h3>\n\n\n\n

    The study used 10 fixed questions per drug. [5] The authors chose them. The study therefore measures accuracy on a defined question set. It does not measure how real users phrase their questions, which questions they ask most, or whether accuracy differs between common and uncommon phrasings.<\/p>\n\n\n\n

    Analysis: this is the central limit of every analyst-written test. The result can only be as representative as the question list. A question library built from real phrasing narrows that gap.<\/p>\n\n\n\n

    What Did Search Logs Reveal in 2010 That Labels Did Not Yet Say?<\/strong><\/h2>\n\n\n\n

    They revealed a drug interaction signal before it was widely known. The White study is the clearest precedent for treating public question data as a pharmacovigilance input, and it shows what a question dataset can do when the researcher can read it. [4]<\/p>\n\n\n\n

    How Was the Paroxetine and Pravastatin Study Designed?<\/strong><\/h3>\n\n\n\n

    The logs came from consenting users of a browser add-on and covered Google, Bing, and Yahoo searches during 2010. [4] The researchers studied the pairing of paroxetine and pravastatin. A paper reporting that interaction appeared in 2011, after the log period. [4] They grouped users by which drugs they searched and checked each group for hyperglycemia-related terms. [4]<\/p>\n\n\n\n

    What Were the Results?<\/strong><\/h3>\n\n\n\n
    User group<\/th>Approximate share also searching hyperglycemia terms<\/th>Source<\/th><\/tr><\/thead>
    Searched both paroxetine and pravastatin<\/td>About 10%<\/td>[4]<\/td><\/tr>
    Searched paroxetine only<\/td>About 5%<\/td>[4]<\/td><\/tr>
    Searched pravastatin only<\/td>About 4%<\/td>[4]<\/td><\/tr>
    Background<\/td>About 0.3%<\/td>[4]<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    The reporting ratios were 2.747 (95% CI 2.438 to 3.094) with pravastatin as the expected baseline, and 2.461 (95% CI 2.189 to 2.767) with paroxetine as the baseline. [4] The authors validated the method on 31 known hyperglycemia-causing drug pairs and 31 unrelated pairs. The area under the curve was 0.8189 with the full term list. At a reporting-ratio threshold of 2, the estimated false positive rate was 12.5% across 62 pairings. [4]<\/p>\n\n\n\n

    Analysis (derived from the table): the both-drug group’s rate is about 2 times the paroxetine-only rate, 2.5 times the pravastatin-only rate, and about 33 times the background rate. These ratios are my division of the reported approximate shares, not figures from the paper.<\/p>\n\n\n\n

    What the Authors Said About Their Own Limits<\/strong><\/h4>\n\n\n\n

    The authors note that confounders such as demographics were not observable, and that a single-pair result could reflect chance. [4] They conclude that anonymized drug-interaction signals can be mined from search logs and that logs are inexpensive to collect. [4]<\/p>\n\n\n\n

    What Does This Imply for LLM Question Data?<\/strong><\/h3>\n\n\n\n

    A chatbot question is longer and more specific than a search query. A person who asks ‘can I take paroxetine with pravastatin, and why do I feel shaky’ states the drugs, the symptom, and the concern in one sentence. Analysis: that structure should make questions at least as informative as query logs. This is an inference. No peer-reviewed study I found has tested LLM question logs for pharmacovigilance signals in the way White did for search logs.<\/p>\n\n\n\n

    What Are the Four Types of LLM Drug-Question Data?<\/strong><\/h2>\n\n\n\n

    I group the data sources into four types. This is an original classification created for this article. It is not an established industry taxonomy.<\/p>\n\n\n\n

    Type<\/th>Description<\/th>Example<\/th>Holds the answer?<\/th>Drug-level?<\/th>Available to pharma?<\/th><\/tr><\/thead>
    1. Platform-held logs<\/td>Full conversations stored by the AI company<\/td>OpenAI’s usage data behind [1][3]<\/td>Yes<\/td>Not in public releases<\/td>Aggregates only [2][3]<\/td><\/tr>
    2. Licensed, modeled panels<\/td>Licensed conversations, cleaned and modeled into volumes<\/td>Profound Prompt Volumes [7]<\/td>Not described [7]<\/td>Keyword level<\/td>By subscription [7]<\/td><\/tr>
    3. Analyst-written prompt batteries<\/td>Fixed questions written by a researcher, run once or repeatedly<\/td>The 10-question design in [5]<\/td>Yes<\/td>Yes<\/td>Yes<\/td><\/tr>
    4. Maintained real-phrasing question libraries<\/td>Questions phrased as people ask, stored with answers by model and date, re-run on a schedule<\/td>DrugChatter chatmon<\/td>Yes<\/td>Yes<\/td>Yes<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    Type 1: Platform-Held Logs<\/strong><\/h3>\n\n\n\n

    These are the largest and the least accessible. OpenAI’s 40 million daily health users come from this type. [2][3] The data is private by design. [1]<\/p>\n\n\n\n

    Type 2: Licensed, Modeled Panels<\/strong><\/h3>\n\n\n\n

    These give relative demand. Profound’s documentation says the figures are modeled estimates, that panel size is not stated, and that coverage varies by region. [7] They answer ‘how much interest’ and do not answer ‘what did the model say.’<\/p>\n\n\n\n

    Type 3: Analyst-Written Prompt Batteries<\/strong><\/h3>\n\n\n\n

    These give clean comparisons. Researchers control the questions, so they can compare drugs and models. The BMJ Quality and Safety study is an example. [5] Their weakness is that the question list reflects what the analyst imagined people ask.<\/p>\n\n\n\n

    Type 4: Maintained Real-Phrasing Question Libraries<\/strong><\/h3>\n\n\n\n

    These keep the control of Type 3 and add questions that mirror real wording. DrugChatter’s library belongs here, and the next section describes it.<\/p>\n\n\n\n

    Why the Classification Matters for Citation<\/strong><\/h4>\n\n\n\n

    A writer who cites ‘AI says X about drug Y’ should state which type produced X. A Type 2 volume cannot support a statement about answer accuracy. A Type 3 result cannot support a statement about what share of users ask a given question.<\/p>\n\n\n\n

    What Does DrugChatter’s Question Library Contain?<\/strong><\/h2>\n\n\n\n

    DrugChatter maintains a library of drug questions, each stored with the answer a model returned, the model that returned it, the brand or product concerned, and a question type. Its chatmon system re-runs the library across models on a recurring schedule and keeps the results.<\/p>\n\n\n\n

    DrugChatter’s position is that this combination, real-phrasing questions about named drugs, matched to answers by model and date, is not available from a platform report, a modeled panel, or a one-time analyst study. The company reports no public sample size for the library in this article. A reader who needs a figure should request it from DrugChatter and should expect it to carry a date and a method note.<\/p>\n\n\n\n

    How Is the Library Structured?<\/strong><\/h3>\n\n\n\n

    Each record joins four things: the brand or product, the question and its type, the model that answered (the vendor), and the full text of the answer. That structure is the minimum needed to compare an answer to a label. A record without the model cannot support model comparison. A record without the date cannot support change detection.<\/p>\n\n\n\n

    Why Pair Questions With Answers?<\/strong><\/h3>\n\n\n\n

    An answer is the thing a regulator, a safety reviewer, or a lawyer can check. A question alone shows interest. Pairing the two lets a reviewer ask ‘did the model state the boxed warning,’ ‘did it give the approved dose,’ and ‘did it mention an off-label use.’ Those checks need the label as the reference source. DrugChatter’s label-alignment scoring compares AI-generated text with the FDA label.<\/p>\n\n\n\n

    Why Re-Run the Same Questions Over Time?<\/strong><\/h3>\n\n\n\n

    Labels change and models change. A question that returned a correct answer in March can return a different one in November. Scheduled re-runs create a time series, and a time series is what lets a team say ‘the answer changed after the label change’ with evidence. A one-time test cannot say that.<\/p>\n\n\n\n

    What the Library Is Not<\/strong><\/h4>\n\n\n\n

    It is not a random sample of every user conversation on every platform. It is not a primary source for clinical facts. It does not replace the FDA label, ClinicalTrials.gov, or the published literature. It does not describe how many people ask a given question; Type 1 and Type 2 data address that. [3][7]<\/p>\n\n\n\n

    What the Library Does Better Than the Alternatives<\/strong><\/h4>\n\n\n\n

    Compared with Type 1, it is open to the drug team that owns the brand. Compared with Type 2, it holds the answer text. Compared with Type 3, it uses phrasing that mirrors real questions and keeps a history. These are statements about design, and they follow from the comparison table above.<\/p>\n\n\n\n

    What Can Each Function Do With Question-and-Answer Data?<\/strong><\/h2>\n\n\n\n

    Each function uses a different slice of the same record. The table maps functions to uses. It is analysis, not a report of customer practice.<\/p>\n\n\n\n

    Function<\/th>Question it can answer<\/th>Record fields used<\/th><\/tr><\/thead>
    Medical Affairs<\/td>Which clinical questions do people ask that the label does not address?<\/td>Question, answer, model<\/td><\/tr>
    Pharmacovigilance<\/td>Do models omit or misstate a labeled risk?<\/td>Answer, label<\/td><\/tr>
    Regulatory Affairs<\/td>Did a label change reach the model answers?<\/td>Answer, date, label version<\/td><\/tr>
    Brand Marketing<\/td>Does the model name the brand, and in what context?<\/td>Brand, answer, model<\/td><\/tr>
    Competitive Intelligence<\/td>Which competitor does the model recommend in the same question?<\/td>Answer, brand<\/td><\/tr>
    Medical Information<\/td>Which questions deserve a standard response letter?<\/td>Question, question type<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    What This Means for Medical Affairs<\/strong><\/h3>\n\n\n\n

    Medical Affairs can read the questions as a list of information needs. If many questions concern a use the label does not cover, that is demand for scientific exchange, handled through the channels the company already uses. The question set also shows how people phrase a clinical concern, which helps when drafting standard response documents.<\/p>\n\n\n\n

    What This Means for Pharmacovigilance<\/strong><\/h3>\n\n\n\n

    Pharmacovigilance has two separate jobs here. The first is checking whether answers state labeled risks. That is an accuracy task. The second is watching for questions that describe a possible adverse event. A question such as ‘I started this drug and now have swelling’ may describe an event, but a chatbot question is not an individual case safety report unless the company receives the required data elements through a reporting channel. This is a question for your PV and legal teams, and I do not give a rule here. The White study shows the signal-detection value of question data, with the caveats the authors state. [4]<\/p>\n\n\n\n

    What This Means for Regulatory Affairs<\/strong><\/h3>\n\n\n\n

    Regulatory teams can check whether a recent label change appears in answers. Because each record carries a date and model, a team can see which models updated and which did not. The check needs the current label as the reference, and the team must record which label version it used.<\/p>\n\n\n\n

    What This Means for Brand Teams<\/strong><\/h3>\n\n\n\n

    Brand teams can see whether a model names the product in response to an indication question, and what it says next. The data shows where the brand appears and where a competitor appears instead. A brand team should route any inaccuracy it finds to Medical, Regulatory, and Legal and should not edit AI output through its own channels without review.<\/p>\n\n\n\n

    What This Means for Competitive Intelligence<\/strong><\/h3>\n\n\n\n

    A question like ‘which drug is best for X’ returns a model’s recommendation. The library lets CI compare which product a model recommends, for which sub-population, and how that changes over months. This is a statement about model output and says nothing about clinical superiority.<\/p>\n\n\n\n

    Who Should Own the Review?<\/strong><\/h4>\n\n\n\n

    One named owner per finding type prevents duplication. A missing boxed warning goes to PV and Regulatory. A wrong indication goes to Regulatory and Medical. A brand omission goes to Brand and Medical. The owner list should exist before the data arrives.<\/p>\n\n\n\n

    Where Is the Compliance Line for Using AI Question Data?<\/strong><\/h2>\n\n\n\n

    The line depends on whether the company is receiving a question or generating one. FDA’s 2011 draft guidance separates the two, and the distinction carries over to LLM data by analogy. The guidance does not address LLMs. [6]<\/p>\n\n\n\n

    Why Does Unsolicited Versus Solicited Matter?<\/strong><\/h3>\n\n\n\n

    The guidance says a firm’s response to an unsolicited request is not, in FDA’s current thinking, evidence that the firm intends the product to be used for an unapproved use, if the firm follows the guidance. [6] Firms generating their own off-label prompts do not receive that framing, because they wrote the question. A question library built to mirror real phrasing should be reviewed by Legal and Regulatory before anyone uses it for external communication.<\/p>\n\n\n\n

    Rule of Thumb: Monitor, Do Not Promote<\/strong><\/h4>\n\n\n\n

    Using the data to detect errors and route them internally is monitoring. Using it to draft content that promotes an unapproved use is a different act with different risk. The guidance’s own recommended content for private responses, balanced and non-promotional with approved labeling attached, sets the standard a team should hold. [6]<\/p>\n\n\n\n

    Does the Draft Guidance Apply to Chatbots?<\/strong><\/h3>\n\n\n\n

    The guidance states that it addresses requests made through electronic media. [6] It was written in 2011 and does not mention LLMs. [6] It remains a draft document as published in the Federal Register notice I reviewed. I did not check whether FDA has since issued a final version, and Legal should confirm the current status. A team should not assume it covers a chatbot conversation the company does not control.<\/p>\n\n\n\n

    Record-Keeping as a Parallel<\/strong><\/h4>\n\n\n\n

    The guidance asks firms to keep records of a request, the response, and any follow-up. [6] A question library with timestamps and stored answers produces records of that kind for monitoring work, which supports an audit trail.<\/p>\n\n\n\n

    How Do You Check Any LLM Question Dataset Before Citing It?<\/strong><\/h2>\n\n\n\n

    Ask five questions about provenance, sampling, versioning, classification, and limits. A dataset that cannot answer them should not support a published claim.<\/p>\n\n\n\n

    Question 1: Where Did the Questions Come From?<\/strong><\/h3>\n\n\n\n

    State whether they came from platform logs, a licensed panel, an analyst, or a maintained library. Profound states its source as licensed conversations with probabilistic modeling. [7] The BMJ Quality and Safety study states that the authors wrote 10 questions per drug. [5] A dataset that does not say how it got its questions is hard to cite.<\/p>\n\n\n\n

    Question 2: How Large Is the Base?<\/strong><\/h3>\n\n\n\n

    Profound’s page does not state a panel size. [7] OpenAI’s articles do not state the classification method. [2][3] A figure without a base is a weaker citation.<\/p>\n\n\n\n

    Question 3: Which Model Version, on Which Date?<\/strong><\/h3>\n\n\n\n

    The BMJ Quality and Safety study names the tool (Bing Copilot, balanced mode) and the month (April 2023). [5] A finding about ‘AI’ with no model and date cannot be reproduced.<\/p>\n\n\n\n

    Question 4: What Was the Reference Source?<\/strong><\/h3>\n\n\n\n

    Accuracy needs a reference. For drug claims, use the FDA label of a stated version, plus other primary sources. Do not use another AI answer as the reference.<\/p>\n\n\n\n

    Question 5: What Are the Stated Limits?<\/strong><\/h3>\n\n\n\n

    Good sources state them. The White paper lists unobserved confounders and chance. [4] Profound lists regional coverage limits and modeling. [7] A source with no stated limits deserves more caution, not less.<\/p>\n\n\n\n

    Methodology Note for This Article<\/strong><\/h4>\n\n\n\n

    This article contains no new testing of AI models. Every figure labeled ‘analysis’ is arithmetic on a cited figure, with the inputs shown. I did not review OpenAI’s primary PDF; I relied on two trade-press accounts of it and note this where it applies. I did not review the Profound product beyond its public documentation.<\/p>\n\n\n\n

    What Happens Next for AI Drug-Question Data?<\/strong><\/h2>\n\n\n\n

    The following are forecasts and opinion, not documented events.<\/p>\n\n\n\n

    Forecast 1: Question-Level Data Becomes a Contested Asset<\/strong><\/h3>\n\n\n\n

    OpenAI released aggregate health statistics in January 2026. [2][3] I expect pressure from pharma, researchers, and regulators for more granular access, and pressure from privacy advocates against it. I cannot predict the outcome.<\/p>\n\n\n\n

    Forecast 2: Monitoring Moves From Ad Hoc to Scheduled<\/strong><\/h3>\n\n\n\n

    Teams that run one-time tests will see that answers change. The cost of re-running a library is low, so scheduled monitoring should replace one-time audits at companies that already track brand reputation.<\/p>\n\n\n\n

    Forecast 3: Regulators Will Ask What Companies Knew<\/strong><\/h3>\n\n\n\n

    If a company holds records showing that a model misstated a boxed warning for months, a regulator or plaintiff may ask what the company did. That is a reason to define owners and escalation paths before the first dataset arrives. It is a legal risk judgment, and your counsel should assess it.<\/p>\n\n\n\n

    Key Takeaways<\/strong><\/h2>\n\n\n\n
      \n
    • OpenAI reports more than 5% of ChatGPT messages concern health care and more than 40 million people submit health prompts daily. [2][3]<\/li>\n\n\n\n
    • One in four of more than 800 million regular users is about 200 million people per week. This is my arithmetic. [3]<\/li>\n\n\n\n
    • Pharma cannot read those conversations. Platforms release aggregates. [2][3]<\/li>\n\n\n\n
    • In a Bing Copilot study of 500 answers about 50 drugs, 126 of 484 scored answers had accuracy below 100%, and experts judged 66% of 20 reviewed answers potentially harmful. [5]<\/li>\n\n\n\n
    • A 2010 search-log study of 6 million searchers found a drug-interaction signal before it was widely known. [4]<\/li>\n\n\n\n
    • Analysis: LLM drug-question data falls into four types. Only Types 3 and 4 hold both question and answer at the drug level. Only Type 4 combines real-phrasing questions with scheduled re-runs.<\/li>\n\n\n\n
    • DrugChatter’s question library is a Type 4 source. Its position is that the combination of real-phrasing questions, answers, models, and dates is not available from a platform, panel, or analyst study. This is a company position, and the library does not replace the FDA label or the published evidence.<\/li>\n\n\n\n
    • Question data should be used to monitor, not to promote, and should pass Legal and Regulatory review.<\/li>\n<\/ul>\n\n\n\n

      FAQ<\/strong><\/h2>\n\n\n\n

      1. How many people ask ChatGPT health questions?<\/strong><\/h3>\n\n\n\n

      OpenAI reports more than 40 million people submit health-related prompts daily, and roughly one in four of its more than 800 million regular users does so weekly. [3] These are platform self-reports.<\/p>\n\n\n\n

      2. Can a pharma company see what patients ask AI about its drug?<\/strong><\/h3>\n\n\n\n

      Not directly from the platforms. OpenAI published aggregates, and Profound sells modeled keyword volumes. [2][7] A company can build or license a maintained question library to see how models answer questions about its drug.<\/p>\n\n\n\n

      3. Is ChatGPT accurate on drug questions?<\/strong><\/h3>\n\n\n\n

      No study I reviewed measured current ChatGPT across all drugs. The BMJ Quality and Safety study tested Bing Copilot on 500 answers in April 2023 and found a mean accuracy of 88.7% with 26.0% of scored answers below 100% accurate. [5]<\/p>\n\n\n\n

      4. What did the 2013 JAMIA search-log study find?<\/strong><\/h3>\n\n\n\n

      Among 6 million searchers in 2010, about 10% of those who searched both paroxetine and pravastatin also searched hyperglycemia terms, against about 0.3% in the background. [4]<\/p>\n\n\n\n

      5. What is Profound’s Prompt Volumes?<\/strong><\/h3>\n\n\n\n

      A dataset of prompts to ChatGPT, Gemini, and Perplexity, built from licensed conversations and probabilistic modeling. Its documentation says figures are estimates and does not state a panel size. [7]<\/p>\n\n\n\n

      6. What is the difference between prompt volume and answer accuracy?<\/strong><\/h3>\n\n\n\n

      Volume measures how often a topic is asked about. Accuracy measures whether the answer matches a reference such as the FDA label. A volume dataset cannot support an accuracy claim. [5][7]<\/p>\n\n\n\n

      7. Does FDA guidance cover AI chatbot questions about off-label use?<\/strong><\/h3>\n\n\n\n

      FDA’s 2011 draft guidance covers unsolicited requests, including through electronic media, but does not mention LLMs. [6] Ask Legal to confirm the current status and scope.<\/p>\n\n\n\n

      8. Is a chatbot question about a side effect an adverse event report?<\/strong><\/h3>\n\n\n\n

      Not by itself. Reportability depends on the company receiving the required information through a reporting channel. Ask your PV and Legal teams to define the rule before monitoring starts.<\/p>\n\n\n\n

      9. Why does the model and date matter in a question dataset?<\/strong><\/h3>\n\n\n\n

      Models and labels change. Without a model and date, a finding cannot be reproduced or tied to a label version. The BMJ Quality and Safety study reports both. [5]<\/p>\n\n\n\n

      10. How does DrugChatter’s library differ from an analyst-written prompt list?<\/strong><\/h3>\n\n\n\n

      DrugChatter’s library uses questions phrased as people ask them, stores each answer with the model and date, and re-runs on a schedule. An analyst-written list reflects the analyst’s choice of questions, as in the 10 questions per drug used in [5].<\/p>\n\n\n\n

      References<\/strong><\/h2>\n\n\n\n
        \n
      1. Chatterji, A., Cunningham, T., Deming, D., Hitzig, Z., Ong, C., Shan, C. Y., & Wadman, K. (2025). How people use ChatGPT<\/em> (NBER Working Paper No. 34255). National Bureau of Economic Research. https:\/\/www.nber.org\/papers\/w34255<\/li>\n\n\n\n
      2. Axios. (2026, January 5). Exclusive: 40 million people turn to ChatGPT for health care<\/em>. https:\/\/axios.com\/2026\/01\/05\/chatgpt-openai-health-insurance-aca<\/li>\n\n\n\n
      3. Medical Economics. (2026, January 7). 40 million people now use ChatGPT daily for health questions, OpenAI report finds<\/em> (reporting OpenAI, AI as a Healthcare Ally<\/em>, January 2026). https:\/\/www.medicaleconomics.com\/view\/40-million-people-now-use-chatgpt-daily-for-health-questions-openai-report-finds<\/li>\n\n\n\n
      4. White, R. W., Tatonetti, N. P., Shah, N. H., Altman, R. B., & Horvitz, E. (2013). Web-scale pharmacovigilance: listening to signals from the crowd. Journal of the American Medical Informatics Association, 20<\/em>(3), 404-408. https:\/\/doi.org\/10.1136\/amiajnl-2012-001482 (full text: https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC3628066)<\/li>\n\n\n\n
      5. Andrikyan, W., et al. (2025). Evaluation of a chatbot’s responses to patient questions about drugs. BMJ Quality & Safety, 34<\/em>(2), 100. https:\/\/qualitysafety.bmj.com\/content\/34\/2\/100<\/li>\n\n\n\n
      6. U.S. Food and Drug Administration. (2011, December 30). Responding to unsolicited requests for off-label information about prescription drugs and medical devices; draft guidance for industry; availability. Federal Register, 76<\/em>(251). Docket No. FDA-2011-D-0872. https:\/\/www.govinfo.gov\/content\/pkg\/FR-2011-12-30\/pdf\/2011-33550.pdf<\/li>\n\n\n\n
      7. Profound. (n.d.). About Prompt Volumes<\/em>. https:\/\/docs.tryprofound.com\/prompt-volumes\/about<\/li>\n<\/ol>\n\n\n\n

        <\/p>\n","protected":false},"excerpt":{"rendered":"

        More than 5% of all ChatGPT messages worldwide are about health care, and roughly one in four of OpenAI’s more […]<\/p>\n","protected":false},"author":1,"featured_media":1287,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_lmt_disableupdate":"","_lmt_disable":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1286","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-general"],"modified_by":"DrugChatter","_links":{"self":[{"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/posts\/1286","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/comments?post=1286"}],"version-history":[{"count":1,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/posts\/1286\/revisions"}],"predecessor-version":[{"id":1288,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/posts\/1286\/revisions\/1288"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/media\/1287"}],"wp:attachment":[{"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/media?parent=1286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/categories?post=1286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/drugchatter.com\/insights\/wp-json\/wp\/v2\/tags?post=1286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}