ChatGPT Health: do we need to be concerned when commercial AI moves into personal health?

OpenAI's launch of ChatGPT Health in January 2026 sparked a fluffy of LinkedIn conversations, with many voices wading in pontificating as to whether we should we allow a commercial AI platform to become personal health infrastructure. So here is my take on this contentious subject.

For those of you who have missed the memo, ChatGPT Health, is Open AI's new version (launched so far in the US) that allows users to connect their medical records and wellness data directly and receive personalised health interpretation. Over 200 million people already use ChatGPT weekly for health queries. In the US, they can now do so with their own clinical data feeding the conversation.

In the words of OpenAI:

You can now securely connect medical records and wellness apps—like Apple Health, Function, and MyFitnessPal—so ChatGPT can help you understand recent test results, prepare for appointments with your doctor, get advice on how to approach your diet and workout routine, or understand the tradeoffs of different insurance options based on your healthcare patterns.

This launch is being presented as progress, making health information more accessible, immediate and personalised. But are we outsourcing a fundamental function of healthcare, medical interpretation and explanation, to a commercial entity that has no clinical accountability, is so far operating outside health regulation and who is extracting value from population health behaviour without contributing to public infrastructure.

OpenAI positions ChatGPT Health as an interpretation tool, not a diagnostic one. This legal distinction enables the platform to operate outside the current regulatory structures, allowing OpenAI to avoid medical device regulation while performing a medical adjacent function. But in reality this distinction is clinically meaningless. It transfers risk from regulated clinical providers to an unaccountable platform

When someone uploads their blood results and asks "should I be worried about my liver function?" they are seeking a diagnostic explanation The response they receive, contextualised to their age, medication history, and prior results, is essentially a diagnostic function, even if wrapped in careful language.

Large language models are fallible. They produce errors that sound confident and plausible, know as halucinations. In a health context, such an inaccuracy can have consequences that extend far beyond misunderstanding. When the first serious harm occurs this liability vacuum will become apparent. By then, millions will already be using the platform.

Health systems are already stretched. ChatGPT Health is being sold as a solution: better-informed patients, more efficient consultations and reduced pressure on services.This assumes that accessible health interpretation reduces demand. The opposite is more likely.

As I wrote about last week, the Jevons Paradox, where efficiency improvements increase rather than decrease demand is more likely. Making health information more comprehensible is likely to not reduce health anxiety or service use. It will make people more health-aware, more uncertain, and more likely to seek clinical validation of AI-generated health information.

Will this mean patients arrive at appointments armed with data interpretations, symptom explanations, and questions produced by an algorithm optimised for engagement, not necessarily for clinical appropriateness? Will clinicians need to spend more time explaining, correcting, or reassuring their patients? Will the time saved by better-prepared patients be simply consumed by managing AI-influenced expectations?

The net effect may be an escalation in demand and if that occurs, a tool marketed as reducing pressure on the NHS could become a driver of it.

Making all health data constantly interpretable may not necessarily empower people. It may increase health anxiety and expand the boundaries of what feels medically significant.

The quantified self movement, a global social movement that promotes self-tracking (or lifelogging), of physical and mental health, behaviour, and environmental data, to allow a person to get to know and overtime to improve their life. There many examples that this has generated harm alongside benefit, through increased health preoccupation, hypervigilance, and reduced overall wellbeing.

ChatGPT Health accelerates this, turning every data point into a prompt for further interrogation. The result may not be better informed patients but medicalised, anxious and dependent ones.

There is a huge gap in published evidence that AI-generated health interpretation improves clinical outcomes, reduces health anxiety, or makes services more efficient.

The assumption is that better comprehension improves patient empowerment, and this reduces unnecessary demand, that real-time data supports better decision-making.

It might, but as yet we don't know.

While users receive individual explanations, OpenAI learns from every health interaction across a population. So even if personal data is encrypted, the interaction patterns, query types, symptom correlations, and aggregate insights create proprietary intelligence and commercial value extracted from people's health concerns without contributing to the public infrastructure that generates the data in the first place.

If the NHS provided the same service ,it would be accountable to the public, governed by health regulation, and any learning would feed back into overall system improvement. When OpenAI does it, the value flows out.

The long-term risk is not just about individual privacy. It is about whether commercial platforms should be permitted to build monopolistic health intelligence by positioning themselves as essential intermediaries between people and their own data.

The groups that are positioned to benefit most, under-served communities, people in rural areas and those with limited health literacy are also at the highest risk. Such as:

• Those with limited digital or health literacy may lack the skills to judge reliability or inaccuracies in AI outputs.
• Under-represented populations may experience worse accuracy because model training data reflect existing biases.
• People in areas with poor clinical access may over relay on AI as a substitute for professional help rather than a supplement.
• Individuals with mental health vulnerabilities may be disproportionately influenced by the tone and health concerns raised through AI-generated responses.

Without regulatory safeguards, this technology could widen the disparities it is marketed as closing. Achieving equity requires transparency about performance variation, accountability when harm occurs.

None of these exist yet.

If ChatGPT Health delivers faster responses, and clearer explanations than NHS digital services, why would people use NHS digital tools? Why would systems invest in patient-facing platforms if OpenAI does it better commercially?

This is not theoretical. Google steadily became the default interface for health queries because it was faster and more convenient than navigating official sources of health information, which has resulted in misinformation at scale.

However, as AI can now reference personal data, creating a stickiness that search engines never achieved. Once millions rely on ChatGPT for health interpretation, OpenAI has leverage. The switching cost in healthcare is not just inconvenience; it is loss of contextual information continuity and an understanding your own health.

We also need to look internally and ask what is it about our health systems that is likely to drive people on mass to upload their medical records to a commercial AI platform in the first place?

The demand is not irrational. Health systems have systematically failed to provide accessible, timely interpretation of clinical information. People receive test results through portals with no context. They have to wait weeks for an appointment to discuss a health concern and they are given diagnoses without adequate time for questions. Clinical language remains opaque. Follow-up explanations are delayed or unavailable. Data are fragmented across apps and devices that do not communicate.

ChatGPT Health does not create this gap. It exploits it.

Health systems that fail to provide accessible information and interpretation transfer that responsibility to patients or, increasingly, to unaccountable platforms.

If we reject commercial AI as a long-term solution, we must address the systemic gaps that make it attractive.

The risks outlined here are not arguments against innovation. They are considerations that we need to have an answer to, and the structures to manage the risks safely, before we allow widespread adoption

Safe deployment of AI health system requires:

• Trial evidence showing that AI interpretation improves clinical relationships and outcomes rather than complicating them
• Independent clinical validation of accuracy, including demographic performance variation and failure mode analysis
• Regulatory classification that brings consumer health AI within the scope of medical device oversight
• Transparent data governance with enforceable limits on secondary use and clear accountability for cross-border data transfer
• Liability frameworks that establish who is accountable when harm occurs
• Interoperability with public health infrastructure rather than commercial displacement of it

Until these exist, ChatGPT Health is an unregulated clinical information service with a subscription business model operating without clarity on liability and without effective regulation.

Digital progress is not the same as digital justice, which requires technology to be accountable, equitable, and aligned with the values of universal healthcare.

The question for health system leaders is not whether to permit its embedding in clinical workflows before we understand the full implications.

The answer is not yet.