Are We Measuring the Wrong Things in Digital Health Adoption?

We measure what we can count. But in digital health, that may be a problem.

Too often, digital health adoption is reported with impressive-sounding metrics:

• "4,000 new users onboarded."
• "Completion rates up 36%."
• "60% of patients accessed the tool at least once."

But these figures tell us very little about whether digital health is actually working, for patients, for clinicians, or for the wider health system.

In the race to scale digital health, we've often confuse usage with impact. But healthcare is not a SaaS platform. Log-ins don't equate to value. And questionnaire completion rates don't tell us if someone understood the content, trusted the process, or felt supported in their care.

This isn't just a measurement problem, it's a mindset problem.

In many NHS digital evaluations, we rely heavily on quantitative analytics:

• Number of users
• Sessions per user
• Drop-off points
• Click-through rates
• Task completions

While useful, these numbers are proxies for engagement, not proof of meaningful adoption.

Take Lucy for example. Lucy is a 29-year-old junior nurse on rotating shifts, referred by her GP for social anxiety that spikes during team briefings. She's using the digital CBT tool to turn theory into quick, digital routines that coach her to speak more confidently at work. Aisha might log in, click through a CBT module, and even "complete" it. But did she feel confident applying what she has learned? Did she understand how the platform supported here? Did her digital experience build trust or erode it?

In one example project, a mental health platform had a 72% completion rate. But interviews with users uncovered that many users clicked through rapidly just to finish. On deeper exploration, several users reported less confidence after using it. The platform wasn't intuitive and it just didn't connect to their real-world needs.

These examples highlight that we need to start asking better questions:

• Did the technology make care feel more human or less?
• Did it reduce friction, or just shift the burden elsewhere?
• Did it help the clinician or add another login to their day?

To understand true digital adoption, we need to track indicators that are harder to measure but infinitely more meaningful.

1. Comprehension

Do users, both clinicians and patients alike, understand what the tool does and how to use it?

Especially as AI-powered triage and decision tools enter mainstream use, the ability to explain what the technology is doing will be increasingly essential to establish both safety and trust.

2. Confidence

Not just: "Did the user complete the journey?" But: "Could they do it again next time, unaided?"

Confidence is a stronger predictor of future use than first-time completion.

3. Trust

Does the user believe their data is secure? That the platform is acting in their best interests? That decisions are fair, unbiased, and transparent?

Trust isn't a static condition, it's earned and re-earned through every interaction.

4. Retention with Purpose

Are users returning to the tool because it adds value, or because they're forced to?

We see this in clinician adoption especially: repeated log-ins don't mean satisfaction. They often mean frustration, obligation and can lead to burnout.

5. Dropout Recovery

Every system has drop-off. But do we notice? Do we care?

A proactive and supportive recovery strategy, whether a follow-up nudge, call, or support session, can be the difference between digital disengagement and re-engagement.

The NHS's digital transformation agenda rightly emphasises inclusivity, usability, and impact. But we won't achieve those goals if our evaluation frameworks stop at surface-level analytics.

Reports like What Good Looks Like and A Plan for Digital Health and Social Care outline ambitions around access and digitisation. But if we want to shift from "digital as infrastructure" to "digital as an enabler of better care," we must evolve how we measure success.

Success should be measured by:

• Patient empowerment, not just patient activity
• Clinician workflow improvements, not just tool installation
• Meaningful usage across demographics, not just aggregate stats

To move forward, we need three shifts:

Policy alignment – NHS standards and funding frameworks should require qualitative, equity-sensitive evaluation, not just usage stats.

Co-design of metrics – We need to involve patients and clinicians in defining what success looks like, to them.

Investment in insights – Go beyond dashboards. Invest in lived experience evaluation, human-centred analytics, and behavioural insights.

Because if we only measure what's easy, we will miss what's essential.

Digital health isn't a numbers game. It's a trust game. And if we can't measure trust, comprehension, and confidence, we're not measuring adoption. We're measuring log-ins.