Share Tweet

How data science and digital experimentation could help healthcare learn what works.

I recently presented at Katalitix on how A/B testing and experimentation, commonly used in digital industries, could be applied to healthcare to improve patient engagement, healthcare delivery, and potentially health outcomes.

The presentation drew on my previous experience as a data scientist in an experimentation team, where we used A/B testing to understand how changes to online games influenced user behaviour. Following a discussion with Prof Wim Delva about the potential to apply these methods in healthcare, I explored how the same experimental principles could help answer practical questions about patient behaviour and digital health interventions.

The central idea is that healthcare often involves decisions about how best to encourage people to take action. For example, could a different medication reminder improve adherence? Could the way a screening or vaccination campaign is communicated influence whether people attend? Could a digital health application help patients remain engaged with treatment over time? Rather than relying solely on assumptions about what might work, experimentation provides a structured way to test alternatives and learn from the results.

Randomised controlled trials (RCTs) have long helped researchers establish whether healthcare interventions are effective. As more healthcare interactions move online, the principles behind these trials can also inform digital experimentation.

The science behind the test

A/B testing compares two or more versions of an intervention. People are randomly assigned to different versions, and outcomes are measured to assess which approach performs better. The methodology involves more than comparing results. It requires appropriate randomisation, adequate sample sizes, data quality checks, and statistical methods to quantify uncertainty and improve the reliability of conclusions.

The presentation covered the full experimentation lifecycle, including developing a hypothesis, designing an experiment, monitoring results, analysing outcomes, and using the findings to inform future decisions. It also introduced statistical and technical approaches used in experimentation, including sample ratio checks, power analysis, methods to improve precision, sequential testing, and Bayesian methods to balance exploring different interventions with directing more exposure towards those performing better.

These methods are well established in digital industries such as technology, e-commerce, social media, streaming, travel, and gaming. The opportunity is to explore where they could also support healthcare, particularly as patients increasingly interact with services through websites, mobile applications, digital campaigns, and connected devices.

Connecting behavioural science and data science

A key theme of the presentation was the role of behaviour in determining whether healthcare interventions achieve their intended purpose. Making health information available or prescribing an effective treatment does not automatically mean that people will act on the information or consistently follow the treatment.

Gaming offers examples of how goals, reminders, feedback, progress tracking, and rewards can encourage participation and sustained engagement. These principles may offer useful ideas for healthcare, provided they are adapted appropriately to patients' needs and clinical objectives.

One example discussed was research by Taj and colleagues, who adapted the game 2048 to simulate medication-taking behaviour. The study found that a reminder intervention significantly improved adherence, illustrating how controlled experiments can help investigate the behavioural factors influencing health.

Taking experimentation into digital healthcare

The potential applications extend across several stages of the patient journey:

• Medication adherence and patient support: Testing reminder wording, timing, and progress feedback to understand which approaches help patients follow treatment plans.
• Digital health campaigns: Comparing messages and formats used to communicate screening, vaccination, and health education initiatives.
• Patient communications: Exploring how information is presented to support understanding, engagement, and appropriate action.
• Healthcare websites and applications: Testing content, navigation, and digital experiences to identify ways of making health services easier to access and use.
• Rehabilitation and chronic disease management: Exploring how goals, feedback, and progress tracking support continued participation in rehabilitation or self-management activities.

The presentation also considered the wider use of gamification in health technology, including fitness trackers, rehabilitation platforms, and medication adherence applications. Repeated interactions with these tools can generate longitudinal data on behaviours and treatment responses. For example, information on medication-taking, rehabilitation activity, or regular blood glucose measurements may help researchers and healthcare teams understand patterns over time.

A further industry example discussed was a digital companion application introduced alongside a rheumatoid arthritis therapy. The application provided patient education and treatment support and was associated with an 8% increase in revenue through better use of the therapy. This illustrates the potential commercial value of patient support tools, although revenue is not itself evidence of improved clinical outcomes.

Innovation with responsibility

Although the underlying experimental principles are transferable, healthcare differs from commercial digital environments in important ways. An intervention that increases clicks, app usage, or engagement does not necessarily improve a patient's health. Experiments must therefore be designed around meaningful outcomes and conducted with appropriate attention to medical accuracy, patient safety, privacy, equity, and ethical oversight.

The aim is not simply to optimise engagement, but to understand which interventions genuinely help patients and healthcare systems. A/B testing should complement, rather than replace, clinical research and the evaluation required to establish safety and effectiveness.

The broader opportunity

The presentation concluded by considering how behavioural science, data science, and experimental methods could work together to support more responsive healthcare services. The opportunity is to move beyond collecting data towards using it systematically to evaluate interventions, improve decisions, and refine how care is delivered.

The methodology remains the same: define, test, measure, and learn. However, the responsibility behind what we test is greater.

The discussion and Q&A at Katalitix explored how these ideas might be relevant to the company's clients, including opportunities to investigate healthcare system efficiencies and improve digital patient experiences.

Ultimately, the question is how experimentation can be applied responsibly to help healthcare learn what works, for whom, and in what context, with patients and their health outcomes at the centre.

With better data, we can make better decisions and drive innovation towards better health.

By Thenuja Pillay
Data Scientist and MSc Candidate
South African Centre for Epidemiological Modelling and Analysis (SACEMA)

News date: 2026-10-27

Links: