Nostalgia can trick us into thinking that things were better in the “good old days”. But the world of science is rarely a place where sentimentality should cloud better judgement. Rather, it is a place where questions, analysis and evidence pave the way for progress.
Arguably, nowhere is this more important that in clinical trials, where an appetite for challenging assumption, uncovering weakness and improving understanding has the potential to trigger developments that, ultimately, increase the likelihood of new and more effective therapies reaching patients in need.
Many areas of clinical trials have benefited from this mindset of continual improvement over time. An obvious example is the use of sealed envelopes in the candidate randomization process. Decades ago, trial co-ordinators commonly used sealed opaque envelopes to allocate patients to treatment groups. Essentially, envelopes containing details of the assignment were mixed into a disordered sequence and then used to assign participants to a treatment one by one as they sign on to the trial. Once assigned, individuals were expected take the assigned treatment – or placebo – according to protocol.
While straightforward and simple to implement, over time it became apparent that this approach was inherently flawed. The underlying belief that assignment was safely hidden from investigators and participants proved not to be true in all situations. Poor levels of opacity or physical manipulation of the envelopes – both deliberate and accidental – could expose the nature of the assignment within. This leads to the obvious risk of co-ordinators having knowledge of the assignment or potentially being able to discern an identifiable pattern in the randomization, opening the door to selective enrolment (selection bias) and allocation that does not conform with the goal of true randomization.
If these scenarios are played out, trial findings can be skewed and judgements on efficacy and safety are undermined. Indeed, research has indicated that in trials where randomization is inadequate or unclear, treatment effects can be overestimated by up to 40%.[1]
While strategies would often be introduced to mitigate against the weaknesses of sealed envelopes, the availability of better alternatives led to change and progress. Specifically, trial co-ordinators were able to draw on advances in technology, leading to the increasing deployment of web-based randomization systems (IWRS) and more complex techniques, such as stratified randomization, to address the potential for human error. Over time, as these methods proved their worth, they came to dominate, and sealed envelopes fell from favour. Today, they are largely obsolete, with regulatory agencies such as the US Food & Drug Administration (FDA) and European Medicines Agency (EMA) advocating the use of centralized, digitized approaches as best practice.[2]
Taking a step back to reflect on this transition, it is clear that the clinical trials landscape has been significantly improved by the recognition of a flawed approach and the introduction of digital methods that are better suited to the task. This represents a natural progression that is rooted in logic and made possible by advances in technology.
But randomization is not the only aspect of a clinical trial that would benefit from such a rethink. Similar advances are yet to be seen, for example, in the area of adherence. Here, being able to rely on a robust measure of a participant’s exposure to the assigned treatment is critical in order to have a reliable gauge of exposure and, therefore, to evaluate efficacy and safety with real accuracy. But, despite this, analogue self-reporting methods, such as pill count, remain the dominant model.
Just as with sealed envelopes, self-reporting methods might be straightforward and simple to implement, but they are also inherently prone to human error and bias. The information that a participant submits to co-ordinators regarding their adherence to the prescribed protocol within a questionnaire or diary can be unreliable through deliberate misreporting and poor recall. Separately, pill counts only provide an indication of patient activity rather than whether they have faithfully adhered to the protocol as directed.
Here again, advances in technology have led to the availability of better alternatives. Digital measurement of adherence in trials represents a far more sophisticated approach through a combination of electronically enabled packaging and complementary software solutions incorporating sophisticated algorithmic analysis. By monitoring adherence far more closely, these digitised approaches allow investigators to be immediately alerted to erratic dosing patterns and events, giving them far greater control over the trial and bringing far more integrity to the output data. Indeed, research has shown that the MEMS system developed by AARDEX demonstrates 97% accuracy in the tracking and collection of participant medication events compared with 60% for pill count and 27% for self-report.
In the 21st century, adherence can, therefore, be seen as an illogical anomaly in the wider drug development landscape. For the clinical trial community, this raises many important questions around the need for modernisation and change – a message that comes through loud and clear in the heart of the World Health Organization’s recently published Guidance for Best Practices for Clinical Trials, which states that “digital technology…is needed to improve efficiency and enable research connectivity in the longer term”.[3]
Old methods might be tried, but it is clear they can’t necessarily be trusted when it comes to something as important as adherence. On this basis, now is the time to let go of the “good old ways” and to embrace innovation, progress and improvement.
[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC2267325/
[2] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/integrating-randomized-controlled-trials-drug-and-biological-products-routine-clinical-practice
