Why drug companies miss their data and safety targets
It’s been a long time since I posted; but this is one I’ve been wanting to share for a while. I have a sign over my desk in the office :
Never mistake activity for achievement — John Wooden
John Wooden was the legendary UCLA basketball coach who won NCAA national championships in a 12-year period as head coach for the UCLA Bruins, including a record seven in a row. Wooden coached players at UCLA who would later become stars — players like layers Kareem Abdul-Jabbar and Bill Walton. Wooden understood and trained his players that just being busy on the court doesn’t count.
Why clinical protocol compliance matters
Compliance to a clinical protocol matters to drug companies that want to bring good products to patients. Non-compliance to the dosing protocol when discovered by FDA will result in the drug not being approved. When discovered in an interim analysis, it can result in trashing the study. When discovered in use by patients, it can result in avoidable hospitalizations.
Compliance to a clinical protocol is both a feeling and a reality.
We feel compliance when we feel protected from process issues, free from data quality issues and safe from adverse events. In this way, protocol compliance is merely a state of mind. None of these threats to the success require unblinding your clinical trial. But there’s the reality of protocol compliance, a reality that has nothing to do how we feel.
We need to ground that feeling of compliance in the reality of compliance and not merely in placebos of dashboards.
Monitoring clinical operations
There is an archaic monitoring activity still conducted in most clinical trials today called SDV — source document verification. SDV involves study monitors manually comparing pieces of paper with digital records in the EDC system. 30 years ago, when we were transcribing from paper forms, this made sense.
With electronic source, wearable devices and mobile patient apps, SDV is no longer needed. That’s the good news. The bad news is that the industry is still using a batch paradigm where data managers extract data and produce periodic reports and dashboards in order to monitor the clinical operations.
Distributed systems cannot be understood with dashboards
Monitoring clinical trials employs a traditional paradigm of monitoring clinical ops activities. With COVID, there is a growing trend to patient-centric and decentralized trials where patients check in from home using a mobile and/or wearable devices. More good news and bad news. Good news because we’re collecting data directly or passively from patients using digital channels.
Bad news, because monitoring clinical operations activities sidesteps the problem that 90–95% of the data in a decentralized trial is generated by a distributed, unmanaged system of patients and connected devices.
As anyone who has ever tried to personally comply to a treatment and/or manage a distributed system with over 1000 dimensions and 100s of components can testify:
High cardinality and high-dimensional systems of unreliable moving parts cannot be managed with dashboards.
Enough for the connected device clocks to drift for your patient alerts to lose meaning.
Observability for clinical data
Monitoring is for clinical operations (the ‘operating system’ of clinical trials)
Instrumentation is for clinical end-points (humans and code in clinical trials)
Observability is for understanding systems of patients in a clinical trial
We have 25 years of GCP and clinical monitoring best practices. We have analytics and reports and systems of queries and discrepancies that are optimized for detecting data entry problems.
The next step is to instrument the end points and achieve observability for clinical data towards understanding these highly complex systems of humans and devices called clinical trials.