Why clinical trials fail (not the LinkedIn version)

While most management books and LinkedIn posts focus on how to do things correctly, so you don’t screw up, I will provide insight into what you must do after you have screwed up.

The good news is, I have plenty of experience at that and so does every other entrepreneur.

In this post, I will share the results of internal research we performed on clinical trials in the US, Israel, Europe, India and Brazil, looking for the things that went wrong. We found 57 blockers. None are technology-related. All are rooted in leadership skills.

I hope I can help people who develop revolutionary cures preempt solutions by applying fundamental leadership skills.

In 2020 we all became clinical trials followers

Starting Feb 2020, with the outbreak of the COVID-19 pandemic, the importance and relevance of clinical trials to our daily lives, suddenly became clear and obvious. Billions of people hung on every shred of news regarding Pfizer, AstraZeneca, and Moderna clinical trials progress in releasing a safe and effective vaccine. Discussion online, and in person — peaked.

When the AstraZeneca vaccine trial fumbled on mis-dosing of 20% of their patient population, there were those who rejoiced, and those who were dismayed. How could such a well-run and sophisticated pharmaceutical company get something as basic as dosing, wrong? I’m not an insider and I don’t know. The bottom line is that AstraZeneca did not make to the FDA finish line with Moderna and Pfizer.

Clinical trials are complex scientific experiments with thousands of moving parts. Vaccine trials happen to be relatively simple in comparison with oncology trials — but they are still complex operations. A lot of things can go wrong.

Patient diversity and patient-centricity

During the pandemic — a number of VC-funded software companies started working on so-called decentralized clinical trials (DCT). Translated into plain English — this means bringing a clinical trial to patients at home, instead of bringing patients into the hospital.

DCT is a wonderful idea and helps studies become more accessible and more diverse.

During the pandemic, bringing trials to patients became a necessity, and hundreds of trials were performed with some level of decentralization.

For the record — over 12,000 new clinical trials are initiated each year. DCT represents less than 5% of clinical trials today. Paraphrasing Andy Grove, former CEO of Intel:

The importance of any new technology is inversely proportional to the level of media exposure. Once a technology like a phone mainstreams, the media attention drops to zero and moves to sales and marketing.

Diversity and accessibility to treatment are super-important, but I hope you agree with me, that we can only achieve diversity and accessibility if the clinical trials actually execute on-time.

What are the blockers for clinical trials?

No question can be more relevant for patients waiting for cures to Alzheimer, Parkinson, autism, cancer, and a long list of chronic diseases.

If we can remove blockers for clinical trials, we can do more for patients and patient-diversity. We can address 100% of the clinical trial industry and create hope for billions of people.

Who cares if Medable (a West Coast DCT company) is a unicorn? Will unicorn status save lives or remove blockers to life science R&D for the 95% of clinical trials that are done in medical centers and not at home?

My name is Danny Lieberman and I’m the CEO of flaskdata.io. We’ve collected over 10M signals on our Digital CRO platform in the past 4 years and we have some perspective as to blockers to clinical trial success.

The blockers to bring a new treatment to market are not patient-centricity and patient-diversity.

The blockers are all process and project management-related.

In plain English — people, blocking and tackling.

We surveyed the blockers in clinical trials encountered by our customers in clinical trials in Europe, Israel, the US, India, and Brazil in the past 4 years.

We found 57. That’s a lot.

God is in the details

I know that many of these are technical, but god is in the details. If a Principal Investigator at a site in Detroit cannot sign off the patient because she cannot login, then that’s a blocker

If a site coordinator in Miami does not have demo accounts for the patient app, how is she going to properly support them?

So here goes.

Blockers in clinical trials

  1. The life science company is a startup and they lack of experience in clinical trials
  2. Uncertainty regarding choice of CRO (or DIY)
  3. Uncertainty regarding regulatory pathway
  4. Uncertainty regarding the SAP (statistical analysis plan) — you’d be surprised how many customers gloss over this.
  5. Uncertainty regarding the data management plan
  6. Uncertainty regarding the study design/protocol
  7. Uncertainty regarding the case report forms/clinical data model. You’re a medical scientist not a computer scientist.
  8. Uncertainty about when the study will start / end
  9. Uncertainty about probability of success
  10. Uncertainty when IRB (ethics committee) approval will be received
  11. Uncertainty regarding site selection. Surprising things can happen — like wanting to use sites in the country of Georgia and then get stuck for 3 months because of national elections shutting down the government.
  12. Uncertainty regarding patient recruiting
  13. Waiting until the last month or 2 before the IRB to get quotes from vendors
  14. Working with a consultant who is not the decision maker
  15. Uncertainty regarding total cost of ownership of the project
  16. Unsure about the impact of changes
  17. Trying to get the lowest price
  18. Quote process takes time and management attention
  19. Delays agreeing on the contract with the eClinical vendor
  20. Delays signing the contract due to lawyers
  21. Lack of attention to complex and lengthy vendor-assessment processes; it’s hard and the executive management is busy running the company. Not every project is COVID-19 with money no object.
  22. The life science company decides to make changes to protocol / case report forms after receiving the vendor quotes
  23. The company clinical team decided to change document/protocol names
  24. This creates mismatches between the protocol and the case report forms
  25. And generated lots of email traffic which creates more confusion and delays
  26. There is uncertainty regarding the risk assessment of the study protocol. The uncertainty ranges from “What is risk assessment, to who will do the risk assessment, who will fund it and how long will it delay the project and drug to market?
  27. The pharma company, surprisingly so, may have a lack of clarity regarding GCP (Good Clinical Practice) risk and how to mitigate. Surprisingly enough, the best practice in the industry is looking at pieces of paper.
  28. What kinds of alerts on GCP violations are needed by the life science company during the clinical trial?
  29. What data management reports are needed to run the study?
  30. Is there an issue of accountability because you are working with an external consultant who may not be available for timely resolution of questions?
  31. Are you vague regarding patient randomization and drug supply chain?
  32. Is there vagueness regarding data validity checks in the database ?
  33. Is the patient-reported outcomes app not validated with patients before onboarding?
  34. Are the case report forms not validated with site coordinators/investigators before onboarded?
  35. Does the drug company have a clear picture of the cycle time for designing the clinical data database model?
  36. What about additional cycle times for developing case report forms, data management reports and data extracts for the biostatisticians?
  37. How long do you have to wait for the vendor to develop data validation checks?
  38. And what about internal EDC vendor team delays?
  39. Do you have a perception that the vendor EDC team is not updating you on a regular basis? You are probably right in assuming that they have some staffing issues.
  40. Do you have a perception that your clinical trial is delayed due to poor CRO execution, because they took their A-team off the project and sent them over to Pfizer? Yep.
  41. Are you running delayed user acceptance testing of the eClinical platform because your team is blocked with other clinical operations-related issues?
  42. Are you unclear regarding delivery-time impact of changes to the clinical protocol and case report form during development?
  43. Did you just now, think about multi-lingual support for non-English-speaking patients (Spanish, Russian, Arabic, Amharic….)
  44. You never asked what additional data reporting tools that the vendor could provide you that would save you time and money downstream?
  45. Did you ever define roles and responsibilities for a clinical data management team?
  46. You know you need training on the alphabet soup of the clinical trial platform supplied by 4 different eClinical vendors — EDC, ePRO, CTMS, eCOA, CTMF, RTSM? How and when will it all happen?
  47. Are you waiting for the last minute for training and then going to your research sites with a baseball bat?
  48. Did you get insufficient training for user management and it slows you down when you need to onboard new users?
  49. Did some of your sites and investigators miss out on the training sessions and makeups?
  50. Your clinical research sites don’t have single sign-on and they have to deal with Post-Its and multiple logins — slowing them down in caring for patients.
  51. Your research sites don’t how to use the patient mobile app themselves
  52. At some point you will have delays due to technical issues with the statistician not being able to parse data extract files
  53. You might discover at the end of the clinical trial that you did not collect the primary efficacy endpoint or safety endpoints. Argh.
  54. The clinical protocol amendments were not properly documented and implemented in the EDC. Now you have issues with reconciling the misconfigurations and squaring away the statistical analysis — since the data structure at the beginning of the study does not match the data structure at the end of the study
  55. Your investigators and study monitors are not able to login because accounts were locked or they forgot how to use the system, because it’s 6 months after training.
  56. You need reports for patient compliance and you need someone else to extract data, process and prepare the report. If you think you are being held hostage by your CRO, you are right.
  57. The data manager assigned by the CRO to manage your data and produce reports left 2 weeks before the end of the study. Talk about being up the creek without a paddle.

After you have screwed up

The answer is extreme ownership ( Jocko Willink’s book is a great resource — Extreme Ownership — how US Navy Seals lead and win

I’ll give a very short version here:

The team leader bears full responsibility for explaining the mission, developing the tactics, and securing the training and resources to enable the team to properly and successfully execute.

  • If an individual on the team is not performing at the level required for the team to succeed, the leader must train and mentor that under-performer.
  • If the under-performer continually fails to meet standards, then you must be loyal to the team and the mission before an individual.
  • If under-performers cannot improve, the leader must make the tough call to terminate them and hire others who can get the job done.
  • It is all on the leader.

Let’s take an example from one of the 57 blockers.

#52 delays due to technical issues with the statistician not being able to parse data extract files

If you’re a small biotech, then the CEO owns the project. He or she need to write into the projects tasks: Extract the data in a test run 4 weeks before the end of the trial and make sure that the statistician can read the data.

Preempt a solution instead of debriefing a situation.

I hope you get the idea.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Danny Lieberman

Danny Lieberman

I am a physicist by training, serious amateur musician and everyday biker. Working in cybersecurity and AI-driven monitoring of clinical trials.