Skip to main content
Playbook

How To Measure What Matters in Clinical Trials

A comprehensive guide to patient-first study design strategies that deliver stronger evidence

Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Playbook

How To Measure What Matters in Clinical Trials

No items found.
Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Playbook

How To Measure What Matters in Clinical Trials

A comprehensive guide to patient-first study design strategies that deliver stronger evidence

No items found.
Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Playbook

How To Measure What Matters in Clinical Trials

No items found.
Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Playbook

How To Measure What Matters in Clinical Trials

No items found.
Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Playbook

How To Measure What Matters in Clinical Trials

A comprehensive guide to patient-first study design strategies that deliver stronger evidence

Download the PDF
No items found.

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1
Back to Top
Playbook

How To Measure What Matters in Clinical Trials

A comprehensive guide to patient-first study design strategies that deliver stronger evidence

Download the PDF

Introduction

“Measuring what matters” is essential to fulfilling the ultimate purpose of clinical research: therapeutic innovation that delivers meaningful, real-world impact. The industry broadly agrees that understanding what matters to patients—and incorporating those insights into study design—is the key to successfully executed studies.

However, it’s important to note that designing a patient-centric trial isn’t easy. Sponsors have many stakeholders beyond patients whose needs they must consider—to say nothing of the regulatory guidelines that must be followed. Moreover, sponsors often feel more comfortable with legacy trial methods; they rely on traditional study designs that can be overly burdensome and deliver inaccurate outcomes.

Given what can sometimes be competing demands, compounded by the complexity of conducting a clinical trial, it’s no surprise the industry often fails to capture the robust evidence required to bring a therapeutic intervention to market.2

Consider this industry statistic: More than half—54 percent—of therapeutics fail to prove efficacy in Phase III trials.3 Of the therapeutics that do make it to market, 41 percent provide no health gains over older comparators.4

To design stronger and more successful studies, the industry needs to start thinking outside the box. This requires a reduced dependence on historic models and a greater attention to methods that best support patients’ needs.

This playbook is designed to help you accomplish exactly that. Below, we explore strategies for designing clinical trial protocols that measure what matters, focusing on several key areas within the study design process:

‍
‍

Endpoint and Outcome Selection

A lack of patient centricity will become evident in the earliest stages of study design—particularly in endpoint and outcome selection. Research5 demonstrates that many clinical trial endpoints, rather than reflecting patients’ therapeutic priorities, are subject to the “streetlight effect”—they’re chosen because they’re easy to measure.

In addition, researchers often fail to consult patients on what therapeutic improvements matter most to them. But, "[b]ecause they are experts in what it's like to live with their condition," patients are uniquely positioned to advise researchers on which outcomes are the most life-improving.6

Bringing patients into the study design process can help sponsors and study teams to build a more accurate and comprehensive understanding of the disease experience and ensure their endpoints and outcomes are applicable in the real world.

Let’s use multiple sclerosis (MS) as an example. For years, researchers have relied on the Expanded Disability Status Scale and relapse rate as endpoints in MS clinical trials, despite considerable deficiencies in these measurements.7 They are tried-and-true, validated measurements—and they are easy to implement. However, research7(p218) suggests there are more precise measurements available that can deliver stronger evidence of therapeutic efficacy. An accelerometer, for instance, can more accurately measure walking impairments in MS patients, an outcome that is “particularly meaningful, as 70% of patients reported gait impairment to be the most challenging aspect of the disease.”7(p218)
‍

Digital Endpoints

A digital endpoint, as in the previous case, may more accurately measure what matters to patients. Digital endpoints can use sensors to capture existing outcomes in a new way. They can also capture new measurements that were not previously possible. While new outcomes require supplemental validation processes, there is increasing acknowledgement that such investments are often warranted in light of the potential payout of patient-centric trials.1

Sponsors who deem novel endpoints or outcomes to be too risky can still explore potentially promising measures as secondary endpoints. Should trial results point to positive signals from these endpoints, sponsors can then consider their incorporation as primaries in future studies.

The Digital Medicine Society (DiME) keeps a library of sponsor-led studies that have used digital endpoints.8 This list includes close to 400 endpoints and is an excellent starting point for sponsors.

‍

Instrument and Device Selection

Once meaningful endpoints and outcomes are selected, the challenge then becomes finding the most effective methods to actually collect the measurements that matter. Choosing instruments that reduce burden and provide a seamless user experience can help to simplify the capture of different types of data—from subjective quality-of-life reporting to objective vitals capture. Researchers must take care to source and select equipment that strikes the right balance between user-friendliness for patients and precision monitoring to support accuracy.
‍

ePRO

Electronic patient reported outcomes (ePRO) tools are the most reliable instruments for capturing a patient’s lived experiences, including measuring the improvement or deterioration of quality of life. Most ePRO tools allow patients to complete study tasks and report symptoms outside the four walls of a clinic—as they go about their daily lives—delivering more accurate data that’s representative of the real world.
‍

Bring Your Own Device

In recent years, the industry has been shifting toward a “bring your own device” (BYOD) ePRO model. Rightly so, as research9-11 has proven its equivalence—and, in certain cases, its superiority—to provisioned devices. This model enables patients to easily report both episodic and planned events in near real time using their own smartphones, resulting in higher compliance and more accurate data for sponsors.
‍

Questionnaire Design

Beyond technology considerations, it’s important for sponsors to choose questionnaires that feel relevant to patients. For studies that include multiple questionnaires, researchers should seek to reduce or eliminate any redundancies, with the goal of getting “better answers from patients with fewer questions.”12 Consulting with patients can help to achieve this, as well as determining the best cadence for questionnaire tasks to minimize non-compliance.

‍

Other ePRO considerations to help measure what matters:

  • Incorporating push notifications and task reminders supports patient engagement and increases compliance rates.
  • Streamlined user interfaces, with one-tap answers and easy scrolling, enable participants to report on the go.
  • ePRO tools should allow patients to log time-stamped events both online and offline.
  • Demographic constraints, like language and digital literacy, should be accounted for (e.g. increased questionnaire font size for older participants).

Image, Audio, and Video Capture

Image, video, and audio capture can help to provide researchers with a more complete picture of a patient’s symptom experience. For example, a patient can report with a description or subjective rating for the severity of their rash and supplement that outcome with an image for clinical assessment. Alternatively, a photo or video can be submitted in place of a subjective description.

Photo capture has been used in healthcare settings for some time. But, with smartphones, any patient can now capture this data at home.

The same is true for audio. Scientists are now using audio recordings to quantify cough, which was nearly impossible a decade ago. New artificial intelligence (AI) applications can passively detect coughs and record them as and when they occur, providing more precise symptomology outcomes.13

Image, video, and audio capture platforms should ideally include central rater capabilities for independent scoring. Here, too, AI can be used to help reduce bias in scoring and mitigate rater variability.
‍

Sensors and Wearables

Yet another way researchers can gain a more complete picture of a patient’s health status and experience is through sensors and wearables, which can capture continuous or point-in-time data as patients go about their everyday lives. Collecting electrocardiogram data 24/7, for example, might reveal spikes or trends that may not have been spotted with sporadic ECGs during site visits.

Importantly, these devices can capture massive amounts of data, potentially complicating analysis if not properly managed. Site staff shouldn’t have to sift through reams of data or struggle with software that doesn’t easily integrate between different systems.

Complex or high-volume sensors and wearables can be paired with an intelligently designed AI technology that can capture and compile the data in specific increments and present the findings to the study team in the form of easy-to-read dashboards. These platforms should also flag any anomalies, as well as important trends, for the research team. We’ll discuss more about this in the safety section.
‍

Which Device?

Selecting the right sensor or wearable for a study begins with the identification of relevant metrics that reflect the meaningful outcomes and endpoints mentioned earlier. The devices used to measure these metrics must also ensure patient safety and privacy and feature the best UX possible to encourage compliance. And, of course, the data collected must meet reliability and validity requirements. To determine the best sensor or wearable for a given study, sponsors and research teams should consider a device’s:

‍

On-Site Data Capture

Most clinical trials may require a visit with a clinician at some point in the study. Physical site visits should be included in a clinical trial design whenever a study task cannot safely or accurately be conducted remotely. An example of this might be assessments pre- and post-phlebotomy, where physical contact is preferable to accurately evaluate pain, local infection, and symptoms of anemia. However, in an increasing number of cases, sponsors might consider replacing a site visit with a telehealth visit to reduce patient burden and enable more regular data collection.

When choosing sites, it is necessary to clearly differentiate the role of the principal investigator and their study team from that of the care team; overlap between the two teams may bias a study. Sponsors and study teams must take into consideration that patients could feel pressured to participate if the trial is conducted by their own care team. There’s also the question of whether a primary care physician serving simultaneously as a clinician can objectively assess their own patients in a clinical trial setting.

Decentralization can create a layer of separation between research and clinical practice to mitigate biases. In this way, clinicians monitor and measure the efficacy of study products or treatments remotely, distinct from the responsibility of providing clinical care to patients.
‍

Safety and Compliance Measurement

Measuring the safety of a therapeutic doesn’t just matter—it is, of course, critical. Yet, one study found that up to 50 percent of adverse events can go undetected in clinical trials.14
‍

Remote Patient Monitoring (RPM)

When it comes to safety, remote patient monitoring has revolutionized clinical trials for both patients and researchers by facilitating the early detection of potential adverse events (AE).

RPM operates on multiple levels. Data captured remotely flows into a DCT platform, where it can be displayed on dashboards for manual monitoring by a study team. Automatic monitoring through AI that’s programmed to identify data outliers can also alert study teams to any potential health issues. For example, if a patient’s vital sign dips too low, the AI-enabled dashboard flags the event as a potential AE and notifies the study team to follow up.

This RPM process is also applicable to lapses in compliance. With real-time data capture and monitoring, the study team has visibility into any skipped or overdue study tasks. If a patient begins to lapse, a member of the team can reach out and encourage them to complete their tasks.
‍

Virtual Site Team

Remote patient monitoring is optimized when it's paired with a virtual site team that tracks patients’ progress in near real time. In addition to monitoring patient data, virtual site team members also communicate directly with patients, addressing any issues or questions they might have.

Advanced technology has dramatically improved workflows and data management in clinical trials. But, it cannot replace the value patients place on connections with real human beings. When patients feel cared for, even virtually, they are more motivated to engage with the study and more likely comply with the protocol.

‍

Other Study Design Considerations

This playbook has focused primarily on the “measurement” aspect of measuring what matters. But, there are several other key components of the study design process that sponsors need to consider critically in order to make measuring what matters possible. These include:


Cohort Design and Recruitment

Sponsors need to target the right patient populations if they hope to deliver meaningful therapeutic evidence that translates to the real world.

In 2022, the FDA issued new draft guidance for developing plans to enroll more participants from underrepresented racial and ethnic populations in the U.S., with the goal of “facilitat[ing] the development of better treatments and better ways to fight diseases that often disproportionately impact diverse communities.”15

Cohort design is essential to meeting this goal of recruiting a truly representative sample. And, decentralization is a strategic enabler of inclusive recruitment. Real-world data can be leveraged to refine cohorts, and digital recruitment can be used to reach them. These methods, in combination with reducing—or eliminating—site visits, break down geographic boundaries and allow sponsors to find and recruit the best possible patient populations for their studies.
‍

Patient Training

Patients need to understand how to properly use measurement instruments—whether they be wearables or apps—for reliable and accurate data capture. Clear and comprehensive training is crucial and should be easily accessible to patients throughout their clinical trial journey.

It’s also important to tailor educational materials to suit different learning styles and levels of digital literacy. Contrary to popular belief, with proper training, older patient populations are very much able to successfully use technology in clinical trials, as confirmed in multiple studies—including ObvioHealth’s own.

‍

eConsent

Informed consent is one of the most important stages of any clinical trial—from an ethical perspective as well as a compliance one. Consent forms must clearly explain the protocol and state what’s required of patients during the trial process. In addition to being a regulatory requirement, confirmation of a patients’ reasonable understanding of the trial journey will reduce the risk of non-compliance and drop-out later in the study.

In trials with decentralized components, eConsent can be deployed to patients’ smartphones, allowing them as much time as needed to review all documents. Strategies such as breaking the text into shorter snippets and animated screens, along with short quiz questions, can also help to ensure comprehension. And, remote study teams should be available to patients throughout the process to address any questions or concerns.

‍

Conclusion

Understanding the importance of anchoring studies in patients’ therapeutic priorities isn’t enough. The industry needs to take action.

Every sponsor and study team should be approaching the design phase with this question in mind: “How do we measure the efficacy of our therapeutic in ways that will actually translate into meaningful, real-world impact?” The answer to that question should influence every aspect of study design and conduct.

For most clinical trials, traditional methods alone won’t provide the flexibility needed to measure what matters for stronger therapeutic evidence. There will be many cases where decentralized study designs, whether fully virtual or hybrid, better serve this goal.

Partnering with a VRO that customizes trial services and products can help sponsors ensure they are designing their studies to truly support therapeutic innovation and to meet the unique needs of today’s patients, wherever they may be.

‍

‍

References

  1. Levitan B, Getz K, Eisenstein EL, et al. Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci. 2018;52(2):220-229. doi: 10.1177/2168479017716715
  2. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1). doi: 10.1186/s13063-017-1870-2
  3. Hwang TJ, Carpenter D, Lauffenburger JC, Wang B, Franklin JM, Kesselheim AS. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016;176(12):1826-1833. doi: 10.1001/jamainternmed.2016.6008
  4. Darrow J, Kesselheim A. Nearly One-Third Of New Drugs Are No Better Than Older Drugs, And Some Are Worse. Health Affairs Forefront. Published online October 6, 2017. doi: 10.1377/forefront.20171021.268271
  5. Bundgaard JS, Iversen K, Bundgaard H. Patient-prioritized primary endpoints in clinical trials. Scand Cardiovasc J. 2022;56(1):4-5. doi: 10.1080/14017431.2022.2035808
  6. Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. FDA. July 2023. Accessed July 2, 2023. https://www.fda.gov/drugs/development-approval-process-drugs/cder-patient-focused-drug-development
  7. Landers M, Dorsey R, Saria S. Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption. Digit Biomark. 2021;5(3):216-223. doi: 10.1159/000517885
  8. Digital Medicine Society (DiMe). Library of Digital Endpoints. Updated 2023. Accessed July 1, 2023. https://dimesociety.org/get-involved/library-of-digital-endpoints/
  9. Demanuele C, Lokker C, Jhaveri K, et al. Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit Biomark. 2022;6(2):47-60. doi: 10.1159/000525080
  10. Newton L, Knight-West O, Eremenco S, et al. Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings. J Patient-Rep Outcomes. 2022;6(1). doi: 10.1186/s41687-022-00492-5
  11. Byrom B, Doll H, Muehlhausen W, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581-589. doi: 10.1016/j.jval.2017.10.008
  12. National Institutes of Health. Patient-Reported Outcomes Measurement Information System (PROMIS). U.S. Department of Health and Human Services. January 2019. Accessed July 2, 2023. https://commonfund.nih.gov/promis/index
  13. Hyfe AI. AI that decodes the rich health data in sound. Published 2023. Accessed July 1, 2023. https://www.hyfe.ai/ai-models
  14. Wahab IA, Pratt N, Kalisch L, Roughead E. The Detection of Adverse Events in Randomized Clinical Trials: Can we Really Say New Medicines are Safe? Curr Drug Saf. 2013;8(2):104-113. doi: 10.2174/15748863113089990030
  15. Adashi EY, Cohen IG. The FDA Initiative to Assure Racial and Ethnic Diversity in Clinical Trials. J Am Board Fam Med. 2023;36(2):366-368. doi: 10.3122/jabfm.2022.220290R1

Discover More Case Studies

Discover More News

Discover More White Papers

Discover More Blogs

Discover More Playbooks

Discover More Webinars

Discover More Events

No items found.

Patient Engagement Strategies for Clinical Trials

A comprehensive look at tech-enabled, human-supported patient engagement protocols

› Read More
No items found.

How-To Guide for Decentralized Clinical Trials

Most of our prospective clients understand that virtual represents a new frontier in clinical research but often ask how to implement virtual solutions into their studies. Is this you? Maybe you already have a written protocol and want to gauge the possibility of a virtual transition, or perhaps you want first to determine the most effective trial design. Because these are scenarios we see so frequently, we thought we could offer a playbook to help you imagine your studies in a virtual way. Here are four big-ticket items that will impact how your study proceeds.

› Read More