Gathering high-quality patient data poses challenges for study teams across therapeutic areas, but pediatric trials are notoriously difficult in this regard. In studies with a pediatric cohort, it’s not only the patients who are involved in the data collection process: The addition of caregiver-reported data adds yet another avenue of complexity that can compromise data accuracy. These unique challenges—which characterize, but are not confined to, pediatric studies—offer significant opportunities for the industry to:
ObvioHealth’s experience has demonstrated the potential of digital instruments to mitigate—even eliminate—these obstacles by improving data collection and assessment processes. By leveraging advanced ePRO methods such as audio and image capture, these instruments can collect more objective data. That data can then be augmented through AI-assisted expert rating. Adopting this “precision instrumentation” approach to create augmented ePRO can transform clinical trials across therapeutic areas and ultimately enhance clinical practice.
Barriers to Patient-Reported Outcomes
Pediatric research is replete with roadblocks that have massive implications for data accuracy, including:
Logistical challenges for caregivers
The concept of “participant burden” is particularly true for caregivers, who spend an estimated 40-plus hours each week attending to their children’s needs. They need convenient, easy methods to report data and stay compliant with study tasks, particularly when multiple caregivers in the same family are involved in a study.
Because participant-reported data has a human element, caregiver perceptions may vary, and caregivers and children may report similar data in different ways. Another complicating factor: Caregivers have emotional bonds with their children, which can cloud their objectivity, particularly if the children are seriously ill. Caregivers may unknowingly inflate the baseline ratings or the impact of a therapeutic intervention, increasing the study’s placebo effect.
Observable Versus Projected Behavior
Within a clinical trial, each member of the patient-caregiver-clinician triad needs to adopt new skills and behaviors. For caregivers and clinicians, this means disconnecting from their primary role and becoming neutral observers. Caregivers need to gather and transmit observable data, rather than the projected outcomes or behaviors they may want to see.
In response to these challenges, the industry has increasingly adopted electronic patient-reported outcomes (ePRO) to improve data collection and assessment. The results are promising: ePRO has boosted compliance and accuracy among pediatric trial participants and caregivers. The following examples, among many others in medical literature, show that e-diaries and mobile devices can significantly improve data quality and adherence.
A randomized trial of electronic versus paper pain diaries in children: Impact on compliance, accuracy, and acceptability
Tonya M.* Duare Valenzuela, Paul P. Stork
Study Design: 60 children with headaches or JIA (ages 8 to 16) randomized to complete e-diaries (smartphone) or paper diaries at home daily for 7 consecutive days.
Results: 83.3% of children with e-diaries and 46.7% of children with paper diaries were 100% compliant.
Paper diaries contained significantly more errors and omissions than e-diaries (P<0.001).
Brief Report: Assessment of Children’s Gastrointestinal Symptoms for Clinical Trials
Lynn S. Walker, PhD, and Susan C. Sorrells, PharmD
Vanderbilt University School of Medicine and Glaxo SmithKline, Inc.
Study Design: Irritable bowel syndrome symptoms in pediatric population (6 to 10 years). Subjects (n=11) worked with parents to complete daily diaries using smartphones for 1 week.
Results: Subjects were 100% compliant. Parents reported that smartphones were enjoyable and easy to use. Parents and children willing to participate in a similar study in the future.
The adoption of ePRO in decentralized or hybrid clinical trials can effectively mitigate the participant burden and, in turn, increase data accuracy. However, problems remain with ePRO—it alone cannot ensure accurate outcome measurement. The following study measuring blood glucose recording by children highlights the gap.
Compliance with blood glucose monitoring in children with type 1 diabetes mellitus. Journal of Pediatrics
N=18 children (12 to 18) with Type I diabetes
Each conducted blood glucose determinations 4 times per day, 30 minutes before meals and at bedtime for 12 weeks. For the first 6 weeks, participants didn’t know the meters were recording. During weeks 7 to 12, they were aware the devices were tracking.
Table. Compliance with blood glucose monitoring in 18 children with type 1 diabetes mellitus
In addition to over- and under-reporting—as demonstrated in the blood glucose pediatric study—ePRO does not fully solve the inherent problem of subjectivity in patient- and caregiver-reported outcomes. Subjectivity affects not only the accuracy of data submitted during the trial, but also the accuracy of baseline measurements and outcome measurement in general.
As an example, ObvioHealth conducted a study that used an audio recording device to monitor the duration of infant cries. The data from the device was compared to how much and how often parents reported crying via an e-diary. The results found that parents inflated the amount of crying by 2 to 4 times.
The findings of this study illustrate the risks associated with relying too heavily on caregiver reporting. Because caregivers are so deeply entrenched in the care of their children, it’s difficult for them to be objective, hindering researchers’ abilities to achieve accurate outcomes.
What is Augmented ePRO?
In our extensive experience developing digital instruments for pediatric decentralized clinical trials, ObvioHealth has recognized the potential of digital tools to augment ePRO and deliver on the promise of patient-reported data.
Now, with advancements in artificial intelligence, study teams can combine digital tools with AI-enhanced algorithms and technologies to improve data accuracy and outcome measurement. AI-assisted ePRO uses smart capture technologies to enable participants to record and submit source data with ease—much like a banking app guides its user to capture a clear, quality image of a check in order to make a mobile deposit into an account.
An augmented ePRO platform goes even further, however, integrating AI for both the capture of patient-reported outcomes and more accurate assessment of that data by expert raters.
AI-Assisted Audio and Image Capture
AI-assisted ePRO has significant potential to support the collection and submission of unstructured data such as audio, images, and video. Audio, image, and video capture allows patients and caregivers to gather observable—and therefore more objective—data in real time, rather than expecting caregivers to precisely characterize their children’s symptoms and behaviors.
Smart-capture tools also help caregivers and patients alike to record and upload only high-quality data, preventing study teams from receiving unusable submissions. Meanwhile, AI algorithms support study teams in collecting the right data and avoiding missed data points.
AI-Assisted Audio Capture
Advanced audio capture tools—both in the form of unique digital instruments or capture-support applications that leverage built-in smartphone features—allow study teams to gather and assess sounds that might be difficult to quantify or might otherwise go undetected. For instance, study teams can create an AI-supported audio capture feature that employs the microphone of a mobile device, designing it to automatically detect and record the nighttime crying of infants. The only burden left to caregivers is to submit the data via the app when prompted.
For instruments recording constant, unstructured ambient noise, AI algorithms can be applied on multiple levels: to detect the onset of the audio outcome, to measure its duration or intensity, and to separate it from superfluous sound.
Current and future applications of audio capture include:
Current and future applications of audio capture include:
During clinical trials with participant-submitted images, study teams frequently contact patients to request higher-quality images or to intervene when a photo includes protected health information. Low-resolution, out-of-focus, or underexposed images can affect the accuracy of clinical ratings.
AI-driven capture-support applications leverage the built-in features of smartphones to take quality images. Smart-capture capabilities can correct poor lighting, autofocus the camera lens, and flag distracting objects in the background. And, AI-assisted image-capture tools, such as framing indicators, help participants take high-quality “selfies” by focusing the camera on the relevant area of the body, prompting them to take photos from multiple angles and informing them of any face angle misalignments.
Of course, more data is not the end goal—better data is. In addition to streamlining data collection and submission, AI-assisted digital tools can help clinicians analyze and measure patient-reported data for optimized expert rating.
In the case of measuring infant cries, a study team may have 24 hours’ worth of audio recordings to review. An AI-assisted platform allows the study team to cluster the important sounds together, filtering out unwanted noise and training the algorithm in the process. Human annotators can collaborate with AI and machine learning algorithms to streamline the expert review process while delivering more accurate data.
With AI supporting the expert rating process, technology performs the repetitive “heavy lifting” involved in assessing participant submissions, so raters can focus on tasks that require their specific expertise. This capability improves accuracy: When teams can create baseline measurements based on objective data, the data compared to the baseline is subsequently more accurate.
AI-assisted rating of ePRO also helps resolve inter-rater variability, addressing the human subjectivity inherent in expert scoring. An ObvioHealth study recently demonstrated the importance of objectivity in the expert rating of unstructured data. The study collected and compared stool ratings from three separate touchpoints in the ePRO collection and assessment process: caregivers, central raters, and pediatric gastroenterologists. The study's results evidence the truth of inter-rater variability—among caregivers, raters, and clinicians alike—which supports the need for further research into standardized training or automated scoring, with a special emphasis on the Bristol Stool Scale.
Two recent studies from ObvioHealth show the effectiveness of AI-assisted capture and assessment of unstructured data to augment ePRO:
Infant/Toddler Stool Study
A baby nutrition client sought to create a database of stool images that could be used for internal software development purposes.
ObvioHealth recruited 100 mothers of toddlers virtually. We developed the protocol, eCRF, prescreening and eICF. Mothers captured images of their babies’ stools using their smart phones. They were trained to grade them using the Brussels scale. Images were stratified into seven stool consistency types and validated by two expert scorers.
Enrollment target was reached in 12 weeks. Over the course of the study, 2,366 images were collected, and the virtual study design facilitated strong engagement, resulting in a 95% retention rate. Expert scorers identified some biases in mothers’ scores, highlighting a tendency to over-report those within the “normal” range.
Infant Crying/Fussing Study
ObvioHealth was asked to conduct an observational study of infant crying and fussing.
Caregivers recorded their infants’ crying and fussing, using the ObvioHealth app to indicate their perception of each event through daily questionnaires and e-diaries. A device inserted into an infant onesie also recorded infant crying. Mandatory training and ongoing communication with subjects minimized user-interface issues and improved data integrity.
Reported crying and fussing frequency and duration was 2 to 3 times higher than the device audio-recorded events. The results provided insights into human bias.
The benefits of augmented ePRO extend beyond clinical trials. Cataloguing a broader range of unstructured data allows care teams to develop more precise baseline measurements for medical conditions. Clinicians can then leverage these enriched data sets in clinical practice to make more accurate diagnoses.
For instance, technology improvements can support early intervention in biliary atresia, a life-threatening, congenital liver disorder. Clinicians can educate caregivers to check for a change in stool color that could signal biliary atresia or other diseases in their newborn children.
Augmented ePRO is also advancing the treatment of asthma by delivering more nuanced data on symptoms. Researchers can use digital instruments to measure children’s usage of inhalers and record their breathing sounds while using inhalers, providing a comprehensive set of structured and unstructured data for expert assessment.
Pediatric clinical trials have traditionally posed some of the industry’s most complex challenges. The emergence of digital instruments and electronic patient-reported outcomes has mitigated some of these difficulties but not fully resolved the subjectivity inherent in caregiver relationships. Augmented ePRO represents the next level of patient-reported outcomes—leveraging precision instrumentation and the wonders of artificial intelligence to deliver better data and streamline the experience at every touchpoint.