NA

Findings
Fragmented and inconsistent measurement approaches currently hinder the generation of decision-grade evidence for pediatric rare diseases. Small and geographically dispersed patient populations make traditional site-based clinical assessments operationally difficult and burdensome for families. Digital health technologies can capture subtle functional changes and "functional fingerprints" in home settings that are often missed during infrequent clinic visits. Standardized core digital measures across conditions allow for the aggregation of data and the creation of a shared evidence base for rare disorders. Meaningful aspects of health identified by patients and caregivers include motor function, communication, sleep quality, and autonomic stability.

Recommendations
Sponsors should adopt the core set of digital clinical measures to reduce trial timelines, lower development costs, and decrease participant burden. Researchers should prioritize passive and objective data collection to minimize the need for manual tracking by caregivers. Clinical trial designs should transition toward decentralized or hybrid models to improve access for children and families regardless of their location. Stakeholders should use the project's conceptual model to identify and customize digital measures that align with the specific health priorities of their target population. Developers should focus on human-centered design to ensure digital tools are usable and sustainable for pediatric patients and their support networks.

Regulatory Considerations
The FDA and EMA provide specific pathways and interaction opportunities to accelerate the acceptance of digital endpoints in rare disease trials. Digital measures must be validated as "decision-grade" endpoints to meet the evidentiary requirements for regulatory submission and marketing approval. Alignment with industry standards for data elements and interoperability is necessary to ensure data integrity across multi-site studies. Early engagement with regulatory bodies through meetings and formal submissions is critical for confirming the suitability of new digital biomarkers. Compliance with data privacy and ethical standards is paramount when collecting continuous, real-world data from vulnerable pediatric populations.

Findings
In a progressive neurodegenerative illness like Alzheimer's disease, slowing the rate of disease progression is considered a clinically meaningful outcome for patients and their caregivers.
The assessment of what constitutes a clinical benefit is highly dependent on the specific stage of AD being studied, the drug's mechanism of action, and the symptoms present in that patient population.
Direct input from patients and caregivers is critical for understanding disease burden and defining treatment benefits that are truly meaningful from their perspective.
The interpretation of score changes on Clinical Outcome Assessments (COAs) requires full context; an absolute point difference must be considered relative to the study's duration, the expected placebo decline, and the specific disease stage.
Evidence from biomarkers that show an effect on underlying disease pathology provides additional support and increases the persuasiveness of the changes observed on clinical endpoints.

Recommendations
Drug developers should implement multiple "fit-for-purpose" COAs that use different reporters (e.g., clinicians, observers) and methods to generate broad and diverse evidence of a drug's clinical benefit.
Sponsors should utilize both qualitative and quantitative methodologies to explore clinical meaningfulness, including assessing "meaningful within-patient change" throughout the development process.
Developers are encouraged to create and validate new COAs and leverage innovative approaches, such as digital health technologies, to better capture concepts that are relevant to patients, especially in the earliest stages of AD.
Throughout the drug development lifecycle, stakeholders should systematically collect and incorporate patient experience data to ensure that the perspectives, needs, and priorities of patients are meaningfully captured.

Regulatory Considerations
For a drug to gain approval, it must meet the regulatory standard of "substantial evidence of effectiveness," which is typically derived from adequate and well-controlled investigations designed to minimize bias.
The FDA defines clinical benefit as a clinically meaningful effect of a drug on how an individual feels, functions, or survives.
An assessment of clinical benefit is not limited to the primary endpoint; the consistency of findings across multiple endpoints (primary and secondary) is a key consideration during regulatory review.
The accelerated approval pathway may be used for serious conditions with unmet needs based on a surrogate endpoint, but traditional approval requires verification of clinical benefit in confirmatory trials.
The FDA's evaluation includes a benefit-risk analysis, which considers the severity of the disease and the availability of alternative therapies, recognizing that patients and physicians may accept greater risks for life-threatening illnesses.

Findings
Patient experience data provides critical context for regulatory review by illuminating disease burden, unmet medical needs, and the aspects of a condition that matter most to patients.
A systematic approach is necessary to ensure patient experience data is robust enough for regulatory consideration, moving beyond anecdotal evidence to scientifically rigorous data collection.
Early engagement between sponsors and the FDA is a key factor for successfully incorporating patient perspectives into a drug development program.
The value of patient-reported outcomes (PROs) and other clinical outcome assessments (COAs) is highly context-dependent, varying significantly across different diseases and patient populations.

Recommendations
Drug sponsors should leverage the FDA's meeting process to discuss their strategies for collecting and submitting patient experience data early in the development lifecycle.
Sponsors should utilize the repository of meeting reports as a learning resource to understand best practices and common challenges in patient-focused drug development for specific conditions.
Patient advocacy groups should actively participate in these discussions to ensure the full spectrum of patient experiences is captured and communicated to both regulators and developers.
Researchers should develop and validate novel tools and methodologies for capturing and analyzing patient experience data that are meaningful for both clinical and regulatory purposes.

Regulatory Considerations
Patient experience data is a key component of the benefit-risk assessment, providing evidence that can inform regulatory decisions regarding a drug's approval and labeling.
The FDA's review of patient experience data is guided by a commitment to patient-focused drug development, as mandated by the 21st Century Cures Act and supported by user fee agreements like PDUFA.
The scientific rigor of data collection and analysis is paramount; for patient experience data to be influential, it must meet high standards of validity and reliability.
Transparency is a core principle, and the publication of these meeting reports is intended to provide clear examples of how patient input can be effectively integrated into regulatory submissions.

Findings
The document introduces a risk-based credibility assessment framework for establishing and evaluating the credibility of an Artificial Intelligence (AI) model's output when used to support regulatory decisions regarding drug safety, effectiveness, or quality. The framework outlines a 7-step process beginning with defining the question of interest and the Context of Use (COU). Credibility is defined as trust, established through evidence, in the AI model's performance for a particular COU. The credibility assessment is tailored to the AI model risk, which is a combination of model influence (the AI model's evidence contribution relative to other evidence) and decision consequence (the significance of an adverse outcome from an incorrect decision). The document highlights challenges with AI use, including variability in development datasets (training/tuning), the need for methodological transparency due to model complexity, difficulty in quantifying and interpreting uncertainty in model output, and the potential for performance change over time (data drift), which necessitates life cycle maintenance.

Recommendations
Sponsors and interested parties should define the question of interest and clearly define the COU, detailing the AI model's specific role and scope and whether other information will be used. They should assess the AI model risk (low, medium, or high) to ensure that subsequent credibility assessment activities (Step 4) are commensurate with that risk and tailored to the COU. For Step 4, the credibility assessment plan should include a description of the model, model development process (including inputs, architecture, feature selection, and rationale), and data used (training and tuning data). Development data must be deemed fit for use (relevant and reliable) to mitigate issues like algorithmic bias. The plan should also detail the model evaluation process using independent test data and include performance metrics with confidence intervals, an estimate of uncertainty, and a description of model limitations. Early engagement with the FDA is strongly encouraged to discuss model risk and the adequacy of the credibility assessment plan.

Regulatory Considerations
The risk-based credibility assessment framework is intended to help organize and document information for regulatory submissions. The required stringency of assessment activities and the level of documentation should be commensurate with the AI model risk. For AI models whose performance can change over time (e.g., in pharmaceutical manufacturing or postmarketing), sponsors must implement life cycle maintenance plans to monitor performance and manage changes in a risk-based manner. Changes to AI models should be evaluated through the manufacturer's change management system and may require re-execution of parts of the credibility assessment plan. Early engagement can be facilitated through formal meetings (e.g., Pre-IND) or other specialized programs listed in the guidance, such as the Center for Clinical Trial Innovation (C3TI), the Model-Informed Drug Development (MIDD) Paired Meeting Program, and the Emerging Technology Program (ETP) or Advanced Technologies Team (CATT).

Findings
Traditional, site-based clinical trials often create significant burdens for participants, which can hinder recruitment, retention, and the enrollment of diverse populations.
A lack of early and sustained patient engagement in trial design can lead to research protocols that are misaligned with patient needs and endpoints that are not meaningful to them.
The underrepresentation of diverse racial, ethnic, and other demographic groups in clinical trials limits the generalizability of study results and can perpetuate health disparities.
Emerging digital health technologies (DHTs) and real-world data (RWD) present significant opportunities to make clinical trials more efficient, patient-centric, and inclusive, but their adoption has been inconsistent.

Recommendations
Sponsors and research teams should engage patients and patient advocacy groups as active partners throughout the entire clinical trial lifecycle, from design to dissemination.
Decentralized clinical trial (DCT) elements should be incorporated to reduce patient burden, improve access for diverse populations, and enhance the quality of data collection.
Trial sponsors must develop and implement proactive strategies to enhance the diversity and inclusion of trial participants to ensure results are applicable to all patient populations.
Novel endpoints derived from DHTs should be developed and validated to capture more objective, real-world measures of how patients feel, function, and survive.
Multi-stakeholder collaboration between industry, academia, patient groups, and regulators is essential to address systemic challenges and improve the clinical trial enterprise.

Regulatory Considerations
Early and frequent communication with regulators, such as the FDA, is critical when implementing novel approaches like DCTs or developing new digital endpoints for pivotal trials.
Regulatory frameworks must support the use of innovative technologies and trial models while ensuring data integrity, reliability, and patient safety.
The use of a single Institutional Review Board (IRB) for multi-site trials is a key regulatory-supported mechanism for streamlining ethics review and increasing trial efficiency.
When using DHTs and decentralized methods, robust plans for data quality, privacy, and security are necessary to meet regulatory standards for trial data submission.

Findings
The use of Artificial Intelligence (AI) and Machine Learning (ML) is being applied to a broad range of drug development activities with the potential to accelerate the process and make clinical trials safer and more efficient. The inclusion of AI/ML is most common in the clinical development/research phase of regulatory submissions. Concerns exist that AI/ML algorithms could amplify errors and preexisting biases in underlying data sources, which raises issues related to generalizability and ethical considerations. Other challenges include limited explainability due to model complexity and proprietary reasons, as well as managing risks related to data quality, reliability, and representativeness. The FDA recognizes that a careful, risk-based assessment of the specific context of use (COU) is needed when evaluating AI/ML.

Recommendations
Stakeholders should adhere to practices in three key areas: human-led governance, accountability, and transparency; quality, reliability, and representativeness of data; and model development, performance, monitoring, and validation. A risk management plan should be applied to identify and mitigate risks based on the COU, guiding the level of documentation and transparency. Practices are needed to ensure the integrity of AI/ML and address issues like bias and missing data. For models, developers should use pre-specification steps and clear documentation for development and assessment criteria. Models must be monitored over time for reliability and consistency, and Real-World Data (RWD) performance can provide valuable feedback, including for potential re-training.

Regulatory Considerations
The FDA encourages early engagement through mechanisms like the Critical Path Innovation Meetings (CPIM), ISTAND Pilot Program, and Emerging Technology Program to discuss relevant AI/ML methodologies or technologies. The Verification and Validation (V&V 40) risk-informed credibility assessment framework and the principles for Good Machine Learning Practices (GMLP), while not specific to drug development, are helpful guides for evaluating models. The industry is exploring the use of a Predetermined Change Control Plan (PCCP) mechanism for AI/ML-based devices to proactively specify and manage modifications, enhancing adaptability. In general, a risk-based approach should guide the level of evidence and record keeping needed for the verification and validation of AI/ML models for a specific COU.

Findings
Coordination challenges with multiple locations in DCTs.
Variability in data collection across decentralized locations and remote tools.
Challenges in implementing certain statistical approaches in DCTs.
Need for DHTs to be accessible and suitable for all trial participants.
Ensuring compliance with local laws and regulations.

Recommendations
Develop clear protocols for integrating decentralized elements into clinical trials, specifying remote and in-person activities.
Use digital health technologies (DHTs) and electronic systems to streamline data acquisition, informed consent, and investigational product tracking.
Provide training for all stakeholders, including trial personnel, local health care providers, and participants, on decentralized processes.
Implement robust safety monitoring plans to address adverse events in decentralized settings.
Ensure compliance with local and international laws governing telehealth, data privacy, and investigational product use.

Regulatory Considerations
Maintain compliance with FDA requirements under 21 CFR parts 312 and 812 for drug and device trials, respectively.
Document all trial activities and data flows in trial protocols and data management plans, ensuring traceability and integrity.
Ensure informed consent processes meet FDA standards and provide clear communication to participants about decentralized trial activities and data handling.
Address investigational product accountability by documenting IP distribution, storage, and return or disposal.
Design electronic systems for decentralized trials to comply with 21 CFR part 11 requirements for data reliability, security, and confidentiality.

Findings
Decentralized Clinical Trials (DCTs), which shift activities away from sites, rely heavily on technology to reduce participant burden and improve access to trials. Digital platforms are essential for this shift, providing centralized data capture, remote monitoring, and streamlined workflows. Benefits include allowing participants to be monitored remotely, which can improve self-management and clinical outcomes, and giving researchers better insight into the real-world variability of disease activity. Currently, commercial platforms are often limited in functionality and face major challenges due to a lack of interoperability and specific data standardization protocols for clinical trial platforms, making it difficult to integrate third-party modules.

Recommendations
The paper strongly recommends the adoption of unified, integrated, and DCT-specific digital platforms to fully realize the benefits of decentralization. Platform developers should adopt international standards for health data exchange, such as HL7 FHIR and CDISC standards (PRM, CDASH, ADaM), to address the lack of data standardization and improve interoperability and modularity. Platforms should incorporate features that enhance participant engagement and adherence, such as customization options, simple user interfaces (UIs), push notifications, gamification, and allowing access to participant data . Security and governance teams are paramount to manage risks associated with malware, lost devices, and ensuring compliance with local legislation and data security protocols.

Regulatory Considerations
Digital platform design must maintain digital security and compliance with local legislation and data standards. The paper notes that a fully integrated, unified digital platform in a best-case scenario would use pre-existing standards (like CDISC and HL7) to guarantee interoperability. Adopting these standards and recommendations for data sharing, privacy, and security, as recommended by organizations like the Healthcare Information and Management Systems Society, is critical for future digital components used in DCTs. Improved data integrity and accountability in platforms could be further explored using technologies like blockchain to create an immutable ledger.

Findings
A proactive risk assessment is essential for optimizing study quality by identifying and mitigating risks to human subject protection and data integrity before and during a trial. Monitoring should be comprehensive, addressing not only likely risks identified initially but also less probable, high-impact risks and unanticipated issues that emerge. The effectiveness of a monitoring strategy depends on tailoring its timing, frequency, and methods to study-specific factors like complexity and site experience. Centralized monitoring, as part of a risk-based approach, can detect systemic issues like data omissions or protocol deviations more rapidly than traditional on-site visits alone.

Recommendations
Sponsors should formally document their risk assessment methodologies and ensure these assessments directly inform the creation and revision of monitoring plans. Monitoring plans must be detailed, outlining the study design, specific data sampling strategies, and clear protocols for escalating significant issues. When significant problems are identified, sponsors must conduct a timely root cause analysis and implement corrective and preventive actions. All monitoring activities, findings, and subsequent actions should be thoroughly documented and communicated to sponsor management, clinical site staff, and other relevant parties.

Regulatory Considerations
FDA regulations mandate sponsor oversight and proper monitoring but do not prescribe specific methods, providing the flexibility for sponsors to adopt a risk-based approach. The FDA may request a sponsor's risk assessment and monitoring plan documentation during an inspection. This guidance represents the Agency's current thinking and is nonbinding, allowing sponsors to use alternative approaches if they satisfy regulatory requirements. A key focus of monitoring should be to ensure critical trial processes, such as the maintenance of blinding, are protected to maintain overall data and trial integrity.

Findings
The clinical research ecosystem has longstanding diversity gaps, making targeted DEI strategies essential for equitable healthcare innovation.
Digital tools, including virtual visits, digital outreach campaigns, and AI-driven analytics, can increase access to trials for underrepresented populations.
Real-world data (RWD) and real-world evidence (RWE) help identify diverse participant pools and optimize recruitment strategies.
eConsent and educational resources improve patient engagement and retention by making clinical trials more transparent and accessible.
Trust-building measures, such as community partnerships and patient advocacy collaborations, are critical for long-term success in diversifying clinical trials.

Recommendations
Clinical trial sponsors should integrate digital tools at each stage of trial design to enhance participant diversity and reduce barriers to participation.
AI/ML and real-world data should be leveraged to identify, recruit, and retain diverse patient populations in a data-driven manner.
Digital engagement strategies, including social media outreach and mobile-friendly platforms, should be employed to improve awareness and accessibility.
Transparent communication, including clear eConsent processes and on-demand educational materials, should be prioritized to foster participant trust.
A comprehensive tracking system should be implemented to measure progress on diversity goals, ensuring accountability in clinical trial execution.

Regulatory Considerations
The FDA Diversity Plan requirement should be incorporated into clinical trial planning, with measurable targets for diverse participant inclusion.
Digital tools used for recruitment and engagement must comply with HIPAA, GDPR, and other privacy regulations to protect participant data.
The use of real-world evidence (RWE) in regulatory submissions should be expanded to demonstrate the efficacy of digital recruitment and retention strategies.
Standardized DEI reporting frameworks should be established to ensure regulatory bodies can assess the impact of diversity initiatives in clinical research.
Clinical trials utilizing digital tools should align with decentralized clinical trial (DCT) regulatory guidance to maximize accessibility and equity.

Findings
A need for additional regulatory clarity specific to DHT-derived endpoints.
The official clinical outcome assessment qualification process is impractical for the biopharmaceutical industry.
A lack of comparator clinical endpoints.
A lack of validated DHTs and algorithms for concepts of interest.
A lack of operational support from DHT vendors.

Recommendations
Engage key stakeholders early.
Incorporate DHT-derived endpoints in early-phase trials and observational studies.
Invest in COA development initiatives.
Engage technology manufacturers early in the development process.

Regulatory Considerations
The EMA published a Q&A document on DHT use in clinical trials.
The FDA released guidance on collecting patient data remotely using DHTs.
The FDA established the Digital Health Center of Excellence to facilitate early regulatory engagement.