Findings
The document introduces a risk-based credibility assessment framework for establishing and evaluating the credibility of an Artificial Intelligence (AI) model's output when used to support regulatory decisions regarding drug safety, effectiveness, or quality. The framework outlines a 7-step process beginning with defining the question of interest and the Context of Use (COU). Credibility is defined as trust, established through evidence, in the AI model's performance for a particular COU. The credibility assessment is tailored to the AI model risk, which is a combination of model influence (the AI model's evidence contribution relative to other evidence) and decision consequence (the significance of an adverse outcome from an incorrect decision). The document highlights challenges with AI use, including variability in development datasets (training/tuning), the need for methodological transparency due to model complexity, difficulty in quantifying and interpreting uncertainty in model output, and the potential for performance change over time (data drift), which necessitates life cycle maintenance.

Recommendations
Sponsors and interested parties should define the question of interest and clearly define the COU, detailing the AI model's specific role and scope and whether other information will be used. They should assess the AI model risk (low, medium, or high) to ensure that subsequent credibility assessment activities (Step 4) are commensurate with that risk and tailored to the COU. For Step 4, the credibility assessment plan should include a description of the model, model development process (including inputs, architecture, feature selection, and rationale), and data used (training and tuning data). Development data must be deemed fit for use (relevant and reliable) to mitigate issues like algorithmic bias. The plan should also detail the model evaluation process using independent test data and include performance metrics with confidence intervals, an estimate of uncertainty, and a description of model limitations. Early engagement with the FDA is strongly encouraged to discuss model risk and the adequacy of the credibility assessment plan.

Regulatory Considerations
The risk-based credibility assessment framework is intended to help organize and document information for regulatory submissions. The required stringency of assessment activities and the level of documentation should be commensurate with the AI model risk. For AI models whose performance can change over time (e.g., in pharmaceutical manufacturing or postmarketing), sponsors must implement life cycle maintenance plans to monitor performance and manage changes in a risk-based manner. Changes to AI models should be evaluated through the manufacturer's change management system and may require re-execution of parts of the credibility assessment plan. Early engagement can be facilitated through formal meetings (e.g., Pre-IND) or other specialized programs listed in the guidance, such as the Center for Clinical Trial Innovation (C3TI), the Model-Informed Drug Development (MIDD) Paired Meeting Program, and the Emerging Technology Program (ETP) or Advanced Technologies Team (CATT).

Findings
Coordination challenges with multiple locations in DCTs.
Variability in data collection across decentralized locations and remote tools.
Challenges in implementing certain statistical approaches in DCTs.
Need for DHTs to be accessible and suitable for all trial participants.
Ensuring compliance with local laws and regulations.

Recommendations
Develop clear protocols for integrating decentralized elements into clinical trials, specifying remote and in-person activities.
Use digital health technologies (DHTs) and electronic systems to streamline data acquisition, informed consent, and investigational product tracking.
Provide training for all stakeholders, including trial personnel, local health care providers, and participants, on decentralized processes.
Implement robust safety monitoring plans to address adverse events in decentralized settings.
Ensure compliance with local and international laws governing telehealth, data privacy, and investigational product use.

Regulatory Considerations
Maintain compliance with FDA requirements under 21 CFR parts 312 and 812 for drug and device trials, respectively.
Document all trial activities and data flows in trial protocols and data management plans, ensuring traceability and integrity.
Ensure informed consent processes meet FDA standards and provide clear communication to participants about decentralized trial activities and data handling.
Address investigational product accountability by documenting IP distribution, storage, and return or disposal.
Design electronic systems for decentralized trials to comply with 21 CFR part 11 requirements for data reliability, security, and confidentiality.

Findings
There is a need for comprehensive validation and verification processes for DHTs.
Ensuring data security and privacy is a significant concern.
Usability issues for diverse populations need to be addressed.
There is a lack of clarity on whether certain DHTs meet the definition of a device under the FD&C Act.
The guidance does not establish legally enforceable responsibilities.

Recommendations
Ensure DHTs are fit-for-purpose for clinical investigations.
Implement robust data security measures to protect participant information.
Conduct usability evaluations to ensure DHTs can be used by intended populations.
Engage with FDA early to discuss the use of DHTs in clinical investigations.
Develop a risk management plan to address potential issues with DHT use.

Regulatory Considerations
Verification and validation should be addressed regardless of device classification.
Sponsors should ensure compliance with data protection and privacy regulations.
FDA evaluates DHT data based on endpoints, medical products, and patient populations. Sponsors can engage with FDA’s Q-Submission Program for feedback on DHT usage in clinical trials.
Sponsors should understand the legal implications of using DHTs in clinical investigations.
The guidance provides recommendations but does not establish legally enforceable responsibilities.

Key Principles
The guidance establishes a predictable and efficient framework for formal interactions between the FDA and sponsors. Its core principle is that timely, high-quality communication is critical to a streamlined drug development process. The document clarifies that different stages of development require different types of meetings (e.g., Type A, B, and C), each with specific timelines and objectives. A key principle is that productive meetings depend on the sponsor providing a comprehensive meeting package in advance, allowing the FDA to prepare and provide substantive feedback.

Recommendations for Sponsors
Sponsors are strongly recommended to engage with the FDA early and throughout the drug development process. To ensure a productive meeting, sponsors should clearly articulate the purpose of the meeting, provide specific questions, and submit a well-organized and complete meeting package by the specified deadline. It is recommended that sponsors carefully consider the type of meeting that is most appropriate for their stage of development and the nature of the questions they have. Following the meeting, sponsors should adhere to the timelines and procedures for submitting meeting minutes for the official record.

Regulatory Considerations
This guidance is a key component of the regulatory framework under the Prescription Drug User Fee Act (PDUFA). Adherence to the procedures outlined in this document is a matter of regulatory compliance. The formal meetings described are a critical part of the Investigational New Drug (IND) and Marketing Authorization Application processes. The meeting process is designed to provide regulatory clarity, reduce the risk of clinical holds or refuse-to-file actions, and ultimately support a more efficient and predictable path to drug approval. The written record of these meetings serves as an important part of the administrative file for a product's development program.

Findings
Complex clinical trials involve unique challenges in design, operational feasibility, and regulatory compliance, necessitating early engagement with stakeholders.
Master protocols streamline trial processes by integrating shared scientific frameworks across sub-protocols, enhancing efficiency and data integrity.
Bayesian approaches, while promising, require transparency and rigorous validation to ensure robustness in trial outcomes.
The use of biomarkers and related assays in CCTs introduces added complexity, particularly concerning regulatory status and performance validation.
Effective risk-based quality management systems are essential to safeguard participant safety and maintain trial reliability.

Recommendations
Develop clear and detailed master protocols to define the shared framework, communication plans, and statistical methodologies for CCTs.
Employ risk-based quality management strategies, including robust risk assessment and targeted training for site personnel.
Ensure early and continuous engagement with regulators, investigators, and patients to address design complexities and operational challenges.
Pre-specify statistical plans and evaluation frameworks for Bayesian methods, adaptive designs, and biomarker integration.
Establish mechanisms for transparent reporting and management of safety data across sub-protocols while safeguarding trial integrity.

Regulatory Considerations
Adhere to EU CTR and IVD regulations, ensuring compliance in the use of biomarkers, companion diagnostics, and related assays.
Include comprehensive documentation of trial design, including shared frameworks, sub-protocols, and statistical methodologies, in submissions.
Implement robust data governance frameworks to ensure ALCOA++ (attributable, legible, original, accurate, complete, consistent) standards for regulatory submissions.
Plan for periodic reassessment of benefit-risk ratios during the trial, particularly when modifications or new data emerge.
Establish independent Data Monitoring Committees (DMCs) for long-term and complex trials to oversee safety and interim analyses.

Findings
Scope of Requirements: The requirement applies to NDAs, ANDAs, certain BLAs, and INDs.
Study data must conform to FDA-supported standards listed in the Data Standards Catalog.
Noncommercial INDs (e.g., investigator-sponsored or expanded access INDs) are exempt but may voluntarily comply.
Supported Standards: FDA currently supports standards like SDTM, ADaM, and SEND for tabulation and analysis.
Controlled terminology standards (e.g., MedDRA, CDISC Controlled Terminology) are critical for semantic data interoperability.
Implementation Timelines: New standards become mandatory 24 months after the transition date announced in the Federal Register.
Updates to existing standards are required for studies starting 12 months after their transition date.
Waivers: Waivers may be granted to allow submission using unsupported standard versions, but not for non-standardized data formats.
FDA-Sponsor Interactions: Sponsors should engage with the FDA early in the development process to align on data standardization plans.
Pre-submission technical reviews and Type C meetings can be used to resolve data standardization issues.

Recommendations
Ensure compliance with FDA-supported standards as listed in the Data Standards Catalog.
Begin using the latest supported standards early in the study lifecycle to avoid non-compliance.
Engage with FDA during early-phase development to confirm data standardization plans.
Use tools like the Study Data Technical Conformance Guide for additional implementation support.
Submit waiver requests early if specific standard versions cannot be used.

Regulatory Considerations
Submissions that do not meet the electronic format and data standard requirements may be refused filing (NDAs and BLAs) or refused receipt (ANDAs).
Compliance with standardized formats is mandatory unless explicitly exempted or a waiver is granted.
Updates to supported standards are announced in the Federal Register, with defined implementation periods to allow sponsors to transition.
Sponsors must include critical files like demographic datasets and define.xml files in their submissions to demonstrate standard conformance.

Findings
Early and ongoing engagement with regulatory bodies is essential to align endpoint development with regulatory expectations.
There are distinct pathways for drugs and medical devices, with specific meeting types (e.g., Type B and Type C meetings) available for each.
Qualification programs help establish the utility and validity of digitally-derived endpoints across different drugs, devices, or diseases.
Regulatory agencies provide detailed feedback on analytical and clinical validation, ensuring endpoints meet clinical relevance and reliability standards.
The document emphasizes the importance of understanding and navigating distinct regulatory frameworks (e.g., IND/NDA for drugs and IDE/510(k) for devices).

Recommendations
Engage with regulatory bodies, such as the FDA and EMA, early in the development process to obtain critical input.
Utilize structured programs, like the Drug Development Tool (DDT) and Medical Device Development Tools (MDDT) qualification pathways, to validate endpoints.
Schedule appropriate regulatory meetings, including Type B and Type C meetings for drugs or Q-Submission and Agreement Meetings for devices.
Consider utilizing general advisory sessions (e.g., Critical Path Innovation Meetings or Innovation Task Force Briefings) to enhance endpoint development strategies.
Document and align endpoint development with regulatory frameworks, ensuring compliance with safety, efficacy, and performance standards.

Regulatory Considerations
Use FDA’s IND/NDA and IDE/510(k) pathways for endpoint validation, tailoring engagement to the specific type of medical product.
Schedule Type B and Type C meetings for focused discussions on endpoint development, including context of use and validation.
Engage with EMA through pre-submission meetings for scientific advice, ensuring endpoints meet requirements for clinical relevance and robustness.
Leverage qualification advice meetings with EMA for methodologies applicable across multiple products or diseases.
Seek assistance from regulatory initiatives, such as the FDA’s Digital Health Center of Excellence or EMA’s Qualification Advice Programs, for specialized guidance.

Findings
The process of obtaining Informed Consent (IC) involves providing adequate information to facilitate comprehension and must allow subjects the opportunity to ask questions, continuing throughout the research. Electronic Informed Consent (eIC) systems, which can use various electronic media, are increasingly used to supplement or replace paper-based IC processes. The eIC process may be conducted on-site or remotely, but the legal responsibility for obtaining consent cannot be delegated to the electronic system. For FDA-regulated clinical investigations, electronic signatures must comply with 21 CFR Part 11 to be considered equivalent to a handwritten signature.

Recommendations
Presentation & Comprehension: eIC information should be easy to navigate, convey information in understandable language, and may use interactive electronic-based technology (e.g., diagrams, video) to facilitate comprehension. Optional questions can be used to assess a subject's understanding of key study elements.
Remote Consent: If consent is obtained remotely, the electronic system must include a reliable method to verify the identity of the subject (e.g., official identification, biometric methods).
Signature & Documentation: Electronic signatures are permitted and can be created using methods like biometrics or username/password, provided they are uniquely linked to the individual. The subject must be given a copy of the signed eIC, which can be electronic or paper.
Privacy & Security: The eIC system must be secure with restricted access and include methods to ensure confidentiality of subject information. If HIPAA applies, information must be encrypted unless otherwise documented.

Regulatory Considerations
IRB Responsibility: IRBs must review and approve all eIC materials and any subsequent amendments, including optional comprehension questions and the usability of the eIC materials. IRBs must maintain records (electronic or hard copy) of the approved versions of the eIC materials.
Submissions & Inspection: For IDE applications, copies of all eIC materials must be submitted to the FDA. During inspections, investigators must have site-specific signed eICs, amendments, and materials available (electronic or paper) for FDA review.
HIPAA: HIPAA authorizations may be obtained electronically, provided the signature is legally valid, and a copy must be provided to the subject.