Findings
The traditional medical device regulatory paradigm is not designed for the adaptive nature of AI/ML technologies, which can learn and change after they are on the market. A key benefit of AI/ML is its ability to improve performance by learning from real-world data, but this also presents a unique regulatory challenge. To ensure patient safety and device effectiveness, a new, flexible regulatory framework is required that can accommodate these iterative improvements. Transparency and robust monitoring are essential to manage the risks associated with evolving algorithms.

Recommendations
The FDA proposes a "Predetermined Change Control Plan" (PCCP) to be included in premarket submissions. This plan would specify the anticipated modifications to the device (the "what") and the methodology for implementing and validating those changes (the "how"). The development of "Good Machine Learning Practice" (GMLP) is encouraged to ensure that AI/ML algorithms are developed and validated using best practices. Manufacturers should implement robust real-world performance monitoring to ensure that their devices remain safe and effective after deployment.

Regulatory Considerations
The FDA is developing a new regulatory framework tailored to the unique aspects of AI/ML-based SaMD, which will leverage a TPLC approach. The agency has issued an "AI/ML SaMD Action Plan" that outlines its multi-pronged approach, including issuing draft guidance on PCCPs and promoting the harmonization of GMLP. The FDA is actively collaborating with stakeholders to foster innovation while ensuring patient safety. The agency maintains a public list of authorized AI/ML-enabled medical devices to enhance transparency.

Findings
The use of Artificial Intelligence (AI) and Machine Learning (ML) is being applied to a broad range of drug development activities with the potential to accelerate the process and make clinical trials safer and more efficient. The inclusion of AI/ML is most common in the clinical development/research phase of regulatory submissions. Concerns exist that AI/ML algorithms could amplify errors and preexisting biases in underlying data sources, which raises issues related to generalizability and ethical considerations. Other challenges include limited explainability due to model complexity and proprietary reasons, as well as managing risks related to data quality, reliability, and representativeness. The FDA recognizes that a careful, risk-based assessment of the specific context of use (COU) is needed when evaluating AI/ML.

Recommendations
Stakeholders should adhere to practices in three key areas: human-led governance, accountability, and transparency; quality, reliability, and representativeness of data; and model development, performance, monitoring, and validation. A risk management plan should be applied to identify and mitigate risks based on the COU, guiding the level of documentation and transparency. Practices are needed to ensure the integrity of AI/ML and address issues like bias and missing data. For models, developers should use pre-specification steps and clear documentation for development and assessment criteria. Models must be monitored over time for reliability and consistency, and Real-World Data (RWD) performance can provide valuable feedback, including for potential re-training.

Regulatory Considerations
The FDA encourages early engagement through mechanisms like the Critical Path Innovation Meetings (CPIM), ISTAND Pilot Program, and Emerging Technology Program to discuss relevant AI/ML methodologies or technologies. The Verification and Validation (V&V 40) risk-informed credibility assessment framework and the principles for Good Machine Learning Practices (GMLP), while not specific to drug development, are helpful guides for evaluating models. The industry is exploring the use of a Predetermined Change Control Plan (PCCP) mechanism for AI/ML-based devices to proactively specify and manage modifications, enhancing adaptability. In general, a risk-based approach should guide the level of evidence and record keeping needed for the verification and validation of AI/ML models for a specific COU.

Findings
AI/ML technologies offer dynamic learning capabilities but require careful regulation to ensure safety and effectiveness.
The FDA recognizes that traditional regulatory paradigms may not align with the adaptive nature of AI/ML and is developing frameworks to address this.
Guidance documents, such as the AI/ML SaMD Action Plan and predetermined change control plan (PCCP) recommendations, provide a structured approach for handling software updates.
Collaboration across FDA centers (CDRH, CBER, CDER) facilitates consistent regulatory practices for AI/ML across medical products.
Transparency and real-world data integration are key focuses in regulating AI/ML technologies.

Recommendations
Manufacturers should use FDA's premarket pathways, including 510(k), De Novo, or PMA, for AI/ML-enabled SaMD.
Apply Good Machine Learning Practices (GMLP) during development to ensure algorithm reliability, transparency, and patient safety.
Include a predetermined change control plan (PCCP) in submissions to allow for iterative updates without requiring resubmissions.
Follow lifecycle management practices to maintain AI/ML system performance after deployment.
Engage with FDA early in development to align on appropriate regulatory strategies for novel AI/ML implementations.

Regulatory Considerations
AI/ML-driven SaMD updates may require premarket review, depending on the significance of changes and associated risks.
The FDA has outlined principles for transparency, including clear labeling and documentation of AI/ML system capabilities and limitations.
Guidance documents like the "Good Machine Learning Practice" and "Marketing Submission Recommendations for PCCP" should be followed for compliance.
Collaboration between FDA centers ensures alignment on the use of AI in combination products and broader healthcare applications.
Lifecycle management strategies must account for real-world data to ensure continuous learning and safe AI/ML system updates.

Findings
Enhanced documentation is required for high-risk device software where flaws could result in serious injury or death.
Risk management plans should include robust risk assessments, including residual risk evaluations.
Verification and validation activities are critical to confirm software functionality and mitigate risks.
The lack of traceability between software design and requirements can undermine device safety and effectiveness.
Unresolved software anomalies must be carefully documented and justified based on a risk assessment.

Recommendations
Use a risk-based approach to determine whether basic or enhanced documentation levels are required for premarket submissions.
Include comprehensive risk management documentation, detailing hazard identification, risk control measures, and residual risk evaluations.
Provide detailed system and software architecture diagrams, highlighting relationships between modules and external systems.
Document unresolved software anomalies and justify their impact on safety and effectiveness using a risk-based rationale.
Align software development, configuration management, and maintenance practices with FDA-recognized standards like ANSI/AAMI/IEC 62304.

Regulatory Considerations
Adherence to 21 CFR Part 820 Quality System regulations, emphasizing design controls and risk management.
Submission of risk management files and unresolved software anomalies as part of premarket documentation.
Use of system and software architecture diagrams to demonstrate software functionality and risk mitigation.
Implementation of cybersecurity measures as part of software validation and risk management processes.
Documentation of premarket changes and interactions between device functions and external systems, particularly in multi-function devices.

Findings
AI-DSFs undergo iterative improvements, necessitating a structured framework for modifications to ensure safety and effectiveness.
PCCPs enable manufacturers to streamline modifications by avoiding repeated marketing submissions, reducing regulatory burden.
Critical elements of a PCCP include data management practices, re-training protocols, performance evaluation, and user update procedures.
Comprehensive risk management and transparency are essential to address potential biases and maintain user trust.
Certain modifications, such as those significantly affecting safety or effectiveness, may still require a new marketing submission.

Recommendations
Structure PCCPs with a clear description of planned modifications, a detailed modification protocol, and a robust impact assessment.
Include methods for data collection, re-training, and performance evaluation aligned with quality system regulations.
Specify user update procedures to communicate changes transparently and ensure safe device use.
Address cybersecurity risks and bias mitigation strategies in modification protocols.
Use the FDA Q-Submission Program to discuss PCCPs prior to submitting marketing applications for AI-DSFs.

Regulatory Considerations
Adherence to 21 CFR Part 820 Quality System Regulations, including design controls and risk management.
PCCPs must include modifications that would otherwise require a PMA supplement or new 510(k) submission.
Modifications implemented under PCCPs must conform to FDA-reviewed protocols and be documented in the device master record.
Transparency to users via device labeling updates and public summaries of authorized PCCPs is required.
Modifications outside the scope of an authorized PCCP or deviations from the protocol require new FDA marketing submissions.