Findings
The traditional medical device regulatory paradigm is not designed for the adaptive nature of AI/ML technologies, which can learn and change after they are on the market. A key benefit of AI/ML is its ability to improve performance by learning from real-world data, but this also presents a unique regulatory challenge. To ensure patient safety and device effectiveness, a new, flexible regulatory framework is required that can accommodate these iterative improvements. Transparency and robust monitoring are essential to manage the risks associated with evolving algorithms.

Recommendations
The FDA proposes a "Predetermined Change Control Plan" (PCCP) to be included in premarket submissions. This plan would specify the anticipated modifications to the device (the "what") and the methodology for implementing and validating those changes (the "how"). The development of "Good Machine Learning Practice" (GMLP) is encouraged to ensure that AI/ML algorithms are developed and validated using best practices. Manufacturers should implement robust real-world performance monitoring to ensure that their devices remain safe and effective after deployment.

Regulatory Considerations
The FDA is developing a new regulatory framework tailored to the unique aspects of AI/ML-based SaMD, which will leverage a TPLC approach. The agency has issued an "AI/ML SaMD Action Plan" that outlines its multi-pronged approach, including issuing draft guidance on PCCPs and promoting the harmonization of GMLP. The FDA is actively collaborating with stakeholders to foster innovation while ensuring patient safety. The agency maintains a public list of authorized AI/ML-enabled medical devices to enhance transparency.

Findings
AI-enabled medical devices require robust risk assessment to address data drift, bias, and transparency challenges.
The total product lifecycle (TPLC) approach is essential for managing AI-enabled devices, ensuring continuous oversight and updates.
There is a need for improved standardization in AI model validation and performance monitoring to ensure consistency in regulatory submissions.
Effective data management practices, including dataset representativeness and bias control, are critical for AI model development.
Cybersecurity vulnerabilities in AI-enabled medical devices must be proactively addressed to prevent risks to patient safety and data integrity.

Recommendations
AI-enabled device manufacturers should integrate Good Machine Learning Practice (GMLP) principles throughout the device lifecycle.
Marketing submissions should include comprehensive documentation of AI model development, validation, and performance monitoring.
Developers should implement transparency measures, such as model interpretability and explainability, to enhance user trust and understanding.
AI models must undergo rigorous bias evaluation to ensure equitable performance across diverse patient populations.
A predetermined change control plan (PCCP) should be established to allow safe and effective AI model updates post-market without additional FDA submissions.

Regulatory Considerations
FDA encourages early engagement through the Q-Submission Program for AI-enabled device manufacturers.
Compliance with FDA-recognized consensus standards, such as ANSI/AAMI/ISO 14971 for risk management, is recommended.
AI-enabled devices must meet labeling requirements, ensuring that users clearly understand model inputs, outputs, and performance metrics.
Post-market surveillance and continuous monitoring of AI model performance are necessary to ensure ongoing safety and effectiveness.
Cybersecurity measures must be included in regulatory submissions, detailing safeguards against data breaches and unauthorized model modifications.

Findings
Sites must establish effective data privacy and security plans, especially considering regional and global regulations like GDPR.
Risk mitigation is critical, including plans to address unanticipated issues and potential patient disengagement due to technology challenges.
Budgeting and contracting often involve additional considerations, such as storage, training, and technical support requirements for connected devices.
Sites require adequate training to ensure staff and patients are prepared to use connected devices efficiently.
Companion applications or services often play an essential role in device functionality and data transmission.

Recommendations
Develop a clear plan for data pathways, including storage, security, and regulatory compliance.
Establish detailed risk mitigation and management strategies to handle unexpected challenges.
Ensure comprehensive training programs for site staff and patients to enhance device usability.
Incorporate device storage and resource allocation into budgeting and contracting processes.
Facilitate effective communication with sponsors and vendors to resolve operational and technical issues promptly.

Regulatory Considerations
Ensure connected devices comply with CFR 21, Part 11, and other relevant data collection and transmission regulations.
Understand and adhere to local and regional data privacy laws, such as GDPR, when managing patient data.
Verify that appropriate licenses and regulatory approvals are in place for device data transmission and storage.
Assess and address shipping and handling regulations for devices, ensuring safe and compliant transportation.

Findings
Adherence: Participants achieved an overall adherence rate of 90.18%, demonstrating the feasibility of home-based data collection over a 30-day period.
Participant Feedback: Most participants found the technology easy to use, though some reported difficulties with specific devices, such as sleeping with a wearable watch.
Device Selection: Precision, consistency, and participant preferences guided the selection of spirometry devices, with single-blow spirometry favored for ease of use.
Accuracy: Home spirometry measurements underestimated forced vital capacity (FVC) compared to historical in-clinic data, possibly due to device differences or disease progression.
Future Participation: Nine out of ten participants expressed interest in joining longer virtual studies using similar technologies.

Recommendations
Evaluate Adherence and Usability: Conduct feasibility studies to assess adherence rates and identify usability challenges before full-scale implementation.
Incorporate Participant Feedback: Use cross-over designs to gather participant preferences and feedback on device usability, data sharing, and frequency of data collection.
Validate Accuracy and Consistency: Ensure that DHTs provide precise, reliable measurements comparable to in-clinic standards and assess their performance in real-world settings.
Optimize Technology for Long-Term Use: Address issues such as wearability and participant burden to improve device acceptance and compliance.
Refine Training and Communication: Provide clear instructions and training to participants, setting expectations for using and troubleshooting the technologies.

Regulatory Considerations
Validate Home-Based Data Collection: Demonstrate that data collected remotely with DHTs are accurate, reliable, and clinically relevant for trial endpoints.
Pilot Studies for Regulatory Submissions: Use feasibility data to strengthen regulatory submissions, ensuring endpoints are validated for use in pivotal trials.
Address Technology Limitations: Acknowledge and mitigate potential discrepancies between home and clinic data, using feasibility study insights to refine protocols.