Findings
The traditional medical device regulatory paradigm is not designed for the adaptive nature of AI/ML technologies, which can learn and change after they are on the market. A key benefit of AI/ML is its ability to improve performance by learning from real-world data, but this also presents a unique regulatory challenge. To ensure patient safety and device effectiveness, a new, flexible regulatory framework is required that can accommodate these iterative improvements. Transparency and robust monitoring are essential to manage the risks associated with evolving algorithms.

Recommendations
The FDA proposes a "Predetermined Change Control Plan" (PCCP) to be included in premarket submissions. This plan would specify the anticipated modifications to the device (the "what") and the methodology for implementing and validating those changes (the "how"). The development of "Good Machine Learning Practice" (GMLP) is encouraged to ensure that AI/ML algorithms are developed and validated using best practices. Manufacturers should implement robust real-world performance monitoring to ensure that their devices remain safe and effective after deployment.

Regulatory Considerations
The FDA is developing a new regulatory framework tailored to the unique aspects of AI/ML-based SaMD, which will leverage a TPLC approach. The agency has issued an "AI/ML SaMD Action Plan" that outlines its multi-pronged approach, including issuing draft guidance on PCCPs and promoting the harmonization of GMLP. The FDA is actively collaborating with stakeholders to foster innovation while ensuring patient safety. The agency maintains a public list of authorized AI/ML-enabled medical devices to enhance transparency.

Findings
Cybersecurity is an integral part of medical device safety and effectiveness, and manufacturers are responsible for addressing it throughout the entire device lifecycle. The FDA considers a device's cybersecurity as part of its benefit-risk assessment for both premarket and postmarket activities. A lack of robust cybersecurity controls can lead to patient harm, compromised device functionality, and breaches of data privacy. The dynamic nature of cybersecurity threats requires ongoing monitoring, risk management, and timely implementation of mitigation strategies.

Recommendations
Manufacturers should build cybersecurity into devices from the design phase ("secure by design") and conduct a thorough risk analysis to identify and mitigate potential vulnerabilities. Premarket submissions should include comprehensive documentation of the device's cybersecurity controls, a risk management plan, and a plan for postmarket surveillance and response. Manufacturers should establish a robust postmarket surveillance program to monitor for, identify, and address new cybersecurity threats in a timely manner. Clear and informative labeling is essential to help users understand and manage cybersecurity risks.

Regulatory Considerations
The FDA has the authority to take action against devices with inadequate cybersecurity that pose a risk to public health. The agency recommends that manufacturers use the Q-submission process to discuss specific cybersecurity questions related to their device submissions. Compliance with recognized standards and best practices for cybersecurity is strongly encouraged. Manufacturers must report certain cybersecurity incidents to the FDA as part of their postmarket reporting requirements. The FDA collaborates with other government agencies and stakeholders to promote a coordinated approach to medical device cybersecurity.

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

Findings
The MS digital health space is still largely uncharted.
Balancing the needs and desires of different users when creating a digital solution is challenging.
Insufficient adherence to remote digital health solutions presents a challenge to long-term engagement.
Creating a digital solution that is both meaningful to end users and aligned with regulatory standards involves challenges and compromises.

Recommendations
Employ an iterative development process to continually refine digital health solutions.
Collaborate closely with healthcare professionals and patients throughout the design process.
Use behavioral science strategies to enhance user engagement and adherence.
Ensure that digital solutions are scientifically robust and meet regulatory standards.
Implement a prescription-based model to improve adherence and integration into clinical practice.

Regulatory Considerations
Conduct technical verification and clinical validation for each assessment in digital health solutions.
Ensure data privacy and cybersecurity measures are robust and comply with local regulations.
Maintain ongoing post-marketing surveillance to monitor safety and effectiveness.
Adapt solutions to meet diverse regulatory requirements across different geographies.

Findings
The MDR/IVDR enhance the focus on cybersecurity for devices incorporating electronic programmable systems and software. Cybersecurity risk is inherently linked to patient safety and effectiveness; manufacturers must reduce all risks, including security risks with safety impacts, to an acceptable level. The management of security risks should be integrated into the product's overall Risk Management System. Due to the rapid change in the threat landscape, security maintenance is a critical, ongoing requirement across the entire product lifecycle. Other EU legislation, such as GDPR (data protection) and the NIS Directive (network security), also apply in parallel.

Recommendations
Manufacturers must follow a "Secure by design" strategy throughout the Design and Development phase, adopting a "Defense-in-Depth strategy". This includes:
Risk Management: Conduct a Security Risk Assessment (using techniques like Threat Modelling) to identify vulnerabilities and their potential impact on safety and effectiveness.
Risk Control: Prioritize mitigating risks in this order: eliminate/reduce risks through safe design; take adequate protection measures (e.g., encryption, authentication, alarms); provide information for safety and training .
Minimum IT Requirements: Clearly set out the minimum hardware, IT network, and IT security requirements for the device's operating environment and communicate these in the Instructions for Use. Devices should be as autonomous as possible in terms of security and avoid sole reliance on the operating environment.
Vigilance: Establish a robust Post-Market Surveillance (PMS) System to actively collect information, review data, and timely implement corrective actions (e.g., security updates/patches) for security vulnerabilities and incidents throughout the device's lifespan. Manufacturers must report all serious incidents and Field Safety Corrective Actions (FSCA).

Regulatory Considerations
Manufacturers must ensure that technical documentation includes information demonstrating conformity with all general safety and performance requirements, including justification and verification/validation of security solutions. Instructions for Use must include information on residual risks related to IT security and detailed instructions for secure installation, configuration, operation, and deployment of security updates. The entire process is a continuous, iterative cycle, requiring regular updates to technical documentation, risk management, and clinical evaluation

Findings
High-quality data must be complete, accurate, timely, and fit for purpose, ensuring reliability for RWE generation.
Effective governance is critical to ensure transparency, ethical standards, and stakeholder engagement in managing RWD.
Data capture challenges include standardization, provenance tracking, and interoperability, particularly for EHR-based data.
Data curation is iterative and involves organizing, assessing, and preparing raw data to meet study-specific needs.
The maturity model identifies five stages of organizational data capabilities, emphasizing consistency, completeness, and automation.

Recommendations
Implement robust governance frameworks to address transparency, stakeholder engagement, and ethical considerations in RWD use.
Focus on improving data capture at the point of care through standardization and semantic interoperability.
Use common data models and validated extraction-transformation-loading (ETL) processes to enhance data consistency and reliability.
Prioritize iterative data curation practices, supported by metadata and provenance tracking, to improve fitness for use over time.
Leverage the NESTcc Data Quality Maturity Model to benchmark and enhance organizational capabilities in RWD management.

Regulatory Considerations
Ensure compliance with patient privacy laws such as HIPAA and GDPR, especially when linking data across sources.
Align data capture and curation practices with FDA guidance for RWE generation and medical device evaluation.
Establish clear data use agreements to protect patient data while enabling analysis for regulatory and research purposes.
Document data transformations, including metadata and provenance, to support reproducibility and transparency in regulatory submissions.
Embrace standard terminologies and data dictionaries to facilitate interoperability and regulatory acceptance.

Findings
Human Factors Engineering (HFE) and Usability Engineering (UE) are fundamental for medical device safety and effectiveness. The HFE/UE process focuses on the interactions between people and devices, considering three major components: device users, device use environments, and the device user interface. The most important goal of this process is to minimize use-related hazards and risks. The FDA's HFE requirements are derived from the Quality System Regulation (QSR), specifically relating to Design Input (needs of the user and patient) and Design Validation (conformance to defined user needs). If risk analysis shows that use errors could lead to serious harm, HFE is explicitly required and must be submitted in premarket submissions (PMA, 510(k)).

Recommendations
Manufacturers should follow HFE/UE processes throughout the device development to improve design and minimize potential use errors. This involves an iterative process that runs parallel to product development. Key steps include:
User Research: Understand the intended users (e.g., professionals, patients, lay caregivers) and their characteristics (e.g., physical, cognitive abilities, experience).
Risk Analysis: Focus on potential use errors and identify critical tasks where errors could result in serious harm.
Formative Evaluation: Conduct evaluations during development to generate ideas for test scenarios, identify dangers early, and gather input for user interface improvements.
Design for Safety: Apply the hierarchy of risk control, prioritizing inherently safe design and protective measures (alarms, warnings) over instructions and training.
Usability Validation Testing: Conduct final summative testing with representative users under simulated real-world use conditions to demonstrate the device can be used safely and effectively.

Regulatory Considerations
The FDA recommends that manufacturers submit human factors data in premarket submissions for devices where risk analysis indicates that use errors could result in serious harm. The FDA has provided guidance on the content that should be included in these submissions, such as descriptions of intended users, use environments, user interface, risk analysis of use-related hazards, and results of validation studies. Manufacturers should also continue to monitor user interactions through postmarket surveillance and adverse event reporting.

Findings
Clinical Evaluation Components: Valid Clinical Association: Demonstrates that the SaMD's outputs are clinically meaningful and relevant to the intended healthcare condition. Analytical Validation: Confirms that the SaMD processes input data accurately and reliably to produce the intended output. Clinical Validation: Assesses whether the SaMD achieves its intended purpose in the target population.
Lifecycle Management: Clinical evaluation is an ongoing process that spans pre-market development and post-market monitoring.
Post-market data collection supports continuous improvement, including refining or expanding the SaMD’s intended use.
Risk-Based Approach: The depth and independence of clinical evaluation depend on the SaMD's risk categorization, with higher-risk categories requiring more rigorous oversight and validation.
Real-World Evidence: SaMD manufacturers are encouraged to use real-world performance data for iterative learning, ensuring alignment with evolving clinical needs.
Independent Review: High-risk SaMD (e.g., those used for critical diagnoses or treatments) benefit from independent evaluation to manage bias and validate clinical evidence.

Recommendations
Pre-Market: Generate evidence through clinical trials, literature reviews, and secondary data analysis to demonstrate valid clinical association and analytical validation.
Use a risk-based framework to determine the rigor of clinical evaluation.
Post-Market: Leverage real-world performance data for continuous improvement and risk management.
Monitor safety, effectiveness, and user interactions, adapting the SaMD definition statement as needed.
Regulatory Submissions: Provide a clear SaMD definition statement, including intended use and core functionality.
Include comprehensive validation data, particularly for high-risk SaMD.
Independent Review: Engage third-party reviewers for high-risk SaMD to enhance transparency and confidence in clinical evaluation.
Quality Management: Integrate clinical evaluation activities into the organization’s quality management system to ensure consistency and compliance.

Regulatory Considerations
SaMD manufacturers must comply with jurisdiction-specific pre-market and post-market requirements, such as informed consent for clinical trials and regulatory submissions for significant changes.
Changes to the SaMD’s intended use or performance measures, based on post-market data, may necessitate updated regulatory approvals.
Independent review requirements vary by jurisdiction but are critical for higher-risk SaMD categories.

Findings
Cybersecurity risk management is a shared responsibility involving manufacturers, healthcare organizations, and IT vendors.
Proactive measures, such as threat modeling and vulnerability scanning, are critical to mitigating risks throughout the device lifecycle.
Cybersecurity routine updates and patches are generally considered enhancements and are not subject to 21 CFR Part 806 reporting unless risks are uncontrolled.
Participation in ISAOs is encouraged to foster collaboration and timely sharing of vulnerability and threat information.
Effective remediation plans must address vulnerabilities promptly, with appropriate reporting and user communication.

Recommendations
Monitor cybersecurity signals from diverse sources, including ISAOs, CERTs, and internal investigations, to identify and assess vulnerabilities.
Establish a robust risk management program incorporating the NIST Cybersecurity Framework to address risks from design to obsolescence.
Use tools like the Common Vulnerability Scoring System (CVSS) for assessing exploitability and prioritizing remediation efforts.
Communicate vulnerability and mitigation strategies clearly to users, ensuring they understand risks and appropriate controls.
Report uncontrolled vulnerabilities to FDA under 21 CFR Part 806, unless certain conditions are met (e.g., timely remediation, participation in ISAOs).

Regulatory Considerations
Cybersecurity routine updates addressing controlled risks are not typically subject to FDA reporting requirements under 21 CFR Part 806.
Uncontrolled risks must be remediated promptly, with detailed reporting to FDA, unless alternative measures like ISAO participation and mitigation plans are in place.
Class III devices with periodic reporting requirements must include cybersecurity-related updates and vulnerabilities in annual PMA reports.
Manufacturers must document their risk assessments, remediation plans, and user communications to demonstrate compliance with 21 CFR Part 820.
Threat detection and forensic capabilities should be built into device designs to support postmarket monitoring and risk mitigation.

Findings
Home use devices face unique environmental challenges, including power interruptions, fluid exposure, and travel-related conditions.
Lay users often have limited training and varying physical, cognitive, and emotional capabilities, requiring user-friendly designs and clear instructions.
Effective risk management should include designing risks out of the device wherever possible, supplemented by protective measures and labeling as needed.
Verification, validation, and human factors testing are essential to confirm device performance and usability under realistic home-use scenarios.
Postmarket considerations, such as customer service and Medical Device Reporting (MDR), are vital for maintaining device safety and compliance.

Recommendations
Design devices for diverse environmental conditions, such as variable power supplies, fluid exposure, and extreme temperatures.
Include safeguards like lock-out mechanisms, robust alarm systems, and protective casings to mitigate risks.
Develop user-friendly labeling and instructions, employing narrative formats and visuals to address low literacy or technical proficiency.
Conduct human factors engineering and usability testing to identify and resolve potential design issues, ensuring safe device operation by lay users.
Plan for postmarket support, including accessible customer service and robust systems for adverse event reporting.

Regulatory Considerations
Premarket submissions should document efforts to address environmental and user-related risks, supported by verification, validation, and usability data.
Devices requiring electrical power must meet applicable ANSI/AAMI standards for safety, including those related to electromagnetic compatibility.
Manufacturers must comply with labeling requirements under 21 CFR Parts 801 and 809, ensuring clear communication of warnings, instructions, and limitations.
FDA emphasizes the use of recognized consensus standards, such as IEC 62304 for software lifecycle processes and ANSI/AAMI HE75 for human factors engineering.
Devices must incorporate mechanisms for handling emergencies, including power outages, and provide clear labeling on disposal, maintenance, and troubleshooting.