Findings
AI-enabled medical devices require robust risk assessment to address data drift, bias, and transparency challenges.
The total product lifecycle (TPLC) approach is essential for managing AI-enabled devices, ensuring continuous oversight and updates.
There is a need for improved standardization in AI model validation and performance monitoring to ensure consistency in regulatory submissions.
Effective data management practices, including dataset representativeness and bias control, are critical for AI model development.
Cybersecurity vulnerabilities in AI-enabled medical devices must be proactively addressed to prevent risks to patient safety and data integrity.

Recommendations
AI-enabled device manufacturers should integrate Good Machine Learning Practice (GMLP) principles throughout the device lifecycle.
Marketing submissions should include comprehensive documentation of AI model development, validation, and performance monitoring.
Developers should implement transparency measures, such as model interpretability and explainability, to enhance user trust and understanding.
AI models must undergo rigorous bias evaluation to ensure equitable performance across diverse patient populations.
A predetermined change control plan (PCCP) should be established to allow safe and effective AI model updates post-market without additional FDA submissions.

Regulatory Considerations
FDA encourages early engagement through the Q-Submission Program for AI-enabled device manufacturers.
Compliance with FDA-recognized consensus standards, such as ANSI/AAMI/ISO 14971 for risk management, is recommended.
AI-enabled devices must meet labeling requirements, ensuring that users clearly understand model inputs, outputs, and performance metrics.
Post-market surveillance and continuous monitoring of AI model performance are necessary to ensure ongoing safety and effectiveness.
Cybersecurity measures must be included in regulatory submissions, detailing safeguards against data breaches and unauthorized model modifications.

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

Findings
The Library of Human Factors Resources compiles external documents related to human factors, human-centered design, and usability in the context of Digital Health Technologies (DHTs), especially sensor-based DHTs (sDHTs). The resources in the library are categorized by usability validation activity:
Use-related risk assessment (topics include user tasks, use-errors, use-related hazards, and risk mitigation).
Design considerations (topics relate to product design).
Formative evaluation (topics related to usability evaluation of a prototype product).
Summative evaluation (topics related to usability evaluation of a final-version or marketed product).

Recommendations
The page recommends that users, developers, and evaluators utilize the library's interactive index to find relevant documents to their DHT:
Use the Search by Activity tab to filter by one of the usability validation activities listed above.
Use the human factors topic column to further narrow the search.
Review the document name, issuing body, and product of focus columns to identify the most relevant document.
Click the hyperlink to access the selected document, then use the information in the relevant sections column to locate the specific topics of interest.

Regulatory Considerations
The library is a collection of resources for accelerating sDHT adoption, and it specifically includes documents like regulatory guidance and industry standards. This emphasizes the importance of understanding and applying human factors principles for validation and achieving regulatory acceptance for sDHTs.

Findings
The V3+ Use Specification must contain a detailed description of the user groups, use environments, and the sDHT user interface. The user groups include end-users (individuals from whom data is captured) as well as carepartners, clinicians, researchers, and administrators. Characteristics of users (e.g., demographics, literacy, physical/cognitive capabilities, disease characteristics) and use environments (e.g., temperature, network availability, clutter) must be considered for risk management .

Recommendations
Developers must follow these four steps to create the Use Specification:
Identify all user groups: Create a list of users, including sub-categories (e.g., different types of researchers), and describe the characteristics of each group (e.g., health literacy, physical capabilities) to create detailed descriptions of representative users.
Identify all likely use environments: Create a list of typical environments (e.g., Home, Hospitals) and describe their characteristics (e.g., temperature, noise, network availability), also considering "edge cases" (e.g., extreme weather).
Describe the sDHT user interface: Detail all aspects of the hardware and software (visual, auditory, tactile cues), accessories (e.g., packaging, chargers), and all written materials and training (e.g., instructions for use, helpdesk troubleshooting).
Keep it up to date: The Use Specification is a living document that requires ongoing updates and maintenance throughout the sDHT development and usability validation process.

Regulatory Considerations
The development of the Use Specification is presented as the foundational step for the usability validation component of the V3+ framework. This document directly informs the subsequent Use-Related Risk Analysis.

Findings
The review includes 275 studies, with neurology, musculoskeletal disorders, and cardiology as the most common therapeutic areas.
The studies focused on sensor performance (48%), algorithm development (86%), operational feasibility (46%), and software development (9%).
Gaps in reporting included insufficient details on software used (27%), comparator measures (17%), and participant demographics (e.g., age and gender were missing in 9% and 15% of studies, respectively).
Sixty-seven percent of the studies used wearable sensors, while others incorporated smartphones, tablets, cameras, and implantable devices.
The lack of methodological and reporting standards across studies hinders reproducibility and broader applicability.

Recommendations
Develop methodological and reporting standards to improve consistency across feasibility studies.
Include comprehensive participant demographic data, including sociodemographics and health indicators, to ensure inclusivity and generalizability.
Conduct small feasibility studies to validate sensors, optimize algorithms, and identify operational challenges before launching full-scale trials. Use the database created from this review to inform trial design and technology selection, ensuring alignment with specific research goals.
Encourage collaboration among investigators, sponsors, and regulators to standardize methods and share insights to avoid redundant studies.

Regulatory Considerations
Align sensor verification and algorithm validation processes with regulatory requirements for reliable clinical endpoints.
Ensure secure and ethical data transfer, storage, and sharing practices for compliance with privacy regulations.
Address barriers to participation for underrepresented populations by assessing and reporting equity-related data during feasibility studies.

Findings
Lack of consensus on the type and scope of digital outcome measures.
Partial integration of mobile health technologies into clinical practice.
Challenges in data presentation and interpretation.
Poorly addressed patient compliance and technology illiteracy.
Validation challenges for mobile health technologies.

Recommendations
Target deficits confirmed to be relevant to patients.
Use a combination of devices with an acceptable benefit-to-burden ratio.
Integrate data into patient management platform standards.
Ensure regulatory approval and adoption by healthcare organizations.
Consider pilot use of competing platforms for better integration.

Regulatory Considerations
Understand and overcome regulatory hurdles.
Ensure sustainable financial models.
Consider pilot use of platforms for regulatory integration.

Findings
Human Factors Engineering (HFE) and Usability Engineering (UE) are fundamental for medical device safety and effectiveness. The HFE/UE process focuses on the interactions between people and devices, considering three major components: device users, device use environments, and the device user interface. The most important goal of this process is to minimize use-related hazards and risks. The FDA's HFE requirements are derived from the Quality System Regulation (QSR), specifically relating to Design Input (needs of the user and patient) and Design Validation (conformance to defined user needs). If risk analysis shows that use errors could lead to serious harm, HFE is explicitly required and must be submitted in premarket submissions (PMA, 510(k)).

Recommendations
Manufacturers should follow HFE/UE processes throughout the device development to improve design and minimize potential use errors. This involves an iterative process that runs parallel to product development. Key steps include:
User Research: Understand the intended users (e.g., professionals, patients, lay caregivers) and their characteristics (e.g., physical, cognitive abilities, experience).
Risk Analysis: Focus on potential use errors and identify critical tasks where errors could result in serious harm.
Formative Evaluation: Conduct evaluations during development to generate ideas for test scenarios, identify dangers early, and gather input for user interface improvements.
Design for Safety: Apply the hierarchy of risk control, prioritizing inherently safe design and protective measures (alarms, warnings) over instructions and training.
Usability Validation Testing: Conduct final summative testing with representative users under simulated real-world use conditions to demonstrate the device can be used safely and effectively.

Regulatory Considerations
The FDA recommends that manufacturers submit human factors data in premarket submissions for devices where risk analysis indicates that use errors could result in serious harm. The FDA has provided guidance on the content that should be included in these submissions, such as descriptions of intended users, use environments, user interface, risk analysis of use-related hazards, and results of validation studies. Manufacturers should also continue to monitor user interactions through postmarket surveillance and adverse event reporting.

Findings
HFE/UE is essential for identifying and mitigating use-related risks that could compromise device safety or effectiveness.
Preliminary analyses, such as task and fault tree analyses, help identify critical tasks and use-related hazards early in device development.
Human factors validation testing must represent realistic use scenarios, include diverse user populations, and focus on critical tasks with potential for serious harm.
Residual risks that remain after validation testing must be justified in terms of the device's overall benefits and risk management measures.
Effective risk management prioritizes design modifications over labeling or training as the primary method for addressing use-related hazards.

Recommendations
Incorporate HFE/UE into all stages of device development to address use-related hazards through design improvements.
Conduct comprehensive risk analyses to identify and prioritize critical tasks that may lead to serious harm if performed incorrectly.
Design human factors validation testing to reflect real-world conditions and involve representative user populations.
Address use-related risks primarily through design modifications, with labeling and training as secondary measures.
Submit detailed HFE/UE documentation in premarket applications to facilitate FDA review and approval.

Regulatory Considerations
Submit human factors validation testing data as part of premarket applications for devices where use-related errors could result in serious harm.
Risk management processes must align with standards such as ANSI/AAMI/ISO 14971 and IEC 62366, ensuring comprehensive hazard identification and mitigation.
Conduct additional validation testing if modifications to a marketed device impact user interactions or introduce new risks.
For actual-use testing, ensure compliance with Investigational Device Exemption (IDE) requirements where applicable.
Manufacturers should maintain detailed records of HFE/UE processes, which must be available for FDA review upon request.

Findings
Home use devices face unique environmental challenges, including power interruptions, fluid exposure, and travel-related conditions.
Lay users often have limited training and varying physical, cognitive, and emotional capabilities, requiring user-friendly designs and clear instructions.
Effective risk management should include designing risks out of the device wherever possible, supplemented by protective measures and labeling as needed.
Verification, validation, and human factors testing are essential to confirm device performance and usability under realistic home-use scenarios.
Postmarket considerations, such as customer service and Medical Device Reporting (MDR), are vital for maintaining device safety and compliance.

Recommendations
Design devices for diverse environmental conditions, such as variable power supplies, fluid exposure, and extreme temperatures.
Include safeguards like lock-out mechanisms, robust alarm systems, and protective casings to mitigate risks.
Develop user-friendly labeling and instructions, employing narrative formats and visuals to address low literacy or technical proficiency.
Conduct human factors engineering and usability testing to identify and resolve potential design issues, ensuring safe device operation by lay users.
Plan for postmarket support, including accessible customer service and robust systems for adverse event reporting.

Regulatory Considerations
Premarket submissions should document efforts to address environmental and user-related risks, supported by verification, validation, and usability data.
Devices requiring electrical power must meet applicable ANSI/AAMI standards for safety, including those related to electromagnetic compatibility.
Manufacturers must comply with labeling requirements under 21 CFR Parts 801 and 809, ensuring clear communication of warnings, instructions, and limitations.
FDA emphasizes the use of recognized consensus standards, such as IEC 62304 for software lifecycle processes and ANSI/AAMI HE75 for human factors engineering.
Devices must incorporate mechanisms for handling emergencies, including power outages, and provide clear labeling on disposal, maintenance, and troubleshooting.