Industry spotlight

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

N/A

Findings
The goal of sDHT design is to create tools that are functional, intuitive, accessible, and enjoyable to use, moving beyond merely minimizing use-errors. Human-centered design (HCD) is the preferred term over user-centered design, emphasizing the impact on many user groups beyond just the end-users. “Users” encompass end-users (patients/participants), carepartners, clinicians, investigators, and administrators.

Recommendations
Developers of sDHTs should adhere to the following HCD principles:
Empathetic: Take time to deeply understand users’ needs, behaviors, and emotions, capturing this in the use specification.
Holistic: Consider the entire end-to-end user journey, including hardware, software, accessories, packaging, instructions for use, and training.
Iterative: Employ an iterative approach to designing, prototyping, testing, and refining, using formative evaluations to identify use-errors and gather usability data, capturing this in the use-related risk analysis.
User-centric: Improve usability by capturing user feedback in real-world settings, gradually recruiting larger, more diverse samples that represent the intended use population.
Inclusive: Collaborate with individuals representing all user groups by hiring them as consultants or creating user advisory panels to influence design decisions (co-design).
Multidisciplinary: Ensure the development team includes colleagues from various disciplines to bring diverse perspectives and innovative solutions.

Regulatory Considerations
The document ties the HCD process to risk management and eventual validation by recommending that findings from formative evaluations (used to identify use-errors) be captured in a use-related risk analysis. The approach aligns with the principles of the overarching V3+ framework.

Findings
The V3+ Use Specification must contain a detailed description of the user groups, use environments, and the sDHT user interface. The user groups include end-users (individuals from whom data is captured) as well as carepartners, clinicians, researchers, and administrators. Characteristics of users (e.g., demographics, literacy, physical/cognitive capabilities, disease characteristics) and use environments (e.g., temperature, network availability, clutter) must be considered for risk management .

Recommendations
Developers must follow these four steps to create the Use Specification:
Identify all user groups: Create a list of users, including sub-categories (e.g., different types of researchers), and describe the characteristics of each group (e.g., health literacy, physical capabilities) to create detailed descriptions of representative users.
Identify all likely use environments: Create a list of typical environments (e.g., Home, Hospitals) and describe their characteristics (e.g., temperature, noise, network availability), also considering “edge cases” (e.g., extreme weather).
Describe the sDHT user interface: Detail all aspects of the hardware and software (visual, auditory, tactile cues), accessories (e.g., packaging, chargers), and all written materials and training (e.g., instructions for use, helpdesk troubleshooting).
Keep it up to date: The Use Specification is a living document that requires ongoing updates and maintenance throughout the sDHT development and usability validation process.

Regulatory Considerations
The development of the Use Specification is presented as the foundational step for the usability validation component of the V3+ framework. This document directly informs the subsequent Use-Related Risk Analysis.

Findings
A Use-Related Risk Analysis (URRA) is essential for identifying potential use-errors (actions or lack of action that may result in harm) and their associated use-related hazards (source of potential harm) when using an sDHT. The analysis must focus on user interactions with the sDHT, including all user groups (end-users, clinicians, carepartners, researchers, and administrators). Critical tasks are defined as those use-errors that may result in serious harm.

Recommendations
Developers should follow these five steps for the Use-Related Risk Analysis:
Describe all user tasks: Identify the sequence of actions a user performs to achieve a goal, which can be derived from sources like a task analysis or formative evaluations.
Describe potential use-errors: Identify and document potential actions or lack of actions that may result in harm for each task, noting that “use-error” is preferable to “user-error”.
Describe potential use-related hazards: Determine the source of potential harm resulting from each identified use-error.
Develop a plan to minimize or eliminate known risks: The preferred approach is inherent safety by design (eliminating the error). If not feasible, use protective measures (e.g., warnings) or, as a last resort, provide instructions to users. Identify methods to evaluate the effectiveness of the chosen mitigation strategy.
Keep it up to date: The URRA is a living document requiring ongoing updates throughout the sDHT development and usability validation process.

Regulatory Considerations
The URRA is presented as a fundamental step within the V3+ framework for ensuring device usability and minimizing risk, implicitly setting the groundwork for regulatory compliance related to device safety. The minimization of use-errors, particularly for critical tasks, is a central tenet of device development best practices.

Findings
The Library of Human Factors Resources compiles external documents related to human factors, human-centered design, and usability in the context of Digital Health Technologies (DHTs), especially sensor-based DHTs (sDHTs). The resources in the library are categorized by usability validation activity:
Use-related risk assessment (topics include user tasks, use-errors, use-related hazards, and risk mitigation).
Design considerations (topics relate to product design).
Formative evaluation (topics related to usability evaluation of a prototype product).
Summative evaluation (topics related to usability evaluation of a final-version or marketed product).

Recommendations
The page recommends that users, developers, and evaluators utilize the library’s interactive index to find relevant documents to their DHT:
Use the Search by Activity tab to filter by one of the usability validation activities listed above.
Use the human factors topic column to further narrow the search.
Review the document name, issuing body, and product of focus columns to identify the most relevant document.
Click the hyperlink to access the selected document, then use the information in the relevant sections column to locate the specific topics of interest.

Regulatory Considerations
The library is a collection of resources for accelerating sDHT adoption, and it specifically includes documents like regulatory guidance and industry standards. This emphasizes the importance of understanding and applying human factors principles for validation and achieving regulatory acceptance for sDHTs.