Findings
The quality of evidence for the analytical validation of sensor-based digital health technologies (sDHTs), which is the evaluation of algorithms converting sensor data into a clinically interpretable measure, is often inconsistent and insufficient. The existing V3+ framework codifies the overall evaluation process, which includes verification, usability validation, analytical validation, and clinical validation. To improve the scientific rigor of analytical validation, a hierarchical framework for selecting reference measures is needed because not all potential reference measures are of equal quality. The framework classifies reference measures based on attributes that contribute to reduced measurement variability, with defining and principal measures being the most rigorous due to objective data acquisition and the ability to retain source data.

Recommendations
The proposed framework sequentially moves the investigator through four steps: (1) Compile preliminary information, including the digital clinical measure, context of use (COU), algorithm requirements, and sensor verification evidence . (2) Select an existing reference measure, develop a novel comparator, or identify a set of anchor measures, prioritizing measures with the highest scientific rigor (defining → principal → manual → reported) . (3) Consider the impact of the data collection environment to determine if the analytical validation study can be conducted in the intended use environment with the highest-order measure, or if in-lab validation is necessary, ensuring the results are generalizable . (4) Describe the rationale for key study design decisions to encourage transparency for evaluators, regulators, and payers . Investigators must justify passing over a higher-ranked reference measure, generally only acceptable if the higher-ranked measure poses unacceptable risk or is not applicable to the context of use.

Regulatory Considerations
The principles of the framework for analytical validation apply regardless of the regulatory status of the sDHT (regulated medical device, low-risk general wellness apps, or research product) or its intended use (clinical care or clinical research). The framework is intended to help investigators support the most rigorous claims regarding sDHT performance, which is important for acceptance by evaluators, peer-reviewers, regulators, and payers. The categorization of the digital clinical measure as a digital biomarker or an electronic clinical outcome assessment also does not change the framework's applicability.

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

Findings
The goal of sDHT design is to create tools that are functional, intuitive, accessible, and enjoyable to use, moving beyond merely minimizing use-errors. Human-centered design (HCD) is the preferred term over user-centered design, emphasizing the impact on many user groups beyond just the end-users. "Users" encompass end-users (patients/participants), carepartners, clinicians, investigators, and administrators.

Recommendations
Developers of sDHTs should adhere to the following HCD principles:
Empathetic: Take time to deeply understand users' needs, behaviors, and emotions, capturing this in the use specification.
Holistic: Consider the entire end-to-end user journey, including hardware, software, accessories, packaging, instructions for use, and training.
Iterative: Employ an iterative approach to designing, prototyping, testing, and refining, using formative evaluations to identify use-errors and gather usability data, capturing this in the use-related risk analysis.
User-centric: Improve usability by capturing user feedback in real-world settings, gradually recruiting larger, more diverse samples that represent the intended use population.
Inclusive: Collaborate with individuals representing all user groups by hiring them as consultants or creating user advisory panels to influence design decisions (co-design).
Multidisciplinary: Ensure the development team includes colleagues from various disciplines to bring diverse perspectives and innovative solutions.

Regulatory Considerations
The document ties the HCD process to risk management and eventual validation by recommending that findings from formative evaluations (used to identify use-errors) be captured in a use-related risk analysis. The approach aligns with the principles of the overarching V3+ framework.

Findings
Usability is a multi-domain concept that requires a combination of methods for evaluation. Evaluations fall into two types: formative (for design modification of prototypes) and summative (for demonstrating usability of the final product to a representative user sample). The user experience metrics fall into several domains, including: Satisfaction, Usefulness, Ease of use, Learnability, Efficiency, Memorability, Understandability, Actionability, Readability, and Use-errors. Metrics related to Satisfaction and Usefulness are always subjectively reported by users.

Recommendations
Developers should select metrics based on the specific usability-related domain being evaluated.
Subjective Data (e.g., Satisfaction, Usefulness): Capture through qualitative surveys, quantitative surveys (scales), interviews, focus groups, and think-aloud evaluations .
Objective Data (e.g., Ease of use, Use-errors): Capture through direct or indirect observation (e.g., counting steps/attempts, timing task completion), or by using data generated by the sDHT (e.g., error reports, timestamps, page load times).
Time-based Metrics: Evaluate Learnability (ease of first use), Efficiency (ease with experience), and Memorability (ease after non-use) by measuring ease of use at different points in time .
Information Presentation: If the sDHT presents clinical data or written information (instructions, warnings), evaluate Understandability, Actionability, and Readability .
Use-errors: Objectively capture the number, type, and recoverability of use-errors (actions, or lack thereof, that may result in harm) via observation and sDHT data, noting that "use-error" is preferred to "user-error".

Regulatory Considerations
While this guide does not reference regulatory bodies like the FDA, it is part of the V3+ framework and recommends that researchers prioritize essential documents like the use specification and use-related risk analysis before designing a usability study. Summative evaluations demonstrating usability against a representative user sample under intended use conditions are the standard for demonstrating product fitness.

Findings
Usability gaps in sDHTs remain a barrier to adoption, with many technologies failing to prioritize ease of use, accessibility, and diverse user needs​
Human-centered design is critical for ensuring that digital health solutions are intuitive, functional, and scalable across different healthcare environments​
Standardized usability metrics for evaluating digital health technologies are lacking, leading to inconsistent reporting and validation of usability outcomes​
Use-related risk analysis is essential to identifying and mitigating risks associated with user errors, ensuring the safety and effectiveness of sDHTs​
The V3+ framework provides a structured approach to integrating usability validation into digital health technology development, aligning with global regulatory expectations​

Recommendations
Developers should incorporate human-centered design principles from the outset, ensuring that usability, accessibility, and user needs are central to sDHT development​
Usability validation should be standardized, with clear methodologies for measuring usability, including satisfaction, ease of use, efficiency, and error mitigation​
Regulatory and clinical stakeholders should collaborate on defining best practices for usability evaluation, ensuring that digital endpoints are both meaningful and scalable​
Risk analysis should be iterative, with developers continuously refining their technologies based on real-world user feedback and testing​
The usability validation component of V3+ should be widely adopted to ensure that digital clinical measures meet patient-centered, regulatory, and technical expectations​

Regulatory Considerations
Regulators are emphasizing the need for usability validation to ensure that digital endpoints are both clinically relevant and patient-friendly​
sDHTs must comply with human factors engineering guidelines, aligning with global regulatory frameworks such as ISO 9241-210 and FDA usability requirements​
Data security, privacy, and interoperability must be ensured, particularly as sDHTs become integrated into remote monitoring and decentralized clinical trials​
Real-world evidence (RWE) should support usability validation, helping to bridge the gap between regulatory approval and real-world adoption​
Regulatory bodies should work toward standardizing usability testing methodologies, ensuring consistency across clinical research, digital endpoints, and medical device evaluations​

Findings
For an sDHT to be considered fit-for-purpose, a researcher or healthcare provider must understand the alignment between the sDHT's intended use (What it does, who uses it, where/when/how) and their own context of use . Key information for this assessment comes from the developer's Use Specification (detailing hardware, software, accessories, training) and Use-Related Risk Analysis (detailing warnings, harms from use-errors, and risk avoidance) . Usability validation evidence should cover study objectives, protocols, participant characteristics, metrics, and collection methods.

Recommendations
Researchers/providers should use the checklist to:
The Basics: Compare the sDHT's intended use to their context of use; if there is substantial overlap, existing evidence may be sufficient.
Use Specification/Risk Analysis: Gather detailed descriptions of the sDHT's hardware, software, accessories, written materials, training, cautions, warnings, and potential harms from use-errors to update their own Use Specification and Use-Related Risk Analysis .
Existing Evidence: Access existing usability validation study results (objectives, methods, participant characteristics, metrics, etc.) to determine its applicability and generalizability to their context of use .
Collaboration: Consider establishing a collaborative relationship with the developer to provide feedback for next-generation sDHTs, ensure version control, and potentially collaborate on future usability validation studies .

Regulatory Considerations
The document notes that if the sDHT is a regulated medical device, the intended use statement should already capture the answers to the basic questions. The entire checklist is framed around the V3+ framework, which is designed to ensure the rigor necessary for a product to be considered fit-for-purpose by all stakeholders.

Findings
The field lacks a centralized, standardized database of validated digital health technologies, making it difficult for researchers and clinicians to select appropriate tools.
Non-wearable sensors and software applications are the most common types of DHTs, with 83% of ambient technologies categorized as software or applications.
Most DHTs focus on mild cognitive impairment (MCI) and early Alzheimer’s disease, with fewer technologies validated for moderate or severe dementia stages.
Uneven Distribution of Dementia Subtypes – The review identified a gap in DHT validation for frontotemporal dementia (FTD) and Lewy Body dementia, with Alzheimer’s disease being the predominant focus.

Recommendations
Expand and maintain an open-access database of validated DHTs to improve accessibility and standardization.
Increase research on digital measures applicable to moderate and severe stages of dementia, as well as non-Alzheimer’s dementias.
Promote integration of wearable, ambient, and cognitive assessment tools to generate comprehensive digital phenotypes of patients.
Follow clear guidelines for analytical and clinical validation of DHTs to improve regulatory acceptance and research applicability.
Conduct more usability and feasibility assessments, especially for populations with cognitive decline, to ensure DHTs are accessible and effective in real-world settings.

Findings
There is a lack of systematic approaches to guide the processes of collecting, interpreting, analyzing, and translating health data from wearables into digital biomarkers.
Most wearables have fixed measurement capabilities, limiting their translation to digital biomarkers.
Current guidance lacks study design and conduct elements that involve all stakeholders in an iterative approach for implementing digital biomarkers in practice.
Researchers and health professionals often rely on limited guidance for using wearable data in clinical practice and chronic disease management.

Recommendations
Implement the DACIA framework to provide interdisciplinary guidance on using wearable sensor data for digital biomarker development.
Focus on participant needs as a crucial factor for study success, applicable to both short and long-duration studies.
Involve relevant stakeholders in each key step of the DACIA framework in an iterative manner.
Apply the DACIA framework to explore digital biomarkers using various devices or signal measurements.
Reduce participant burden through support and continuous feedback.

Regulatory Considerations
Not mentioned

Findings
The V3+ Use Specification must contain a detailed description of the user groups, use environments, and the sDHT user interface. The user groups include end-users (individuals from whom data is captured) as well as carepartners, clinicians, researchers, and administrators. Characteristics of users (e.g., demographics, literacy, physical/cognitive capabilities, disease characteristics) and use environments (e.g., temperature, network availability, clutter) must be considered for risk management .

Recommendations
Developers must follow these four steps to create the Use Specification:
Identify all user groups: Create a list of users, including sub-categories (e.g., different types of researchers), and describe the characteristics of each group (e.g., health literacy, physical capabilities) to create detailed descriptions of representative users.
Identify all likely use environments: Create a list of typical environments (e.g., Home, Hospitals) and describe their characteristics (e.g., temperature, noise, network availability), also considering "edge cases" (e.g., extreme weather).
Describe the sDHT user interface: Detail all aspects of the hardware and software (visual, auditory, tactile cues), accessories (e.g., packaging, chargers), and all written materials and training (e.g., instructions for use, helpdesk troubleshooting).
Keep it up to date: The Use Specification is a living document that requires ongoing updates and maintenance throughout the sDHT development and usability validation process.

Regulatory Considerations
The development of the Use Specification is presented as the foundational step for the usability validation component of the V3+ framework. This document directly informs the subsequent Use-Related Risk Analysis.

Findings
A Use-Related Risk Analysis (URRA) is essential for identifying potential use-errors (actions or lack of action that may result in harm) and their associated use-related hazards (source of potential harm) when using an sDHT. The analysis must focus on user interactions with the sDHT, including all user groups (end-users, clinicians, carepartners, researchers, and administrators). Critical tasks are defined as those use-errors that may result in serious harm.

Recommendations
Developers should follow these five steps for the Use-Related Risk Analysis:
Describe all user tasks: Identify the sequence of actions a user performs to achieve a goal, which can be derived from sources like a task analysis or formative evaluations.
Describe potential use-errors: Identify and document potential actions or lack of actions that may result in harm for each task, noting that "use-error" is preferable to "user-error".
Describe potential use-related hazards: Determine the source of potential harm resulting from each identified use-error.
Develop a plan to minimize or eliminate known risks: The preferred approach is inherent safety by design (eliminating the error). If not feasible, use protective measures (e.g., warnings) or, as a last resort, provide instructions to users. Identify methods to evaluate the effectiveness of the chosen mitigation strategy.
Keep it up to date: The URRA is a living document requiring ongoing updates throughout the sDHT development and usability validation process.

Regulatory Considerations
The URRA is presented as a fundamental step within the V3+ framework for ensuring device usability and minimizing risk, implicitly setting the groundwork for regulatory compliance related to device safety. The minimization of use-errors, particularly for critical tasks, is a central tenet of device development best practices.

Findings
Usability gaps in sDHTs remain a barrier to adoption, with many technologies failing to prioritize ease of use, accessibility, and diverse user needs​
Human-centered design is critical for ensuring that digital health solutions are intuitive, functional, and scalable across different healthcare environments​
Standardized usability metrics for evaluating digital health technologies are lacking, leading to inconsistent reporting and validation of usability outcomes​
Use-related risk analysis is essential to identifying and mitigating risks associated with user errors, ensuring the safety and effectiveness of sDHTs​
The V3+ framework provides a structured approach to integrating usability validation into digital health technology development, aligning with global regulatory expectations​

Recommendations
Developers should incorporate human-centered design principles from the outset, ensuring that usability, accessibility, and user needs are central to sDHT development​
Usability validation should be standardized, with clear methodologies for measuring usability, including satisfaction, ease of use, efficiency, and error mitigation​
Regulatory and clinical stakeholders should collaborate on defining best practices for usability evaluation, ensuring that digital endpoints are both meaningful and scalable​
Risk analysis should be iterative, with developers continuously refining their technologies based on real-world user feedback and testing​
The usability validation component of V3+ should be widely adopted to ensure that digital clinical measures meet patient-centered, regulatory, and technical expectations​

Regulatory Considerations
Regulators are emphasizing the need for usability validation to ensure that digital endpoints are both clinically relevant and patient-friendly​
sDHTs must comply with human factors engineering guidelines, aligning with global regulatory frameworks such as ISO 9241-210 and FDA usability requirements​
Data security, privacy, and interoperability must be ensured, particularly as sDHTs become integrated into remote monitoring and decentralized clinical trials​
Real-world evidence (RWE) should support usability validation, helping to bridge the gap between regulatory approval and real-world adoption​
Regulatory bodies should work toward standardizing usability testing methodologies, ensuring consistency across clinical research, digital endpoints, and medical device evaluations​

Findings
Analytical validation is critical for ensuring digital clinical measures align with regulatory and scientific expectations, particularly when no established reference measures exist.
Novel digital measures require flexible validation approaches, as traditional clinical reference measures often do not directly correspond to digital endpoints​
Statistical methodologies must be tailored to the nature of digital measures, using approaches such as factor analysis, regression modeling, and latent variable estimation​
Regulatory engagement is crucial early in the validation process to align evidentiary standards and facilitate market adoption​
The validation process must be context-specific, considering population characteristics, data collection settings, and sensor variability to ensure reliability across diverse applications​.

Recommendations
Developers should follow a stepwise approach in designing validation studies, incorporating existing reference measures, novel comparators, and statistical validation techniques.
Regulatory authorities should provide clearer guidance on acceptable validation methodologies, particularly for novel digital endpoints.
Analytical validation must be tailored to the intended use environment, ensuring that sensor-based measures capture meaningful health outcomes in real-world settings​.
Multi-stakeholder collaboration (regulators, payers, researchers, and patients) should be prioritized to create consensus on validation strategies and market access pathways.
Machine learning and AI models used for digital clinical measures should undergo rigorous evaluation to mitigate bias and improve interpretability in healthcare decision-making​.

Regulatory Considerations
Digital endpoint validation must incorporate both traditional statistical measures and novel validation frameworks, ensuring credibility in regulatory submissions.
FDA and international regulators encourage early engagement to discuss validation plans, data requirements, and evidentiary thresholds for digital measures​.
Real-world evidence (RWE) and real-world data (RWD) should be leveraged to support regulatory submissions and post-market surveillance of digital health innovations​.
Validation studies should align with global regulatory standards, such as ISO, FDA’s digital health guidance, and European Medical Device Regulations (MDR).
Data privacy, security, and compliance with regulations like HIPAA and GDPR are critical considerations when deploying and validating digital clinical measures​