Findings
The FDA classifies CDS software as Non-Device CDS only if it meets four specific criteria related to data inputs, information display, HCP support, and independent reviewability. Software functions that analyze medical images, signals from IVDs, or patterns from signal acquisition systems remain regulated as medical devices. Non-Device CDS must be intended for health care professionals and not for patients or caregivers. Automation bias and the time-critical nature of decision-making are key factors in determining whether an HCP can truly review the basis of a recommendation independently. If software provides a specific diagnostic or treatment directive rather than a list of options, it generally fails to meet the exclusion criteria.

Recommendations
Developers should ensure that software intended as Non-Device CDS provides a plain language description of the underlying algorithm and the data used for validation. The software or labeling must clearly identify the intended HCP user, the patient population, and the required input medical information. To support independent review, the software should highlight the source of its clinical recommendations, such as specific clinical practice guidelines or peer-reviewed studies. Developers are encouraged to use usability testing to verify that HCPs can understand the basis of recommendations without relying primarily on the software’s output. For multiple function products, developers should follow the FDA’s policy for assessing products that contain both device and non-device functions.

Regulatory Considerations
The FDA applies a risk-based approach to software oversight, focusing on functions that acquire or analyze complex medical data like ECG waveforms or genomic sequences. Software intended for time-sensitive or critical medical decisions is typically regulated as a device because the user lacks the time to independently verify the recommendation. The agency intends to exercise enforcement discretion for certain software functions that provide only one clinically appropriate recommendation if all other non-device criteria are met. Sponsors may use the Q-Submission process to discuss alternative approaches or clarify the regulatory status of specific software functions. Existing digital health policies continue to apply to software functions that meet the device definition, including mobile medical applications.

Findings
The quality of evidence for the analytical validation of sensor-based digital health technologies (sDHTs), which is the evaluation of algorithms converting sensor data into a clinically interpretable measure, is often inconsistent and insufficient. The existing V3+ framework codifies the overall evaluation process, which includes verification, usability validation, analytical validation, and clinical validation. To improve the scientific rigor of analytical validation, a hierarchical framework for selecting reference measures is needed because not all potential reference measures are of equal quality. The framework classifies reference measures based on attributes that contribute to reduced measurement variability, with defining and principal measures being the most rigorous due to objective data acquisition and the ability to retain source data.

Recommendations
The proposed framework sequentially moves the investigator through four steps: (1) Compile preliminary information, including the digital clinical measure, context of use (COU), algorithm requirements, and sensor verification evidence . (2) Select an existing reference measure, develop a novel comparator, or identify a set of anchor measures, prioritizing measures with the highest scientific rigor (defining → principal → manual → reported) . (3) Consider the impact of the data collection environment to determine if the analytical validation study can be conducted in the intended use environment with the highest-order measure, or if in-lab validation is necessary, ensuring the results are generalizable . (4) Describe the rationale for key study design decisions to encourage transparency for evaluators, regulators, and payers . Investigators must justify passing over a higher-ranked reference measure, generally only acceptable if the higher-ranked measure poses unacceptable risk or is not applicable to the context of use.

Regulatory Considerations
The principles of the framework for analytical validation apply regardless of the regulatory status of the sDHT (regulated medical device, low-risk general wellness apps, or research product) or its intended use (clinical care or clinical research). The framework is intended to help investigators support the most rigorous claims regarding sDHT performance, which is important for acceptance by evaluators, peer-reviewers, regulators, and payers. The categorization of the digital clinical measure as a digital biomarker or an electronic clinical outcome assessment also does not change the framework's applicability.

Findings
The most promising aspects of mental health for digital measurement are sleep, physical activity, stress, and social behavior, which have the strongest scientific evidence. Core barriers to adoption include high cost and limited access, data privacy concerns, poor technological literacy, and a lack of technology adaptation for specific mental health needs. Essential technology characteristics for "fit-for-purpose" sDHTs include usability, reliable performance, strong data privacy and security, and long battery life.

Recommendations
Research and development should prioritize moving promising measures (sleep, activity, stress, social behavior) to large-scale clinical trials. Algorithms must be refined and clinically validated for mental health indications, and new sensor modalities should be explored. Infrastructure must be developed by creating standards and ontologies for mental health sensor data to ensure interoperability and scalability. To improve access and equity, financial support mechanisms and inclusive, culturally tailored design are critical.

Regulatory Considerations
The report does not provide a separate section for "Regulatory Considerations" but emphasizes that future development and funding should prioritize clinical validation across diverse populations. It notes the importance of a clear understanding of the intended measurement claims and the need for rigorous validation studies to move beyond pilot and feasibility stages to demonstrate real-world clinical utility.

Findings
AI-enabled medical devices require robust risk assessment to address data drift, bias, and transparency challenges.
The total product lifecycle (TPLC) approach is essential for managing AI-enabled devices, ensuring continuous oversight and updates.
There is a need for improved standardization in AI model validation and performance monitoring to ensure consistency in regulatory submissions.
Effective data management practices, including dataset representativeness and bias control, are critical for AI model development.
Cybersecurity vulnerabilities in AI-enabled medical devices must be proactively addressed to prevent risks to patient safety and data integrity.

Recommendations
AI-enabled device manufacturers should integrate Good Machine Learning Practice (GMLP) principles throughout the device lifecycle.
Marketing submissions should include comprehensive documentation of AI model development, validation, and performance monitoring.
Developers should implement transparency measures, such as model interpretability and explainability, to enhance user trust and understanding.
AI models must undergo rigorous bias evaluation to ensure equitable performance across diverse patient populations.
A predetermined change control plan (PCCP) should be established to allow safe and effective AI model updates post-market without additional FDA submissions.

Regulatory Considerations
FDA encourages early engagement through the Q-Submission Program for AI-enabled device manufacturers.
Compliance with FDA-recognized consensus standards, such as ANSI/AAMI/ISO 14971 for risk management, is recommended.
AI-enabled devices must meet labeling requirements, ensuring that users clearly understand model inputs, outputs, and performance metrics.
Post-market surveillance and continuous monitoring of AI model performance are necessary to ensure ongoing safety and effectiveness.
Cybersecurity measures must be included in regulatory submissions, detailing safeguards against data breaches and unauthorized model modifications.

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

Findings
Existing guidelines lack pragmatic application and systematic approach for device selection.
Device choice is dependent on measurement objectives, user population, and available resources.
Current frameworks do not systematically consider verification, validation, feasibility, and protocol design.
Rapid obsolescence of digital devices due to technological advancements.
Need to incorporate social/psychological factors into device selection.

Recommendations
Develop a practical guide with a systematic approach for selecting wearable devices.
Use five core criteria: continuous monitoring capability, device suitability and availability, technical performance, feasibility of use, and cost evaluation.
Prioritize feasibility of use to ensure user needs are incorporated into the selection process.
Adapt guide criteria to accommodate novel innovations.
Foster clarity and transparency in decision-making among researchers, HCPs, and device users.

Regulatory Considerations
Follow FDA guidance for digital health technology usage in clinical investigations.
Consider CTTI recommendations for improving clinical trial quality and efficiency.
Use ePRO Consortium's factors for device suitability in regulatory trials.
Apply international guidelines for specific measurements when available.

Findings
The goal of sDHT design is to create tools that are functional, intuitive, accessible, and enjoyable to use, moving beyond merely minimizing use-errors. Human-centered design (HCD) is the preferred term over user-centered design, emphasizing the impact on many user groups beyond just the end-users. "Users" encompass end-users (patients/participants), carepartners, clinicians, investigators, and administrators.

Recommendations
Developers of sDHTs should adhere to the following HCD principles:
Empathetic: Take time to deeply understand users' needs, behaviors, and emotions, capturing this in the use specification.
Holistic: Consider the entire end-to-end user journey, including hardware, software, accessories, packaging, instructions for use, and training.
Iterative: Employ an iterative approach to designing, prototyping, testing, and refining, using formative evaluations to identify use-errors and gather usability data, capturing this in the use-related risk analysis.
User-centric: Improve usability by capturing user feedback in real-world settings, gradually recruiting larger, more diverse samples that represent the intended use population.
Inclusive: Collaborate with individuals representing all user groups by hiring them as consultants or creating user advisory panels to influence design decisions (co-design).
Multidisciplinary: Ensure the development team includes colleagues from various disciplines to bring diverse perspectives and innovative solutions.

Regulatory Considerations
The document ties the HCD process to risk management and eventual validation by recommending that findings from formative evaluations (used to identify use-errors) be captured in a use-related risk analysis. The approach aligns with the principles of the overarching V3+ framework.

Findings
Usability is a multi-domain concept that requires a combination of methods for evaluation. Evaluations fall into two types: formative (for design modification of prototypes) and summative (for demonstrating usability of the final product to a representative user sample). The user experience metrics fall into several domains, including: Satisfaction, Usefulness, Ease of use, Learnability, Efficiency, Memorability, Understandability, Actionability, Readability, and Use-errors. Metrics related to Satisfaction and Usefulness are always subjectively reported by users.

Recommendations
Developers should select metrics based on the specific usability-related domain being evaluated.
Subjective Data (e.g., Satisfaction, Usefulness): Capture through qualitative surveys, quantitative surveys (scales), interviews, focus groups, and think-aloud evaluations .
Objective Data (e.g., Ease of use, Use-errors): Capture through direct or indirect observation (e.g., counting steps/attempts, timing task completion), or by using data generated by the sDHT (e.g., error reports, timestamps, page load times).
Time-based Metrics: Evaluate Learnability (ease of first use), Efficiency (ease with experience), and Memorability (ease after non-use) by measuring ease of use at different points in time .
Information Presentation: If the sDHT presents clinical data or written information (instructions, warnings), evaluate Understandability, Actionability, and Readability .
Use-errors: Objectively capture the number, type, and recoverability of use-errors (actions, or lack thereof, that may result in harm) via observation and sDHT data, noting that "use-error" is preferred to "user-error".

Regulatory Considerations
While this guide does not reference regulatory bodies like the FDA, it is part of the V3+ framework and recommends that researchers prioritize essential documents like the use specification and use-related risk analysis before designing a usability study. Summative evaluations demonstrating usability against a representative user sample under intended use conditions are the standard for demonstrating product fitness.

Findings
Sites must establish effective data privacy and security plans, especially considering regional and global regulations like GDPR.
Risk mitigation is critical, including plans to address unanticipated issues and potential patient disengagement due to technology challenges.
Budgeting and contracting often involve additional considerations, such as storage, training, and technical support requirements for connected devices.
Sites require adequate training to ensure staff and patients are prepared to use connected devices efficiently.
Companion applications or services often play an essential role in device functionality and data transmission.

Recommendations
Develop a clear plan for data pathways, including storage, security, and regulatory compliance.
Establish detailed risk mitigation and management strategies to handle unexpected challenges.
Ensure comprehensive training programs for site staff and patients to enhance device usability.
Incorporate device storage and resource allocation into budgeting and contracting processes.
Facilitate effective communication with sponsors and vendors to resolve operational and technical issues promptly.

Regulatory Considerations
Ensure connected devices comply with CFR 21, Part 11, and other relevant data collection and transmission regulations.
Understand and adhere to local and regional data privacy laws, such as GDPR, when managing patient data.
Verify that appropriate licenses and regulatory approvals are in place for device data transmission and storage.
Assess and address shipping and handling regulations for devices, ensuring safe and compliant transportation.

Findings
Usability gaps in sDHTs remain a barrier to adoption, with many technologies failing to prioritize ease of use, accessibility, and diverse user needs​
Human-centered design is critical for ensuring that digital health solutions are intuitive, functional, and scalable across different healthcare environments​
Standardized usability metrics for evaluating digital health technologies are lacking, leading to inconsistent reporting and validation of usability outcomes​
Use-related risk analysis is essential to identifying and mitigating risks associated with user errors, ensuring the safety and effectiveness of sDHTs​
The V3+ framework provides a structured approach to integrating usability validation into digital health technology development, aligning with global regulatory expectations​

Recommendations
Developers should incorporate human-centered design principles from the outset, ensuring that usability, accessibility, and user needs are central to sDHT development​
Usability validation should be standardized, with clear methodologies for measuring usability, including satisfaction, ease of use, efficiency, and error mitigation​
Regulatory and clinical stakeholders should collaborate on defining best practices for usability evaluation, ensuring that digital endpoints are both meaningful and scalable​
Risk analysis should be iterative, with developers continuously refining their technologies based on real-world user feedback and testing​
The usability validation component of V3+ should be widely adopted to ensure that digital clinical measures meet patient-centered, regulatory, and technical expectations​

Regulatory Considerations
Regulators are emphasizing the need for usability validation to ensure that digital endpoints are both clinically relevant and patient-friendly​
sDHTs must comply with human factors engineering guidelines, aligning with global regulatory frameworks such as ISO 9241-210 and FDA usability requirements​
Data security, privacy, and interoperability must be ensured, particularly as sDHTs become integrated into remote monitoring and decentralized clinical trials​
Real-world evidence (RWE) should support usability validation, helping to bridge the gap between regulatory approval and real-world adoption​
Regulatory bodies should work toward standardizing usability testing methodologies, ensuring consistency across clinical research, digital endpoints, and medical device evaluations​

Findings
For an sDHT to be considered fit-for-purpose, a researcher or healthcare provider must understand the alignment between the sDHT's intended use (What it does, who uses it, where/when/how) and their own context of use . Key information for this assessment comes from the developer's Use Specification (detailing hardware, software, accessories, training) and Use-Related Risk Analysis (detailing warnings, harms from use-errors, and risk avoidance) . Usability validation evidence should cover study objectives, protocols, participant characteristics, metrics, and collection methods.

Recommendations
Researchers/providers should use the checklist to:
The Basics: Compare the sDHT's intended use to their context of use; if there is substantial overlap, existing evidence may be sufficient.
Use Specification/Risk Analysis: Gather detailed descriptions of the sDHT's hardware, software, accessories, written materials, training, cautions, warnings, and potential harms from use-errors to update their own Use Specification and Use-Related Risk Analysis .
Existing Evidence: Access existing usability validation study results (objectives, methods, participant characteristics, metrics, etc.) to determine its applicability and generalizability to their context of use .
Collaboration: Consider establishing a collaborative relationship with the developer to provide feedback for next-generation sDHTs, ensure version control, and potentially collaborate on future usability validation studies .

Regulatory Considerations
The document notes that if the sDHT is a regulated medical device, the intended use statement should already capture the answers to the basic questions. The entire checklist is framed around the V3+ framework, which is designed to ensure the rigor necessary for a product to be considered fit-for-purpose by all stakeholders.

Findings
The field lacks a centralized, standardized database of validated digital health technologies, making it difficult for researchers and clinicians to select appropriate tools.
Non-wearable sensors and software applications are the most common types of DHTs, with 83% of ambient technologies categorized as software or applications.
Most DHTs focus on mild cognitive impairment (MCI) and early Alzheimer’s disease, with fewer technologies validated for moderate or severe dementia stages.
Uneven Distribution of Dementia Subtypes – The review identified a gap in DHT validation for frontotemporal dementia (FTD) and Lewy Body dementia, with Alzheimer’s disease being the predominant focus.

Recommendations
Expand and maintain an open-access database of validated DHTs to improve accessibility and standardization.
Increase research on digital measures applicable to moderate and severe stages of dementia, as well as non-Alzheimer’s dementias.
Promote integration of wearable, ambient, and cognitive assessment tools to generate comprehensive digital phenotypes of patients.
Follow clear guidelines for analytical and clinical validation of DHTs to improve regulatory acceptance and research applicability.
Conduct more usability and feasibility assessments, especially for populations with cognitive decline, to ensure DHTs are accessible and effective in real-world settings.