Findings
The quality of evidence for the analytical validation of sensor-based digital health technologies (sDHTs), which is the evaluation of algorithms converting sensor data into a clinically interpretable measure, is often inconsistent and insufficient. The existing V3+ framework codifies the overall evaluation process, which includes verification, usability validation, analytical validation, and clinical validation. To improve the scientific rigor of analytical validation, a hierarchical framework for selecting reference measures is needed because not all potential reference measures are of equal quality. The framework classifies reference measures based on attributes that contribute to reduced measurement variability, with defining and principal measures being the most rigorous due to objective data acquisition and the ability to retain source data.

Recommendations
The proposed framework sequentially moves the investigator through four steps: (1) Compile preliminary information, including the digital clinical measure, context of use (COU), algorithm requirements, and sensor verification evidence . (2) Select an existing reference measure, develop a novel comparator, or identify a set of anchor measures, prioritizing measures with the highest scientific rigor (defining → principal → manual → reported) . (3) Consider the impact of the data collection environment to determine if the analytical validation study can be conducted in the intended use environment with the highest-order measure, or if in-lab validation is necessary, ensuring the results are generalizable . (4) Describe the rationale for key study design decisions to encourage transparency for evaluators, regulators, and payers . Investigators must justify passing over a higher-ranked reference measure, generally only acceptable if the higher-ranked measure poses unacceptable risk or is not applicable to the context of use.

Regulatory Considerations
The principles of the framework for analytical validation apply regardless of the regulatory status of the sDHT (regulated medical device, low-risk general wellness apps, or research product) or its intended use (clinical care or clinical research). The framework is intended to help investigators support the most rigorous claims regarding sDHT performance, which is important for acceptance by evaluators, peer-reviewers, regulators, and payers. The categorization of the digital clinical measure as a digital biomarker or an electronic clinical outcome assessment also does not change the framework's applicability.

Findings
The most promising aspects of mental health for digital measurement are sleep, physical activity, stress, and social behavior, which have the strongest scientific evidence. Core barriers to adoption include high cost and limited access, data privacy concerns, poor technological literacy, and a lack of technology adaptation for specific mental health needs. Essential technology characteristics for "fit-for-purpose" sDHTs include usability, reliable performance, strong data privacy and security, and long battery life.

Recommendations
Research and development should prioritize moving promising measures (sleep, activity, stress, social behavior) to large-scale clinical trials. Algorithms must be refined and clinically validated for mental health indications, and new sensor modalities should be explored. Infrastructure must be developed by creating standards and ontologies for mental health sensor data to ensure interoperability and scalability. To improve access and equity, financial support mechanisms and inclusive, culturally tailored design are critical.

Regulatory Considerations
The report does not provide a separate section for "Regulatory Considerations" but emphasizes that future development and funding should prioritize clinical validation across diverse populations. It notes the importance of a clear understanding of the intended measurement claims and the need for rigorous validation studies to move beyond pilot and feasibility stages to demonstrate real-world clinical utility.

Findings
The traditional process of evaluating biomarkers within the context of a single drug development program is inefficient and creates uncertainty for sponsors. This case-by-case approach leads to redundant efforts, slows down the development of novel therapies, and hinders the broad adoption of promising scientific tools. There is a clear need for a centralized, collaborative pathway to formally validate biomarkers, which can de-risk drug development, encourage innovation, and make the process more predictable and cost-effective for all stakeholders.

Recommendations
Drug developers, academic researchers, and other stakeholders should proactively engage with the FDA through the formal Biomarker Qualification Program to validate biomarkers for specific contexts of use. It is recommended to form public-private partnerships and other collaborations to pool resources and data, which strengthens the evidence package for a biomarker's utility. Developers should use the qualification process to establish a biomarker's value early, making it a publicly available and reliable tool that can accelerate the development of multiple drug products.

Regulatory Considerations
The Biomarker Qualification Program provides a distinct regulatory pathway for establishing a biomarker's validity for a specific Context of Use (COU), separate from an individual Investigational New Drug (IND) or New Drug Application (NDA). The process involves a three-stage submission and review cycle: the Letter of Intent, the Qualification Plan, and the Full Qualification Package. Once qualified, a biomarker is publicly listed and can be incorporated into multiple drug development programs without the need for sponsors to re-submit and re-justify the validation data for that specific COU, streamlining subsequent regulatory reviews.

Findings
Cytokine Release Syndrome (CRS) is a common and potentially life-threatening adverse event of immunotherapies, particularly in Oncology, complicating patient care and increasing healthcare costs. Standard-of-care inpatient monitoring for CRS is manual, intermittent, costly, and restrictive, providing an incomplete view of the syndrome’s development and progression. The use of Digital Health Technologies (DHTs) for continuous, remote monitoring of vital signs (like heart rate, respiratory rate, skin temperature, SpO2, and activity) can capture early indicators of CRS up to two hours earlier than standard episodic monitoring. This ability to collect multivariate continuous data is valuable for informing robust model development for CRS risk prediction.

Recommendations
Investigators should deploy DHTs available today to monitor vital signs and symptoms currently observed in the hospital setting, but in an outpatient or home environment. The goal is to develop Early Warning Products that assess the probability of developing CRS, providing clinical decision support. Product developers should follow a strategic roadmap that outlines milestones for building products that are clinically relevant and commercially viable. Researchers should use a common set of digital clinical measures to gather high-quality datasets and ensure comparability across studies to build more robust predictive models. Predictive algorithms should be built on a robust reference measure for analytical validation and be clinically validated with sufficient data.

Regulatory Considerations
The resources are designed to help developers build products that are clinically appropriate, regulatory-acceptable, and commercially viable. Future regulatory submissions for CRS de-risking products will benefit from aligning with this industry-wide dialogue that is being built in collaboration with the FDA. Developing a robust CRS safety biomarker could enhance the safety profile of clinical trials, increase trial access, and streamline regulatory decision-making, possibly through a qualification pathway. Products that aim for a higher level of autonomy, such as a Diagnostic that redefines current CRS grading classes, may require very high clinical evidence and likely stringent regulatory review.

Findings
While verification, analytical validation, and clinical validation have been well-established, usability validation has not been systematically incorporated into digital health technology evaluation.
Variability in device designs, patient populations, and regulatory environments creates barriers to widespread adoption of sensor-based digital health technologies.
Usability problems, such as poor user interfaces and technical errors, can lead to significant data loss in clinical trials and real-world applications.
While some guidance exists for usability in medical devices, there is no unified global standard for assessing usability in digital health products, leading to inconsistencies in implementation.
Stakeholders, including regulators, industry leaders, and researchers, recognize the need for usability validation to ensure the real-world effectiveness of digital health technologies.

Recommendations
Adopt the V3+ framework as a standardized method to ensure that usability is rigorously tested alongside verification, analytical validation, and clinical validation.
Establish clear protocols for usability testing, including use specification development, risk analysis, iterative formative evaluations, and summative evaluations.
Bring together regulators, technology developers, clinicians, and patients to create guidelines that ensure fit-for-purpose digital health solutions.
Work with regulatory agencies such as FDA, EMA, and MHRA to establish harmonized global standards for usability validation.
Encourage the publication of usability study results, including negative findings, to facilitate transparency and continuous improvement in digital health technologies.

Regulatory Considerations
Agencies like FDA and EMA increasingly require usability data for digital health technologies, but standardized methodologies are still evolving.
Usability validation should align with regulatory requirements for medical devices and digital biomarkers, ensuring clinical relevance and data integrity.
Digital health technologies must adhere to HIPAA, GDPR, and other data protection regulations while ensuring seamless usability.
Poor usability can lead to missing or unreliable data, which affects regulatory submissions and real-world evidence generation.
A consistent approach to usability evaluation is needed to support regulatory decision-making and digital health product approvals globally.

Findings
Different Digital Health Technologies (DHTs) estimate sleep staging using data from various sensor-based sources (e.g., EEG, actigraphy, ballistocardiography), each with different properties impacting the estimation. Sleep staging algorithms are often proprietary. DHTs interpret sleep staging at different time intervals, or epochs (e.g., polysomnography uses 30-second epochs). DHT vendors transmit data at different levels, ranging from epoch-level data to pre-calculated summary data (e.g., "total sleep time").

Recommendations
Method and Signals: Ask the vendor about their method of sleep monitoring and which signals are being recorded and used, and understand the strengths and limitations of the technology.
Granularity and Epochs: Inquire about the granularity of sleep data estimated (coarse to fine grain) and the epoch length used for sleep annotations, as this informs interpretation and comparability to other research.
Thresholds and Rules: Ask what rules and thresholds are set for confirming events like sleep onset and offset to ensure certainty in the data and inform future interpretation of results.
Data Level: To align with the Core Digital Measures of Sleep, epoch-level data is preferred for further analysis and comparison between measurement systems. If only summary data is offered, ask for a detailed description of the estimation process.
Algorithms and Evidence: Ask for evidence to support the validity and reliability of the estimated sleep stages, which may include peer-reviewed manuscripts, technical documentation, and conference abstracts.

Regulatory Considerations
While not a regulatory document, the recommendations emphasize the need for vendors to provide evidence for the validity and reliability of their proprietary sleep staging algorithms. This evidence, which can be found in peer-reviewed literature or technical documentation, is crucial for establishing confidence in the results arising from the technology, and can be used for inclusion in, for example, regulatory documents.

Findings
The field lacks a centralized, standardized database of validated digital health technologies, making it difficult for researchers and clinicians to select appropriate tools.
Non-wearable sensors and software applications are the most common types of DHTs, with 83% of ambient technologies categorized as software or applications.
Most DHTs focus on mild cognitive impairment (MCI) and early Alzheimer’s disease, with fewer technologies validated for moderate or severe dementia stages.
Uneven Distribution of Dementia Subtypes – The review identified a gap in DHT validation for frontotemporal dementia (FTD) and Lewy Body dementia, with Alzheimer’s disease being the predominant focus.

Recommendations
Expand and maintain an open-access database of validated DHTs to improve accessibility and standardization.
Increase research on digital measures applicable to moderate and severe stages of dementia, as well as non-Alzheimer’s dementias.
Promote integration of wearable, ambient, and cognitive assessment tools to generate comprehensive digital phenotypes of patients.
Follow clear guidelines for analytical and clinical validation of DHTs to improve regulatory acceptance and research applicability.
Conduct more usability and feasibility assessments, especially for populations with cognitive decline, to ensure DHTs are accessible and effective in real-world settings.

Findings
There is a lack of systematic approaches to guide the processes of collecting, interpreting, analyzing, and translating health data from wearables into digital biomarkers.
Most wearables have fixed measurement capabilities, limiting their translation to digital biomarkers.
Current guidance lacks study design and conduct elements that involve all stakeholders in an iterative approach for implementing digital biomarkers in practice.
Researchers and health professionals often rely on limited guidance for using wearable data in clinical practice and chronic disease management.

Recommendations
Implement the DACIA framework to provide interdisciplinary guidance on using wearable sensor data for digital biomarker development.
Focus on participant needs as a crucial factor for study success, applicable to both short and long-duration studies.
Involve relevant stakeholders in each key step of the DACIA framework in an iterative manner.
Apply the DACIA framework to explore digital biomarkers using various devices or signal measurements.
Reduce participant burden through support and continuous feedback.

Regulatory Considerations
Not mentioned

Findings
Analytical validation is critical for ensuring digital clinical measures align with regulatory and scientific expectations, particularly when no established reference measures exist.
Novel digital measures require flexible validation approaches, as traditional clinical reference measures often do not directly correspond to digital endpoints​
Statistical methodologies must be tailored to the nature of digital measures, using approaches such as factor analysis, regression modeling, and latent variable estimation​
Regulatory engagement is crucial early in the validation process to align evidentiary standards and facilitate market adoption​
The validation process must be context-specific, considering population characteristics, data collection settings, and sensor variability to ensure reliability across diverse applications​.

Recommendations
Developers should follow a stepwise approach in designing validation studies, incorporating existing reference measures, novel comparators, and statistical validation techniques.
Regulatory authorities should provide clearer guidance on acceptable validation methodologies, particularly for novel digital endpoints.
Analytical validation must be tailored to the intended use environment, ensuring that sensor-based measures capture meaningful health outcomes in real-world settings​.
Multi-stakeholder collaboration (regulators, payers, researchers, and patients) should be prioritized to create consensus on validation strategies and market access pathways.
Machine learning and AI models used for digital clinical measures should undergo rigorous evaluation to mitigate bias and improve interpretability in healthcare decision-making​.

Regulatory Considerations
Digital endpoint validation must incorporate both traditional statistical measures and novel validation frameworks, ensuring credibility in regulatory submissions.
FDA and international regulators encourage early engagement to discuss validation plans, data requirements, and evidentiary thresholds for digital measures​.
Real-world evidence (RWE) and real-world data (RWD) should be leveraged to support regulatory submissions and post-market surveillance of digital health innovations​.
Validation studies should align with global regulatory standards, such as ISO, FDA’s digital health guidance, and European Medical Device Regulations (MDR).
Data privacy, security, and compliance with regulations like HIPAA and GDPR are critical considerations when deploying and validating digital clinical measures​

NA

Findings
There is a need for a structured framework to leverage prior work in the use of DHTs in clinical trials.
The current body of evidence supporting DHTs is growing, but there is a lack of clarity on how to effectively utilize this evidence.
The V3 framework provides a process for validating DHTs, but its application across different medical product development programs is inconsistent.

Recommendations
Implement a framework to reuse analytical and clinical validation data for existing DHTs.
Encourage early and continuous communication with regulatory health authorities.
Leverage prior work to share best practices and consistent approaches in employing DHTs.
Use the V3 framework to ensure DHTs are fit-for-purpose in clinical trials.
Develop a strategic approach to incorporate DHTs and digitally derived endpoints within clinical development programs.

Regulatory Considerations
Sponsors should ensure their plans to leverage prior work are endorsed by regulatory health authorities.
Alignment with FDA guidance on digital health technologies is crucial.
The regulatory status of the DHT and its intended use should be clearly defined and considered in clinical trial applications.

N/A