Findings
Usability gaps in sDHTs remain a barrier to adoption, with many technologies failing to prioritize ease of use, accessibility, and diverse user needs​
Human-centered design is critical for ensuring that digital health solutions are intuitive, functional, and scalable across different healthcare environments​
Standardized usability metrics for evaluating digital health technologies are lacking, leading to inconsistent reporting and validation of usability outcomes​
Use-related risk analysis is essential to identifying and mitigating risks associated with user errors, ensuring the safety and effectiveness of sDHTs​
The V3+ framework provides a structured approach to integrating usability validation into digital health technology development, aligning with global regulatory expectations​

Recommendations
Developers should incorporate human-centered design principles from the outset, ensuring that usability, accessibility, and user needs are central to sDHT development​
Usability validation should be standardized, with clear methodologies for measuring usability, including satisfaction, ease of use, efficiency, and error mitigation​
Regulatory and clinical stakeholders should collaborate on defining best practices for usability evaluation, ensuring that digital endpoints are both meaningful and scalable​
Risk analysis should be iterative, with developers continuously refining their technologies based on real-world user feedback and testing​
The usability validation component of V3+ should be widely adopted to ensure that digital clinical measures meet patient-centered, regulatory, and technical expectations​

Regulatory Considerations
Regulators are emphasizing the need for usability validation to ensure that digital endpoints are both clinically relevant and patient-friendly​
sDHTs must comply with human factors engineering guidelines, aligning with global regulatory frameworks such as ISO 9241-210 and FDA usability requirements​
Data security, privacy, and interoperability must be ensured, particularly as sDHTs become integrated into remote monitoring and decentralized clinical trials​
Real-world evidence (RWE) should support usability validation, helping to bridge the gap between regulatory approval and real-world adoption​
Regulatory bodies should work toward standardizing usability testing methodologies, ensuring consistency across clinical research, digital endpoints, and medical device evaluations​

Findings
Different Digital Health Technologies (DHTs) estimate sleep staging using data from various sensor-based sources (e.g., EEG, actigraphy, ballistocardiography), each with different properties impacting the estimation. Sleep staging algorithms are often proprietary. DHTs interpret sleep staging at different time intervals, or epochs (e.g., polysomnography uses 30-second epochs). DHT vendors transmit data at different levels, ranging from epoch-level data to pre-calculated summary data (e.g., "total sleep time").

Recommendations
Method and Signals: Ask the vendor about their method of sleep monitoring and which signals are being recorded and used, and understand the strengths and limitations of the technology.
Granularity and Epochs: Inquire about the granularity of sleep data estimated (coarse to fine grain) and the epoch length used for sleep annotations, as this informs interpretation and comparability to other research.
Thresholds and Rules: Ask what rules and thresholds are set for confirming events like sleep onset and offset to ensure certainty in the data and inform future interpretation of results.
Data Level: To align with the Core Digital Measures of Sleep, epoch-level data is preferred for further analysis and comparison between measurement systems. If only summary data is offered, ask for a detailed description of the estimation process.
Algorithms and Evidence: Ask for evidence to support the validity and reliability of the estimated sleep stages, which may include peer-reviewed manuscripts, technical documentation, and conference abstracts.

Regulatory Considerations
While not a regulatory document, the recommendations emphasize the need for vendors to provide evidence for the validity and reliability of their proprietary sleep staging algorithms. This evidence, which can be found in peer-reviewed literature or technical documentation, is crucial for establishing confidence in the results arising from the technology, and can be used for inclusion in, for example, regulatory documents.

Findings
Digital health measures for ADRD must align with patient and care partner priorities, including functional daily activities such as remembering object locations and maintaining speech fluency.
Sensor placement, data collection modalities, and algorithmic interpretation significantly impact the accuracy and reliability of digital measures.
While digital cognitive and behavioral assessments have strong potential as clinical endpoints, standardization is needed to ensure regulatory acceptance.
Sleep and mobility disruptions in ADRD can be measured with actigraphy, EEG, and ambient sensor-based approaches, but usability considerations are crucial.
Metadata, including environmental conditions and patient comorbidities, must be accounted for to ensure valid interpretations of digital measures in both research and clinical practice.

Recommendations
Researchers and technology developers should adopt standardized ontologies for digital measures to improve consistency across studies and regulatory submissions.
Digital biomarkers should be selected and validated with reference to patient and care partner needs, ensuring they reflect meaningful aspects of health.
Considerations such as sensor placement, data processing methods, and cultural neutrality of cognitive assessments must be accounted for in study designs.
Clinical trials should incorporate digital health technologies as both exploratory endpoints and potential screening tools for ADRD progression.
Further research is needed to refine algorithms for sleep, mobility, and speech-based digital biomarkers to enhance their predictive power for cognitive decline.

Regulatory Considerations
Digital measures of sleep and mobility have been recognized as potential clinical trial endpoints by regulatory agencies such as the FDA.
Standardized reporting and frameworks should be followed to ensure interoperability and data integrity in digital health studies.
Developers must document and validate scoring algorithms used for cognitive and behavioral assessments to meet regulatory expectations.
Data privacy and security regulations must be adhered to, particularly when collecting real-world behavioral and biometric data.
Ongoing validation and real-world evidence generation are necessary to establish digital measures as reliable clinical and regulatory endpoints in ADRD research.

Findings
Sleep disturbances are common across multiple therapeutic areas, making standardized digital measures essential for cross-condition research.
Measurement accuracy varies depending on sensor placement, algorithms, and contextual factors such as sleep environment.
While home-based digital sleep tracking improves accessibility, challenges remain in ensuring consistency with clinical polysomnography.
Digital measures of sleep provide new opportunities for continuous and longitudinal monitoring, but standardization in data collection and interpretation is needed.
Stakeholders, including regulatory agencies, increasingly recognize digital sleep biomarkers, but additional validation is required to ensure widespread adoption.

Recommendations
Researchers and clinicians should integrate core digital sleep measures into study designs to improve data comparability across trials and clinical contexts.
Algorithm transparency and validation protocols should be established to enhance the accuracy of digital sleep monitoring tools.
Regulatory engagement should be prioritized early in the development process to ensure that digital sleep measures meet evidentiary standards.
Multi-stakeholder collaboration, including patient and care partner input, is essential to ensure sleep measures reflect meaningful aspects of health.
Further research is needed to refine wearable and sensor-based technologies to improve real-world applicability and clinical utility of digital sleep biomarkers.

Regulatory Considerations
The FDA and other regulatory bodies increasingly acknowledge sleep measures as potential clinical endpoints, but clear validation frameworks are necessary.
Digital sleep measures should align with industry standards such as HL7 to ensure interoperability and data integrity.
Data privacy and security regulations must be followed, particularly for continuous sleep monitoring in real-world settings.
Post-market validation and real-world evidence generation are critical to support regulatory acceptance of digital sleep biomarkers.
Developers must document the derivation of sleep measures, including algorithmic processing and sensor accuracy, to meet regulatory review requirements.

Findings
Usability gaps in sDHTs remain a barrier to adoption, with many technologies failing to prioritize ease of use, accessibility, and diverse user needs​
Human-centered design is critical for ensuring that digital health solutions are intuitive, functional, and scalable across different healthcare environments​
Standardized usability metrics for evaluating digital health technologies are lacking, leading to inconsistent reporting and validation of usability outcomes​
Use-related risk analysis is essential to identifying and mitigating risks associated with user errors, ensuring the safety and effectiveness of sDHTs​
The V3+ framework provides a structured approach to integrating usability validation into digital health technology development, aligning with global regulatory expectations​

Recommendations
Developers should incorporate human-centered design principles from the outset, ensuring that usability, accessibility, and user needs are central to sDHT development​
Usability validation should be standardized, with clear methodologies for measuring usability, including satisfaction, ease of use, efficiency, and error mitigation​
Regulatory and clinical stakeholders should collaborate on defining best practices for usability evaluation, ensuring that digital endpoints are both meaningful and scalable​
Risk analysis should be iterative, with developers continuously refining their technologies based on real-world user feedback and testing​
The usability validation component of V3+ should be widely adopted to ensure that digital clinical measures meet patient-centered, regulatory, and technical expectations​

Regulatory Considerations
Regulators are emphasizing the need for usability validation to ensure that digital endpoints are both clinically relevant and patient-friendly​
sDHTs must comply with human factors engineering guidelines, aligning with global regulatory frameworks such as ISO 9241-210 and FDA usability requirements​
Data security, privacy, and interoperability must be ensured, particularly as sDHTs become integrated into remote monitoring and decentralized clinical trials​
Real-world evidence (RWE) should support usability validation, helping to bridge the gap between regulatory approval and real-world adoption​
Regulatory bodies should work toward standardizing usability testing methodologies, ensuring consistency across clinical research, digital endpoints, and medical device evaluations​

Findings
Analytical validation is critical for ensuring digital clinical measures align with regulatory and scientific expectations, particularly when no established reference measures exist.
Novel digital measures require flexible validation approaches, as traditional clinical reference measures often do not directly correspond to digital endpoints​
Statistical methodologies must be tailored to the nature of digital measures, using approaches such as factor analysis, regression modeling, and latent variable estimation​
Regulatory engagement is crucial early in the validation process to align evidentiary standards and facilitate market adoption​
The validation process must be context-specific, considering population characteristics, data collection settings, and sensor variability to ensure reliability across diverse applications​.

Recommendations
Developers should follow a stepwise approach in designing validation studies, incorporating existing reference measures, novel comparators, and statistical validation techniques.
Regulatory authorities should provide clearer guidance on acceptable validation methodologies, particularly for novel digital endpoints.
Analytical validation must be tailored to the intended use environment, ensuring that sensor-based measures capture meaningful health outcomes in real-world settings​.
Multi-stakeholder collaboration (regulators, payers, researchers, and patients) should be prioritized to create consensus on validation strategies and market access pathways.
Machine learning and AI models used for digital clinical measures should undergo rigorous evaluation to mitigate bias and improve interpretability in healthcare decision-making​.

Regulatory Considerations
Digital endpoint validation must incorporate both traditional statistical measures and novel validation frameworks, ensuring credibility in regulatory submissions.
FDA and international regulators encourage early engagement to discuss validation plans, data requirements, and evidentiary thresholds for digital measures​.
Real-world evidence (RWE) and real-world data (RWD) should be leveraged to support regulatory submissions and post-market surveillance of digital health innovations​.
Validation studies should align with global regulatory standards, such as ISO, FDA’s digital health guidance, and European Medical Device Regulations (MDR).
Data privacy, security, and compliance with regulations like HIPAA and GDPR are critical considerations when deploying and validating digital clinical measures​

Findings
A need for additional regulatory clarity specific to DHT-derived endpoints.
The official clinical outcome assessment qualification process is impractical for the biopharmaceutical industry.
A lack of comparator clinical endpoints.
A lack of validated DHTs and algorithms for concepts of interest.
A lack of operational support from DHT vendors.

Recommendations
Engage key stakeholders early.
Incorporate DHT-derived endpoints in early-phase trials and observational studies.
Invest in COA development initiatives.
Engage technology manufacturers early in the development process.

Regulatory Considerations
The EMA published a Q&A document on DHT use in clinical trials.
The FDA released guidance on collecting patient data remotely using DHTs.
The FDA established the Digital Health Center of Excellence to facilitate early regulatory engagement.

Findings
There is a need for a structured framework to leverage prior work in the use of DHTs in clinical trials.
The current body of evidence supporting DHTs is growing, but there is a lack of clarity on how to effectively utilize this evidence.
The V3 framework provides a process for validating DHTs, but its application across different medical product development programs is inconsistent.

Recommendations
Implement a framework to reuse analytical and clinical validation data for existing DHTs.
Encourage early and continuous communication with regulatory health authorities.
Leverage prior work to share best practices and consistent approaches in employing DHTs.
Use the V3 framework to ensure DHTs are fit-for-purpose in clinical trials.
Develop a strategic approach to incorporate DHTs and digitally derived endpoints within clinical development programs.

Regulatory Considerations
Sponsors should ensure their plans to leverage prior work are endorsed by regulatory health authorities.
Alignment with FDA guidance on digital health technologies is crucial.
The regulatory status of the DHT and its intended use should be clearly defined and considered in clinical trial applications.

Findings
Nocturnal scratch is a clinically relevant behavior that impacts sleep quality, skin integrity, and overall disease burden in conditions like AD.
Traditional clinical outcome assessments (COAs) often fail to adequately measure scratching behavior, making digital measurement an important complement.
Digital health technologies, including wearables and sensor-based monitoring, enable passive and objective measurement of scratch behavior without relying on patient recall.
Regulatory agencies emphasize the importance of validation, ensuring digital measures are fit-for-purpose and aligned with patient needs.
Privacy, security, and compliance considerations must be prioritized, particularly in decentralized clinical trials using real-world data collection methods.

Recommendations
Digital measurement of nocturnal scratch should be integrated as an endpoint in clinical trials to capture patient-relevant outcomes objectively.
Sensor-based tools must undergo validation processes, including analytical and clinical validation, to ensure accuracy and reliability in different populations.
Stakeholders should align terminology and measurement definitions to support consistency across studies and regulatory submissions.
Usability testing with patients is critical to ensuring that wearable devices are practical and minimally burdensome.
Clinical trials should incorporate data privacy protections and clear informed consent processes to safeguard patient information.

Regulatory Considerations
FDA encourages early engagement to discuss digital endpoints, particularly through the Critical Path Innovation Meeting (CPIM) process.
Digital tools used for clinical investigations should align with 21 CFR Part 11 compliance for electronic records and data integrity.
Sponsors should ensure that digital health technologies used in trials meet validation criteria, including fit-for-purpose assessment and clinical relevance.
Privacy regulations, including GDPR and HIPAA, must be considered when handling patient data collected via wearable sensors.
Post-market monitoring and long-term validation studies are recommended to ensure continued accuracy and reliability of nocturnal scratch measurements.

Open source: Core Digital Measures of Nocturnal Scratch

Findings
Novel digitally-derived endpoints can provide more reliable data, increase trial efficiency, and enhance patient centricity.
Selecting appropriate outcome measures that are meaningful to patients and clinicians is critical to success.
Developing these endpoints requires a resource-intensive, systematic approach to meet stakeholder needs.
Demonstrating validity and utility of novel endpoints poses unique challenges, especially for new measures without established validation standards.
Sharing lessons learned and promoting transparency can advance the field by enabling collaboration and establishing standards.

Recommendations
Focus on measures that are meaningful to patients and clinically relevant by incorporating both patient and clinician perspectives.
Select technology after identifying the appropriate outcome to ensure alignment between the technology and trial objectives.
Engage with regulators early and often to ensure endpoint acceptance and alignment with regulatory requirements.
Include digitally-derived endpoints in early-phase trials and observational studies to validate their fit-for-purpose status.
Encourage knowledge sharing and collaboration among stakeholders to establish shared standards and accelerate adoption.

Regulatory Considerations
Engage with FDA, EMA, or other regulatory bodies during early stages of endpoint development to gather critical input.
Use established regulatory frameworks, such as Investigational New Drug (IND) or Investigational Device Exemption (IDE), for guidance on endpoint use in pivotal trials.
Validate technologies to meet performance characteristics, ensuring outputs correspond to clinical concepts of interest.
Include digitally-derived endpoints in exploratory studies to build evidence for their regulatory approval.
Reference resources such as the FDA and EMA guides for navigating endpoint-related regulatory interactions.

Findings
Digital health measures must be grounded in patient priorities, ensuring that they capture meaningful aspects of health.
Variability in symptoms, patient experiences, and disease progression necessitates adaptable and inclusive digital measurement strategies.
Sensor technologies must be carefully evaluated for accuracy, reliability, and suitability for specific clinical applications.
Digital measures can support multiple endpoints, requiring clear definitions to ensure consistency and interoperability.
The validation of digital measures must integrate statistical and clinical significance to support regulatory acceptance.

Recommendations
Patient perspectives should be prioritized when designing and selecting digital clinical measures.
Digital endpoints should align with clinical goals and be clearly defined to ensure relevance across different conditions.
Technical specifications of sensors must be assessed rigorously to ensure appropriate data quality and integrity.
Developers should collaborate with regulatory agencies early to streamline the validation and approval of digital measures.
Standardized methodologies should be established to ensure consistency in evaluating digital health technologies.

Regulatory Considerations
Digital endpoints should be validated using rigorous scientific and regulatory frameworks to ensure clinical applicability.
Sensor-based measures must comply with data integrity standards and regulatory requirements for digital health technologies.
Interoperability and standardization of digital measures are necessary to facilitate regulatory submissions and cross-study comparisons.
Stakeholders should leverage real-world evidence (RWE) to support regulatory decision-making for digital health innovations.
Privacy and security considerations must be addressed to ensure compliance with HIPAA, GDPR, and other data protection regulations.

Findings
A unified digital measurement lexicon is critical for clear communication across stakeholders during development and evaluation processes.
Stakeholder and patient engagement is essential for identifying meaningful aspects of health (MAH) and defining context-specific concepts of interest (COI).
Establishing proof of concept via observational studies or exploratory trial phases de-risks investment and demonstrates feasibility.
Evaluation frameworks such as V3 ensure that digital measures meet analytical and clinical validation requirements, even in the absence of established comparators.
The lack of robust comparators in underserved conditions creates both challenges and opportunities for the development of novel digital measures.

Recommendations
Engage Early and Continuously: Involve patients, caregivers, clinicians, and other stakeholders early to identify MAH and COI that align with patient needs and trial objectives.
Adopt V3 Framework: Follow the V3 process for verification, analytical validation, and clinical validation to ensure measures are fit for purpose.
Design Iterative Proof of Concept Studies: Use small-scale studies or exploratory trial phases to validate the technical and clinical feasibility of digital measures.
Seek Early Regulatory Engagement: Initiate discussions with regulatory agencies (e.g., FDA or EMA) early in the evaluation process to refine applications and address potential challenges.
Collaborate Across Stakeholders: Foster multi-stakeholder collaboration to pool expertise and resources, especially for challenging therapeutic areas or underserved populations.

Regulatory Considerations
Use tools like FDA's Drug Development Tool Qualification Program or EMA's Innovation Task Force to refine evidence requirements and address legal or technical challenges.
Understand COI and COU: Tailor digital measures to specific contexts of use and intended applications, which determine whether measures are classified as biomarkers or COAs.
Demonstrate that digital measures are valid and reliable through rigorous analytical and clinical validation studies.
Use longitudinal clinical studies to gather evidence supporting the use of digital measures for regulatory decision-making.
Work within consortia to align standards and generate shared evidence, particularly for challenging use cases or rare conditions.