Industry spotlight
PUBLICATION
The V3 framework foundational paper. Digital medicine is an interdisciplinary field, drawing together stakeholders with expertise in engineering, manufacturing, clinical science, data science, biostatistics, regulatory science, ethics, patient advocacy, and healthcare policy, to name a few. Although this diversity is undoubtedly valuable, it can lead to confusion regarding terminology and best practices. There are many instances, as we detail in this paper, where a single term is used by different groups to mean different things, as well as cases where multiple terms are used to describe essentially the same concept. Our intent is to clarify core terminology and best practices for the evaluation of Biometric Monitoring Technologies (BioMeTs), without unnecessarily introducing new terms. We focus on the evaluation of BioMeTs as fit-for-purpose for use in clinical trials. However, our intent is for this framework to be instructional to all users of digital measurement tools, regardless of setting or intended use. We propose and describe a three-component framework intended to provide a foundational evaluation framework for BioMeTs. This framework includes (1) verification, (2) analytical validation, and (3) clinical validation. We aim for this common vocabulary to enable more effective communication and collaboration, generate a common and meaningful evidence base for BioMeTs, and improve the accessibility of the digital medicine field. (Abstract).
PUBLICATION
(DiMe) This viewpoint proposes a hierarchical framework to address inconsistencies and insufficiency in the evidence for the analytical validation of sensor-based digital health technologies (sDHTs). The framework provides a step-by-step approach for investigators to select the most appropriate reference measure for evaluating sDHT algorithms that convert sensor data into a clinically interpretable digital clinical measure. By codifying best practices, the framework aims to close a key gap in the science and help capture the greatest value of sDHTs for patient care and medical product development. The proposed approach builds on the established V3+ framework.
BEST PRACTICES BY DIME
Outlines the V3+ Framework, which includes step-by-step guidance to ensure that sensor-based DHTs meet rigorous engineering standards and reliably deliver accurate and consistent data. The playbook emphasizes the importance of documenting technical testing and verification against predefined specifications, ensuring that DHTs are “fit for purpose” and ready for use in clinical study settings. Contains a dedicated section detailing where to find evidence of analytical validation.
LIBRARY BY DIME
A comprehensive resource cataloging high-quality analytical validation studies for digital health technologies. The library aims to showcase best practices in analytical validation, supporting the implementation of DiMe’s V3+ Framework for evaluating digital health technologies.
LIBRARY BY DIME
An interactive, searchable database cataloging sensor-based digital health technologies, high-quality digital clinical measures, and measurement tools. Evidence can be filtered by validation type.
TOOLKIT BY DIME
Need to validate a novel digital endpoint, or determine whether your reference standard is right for your digital measure? Start here. The Validating Novel Digital Clinical Measures (VNDCM) Toolkit provides structured guidance, templates, and examples to help you plan and execute validation for digital clinical measures that lack prior precedent. Begin with the interactive guide, which provides a high-level decision process for designing an analytical validation study. It introduces a structured framework to select fit-for-purpose reference measures.
Next, follow the steps in the study builder to:
- Design a rigorous analytical validation study for your digital clinical measure of interest
- Use the workbook format to document your study plan
- Present your study plan and results to regulators and other stakeholders
Use the simulation toolkit when your measure of interest and reference measure do not have directly comparable units. It bundles methods for data simulation and an approach to design comparative studies.
PUBLICATION
This paper discusses key considerations for generating evidence for clinical validity through the lens of the type and intended use of a clinical measure. This paper also briefly discusses the regulatory pathways through which clinical validity evidence may be reviewed and highlights challenges that investigators may encounter while dealing with data from biometric monitoring technologies. (Abstract)
PUBLICATION
The DIA Study Endpoints Community Working Group on Mobile Sensor Technology (MST) series addresses considerations that may be useful for selecting MST for use in a clinical trial. This article describes considerations regarding the selection of MST for clinical trials including expectations around technology specifications, verification (bench testing), regulatory clearance and certification status. We identify useful statistical methods needed to establish agreement of the MST with respect to a clinical ‘gold’ standard technology in terms of accuracy and precision, and to combine data across trials, data types or device versions. In addition to describing key considerations, this manuscript also serves as a central location citing those resources where additional detail can be found.
CASE EXAMPLES BY CTTI
The aim of this use case is to describe the pathway by which a novel physical activity endpoint measured using accelerometer technology could be developed for use in conjunction with traditional clinical endpoints in regulatory submission trials in heart failure populations.
Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)
The term “clinically validated” is frequently used in marketing but lacks a clear, standardized meaning, leading to confusion. The rapid development of BioMeTs has outpaced the creation of systematic, evidence-based evaluation frameworks, creating a knowledge gap. Existing validation standards from software, hardware, and clinical development are often applied in silos and are not fully sufficient for modern BioMeTs. Evaluating a BioMeT requires assessing the entire “data supply chain,” from the sensor hardware (verification) and data processing algorithms (analytical validation) to its performance against a meaningful clinical concept (clinical validation).
Recommendations
The digital medicine field should adopt the V3 (Verification, Analytical Validation, Clinical Validation) framework as a foundational evaluation standard for all BioMeTs to ensure they are fit-for-purpose. Technology manufacturers, clinical trial sponsors, and researchers should transparently report their V3 processes and results to overcome “black box” approaches and build a common evidence base. Technology manufacturers are primarily responsible for verification , while the entity developing the algorithm (e.g., manufacturer or sponsor) is responsible for analytical validation. The sponsor or clinical team using the BioMeT for a specific purpose is responsible for clinical validation in that context of use.
Regulatory Considerations
The V3 framework is designed to inform and align with the current regulatory landscape, although the regulatory pathway for a specific BioMeT depends on its intended use and marketing claims, not just its underlying technology. The 21st Century Cures Act and the concept of Software as a Medical Device (SaMD) have created new regulatory paradigms that decouple software from specific hardware. BioMeTs used to support drug development may follow a tool qualification pathway, while those marketed as standalone medical devices are subject to device clearance or approval processes. Stakeholders should engage with regulatory agencies early to determine appropriate validation approaches.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
A Hierarchical Framework for Selecting Reference Measures for the Analytical Validation of Sensor-Based Digital Health Technologies
The quality of evidence for the analytical validation of sensor-based digital health technologies (sDHTs), which is the evaluation of algorithms converting sensor data into a clinically interpretable measure, is often inconsistent and insufficient. The existing V3+ framework codifies the overall evaluation process, which includes verification, usability validation, analytical validation, and clinical validation. To improve the scientific rigor of analytical validation, a hierarchical framework for selecting reference measures is needed because not all potential reference measures are of equal quality. The framework classifies reference measures based on attributes that contribute to reduced measurement variability, with defining and principal measures being the most rigorous due to objective data acquisition and the ability to retain source data.
Recommendations
The proposed framework sequentially moves the investigator through four steps: (1) Compile preliminary information, including the digital clinical measure, context of use (COU), algorithm requirements, and sensor verification evidence . (2) Select an existing reference measure, develop a novel comparator, or identify a set of anchor measures, prioritizing measures with the highest scientific rigor (defining → principal → manual → reported) . (3) Consider the impact of the data collection environment to determine if the analytical validation study can be conducted in the intended use environment with the highest-order measure, or if in-lab validation is necessary, ensuring the results are generalizable . (4) Describe the rationale for key study design decisions to encourage transparency for evaluators, regulators, and payers . Investigators must justify passing over a higher-ranked reference measure, generally only acceptable if the higher-ranked measure poses unacceptable risk or is not applicable to the context of use.
Regulatory Considerations
The principles of the framework for analytical validation apply regardless of the regulatory status of the sDHT (regulated medical device, low-risk general wellness apps, or research product) or its intended use (clinical care or clinical research). The framework is intended to help investigators support the most rigorous claims regarding sDHT performance, which is important for acceptance by evaluators, peer-reviewers, regulators, and payers. The categorization of the digital clinical measure as a digital biomarker or an electronic clinical outcome assessment also does not change the framework’s applicability.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Playbook Digital Clinical Measures
Successful deployment of digital clinical measures requires a shared foundation of standardized methodologies, terminology, and best practices.
The selection of digital measures must prioritize patient-centered outcomes and align with meaningful aspects of health.
Technology validation processes, including the Verification, Analytical Validation, and Clinical Validation (V3) framework, are crucial to ensuring data accuracy and reliability.
Interoperability, data security, and governance remain key challenges for digital health technologies in both research and clinical applications.
Case studies demonstrate the real-world utility of digital clinical measures in clinical research, patient care, and public health initiatives.
Recommendations
Stakeholders should follow a structured, stepwise approach to selecting and validating digital clinical measures, starting with identifying meaningful health aspects.
Digital health tools must undergo rigorous verification and validation to ensure they are fit-for-purpose and meet clinical and regulatory standards.
Patient engagement should be integrated into every stage of digital measure development to ensure the relevance and usability of selected endpoints.
Regulatory and payer engagement should occur early in the process to streamline market access and reimbursement pathways.
Organizations should adopt a proactive approach to data privacy, security, and governance, ensuring compliance with regulations such as HIPAA and GDPR.
Regulatory Considerations
The FDA and other regulatory bodies emphasize the need for clinical validation of digital measures before they can be used as primary endpoints in trials.
Standardization of digital health technologies is critical to regulatory approval, requiring alignment with frameworks such as HL7 and ISO standards.
Data security and privacy regulations must be strictly adhered to, particularly in decentralized clinical trials where remote monitoring is used.
Digital endpoint validation must include real-world evidence (RWE) to support regulatory decision-making and post-market surveillance.
Organizations must consider the evolving regulatory landscape for AI-driven health technologies, ensuring compliance with best practices for algorithmic transparency and bias mitigation.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Analytical Validation Library
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Library of Digital Measurement Products
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
VNDCM Simulation Toolkit
Analytical validation is critical for ensuring digital clinical measures align with regulatory and scientific expectations, particularly when no established reference measures exist.
Novel digital measures require flexible validation approaches, as traditional clinical reference measures often do not directly correspond to digital endpoints
Statistical methodologies must be tailored to the nature of digital measures, using approaches such as factor analysis, regression modeling, and latent variable estimation
Regulatory engagement is crucial early in the validation process to align evidentiary standards and facilitate market adoption
The validation process must be context-specific, considering population characteristics, data collection settings, and sensor variability to ensure reliability across diverse applications.
Recommendations
Developers should follow a stepwise approach in designing validation studies, incorporating existing reference measures, novel comparators, and statistical validation techniques.
Regulatory authorities should provide clearer guidance on acceptable validation methodologies, particularly for novel digital endpoints.
Analytical validation must be tailored to the intended use environment, ensuring that sensor-based measures capture meaningful health outcomes in real-world settings.
Multi-stakeholder collaboration (regulators, payers, researchers, and patients) should be prioritized to create consensus on validation strategies and market access pathways.
Machine learning and AI models used for digital clinical measures should undergo rigorous evaluation to mitigate bias and improve interpretability in healthcare decision-making.
Regulatory Considerations
Digital endpoint validation must incorporate both traditional statistical measures and novel validation frameworks, ensuring credibility in regulatory submissions.
FDA and international regulators encourage early engagement to discuss validation plans, data requirements, and evidentiary thresholds for digital measures.
Real-world evidence (RWE) and real-world data (RWD) should be leveraged to support regulatory submissions and post-market surveillance of digital health innovations.
Validation studies should align with global regulatory standards, such as ISO, FDA’s digital health guidance, and European Medical Device Regulations (MDR).
Data privacy, security, and compliance with regulations like HIPAA and GDPR are critical considerations when deploying and validating digital clinical measures
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Considerations for Analyzing and Interpreting Data from Biometric Monitoring Technologies in Clinical Trials
Limited evidence of clinical validity from pilot trials due to cost, time, and regulatory complexities.
Lack of standards for data integration across different tools and platforms.
Potential biases introduced by pre-existing algorithms.
Opaque data processing methods in BioMeTs.
Recommendations
Develop data, hardware, and software standards for BioMeTs.
Improve regulations for data rights, access, privacy, and governance.
Provide guidance on analytical methodologies for BioMeT data validation.
Regulatory Considerations
Early regulatory interactions with agencies like the FDA and EMA.
Ensuring data quality, integrity, reliability, and robustness.
Understanding regulatory pathways for BioMeTs in clinical trials.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Choosing a Mobile Sensor Technology for a Clinical Trial: Statistical Considerations, Developments and Learnings
The complexity of selecting appropriate technology due to an increasing array of devices and sensors.
Risks associated with choosing inappropriate MSTs, including susceptibility to missing data or erroneous data transmission.
The need for both manufacturers and clinical trial sponsors to ensure analytical validation supports MST use.
Recommendations
Identify a digital outcome that meets an unmet need for the planned trial or population.
Determine whether the technology is fit-for-purpose based on the measure, context of use, and classification as a medical device.
Ensure devices are reliable and reproducible for collecting required data.
Conduct statistical analysis according to a predefined analysis plan.
Consider adaptive designs to reduce resource requirements and increase study success.
Regulatory Considerations
Compliance with medical device classifications such as 510(k)s and CE marks.
Ensure devices and platforms comply with HIPAA, GDPR, and data privacy regulations.
Be aware of potential updates to technology or software that could impact trials.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
Case Study: Developing Novel Endpoints Generated Using Digital Health Technology: Heart Failure
Existing PROs for HF, such as KCCQ and MLHFQ, are insufficiently sensitive and rely on subjective assessments.
Accelerometer technology offers objective and continuous real-world data that may better capture patient activity and health.
Novel endpoints must be validated through analytical and cross-sectional studies, correlating “time walking” with HF severity and clinical outcomes.
Developing and validating these endpoints is more feasible for patients with NYHA class II/III HF due to their moderate activity levels.
Future refinements and central databases of accelerometer data will enhance endpoint development and application.
Recommendations
Use accelerometer-derived metrics, such as “time spent walking per day,” as novel endpoints to complement traditional clinical measures.
Validate novel endpoints through controlled and real-world studies, including correlating them with existing HF measures and clinical outcomes.
Include accelerometer endpoints in exploratory analyses within ongoing HF trials to gather supportive data without requiring regulatory submission.
Establish data standards and centralized databases for accelerometer-derived endpoints to streamline future development.
Collaborate across stakeholders, including patients, clinicians, investigators, and regulators, to align endpoint development with real-world applicability and regulatory requirements.
Regulatory Considerations
Demonstrate that accelerometer-derived endpoints reflect meaningful changes in patient health and correlate with established HF measures.
Validate endpoints in diverse patient populations and real-world settings to support generalizability and regulatory acceptance.
Address missing data and potential biases in accelerometer readings during endpoint analysis and validation.
Ensure endpoints align with regulatory trial design and analysis standards, including blinding and pre-specified analytical plans.
Develop frameworks for incorporating accelerometer-based endpoints into regulatory submissions alongside traditional clinical outcomes.
Some summaries are generated with the help of a large language model; always view the linked primary source of a resource you are interested in.
For reference: review the relevant regulatory guidances
Regulatory spotlight
Features essential guidance, publications, and communications from regulatory bodies relevant to this section. Use these resources to inform your regulatory strategy and ensure compliance.