Findings
There is a need for more rigorous research beyond technology validation to ensure reliable real-world data capture and improved patient outcomes.
Limited translation of AI tools into medical practice despite their success in retrospective studies.
Insufficient application of social factors in clinical decision-making and DHT research.
Need for more rigorous and reproducible research designs with larger sample sizes and longer follow-up times.

Recommendations
Use the study's repository to inform future research by healthcare providers, policymakers, and the life sciences industry.
Consider how data collection methods (active or passive) complement primary study outcomes.
Conduct targeted systematic reviews to assess factors contributing to the digital divide.
Ensure greater consistency in metrics used across DHT research.

Regulatory Considerations
Manufacturers need to demonstrate the ongoing value of their products using real-world evidence.
Regulatory approvals for AI-based products are increasing, particularly for machine learning applications.

Findings
Advantages of MCTs: Investigators highlighted remote data capture, access to real-time data for monitoring, and improved data quality as major benefits. They also noted reduced participant burden due to fewer in-person visits and increased participant engagement through real-time data access.
Challenges of MCTs: Increased site burden due to additional time required for technology setup, troubleshooting, and managing high data volumes was a common theme. Participants faced challenges such as technology unfamiliarity, device management, and potential behavior changes from real-time data access.
Support Needs: Investigators emphasized the need for technical support, staff training, and increased budgetary resources to manage devices and train participants. They also highlighted the importance of clear communication about device selection and capabilities from sponsors.

Recommendations
Sponsors should supply comprehensive device training, including hands-on and supplemental materials, and establish systems for ongoing technical support throughout the trial.
Include funds for device management, staff training, and participant support to accommodate the additional demands of MCTs.
Prioritize user-friendly devices to minimize participant burden and improve adherence.
Collaborate with investigators and participants during trial planning to ensure technologies align with study objectives and participant needs.
Focus on in-person, hands-on training to ensure staff and participants are comfortable with the technologies.

Regulatory Considerations
Devices must meet data security and safety requirements to address Institutional Review Board (IRB) concerns.
Provide detailed information about device safety, storage, and capabilities to ensure compliance with regulatory standards.
Clearly communicate data access levels to participants, minimizing risks of data misinterpretation.

Findings
The lack of standardization in vendor onboarding processes increases operational inefficiencies for sponsors and vendors.
Essential topics such as data security, quality management systems (QMS), and validation studies are under-addressed in ad hoc vendor assessments.
Cybersecurity and patient data privacy, especially compliance with GDPR, HIPAA, and global regulations, require enhanced focus during vendor evaluations.
Tailoring vendor assessments to specific trial requirements and patient populations is critical for effective implementation of digital health tools.
Greater collaboration between sponsors and vendors can improve operational alignment and mitigate risks during trials.

Recommendations
Utilize the 13 vendor assessment categories as a baseline for customizing questionnaires to meet specific project needs.
Establish standardized templates for evaluating data privacy, regulatory compliance, and patient-facing user experience.
Prioritize cybersecurity measures, including penetration testing, access management, and encryption standards, as a core assessment criterion.
Implement continuous feedback loops during vendor selection and onboarding to refine assessment processes and address emerging risks.
Encourage industry collaboration to evolve and expand the open-source framework based on practical implementation experiences.

Regulatory Considerations
Ensure all vendors adhere to relevant global standards, including 21 CFR Part 11, GDPR, and HIPAA, for data security and compliance.
Verify the regulatory status of medical devices and algorithms used in digital health solutions, including certifications such as ISO 13485 and IEC 62304.
Require documentation of informed consent processes and adherence to regional data protection regulations for patient data handling.
Align vendor capabilities with regulatory guidelines for clinical trial endpoints, emphasizing validation studies and clinical relevance.
Maintain transparent and audit-ready documentation for inspections and compliance verifications.

Findings
Key considerations include accuracy, precision, sampling frequency, resolution, and data processing. Metadata and communication protocols must ensure reliable and secure data collection.
Sponsors must assess data access, security, and compliance with regulations like 21 CFR Part 11. Clarity on manufacturer and sponsor responsibilities is essential for maintaining data integrity.
Safety risks should be minimized, especially for vulnerable populations. Specifications should ensure that devices pose minimal risks when used solely for data capture.
Human Factors: Acceptability, tolerability, and usability directly impact participant recruitment and adherence. Feasibility studies can help evaluate these factors in target populations.
Operational Considerations: Firmware updates, failure rates, battery life, and customer support must be planned for to avoid disruptions in data collection and participant experience.
Non-Performance Specifications: Cost and customer service must be accounted for, ensuring smooth implementation and user support.

Recommendations
Tailor DHT selection to trial needs, focusing on measurement accuracy, precision, and reliability.
Engage sponsors, technology manufacturers, and patient groups to align specifications with practical and clinical requirements.
Ensure compliance with regulatory standards and implement robust processes for secure data transfer and storage.
Test DHTs for usability, tolerability, and operational reliability in representative populations before full-scale implementation.
Develop clear protocols for managing firmware updates, device malfunctions, and participant support to ensure trial continuity.

Regulatory Considerations
Ensure all data management processes comply with regulatory requirements like 21 CFR Part 11 and align with FDA guidance.
Validate DHTs within the target population to confirm their reliability and relevance for the specific trial context.
Clearly communicate how data will be used and shared to maintain ethical standards and informed consent compliance.
Minimize participant risks by selecting devices with proven safety profiles and addressing potential vulnerabilities during feasibility testing.

Findings
Feedback on training for both sites and patients is a critical focus, including its clarity, duration, and accessibility.
Common site challenges include managing logistics, technical troubleshooting, and adapting workflows to accommodate patient-facing technologies.
Patient Experience: The questionnaire captures patient compliance, usability issues, and burden, particularly in remote or decentralized trial setups.
Feedback is collected at three distinct points—early (training), mid-study (technical issues), and post-study (overall impressions)—to address evolving challenges.
Strong communication and timely support from sponsors and vendors are emphasized as critical to resolving issues and maintaining trial integrity.

Recommendations
Incorporate PTSFQ into Trial Protocols: Sponsors should integrate this feedback tool into study plans to systematically capture insights from sites.
Provide comprehensive, multi-format training tailored to diverse learning needs for both site staff and patients.
Establish clear, accessible points of contact for technical and logistical support, ensuring rapid resolution of issues.
Use patient feedback to minimize burden, improve device usability, and ensure compatibility with daily routines.
Regularly review feedback from all PTSFQ sections to identify trends, address issues early, and refine technology use in ongoing and future trials.

Regulatory Considerations
Not provided.

Findings
The selection of DHTs must align with the specific goals of the trial, focusing on unmet patient or scientific needs.
A specification-driven approach, rather than solely relying on a technology's regulatory status, ensures alignment with trial requirements.
Verification and validation are distinct processes; both are critical to confirm the reliability and clinical relevance of DHTs.
Pre-trial feasibility studies help identify potential issues, such as wear-time compliance or usability concerns, before full implementation.
DHTs can alter participant interactions and trial workflows, necessitating clear communication, training, and management plans.

Recommendations
Define Measurement Goals Before Selection: Ensure that the decision to use a DHT is based on unmet needs or the promise of reducing trial burdens.
Adopt a Specification-Driven Selection Process: Tailor DHT selection to technical performance, participant needs, and study-specific requirements.
Verify and Validate Technologies Thoroughly: Collaborate with manufacturers to ensure DHTs are tested in both controlled and real-world settings and validated for the target population.
Conduct Feasibility Studies: Test DHTs for tolerability, usability, and compliance within the specific trial context to identify and address issues early.
Prepare for Operational Challenges: Develop a robust management plan with standard operating procedures (SOPs) to address potential failures and ensure smooth implementation.

Regulatory Considerations
The regulatory status of a DHT should not solely drive its selection; instead, focus on its ability to meet trial specifications.
Ensure transparent collaboration with manufacturers to document DHT performance characteristics and limitations.
Validate endpoints and DHT data to align with evidentiary standards for regulatory submissions.
Use feasibility studies and SOPs to ensure that DHTs comply with regulatory and operational requirements during trials.

Findings
Early and iterative engagement with EMA helps developers refine data generation plans, identify multidisciplinary expertise, and ensure the adequacy of early-stage data.
Clearly defining the concept of interest, context of use, and clinically meaningful change is essential for qualifying digital measures.
Comprehensive documentation should cover benefit-risk impacts, reliability, and validity of digital health technologies, avoiding overly detailed technical specifications that could invalidate qualification during updates.
Risk assessments of technology changes and updates, akin to approaches used for manufacturing changes, are crucial during regulatory reviews.
Support for collaborative groups, such as consortia and trade associations, helps aggregate and harmonize data to progress regulatory applications.

Recommendations
Establish early contact with EMA to align on regulatory requirements, optimize data generation, and ensure continuity in assessment teams.
Identify the digital technology's impact on benefit-risk assessment, specifying its purpose as a novel measure or alternative to traditional methods.
Provide evidence of reliability, accuracy, repeatability, and clinical validity, ensuring sufficient detail for regulatory assessment without risking qualification during updates.
Conduct comprehensive risk assessments for changes to technology or software, following principles of ICH guidelines (Q8, Q9, Q10, Q12).
Develop user manuals and training materials to optimize implementation in clinical trials and ensure patient compliance.

Regulatory Considerations
EMA’s Remit: Focus on aspects affecting the benefit-risk assessment of medicinal products, while providing high-level information on unrelated technical parameters.
Alignment with MDR and GDPR: Ensure digital tools comply with applicable legal frameworks, including medical device regulations and data protection requirements.
Treat software and technology updates with a risk-based framework, evaluating their impact on clinical data validity and performance.
Collaborate with consortia to aggregate diverse data sources for confidential regulatory discussions, maximizing evidentiary value.
For medical devices, ensure CE marking or equivalent regulatory compliance before marketing, though it is not required during development.

Findings
Operational inefficiencies in participant recruitment and data acquisition inflate costs and extend timelines.
Disparities in access to research due to geographic and mobility constraints limit participant diversity.
Many digital biomarkers require further validation for use in clinical trials.
Heightened need for security measures to protect against data breaches in digital trials.
Opportunities exist to improve clinical trials using real-world data from EHRs and IoT technologies.

Recommendations
Leverage existing technologies and research platforms to transform clinical trials.
Develop partnerships with technology and computational communities.
Create standard protocol templates for automation in recruitment, retention, and data collection.
Develop validation models for new devices and analyses using existing trials.
Invest in the next generation workforce in medicine, technology, and clinical research.

Regulatory Considerations
Address data privacy and security concerns in digital trials.
Provide guidance for IRBs on consenting requirements, reporting, and oversight in digital trials.
Develop empirical research on the risks and benefits of digital trials.
Educate IRBs on digital technology and its implications for clinical trials.

Findings
There are significant risks associated with connected sensor technologies that exceed current evaluation capabilities, including validation, security practices, data rights and governance, utility and usability, and economic feasibility.
Existing evaluation frameworks are inadequate for the unique challenges posed by digital health technologies.
The regulatory environment for digital specimens is not well-established, leading to ambiguity in oversight.

Recommendations
Develop a systematic and standardized evaluation framework for connected sensor technologies.
Implement a connected sensor technology label to improve transparency and decision-making.
Encourage innovation and modern regulatory oversight through updated guidelines.
Address the evolving distinction between regulated and unregulated digital health technologies.
Enhance communication infrastructure to make information more accessible to stakeholders.

Regulatory Considerations
The regulatory environment for digital health technologies is evolving, with a need for modern oversight.
There is ambiguity in the regulation of digital specimens, requiring clearer guidelines.
The FDA has issued guidances to encourage innovation and efficient oversight.

Findings
The term "clinically validated" is frequently used in marketing but lacks a clear, standardized meaning, leading to confusion. The rapid development of BioMeTs has outpaced the creation of systematic, evidence-based evaluation frameworks, creating a knowledge gap. Existing validation standards from software, hardware, and clinical development are often applied in silos and are not fully sufficient for modern BioMeTs. Evaluating a BioMeT requires assessing the entire "data supply chain," from the sensor hardware (verification) and data processing algorithms (analytical validation) to its performance against a meaningful clinical concept (clinical validation).

Recommendations
The digital medicine field should adopt the V3 (Verification, Analytical Validation, Clinical Validation) framework as a foundational evaluation standard for all BioMeTs to ensure they are fit-for-purpose. Technology manufacturers, clinical trial sponsors, and researchers should transparently report their V3 processes and results to overcome "black box" approaches and build a common evidence base. Technology manufacturers are primarily responsible for verification , while the entity developing the algorithm (e.g., manufacturer or sponsor) is responsible for analytical validation. The sponsor or clinical team using the BioMeT for a specific purpose is responsible for clinical validation in that context of use.

Regulatory Considerations
The V3 framework is designed to inform and align with the current regulatory landscape, although the regulatory pathway for a specific BioMeT depends on its intended use and marketing claims, not just its underlying technology. The 21st Century Cures Act and the concept of Software as a Medical Device (SaMD) have created new regulatory paradigms that decouple software from specific hardware. BioMeTs used to support drug development may follow a tool qualification pathway, while those marketed as standalone medical devices are subject to device clearance or approval processes. Stakeholders should engage with regulatory agencies early to determine appropriate validation approaches.