Findings
Different Digital Health Technologies (DHTs) estimate sleep staging using data from various sensor-based sources (e.g., EEG, actigraphy, ballistocardiography), each with different properties impacting the estimation. Sleep staging algorithms are often proprietary. DHTs interpret sleep staging at different time intervals, or epochs (e.g., polysomnography uses 30-second epochs). DHT vendors transmit data at different levels, ranging from epoch-level data to pre-calculated summary data (e.g., "total sleep time").

Recommendations
Method and Signals: Ask the vendor about their method of sleep monitoring and which signals are being recorded and used, and understand the strengths and limitations of the technology.
Granularity and Epochs: Inquire about the granularity of sleep data estimated (coarse to fine grain) and the epoch length used for sleep annotations, as this informs interpretation and comparability to other research.
Thresholds and Rules: Ask what rules and thresholds are set for confirming events like sleep onset and offset to ensure certainty in the data and inform future interpretation of results.
Data Level: To align with the Core Digital Measures of Sleep, epoch-level data is preferred for further analysis and comparison between measurement systems. If only summary data is offered, ask for a detailed description of the estimation process.
Algorithms and Evidence: Ask for evidence to support the validity and reliability of the estimated sleep stages, which may include peer-reviewed manuscripts, technical documentation, and conference abstracts.

Regulatory Considerations
While not a regulatory document, the recommendations emphasize the need for vendors to provide evidence for the validity and reliability of their proprietary sleep staging algorithms. This evidence, which can be found in peer-reviewed literature or technical documentation, is crucial for establishing confidence in the results arising from the technology, and can be used for inclusion in, for example, regulatory documents.

Findings
Vendor selection should assess 13 categories, including General organizational operation and products, Quality Management Principles, Practices of product or service design, Device supply and provisioning, Account provisioning, Study specific materials, Live trial support, Trial closeout activities, Data handling/processing and data flow (GDPs), Device and Data (sensors + raw data) algorithm accessibility, Interoperability/Integration, Validation/Clinical Relevance/standard of documentation, and Cybersecurity. Key aspects to consider include having Validation and verification of the device and algorithm in place, ability to support multiple countries, an established quality management system. Vendors must assure maintained data integrity and quality, and provide evidence of Good Clinical Practice (GCP) compliance. Robust practices in Good manufacturing practice, Good product development practices (for hardware and software, including software lifecycle documentation), and Good scientific practices are required.

Recommendations
Sponsors should engage with vendors early in study design to tailor the technology capabilities and data requirements to patient needs and preferences. Vendors are typically responsible for device verification and analytical validation, but collaboration with sponsors and other stakeholders on clinical validation is beneficial to establish validation thresholds, specific needs of target clinical populations, and acceptability and usability of the technology. Sponsors should enable a feedback loop from patients back to vendors to improve technology for specific target populations. Sponsors or researchers should prioritize access to high-fidelity and sensor-level data to enable novel research and assessment of additional health aspects. Specific inquiries for vendors should cover: measurements offered (Accelerometry output for scratch detection, sleep measurement, environmental factors, and vitals), device material and safety testing (irritation/sensitization), usability/patient burden (e.g., disturbance during sleep), and applicability to Pediatrics, different ethnic groups, and different skin colors.

Regulatory Considerations
For software development, vendors should document and monitor the software lifecycle for quality, and demonstrate that algorithms have been tested with appropriate datasets. Assurances of GCP compliance are necessary. Vendors should demonstrate that their manufacturing practices ensure devices from different batches provide the same result measurements. Collaboration on clinical validation with sponsors and other stakeholders is a key component to generate the necessary evidence for the Validation/Clinical Relevance/standard of documentation requirement for regulatory purposes.

Findings
The lack of standardization in vendor onboarding processes increases operational inefficiencies for sponsors and vendors.
Essential topics such as data security, quality management systems (QMS), and validation studies are under-addressed in ad hoc vendor assessments.
Cybersecurity and patient data privacy, especially compliance with GDPR, HIPAA, and global regulations, require enhanced focus during vendor evaluations.
Tailoring vendor assessments to specific trial requirements and patient populations is critical for effective implementation of digital health tools.
Greater collaboration between sponsors and vendors can improve operational alignment and mitigate risks during trials.

Recommendations
Utilize the 13 vendor assessment categories as a baseline for customizing questionnaires to meet specific project needs.
Establish standardized templates for evaluating data privacy, regulatory compliance, and patient-facing user experience.
Prioritize cybersecurity measures, including penetration testing, access management, and encryption standards, as a core assessment criterion.
Implement continuous feedback loops during vendor selection and onboarding to refine assessment processes and address emerging risks.
Encourage industry collaboration to evolve and expand the open-source framework based on practical implementation experiences.

Regulatory Considerations
Ensure all vendors adhere to relevant global standards, including 21 CFR Part 11, GDPR, and HIPAA, for data security and compliance.
Verify the regulatory status of medical devices and algorithms used in digital health solutions, including certifications such as ISO 13485 and IEC 62304.
Require documentation of informed consent processes and adherence to regional data protection regulations for patient data handling.
Align vendor capabilities with regulatory guidelines for clinical trial endpoints, emphasizing validation studies and clinical relevance.
Maintain transparent and audit-ready documentation for inspections and compliance verifications.

Findings
Advantages of MCTs: Investigators highlighted streamlined study operations, remote data capture, and higher-quality, real-time data collection as key benefits. MCTs were also noted for their potential to reduce participant burden by enabling remote participation.
Challenges of MCTs: Investigators reported increased operational challenges, such as device setup, maintenance, and troubleshooting. They also noted time burdens for staff and uncertainties regarding data quality, including potential biases and technical malfunctions.
Support Needs: Investigators emphasized the need for technical support, comprehensive training for staff and participants, and adequate budgetary planning to address additional costs associated with MCTs.
Participant Considerations: While MCTs offer convenience and engagement opportunities for participants, challenges include the intrusiveness of data capture, technology adoption barriers, and potential negative impacts of real-time data access on participant behavior.
Recommendations: Investigators stressed the importance of collaborative relationships between sponsors and sites, user-friendly technology selection, and participant-centric trial designs.

Recommendations
Improve Training and Support: Sponsors should provide hands-on training for staff and participants, including troubleshooting support and device-specific materials.
Plan Budgets Appropriately: Include funds for device procurement, staff time, and technology management in trial budgets.
Enhance Technical Support: Sponsors should establish centralized technical support systems to address technology-related issues during trials.
Select Participant-Friendly Technologies: Prioritize devices that are intuitive, minimally intrusive, and suitable for the target population's needs.
Engage Stakeholders Early: Collaborate with investigators, participants, and sponsors during trial planning to align expectations and address potential challenges.

Regulatory Considerations
Data Security: Ensure data collected by mobile technologies comply with privacy and security regulations, and communicate these measures to IRBs.
Device Validation: Validate devices for the intended trial context to ensure reliability and minimize technical risks.
Participant Communication: Clearly inform participants about how their data will be used and provide transparency regarding data access.

Findings
Sufficient resources must be allocated, including infrastructure costs, training, and site reimbursement, to ensure smooth PT deployment.
PTs must be intuitive, validated, and able to withstand technical or environmental challenges to avoid burdening patients and sites.
PTs must comply with data privacy laws (e.g., GDPR, HIPAA) and regulatory standards (e.g., 21 CFR Part 11), and address import restrictions and age limitations.
Scaling PTs requires plans for device maintenance, multilingual support, and consistent availability across geographies and populations.
Sites need adequate training, realistic responsibilities, and clear workflows to avoid overburdening site staff and ensure patient compliance.

Recommendations
Involve key stakeholders (e.g., clinical technologies, regulatory affairs, site relations) early in the planning process to address potential challenges.
Identify risks related to usability, compliance, and data integrity, and establish mitigation strategies before implementation.
Provide tailored training materials for patients and site staff, ensuring clarity and accessibility in multiple formats and languages.
Develop Clear Vendor Contracts: Clearly outline responsibilities for maintenance, data management, and support in vendor contracts to avoid operational ambiguities.
Create Scalability Plans: Address challenges like multilingual support, long-term device maintenance, and cross-region deployment during the initial planning stages.

Regulatory Considerations
Ensure PTs comply with GDPR, HIPAA, and other relevant data protection regulations, particularly in global trials.
Verify if PTs qualify as medical devices and adhere to corresponding regulatory frameworks.
Assess and plan for country-specific import restrictions and data privacy laws to avoid delays.
Validate PTs according to Good Clinical Practice (GCP) guidelines, ensuring reliable data generation and compliance with regulatory standards.
Account for age-related legal restrictions, ensuring PTs are suitable for all intended patient populations.

Findings
There is uncertainty regarding the regulatory acceptability of data collected from wearable devices.
There is a lack of specific regulatory guidance on implementing wearables in clinical trial protocols.
The need for evidence to demonstrate the appropriateness and clinical relevance of new endpoints derived from wearable data.

Recommendations
Identify essential properties of fit-for-purpose wearables and propose evidence needed to support their use.
Extend the FDA's definition of a PerfO to include unsupervised settings.
Ensure that any wearable device adheres to basic properties important to clinical trials, such as source data control, traceability, and security.
Provide evidence supporting the reliability, validity, and interpretability of data generated by wearable devices.

Regulatory Considerations
Market clearance/certification is not a requirement for device selection in clinical trials if evidentiary considerations are satisfied.
The need for a robust framework for adopting wearables in regulatory trials despite the lack of specific guidance.
The evidence needed to support a device and its endpoint depends on the ultimate use of the endpoint.