Findings
The FDA classifies CDS software as Non-Device CDS only if it meets four specific criteria related to data inputs, information display, HCP support, and independent reviewability. Software functions that analyze medical images, signals from IVDs, or patterns from signal acquisition systems remain regulated as medical devices. Non-Device CDS must be intended for health care professionals and not for patients or caregivers. Automation bias and the time-critical nature of decision-making are key factors in determining whether an HCP can truly review the basis of a recommendation independently. If software provides a specific diagnostic or treatment directive rather than a list of options, it generally fails to meet the exclusion criteria.

Recommendations
Developers should ensure that software intended as Non-Device CDS provides a plain language description of the underlying algorithm and the data used for validation. The software or labeling must clearly identify the intended HCP user, the patient population, and the required input medical information. To support independent review, the software should highlight the source of its clinical recommendations, such as specific clinical practice guidelines or peer-reviewed studies. Developers are encouraged to use usability testing to verify that HCPs can understand the basis of recommendations without relying primarily on the software’s output. For multiple function products, developers should follow the FDA’s policy for assessing products that contain both device and non-device functions.

Regulatory Considerations
The FDA applies a risk-based approach to software oversight, focusing on functions that acquire or analyze complex medical data like ECG waveforms or genomic sequences. Software intended for time-sensitive or critical medical decisions is typically regulated as a device because the user lacks the time to independently verify the recommendation. The agency intends to exercise enforcement discretion for certain software functions that provide only one clinically appropriate recommendation if all other non-device criteria are met. Sponsors may use the Q-Submission process to discuss alternative approaches or clarify the regulatory status of specific software functions. Existing digital health policies continue to apply to software functions that meet the device definition, including mobile medical applications.

Findings
The quality of evidence for the analytical validation of sensor-based digital health technologies (sDHTs), which is the evaluation of algorithms converting sensor data into a clinically interpretable measure, is often inconsistent and insufficient. The existing V3+ framework codifies the overall evaluation process, which includes verification, usability validation, analytical validation, and clinical validation. To improve the scientific rigor of analytical validation, a hierarchical framework for selecting reference measures is needed because not all potential reference measures are of equal quality. The framework classifies reference measures based on attributes that contribute to reduced measurement variability, with defining and principal measures being the most rigorous due to objective data acquisition and the ability to retain source data.

Recommendations
The proposed framework sequentially moves the investigator through four steps: (1) Compile preliminary information, including the digital clinical measure, context of use (COU), algorithm requirements, and sensor verification evidence . (2) Select an existing reference measure, develop a novel comparator, or identify a set of anchor measures, prioritizing measures with the highest scientific rigor (defining → principal → manual → reported) . (3) Consider the impact of the data collection environment to determine if the analytical validation study can be conducted in the intended use environment with the highest-order measure, or if in-lab validation is necessary, ensuring the results are generalizable . (4) Describe the rationale for key study design decisions to encourage transparency for evaluators, regulators, and payers . Investigators must justify passing over a higher-ranked reference measure, generally only acceptable if the higher-ranked measure poses unacceptable risk or is not applicable to the context of use.

Regulatory Considerations
The principles of the framework for analytical validation apply regardless of the regulatory status of the sDHT (regulated medical device, low-risk general wellness apps, or research product) or its intended use (clinical care or clinical research). The framework is intended to help investigators support the most rigorous claims regarding sDHT performance, which is important for acceptance by evaluators, peer-reviewers, regulators, and payers. The categorization of the digital clinical measure as a digital biomarker or an electronic clinical outcome assessment also does not change the framework's applicability.

Findings
The traditional medical device regulatory paradigm is not designed for the adaptive nature of AI/ML technologies, which can learn and change after they are on the market. A key benefit of AI/ML is its ability to improve performance by learning from real-world data, but this also presents a unique regulatory challenge. To ensure patient safety and device effectiveness, a new, flexible regulatory framework is required that can accommodate these iterative improvements. Transparency and robust monitoring are essential to manage the risks associated with evolving algorithms.

Recommendations
The FDA proposes a "Predetermined Change Control Plan" (PCCP) to be included in premarket submissions. This plan would specify the anticipated modifications to the device (the "what") and the methodology for implementing and validating those changes (the "how"). The development of "Good Machine Learning Practice" (GMLP) is encouraged to ensure that AI/ML algorithms are developed and validated using best practices. Manufacturers should implement robust real-world performance monitoring to ensure that their devices remain safe and effective after deployment.

Regulatory Considerations
The FDA is developing a new regulatory framework tailored to the unique aspects of AI/ML-based SaMD, which will leverage a TPLC approach. The agency has issued an "AI/ML SaMD Action Plan" that outlines its multi-pronged approach, including issuing draft guidance on PCCPs and promoting the harmonization of GMLP. The FDA is actively collaborating with stakeholders to foster innovation while ensuring patient safety. The agency maintains a public list of authorized AI/ML-enabled medical devices to enhance transparency.

Findings
The document introduces a risk-based credibility assessment framework for establishing and evaluating the credibility of an Artificial Intelligence (AI) model's output when used to support regulatory decisions regarding drug safety, effectiveness, or quality. The framework outlines a 7-step process beginning with defining the question of interest and the Context of Use (COU). Credibility is defined as trust, established through evidence, in the AI model's performance for a particular COU. The credibility assessment is tailored to the AI model risk, which is a combination of model influence (the AI model's evidence contribution relative to other evidence) and decision consequence (the significance of an adverse outcome from an incorrect decision). The document highlights challenges with AI use, including variability in development datasets (training/tuning), the need for methodological transparency due to model complexity, difficulty in quantifying and interpreting uncertainty in model output, and the potential for performance change over time (data drift), which necessitates life cycle maintenance.

Recommendations
Sponsors and interested parties should define the question of interest and clearly define the COU, detailing the AI model's specific role and scope and whether other information will be used. They should assess the AI model risk (low, medium, or high) to ensure that subsequent credibility assessment activities (Step 4) are commensurate with that risk and tailored to the COU. For Step 4, the credibility assessment plan should include a description of the model, model development process (including inputs, architecture, feature selection, and rationale), and data used (training and tuning data). Development data must be deemed fit for use (relevant and reliable) to mitigate issues like algorithmic bias. The plan should also detail the model evaluation process using independent test data and include performance metrics with confidence intervals, an estimate of uncertainty, and a description of model limitations. Early engagement with the FDA is strongly encouraged to discuss model risk and the adequacy of the credibility assessment plan.

Regulatory Considerations
The risk-based credibility assessment framework is intended to help organize and document information for regulatory submissions. The required stringency of assessment activities and the level of documentation should be commensurate with the AI model risk. For AI models whose performance can change over time (e.g., in pharmaceutical manufacturing or postmarketing), sponsors must implement life cycle maintenance plans to monitor performance and manage changes in a risk-based manner. Changes to AI models should be evaluated through the manufacturer's change management system and may require re-execution of parts of the credibility assessment plan. Early engagement can be facilitated through formal meetings (e.g., Pre-IND) or other specialized programs listed in the guidance, such as the Center for Clinical Trial Innovation (C3TI), the Model-Informed Drug Development (MIDD) Paired Meeting Program, and the Emerging Technology Program (ETP) or Advanced Technologies Team (CATT).

Findings
Cytokine Release Syndrome (CRS) is a common and potentially life-threatening adverse event of immunotherapies, particularly in Oncology, complicating patient care and increasing healthcare costs. Standard-of-care inpatient monitoring for CRS is manual, intermittent, costly, and restrictive, providing an incomplete view of the syndrome’s development and progression. The use of Digital Health Technologies (DHTs) for continuous, remote monitoring of vital signs (like heart rate, respiratory rate, skin temperature, SpO2, and activity) can capture early indicators of CRS up to two hours earlier than standard episodic monitoring. This ability to collect multivariate continuous data is valuable for informing robust model development for CRS risk prediction.

Recommendations
Investigators should deploy DHTs available today to monitor vital signs and symptoms currently observed in the hospital setting, but in an outpatient or home environment. The goal is to develop Early Warning Products that assess the probability of developing CRS, providing clinical decision support. Product developers should follow a strategic roadmap that outlines milestones for building products that are clinically relevant and commercially viable. Researchers should use a common set of digital clinical measures to gather high-quality datasets and ensure comparability across studies to build more robust predictive models. Predictive algorithms should be built on a robust reference measure for analytical validation and be clinically validated with sufficient data.

Regulatory Considerations
The resources are designed to help developers build products that are clinically appropriate, regulatory-acceptable, and commercially viable. Future regulatory submissions for CRS de-risking products will benefit from aligning with this industry-wide dialogue that is being built in collaboration with the FDA. Developing a robust CRS safety biomarker could enhance the safety profile of clinical trials, increase trial access, and streamline regulatory decision-making, possibly through a qualification pathway. Products that aim for a higher level of autonomy, such as a Diagnostic that redefines current CRS grading classes, may require very high clinical evidence and likely stringent regulatory review.

Findings
Different Digital Health Technologies (DHTs) estimate sleep staging using data from various sensor-based sources (e.g., EEG, actigraphy, ballistocardiography), each with different properties impacting the estimation. Sleep staging algorithms are often proprietary. DHTs interpret sleep staging at different time intervals, or epochs (e.g., polysomnography uses 30-second epochs). DHT vendors transmit data at different levels, ranging from epoch-level data to pre-calculated summary data (e.g., "total sleep time").

Recommendations
Method and Signals: Ask the vendor about their method of sleep monitoring and which signals are being recorded and used, and understand the strengths and limitations of the technology.
Granularity and Epochs: Inquire about the granularity of sleep data estimated (coarse to fine grain) and the epoch length used for sleep annotations, as this informs interpretation and comparability to other research.
Thresholds and Rules: Ask what rules and thresholds are set for confirming events like sleep onset and offset to ensure certainty in the data and inform future interpretation of results.
Data Level: To align with the Core Digital Measures of Sleep, epoch-level data is preferred for further analysis and comparison between measurement systems. If only summary data is offered, ask for a detailed description of the estimation process.
Algorithms and Evidence: Ask for evidence to support the validity and reliability of the estimated sleep stages, which may include peer-reviewed manuscripts, technical documentation, and conference abstracts.

Regulatory Considerations
While not a regulatory document, the recommendations emphasize the need for vendors to provide evidence for the validity and reliability of their proprietary sleep staging algorithms. This evidence, which can be found in peer-reviewed literature or technical documentation, is crucial for establishing confidence in the results arising from the technology, and can be used for inclusion in, for example, regulatory documents.

Findings
Data analytics are challenging due to diverse metrics and study aims.
Most devices lack built-in software for data output.
There is a lack of comparison and validation studies for different devices and metrics.
Validation of PA metrics is difficult due to the absence of a gold standard.
The integration of various databases is needed but challenging.

Recommendations
Conduct comparison and validation studies between different brands of devices and PA metrics.
Develop standardized metrics for measuring PA.
Improve data integration methods across different studies and databases.
Focus on developing built-in software for devices to facilitate data output.
Encourage research on the validation of PA metrics.

Regulatory Considerations
1Not mentioned

Findings
Challenges in patient adherence and acceptability of digital endpoints.
Issues with algorithm validation and use in diverse populations.
Barriers due to proprietary software and lack of transparency.
Vast heterogeneity in digital endpoints and lack of standards.
Need for ongoing ethical support and consideration of environmental impact.

Recommendations
Foster multi-stakeholder cooperation and open-forum discussions.
Integrate patient needs into the design of digital health technologies.
Include implementation science expertise in research proposals.
Develop standards for selecting and reporting digital endpoints.
Provide ongoing ethical support throughout the research lifecycle.

Regulatory Considerations
Early engagement with regulators is crucial.
Understanding regulatory requirements for clinical trials versus clinical care.
Need for harmonised terminology and guidelines for digital endpoints.

Findings
Misclassification of wakefulness during sleep periods and issues with tracking outside main sleep bouts.
Bias in performance evaluation studies due to limited representation of diverse populations.
Hidden complexities in consumer-grade devices related to data access, fees, privacy, and security.

Recommendations
Carefully interpret study results based on wearable sleep-tracking technology data.
Address biases in study populations by including diverse cohorts.
Ensure proper preprocessing of data from consumer-grade devices.
Avoid inserting personally identifiable information in device settings.
Evaluate issues related to specific populations like minors.

Regulatory Considerations
Complexity of privacy laws across different countries.
Need for strategies to protect personal information in device settings.
Consideration of specific population issues, such as minors, in regulatory frameworks.

Findings
Standard assessments lack sensitivity in early stages of neurodegenerative diseases.
Challenges with the validity and quality of RMT measurements.
Issues related to equity and inclusion in deploying digital tools.
Importance of considering feasibility, acceptance, usability, and ecological validity of digital endpoints.

Recommendations
Develop regulatory strategies early on.
Ensure equity and inclusion in deploying digital tools.
Address challenges related to the validity and usability of digital endpoints.
Promote public-private partnerships to address privacy and security concerns.
Involve patients and stakeholders in the design and implementation of digital tools.

Regulatory Considerations
Acceptance of digital endpoints by regulatory authorities is crucial.
Validation with current gold standards and clinically meaningful legacy endpoints.
Ensure data security and privacy.

Findings
Digital health technologies can provide more granular, continuous, and sensitive measures compared to traditional clinical assessments.
Digital measurements can detect treatment responses earlier and with smaller sample sizes than traditional methods.
There is a lack of standardized endpoints and insufficient data to contextualize findings from digital measurements.

Recommendations
Utilize digital health technologies to increase research efficiency and reduce trial participation burden.
Develop frameworks for regulatory acceptance of digital endpoints.
Continue research to establish meaningful changes9 and standardize endpoints based on digital measurements.

Regulatory Considerations
Establish evidentiary criteria for using digital measurements as surrogate endpoints.
Address the need for regulatory frameworks to support the use of digital health technologies in clinical trials.

Findings
Not all CDS software is regulated as a medical device; the FDA applies specific criteria to determine its classification.
CDS software functions are excluded from the device definition if they meet all four criteria in section 520(o)(1)(E) of the FD&C Act.
Automation bias in decision-making poses a risk, particularly in time-critical scenarios, and influences regulatory considerations.
Clear labeling and transparency about the basis for recommendations are essential for enabling HCPs to make independent decisions.
Software functions that provide specific diagnostic outputs or time-critical directives typically fail to meet the criteria for Non-Device CDS.

Recommendations
Clearly define the intended use, user population, and input medical information for CDS software in labeling.
Ensure that software provides transparent and plain language descriptions of algorithms, data sources, and validation results.
Avoid presenting specific treatment or diagnostic directives to ensure the software supports rather than replaces HCP judgment.
Include sufficient information to allow HCPs to independently review and understand the basis for software recommendations.
Engage with the FDA early in the development process for software functions with potential regulatory oversight.

Regulatory Considerations
CDS software functions that meet all four criteria under section 520(o)(1)(E) of the FD&C Act are excluded from FDA’s definition of a device.
Software intended for time-critical decision-making or replacing HCP judgment is generally considered a device.
Developers must ensure that software labeling and functionality align with the criteria for Non-Device CDS.
Transparency in data sources, algorithm logic, and validation methods is required to enable independent HCP decision-making.
The FDA may request additional information or oversight for software that poses significant risks to patient safety.