Skip to content

Findings
Usability is a multi-domain concept that requires a combination of methods for evaluation. Evaluations fall into two types: formative (for design modification of prototypes) and summative (for demonstrating usability of the final product to a representative user sample). The user experience metrics fall into several domains, including: Satisfaction, Usefulness, Ease of use, Learnability, Efficiency, Memorability, Understandability, Actionability, Readability, and Use-errors. Metrics related to Satisfaction and Usefulness are always subjectively reported by users.

Recommendations
Developers should select metrics based on the specific usability-related domain being evaluated.
Subjective Data (e.g., Satisfaction, Usefulness): Capture through qualitative surveys, quantitative surveys (scales), interviews, focus groups, and think-aloud evaluations .
Objective Data (e.g., Ease of use, Use-errors): Capture through direct or indirect observation (e.g., counting steps/attempts, timing task completion), or by using data generated by the sDHT (e.g., error reports, timestamps, page load times).
Time-based Metrics: Evaluate Learnability (ease of first use), Efficiency (ease with experience), and Memorability (ease after non-use) by measuring ease of use at different points in time .
Information Presentation: If the sDHT presents clinical data or written information (instructions, warnings), evaluate Understandability, Actionability, and Readability .
Use-errors: Objectively capture the number, type, and recoverability of use-errors (actions, or lack thereof, that may result in harm) via observation and sDHT data, noting that “use-error” is preferred to “user-error”.

Regulatory Considerations
While this guide does not reference regulatory bodies like the FDA, it is part of the V3+ framework and recommends that researchers prioritize essential documents like the use specification and use-related risk analysis before designing a usability study. Summative evaluations demonstrating usability against a representative user sample under intended use conditions are the standard for demonstrating product fitness.