Abstract

Decision support system (DSS) development and evaluation in healthcare is both essential and challenging. In the United States, the U.S. Food and Drug Administration encourages comprehensive evaluation of a DSS before being considered for approval to use in healthcare settings. Development of contextually relevant evaluation scenarios requires substantial effort to accurately reflect actual care. Use of a consistent methodology such as cognitive systems engineering (CSE) ensures solution features can be traced back through requirements and analyses to the original data. Completing evaluations in settings with staff, equipment and procedures as close as possible to actual practice ensures results have high validity. This chapter reports on two different projects using usability assessment to improve healthcare. One project evaluated a Burn Intensive Care Unit (ICU) DSS at both the individual and the team levels at one site. The other project evaluated a telemedicine DSS during prolonged pre-hospital care by junior physicians and medics under simulated conditions at two research sites. Both provided quantitative and qualitative data that can be used to make informed decisions on the effect of DSS on human performance.