Endsley᾿s Model

Although there are competing models with various degrees of overlap, Endsley᾿s (1995) model is dominant. It describes SA at three hierarchical but nonlinear levels (Endsley, 2015a). Level 1 is the perception of relevant elements in the environment. At level 1 SA, an individual attends to individual pieces of relevant information. Level 2 SA is the comprehension of the current situation; at this level, an individual connects pieces of information together in order to apply elements to the situational context. Level 3 SA is the projection of future status; it is knowledge of the state of elements in the future.

Team SA

The success of SA as a construct of applied cognition has come with challenges. While it has found particular success in aviation settings, a continued challenge has been the application of SA to teamwork. Because teamwork is more than the sum of its parts (Durso & Sethumadhavan, 2008) and teamwork in a sociotechnical system is affected by work system components, several approaches to measuring team cognition exist. In describing team-level SA, two approaches have received significant attention in the literature.

First is Endsley᾿s concept of team SA. Team SA has been defined as “the degree to which every team member possesses the SA needed for his or her job” (Endsley, 1995, p. 39). It suggests a largely additive process in which individual practitioners are contributors to the team through a process of building individual SA. Shared SA is the overlap of the SA of individuals; it is where the SA of multiple individuals is equivalent. Shared SA is “the degree to which team members have the same SA on shared SA requirements” (Endsley & Jones, 2001, p. 48).

In this model, SA is held exclusively within human team members. This model extends beyond individual SA in the mechanisms by which individuals build SA. Team process, identified by Salas et al. (1995), is a component of teamwork. Team process includes the communication and coordination behaviors that team members engage in as they perform taskwork. Endsley and Jones (2001) extended this work by identifying the requirements, devices, processes, and mechanisms of team SA. Communication, shared environments, and shared displays are the devices of team SA (Endsley & Jones, 2001). A limitation of this approach is that it can be difficult to model the complex impact of automation and other elements of the sociotechnical system if they do not cause an observable change in one individual᾿s SA.

Distributed SA

Stanton et al. (2006) argued for a different approach. Their concept of distributed SA treats SA as an artifact of the sociotechnical system. In this view, SA can be held in technological artifacts as well as by human team members. An example, such as that of a pulse oximeter, demonstrates the differences between these two perspectives. Is the status of the pulse oximeter sensor critical to one of the EMTs, or is it sufficient for the monitor to run without direct attention? Assuming that the oximeter provides an alert, is perfectly accurate, and is not being read, information held in the monitor would not be part of team SA. It would, however, be considered a part of distributed SA in that it is relevant information held by a technological agent (the meter). When the EMT must attend to an alert or read the meter, this information becomes part of that EMTs individual SA or, when told to another team member, part of shared SA. Under the distributed SA perspective, reading the value from the meter causes that information to be shared between the technological agent and the EMT. Endsley (2015a, p. 26) argued against a distributed approach to SA, suggesting that only automation that is a “cognizant and independent decision maker” could be considered to have SA. Since the technology used in ambulances is becoming ever more sophisticated and decisions in sociotechnical systems are so tightly coupled, this line is not a clear one. Distributed SA is attractive because it provides a way to describe the benefits of automating information processing. However, distributed SA measures the degree to which information is managed within the sociotechnical system.

We repeat an argument indicating that these perspectives may be partially reconciled by quantifying both overlapping and nonoverlapping information and allowing for agents to be diverse in their information processing (Cain & Schuster, 2014; Ososky et al., 2012). That is, both people and technology may contribute to SA, but cognition is a unique property of people. Individual SA describes the knowledge held by one individual, with some of this information shared by others. Individual SA that overlaps is shared; individual SA held by one agent, human or technological, but relevant to others is complementary SA.

While it is important for researchers and practitioners to understand the theoretical complexity surrounding SA, much of the theoretical debate is of limited relevance to the practitioner. What is critical, regardless of the theoretical approach, is that SA reflects goal-directed knowledge. The proper definition of goals is critical. If goals include good decision-making on the part of an individual human, then an individual᾿s SA must be defined based on the information needed for individual decision-making. However, we have made a case for a sociotechnical system approach to SA in EMS regardless of the theoretical approach.

Issues In Ems Decision-Making And Recommendations

Issue: SA Definition Confusion and Multiple Measurement Techniques

Both SA and human performance have been defined in many ways, leading to confusion about the use of SA. Parasuraman et al. (2008) argued that SA is distinct from generalized knowledge, task performance, and the quality of decision-making. They suggested that SA is distinctly useful in atypical circumstances. In post hoc investigations of accidents in domains such as aviation and driving, SA has been frequently found to be a contributing factor (Jones & Endsley, 1996; Durso & Sethumadhavan, 2008). Some evidence for this claim is found in empirical work examining differences between experts and novice pilots. Individual SA is a construct with well-established predictive validity (Durso et al., 2006) and a widely cited model, but it is difficult to model in general terms without context (Wickens, 2015). In EMS, Endsley᾿s model can be used to diagnose the cognitive performance of practitioners within the sociotechnical system despite debates about the meaning and validity of subcomponents of Endsley᾿s model (e.g., Flach, 1995; Hoffman, 2015). Therefore, practitioners should weigh the relatively small effort of conducting goal-directed task analysis against the utility of capturing goal-relevant knowledge, since SA predicts a variety of diverse performance outcomes. While it is tempting to identify everything that could possibly be known in an environment and reward those who hold the greatest amount of information during task performance, SA is an appropriate metric only when it targets specific goals rather than all possible outcomes. Appropriate measurement SA should focus exclusively on the knowledge needed to immediately achieve such goals. The goal-oriented nature of SA distinguishes it from trivia.

There are a variety of measurement techniques available. The situation awareness global assessment technique (SAGAT) is an example of a measurement technique in which a task or simulation is paused and then individuals are prompted to answer questions that sample SA (Endsley, 1988, 2000). SPAM, the situation present assessment method (Durso & Dattel, 2004), is a variation of this technique in which questions are made available randomly with a signal. The individual pushes a button to view and respond to the question as soon as they are able, but the task environment continues uninterrupted. Questions could be offered on a tablet or other mobile device. Neither SAGAT nor SPAM measurement would be suitable in an operational environment, but both are well suited to simulation and training environments.

SPAM offers a way to simultaneously measure workload; question response latency can be used as a proxy measure of workload. When operators are under high demand, they may take longer to respond to the probe questions (Durso et al., 2006). Quantifying the time to question can provide a supplementary workload measure without requiring a separate questionnaire, although this method may be less sensitive than other assessments of workload (Pierce et al., 2008).

SA measures that are not domain specific also exist, but they suffer from validity issues. The most common of these, the situation awareness rating technique (SART) (Taylor, 1990) is a self-report questionnaire that asks an individual to report their levels of awareness along several dimensions: supply of attentional resources, demand on attentional resources, and understanding of the present situation. It is administered after task performance. If an individual lacks SA but does not know what they do not know, they may rate their SA highly. Further, high-performing individuals may be better able to think about their own SA losses and could rate themselves lower than our oblivious individual. In this way, SART is more of a metacognitive measure than an SA measure. Empirical data have provided support to these theoretical criticisms (Endsley, 1988). Although the SART is the least costly measure to implement, the validity issues make it less useful for operational use in EMS.

Recommendation: Unless such measures already exist, use goal-directed task analysis to facilitate the use of domain-specific measures of SA. Goal-directed task analysis is a method for describing the knowledge requirements in a situation (Endsley & Jones, 2012). This method requires access to high-performing individuals to identify their goals and subgoals. The result of this analysis can be represented as a hierarchical list of information requirements that can be classified at the three levels of SA. This technique can also identify shared SA requirements (Bolstad et al., 2002). It is important that a continuum of expertise is represented in goal-directed task analysis, as experience brings more efficient use of working memory and better strategies (Endsley, 2015b).

Goal-directed task analysis is a prerequisite for defining ideal SA in a new environment. While goal-directed task analysis results in a hierarchical list of information without description of how that information is obtained, it is inherently bound to an operational context. Consequently, SA measures may need revision as prehospital teams vary in their composition or as technology affects what knowledge must be held by team members in the prehospital environment.

Issue: Individual SA Provides an Incomplete Picture

Individual SA will likely predict the quality of patient care, but it is not the whole story. To properly capture SA, practitioners and researchers should take care to broadly consider SA in the sociotechnical system. Because SA is inherently tied to tasks, the boundaries of the sociotechnical system (i.e., the situation) must be carefully delineated. In EMS, the activities within the ambulance are at one point along the care continuum but exclude events related to dispatch and transfer to the emergency department. Within the situation, individual decision makers should be identified. Individual decision makers will include EMS practitioners, but they should also include other human team members such as physicians and dispatchers. Additionally, the contribution of any technology capable of decision-making should also be included.

Recommendation: Capture team process to contextualize individual SA metrics. Despite theoretical debates, application of Endsley᾿s (1995) model is likely to be a useful approach. However, an analysis that identifies gaps in individual SA can be misleading without identifying deficiencies in team process behaviors, including how information is communicated (Cooke et al., 2007). As members of a team, loss of SA by an individual is unlikely to be an isolated problem. Both the antecedents of SA loss and the impacts of SA loss are intertwined with other elements of the sociotechnical system. Consequently, the goal should not be to single out low-performing individuals but to understand how individual performance is hindered or facilitated by elements of the sociotechnical system.

At one end of a spectrum, the simplest approach to measurement is quantitative assessment of the SA provided by human team members and technological decision aids. Individual SA measurement can be augmented by quantitative or qualitative analysis of team process behaviors. At the other end of this spectrum is a distributed approach to SA measurement, which focuses to the greatest degree on the interactions among elements in the sociotechnical system. A distributed approach to SA measurement may be possible with a well-defined situation model and a thorough understanding of the contribution of technology to individual SA. At present, empirical work is still needed to support a comprehensive understanding of EMS as a sociotechnical system.

Issue: Automation in the Ambulance Can Have Unanticipated Impacts on SA

Since its inception, research on SA has been applied to human–technology interaction, starting with the aviation cockpit. Other domains in which SA has been effectively applied frequently include automated tools. EMS is no exception with increasingly sophisticated medical equipment becoming a part of prehospital care. An example is the increasing computerization of medical equipment and the use of diagnostic aids, such as a recent National Institutes of Health Stroke Scale Assessment administered on a tablet (Padrick et al., 2015).

In other domains, empirical results have suggested, on the surface, that automated tools reduce workload and increase performance. Subsequent research has suggested that SA moderates this relationship (Wickens et al., 2010). However, high levels of automation can lead to a loss of SA (Endsley & Kiris, 1995). Consequently, SA is useful in diagnosing the impact of automation on decision-making performance. Several recurring issues in human–automation interaction are common in sociotechnical systems across domains. Fortunately, an understanding of the potential issues introduced by automation can inform automation design to minimize their impact.

First, automation may increase workload by placing additional demands on the practitioner. In doing so, SA is likely to be reduced (Vortac et al., 1993). Since critical decisions are regularly made under conditions of high mental and physical workload, it is important that new tools do not place an additional burden on already taxed resources. Another issue is the out-of-the-loop performance problem (Endsley & Kiris, 1995), which characterizes a situation of high-performing, but imperfect automation. Automation can be imperfect by occasionally failing outright, such as in the case of a malfunctioning sensor that reports either no value or an incorrect value. Automation can also be imperfect in that it fails to be usable in complex, high-workload situations. In either case of imperfect automation, human operators find themselves largely able to rely on the automation, except during rare occurrences. In these rare occurrences, humans are called “back into the loop” and must take over physical or cognitive tasks typically performed by the automation. People are particularly bad at a rapid, unanticipated shift to manual control. The result is a loss of SA.

Recommendations: Avoid EMS automation that overpromises and underdelivers. Provide training to users of automation on the true capabilities and limitations of automation to encourage appropriate trust. Train for mitigation procedures under automation failure.

Notably, such a loss of SA might not be observed in the case of poorly functioning automation. When working with automation, people base trust attributions on their perceptions of automation performance (Johnson et al., 2009; Muir & Moray, 1996; Oleson et al., 2011, p. 176). Automation which is known to be unreliable may be relied upon less (Yeh & Wickens, 2001). Although this may seem to mitigate the out-of-the-loop performance problem, it may be more accurately described as a “stuck-in-the-loop” performance problem. Such automation is either appropriately disregarded, in which case it provides no benefit, or inappropriately underutilized, a condition called disuse (Parasuraman & Riley, 1997). When human operators of otherwise useful automation elect not to use it, the problem is a lack of trust in automation (Lee & See, 2004; Parasuraman & Riley, 1997). Trust in automation is “the attitude that an agent will help achieve an individual᾿s goals in a situation characterized by uncertainty and vulnerability” (Lee & See, 2004, p. 54). This state of affairs can become evident during goal-directed task analysis. Experts, as part of task analysis, may identify technology that is rarely used as intended due to its perception as being ineffective. When trust is appropriately calibrated, it is adaptive. Trust allows people to appropriately use automated tools.

To encourage appropriate calibrated trust, it is important that the performance of automation in the prehospital environment matches practitioner perceptions of its performance. All other factors being equal, trust in automation will be higher with higher levels of automation reliability. Training that overstates the capabilities of automation may have the effect of reducing use of the automation. For many forms of automation, perfect reliability is not possible. If automation can require unanticipated human intervention, especially during periods of time pressure and high workload, practitioners should be trained for these scenarios to minimize the effects of the out-of-the-loop performance problem.