Action Selection in Human Performance
Information Processing and Action
actions selected by a third type of behavior, known as skillbased behavior
This example typifies applying the brake of a car upon seeing a yellow light, shutting down a piece of equipment when the emergency alert goes off, or pressing a key (or set of keys) on a keyboard, after seeing (or hearing) an element of the message that is to be transcribed.
(Consider the unfortunate sprinter who errs in the skill-based response to the starting gun by committing a false start.) However, much greater emphasis in skill-based behavior is placed on response time (RT).
One of the most important is the degree of uncertainty about what stimulus event will occur and therefore the degree of choice in the action to make. For the sprinter at the starting line of a race, there is no uncertainty about the stimulus— the sound of the starting gun—nor is there a choice of what response to make: to get off the blocks as fast as possible. On the other hand, for the driver of an automobile, wary of potential obstacles in the road, there is both stimulus uncertainty and response choice. An obstacle could be encountered on the left, requiring a swerve to the right; on the right, requiring a swerve to the left; or perhaps at dead center, requiring that the brakes be applied. The situation of the sprinter illustrates the simple RT task, the vehicle driver the task of choice RT.
Variables Influencing SimpleRT
Stimulus Modulity: Several investigators have reported that simple RT to auditory stimuli is about 30 to 50 msec faster than to visual stimuli presented in foveal vision (roughly 130 msec and 170 msec, respectively; Woodworth & Schlossberg, 1965). This difference has been attributed to differences in the speed of sensory processing between the two modalities.
Stimulus Intensity: Simple RT decreases with increases in intensity of the stimulus to an asymptotic value, following a function.
Stimulus Intensity: Simple RT decreases with increases in intensity of the stimulus to an asymptotic value, following a function.
Temporal Uncertainty: The degree of predictability of when the stimulus will occur is called temporal uncertainty. This factor can be manipulated by varying the warning interval (WI) occurring between a warning signal and the imperative stimulus to which the subject must respond. In the case of the sprinter, two warning signals are provided: “Take your mark” and “Set.” The gunshot then represents the imperative stimulus. If the warning interval (WI) is short and remains constant over a block of trials, then the imperative stimulus is highly predictable in time and the RT will be short. I
Expectancy: if we look at individual RTs to different warning intervals within the varied set, then RT following a short WI is longer than that following a longer RT (Drazin, 1961). This difference is due to Chapter 9 • Selection of Action 287 expectancy. The longer you wait, the more “primed” you are for an action (the lower the criterion), and so when the signal occurs, you act faster; but at the possible cost of an error (the false start of the sprinter after a long pause before the starting gun).
Control and Coordination
Factors Affecting Action Selection
Reaction Time Fundamentals
Simple vs. Choice Reaction Time
Additive Factors Method
Factors Influencing Reaction Time
Reaction Time and Information Theory
Practical Relevance of RT
Hick–Hyman Law
The constant ‘’b’’ reflects the slope of the function—the amount of added processing time that results from each added bit of stimulus information to be processed. The constant ‘’a’’ describes the sum of those processing latencies that are unrelated to the reduction of uncertainty. These would include, for example, the time taken to encode the stimulus and to execute the response
It is intuitive that the more complex decisions or choices require longer time to initiate. A straightforward example is the difference between simple RT, and choice RT in which there is uncertainty about which stimulus will occur and therefore about which action to take. The actual function that related the amount of uncertainty or degree of choice to RT was first presented by Merkel (1885). He found that RT was a negatively accelerating function of the number of stimulus-response alternatives. Each added alternative increases RT, but by a smaller amount than the previous alternative.
Three variables influence the information conveyed, and RT, by a stimulus: the number of possible stimuli, the probability of a stimulus, and its context or sequential constraints
n RT = a + bHs
Hs —> the information in the stimulus
Thus, the Hick-Hyman Law seems to capture the fact that, in many circumstances, the human has a relatively constant rate of processing information, defined by the inverse slope (1/b) of Figure 9.2: a constant number of bits/second.
Hyman further demonstrated that the function was still linear when the average information transmitted by stimuli during a block of trials was manipulated by varying the probability of stimuli and their sequential expectancy. If probability is varied, then when N alternatives are equally likely, information is maximum (i.e., four alternatives yield two bits). When the probabilities are imbalanced, the average information is reduced. Hyman observed that the mean RT for a block of trials is shortened by this reduction of information in such a way that the new, faster data point still lies along the linear function of the Hick-Hyman law.
Choice RT is also strongly influenced by expectancy (which, in turn, is influenced by the probability of the stimulus event). If we expect to make a right turn because we always do, we will be fast in initiating that action and slow when a left turn is suddenly signaled. In information theory terms, the expected event contains less information than the surprising one. If there are two events, the occurrence of an expected one (e.g., that which occurs 80 percent of the time) conveys less than one bit, whereas the surprising one conveys more than one bit. But if we measure RT to each of these events, the RT measure will still fall directly on the line predicted by the Hick-Hyman law as in Figure 9.2b.
Departures from Hick–Hyman Law
Experimental Basis
Design Implications
Applications
Stimulus–Response Compatibility
Spatial Compatibility
Control–Display Ratio
Population Stereotypes
Compatibility and Learning
S–R Compatibility and Workload
Compatibility in Interface Design
Speed–Accuracy Relationship
Fitts’s Law
Interpretation of Fitts’s Law
When the target is fixed in space, like the position of a word on a screen to be edited, then movement of the cursor to the target follows a well-established law in human performance known as Fitts’ Law (Fitts, 1954; Card, English, & Burr, 1978). This law predicts that movement time is proportional to both the distance traveled and the required precision of the target (smaller target, greater precision). More specifically, there is a linear relation between movement time and the Index of Difficulty (ID) of the movement, such that:
This law can be used effectively to predict values like the time required to move a mouse to a “button” on a computer screen as a function of the button’s size or to predict the time to move the foot to pedals of different sizes and separation (Drury, 1975)
Applications of Fitts’s Law
Speed–Accuracy Optimization
In RT tasks, and in speeded performance in general, people often make errors. Furthermore, they tend to make more errors as they try to respond more rapidly. This reciprocity between time and errors is referred to as the speed-accuracy trade-off
THE SPEED-ACCURACY OPERATING CHARACTERISTIC
Design for SAT Balance
The speed-accuracy operating characteristic, or SAOC, is a function that represents RT performance in a manner analogous to the receiver operating characteristic (ROC) representation of signal detection performance.
RT and error rate represent two dimensions of the efficiency of processing information. These dimensions are analogous in some respects to the dimensions of hit and false-alarm rate in signal detection
Human Control Models
stem designers should also be aware that certain design features seem automatically to shift performance along the SAOC. For example, redundant presentation across modalities (simultaneous text and speech) appears to improve accuracy, but sometimes slows the speed of processing (Wickens, Prinet, et al., 2011). Presenting more information, of greater precision, on a visual display will often lead to more accurate performance (assuming that information is used by the operator) but at a greater cost of time. For example, magnifying the displayed error in a target-aiming task will prolong the aiming response
There is an important exception to the SATO, which might be described as the speed-accuracy trade-on (SATON). For example, good design can produce faster and more accurate performance than poor design (e.g., stimulus-response compatibility violations as we see later in this chapter). Beilock et al. (2008) studied the SATON as reflected in the expertise effect in sports. Here experts (but not novices) may be more accurate if less time is given for an action (e.g., golf putting)
SPEED-ACCURACY MICRO-TRADE-OFF
A different way of looking at the speed-accuracy relationship is to compare the accuracy of fast and slow responses within a block of trials, using the same system (or experimental condition). (Alternatively one can compare the mean RT of correct and error responses.) This comparison describes the speed-accuracy micro-trade-off. Its form depends on what varies most from trial to trial. On the one hand, when the criterion varies, this produces a pattern typical of the macro trade-off (faster responses are more error prone). Indeed, sometimes the criterion can be so low that a response is essentially a “fast guess” in which a random response is initiated as soon as the stimulus is detected (Gratton et al., 1988; Pachella, 1974).
RT is lengthened as a set of stimuli are made less discriminable from one another.
The advantage of repetitions over alternations, referred to as the repetition effect appears to be enhanced by increasing N.
) RT is lengthened as the confusability between the responses is increased.
RT is lengthened by the complexity of the response.
Consistent results suggest that practice decreases the slope of the Hick-Hyman law function relating RT to information.
Executive control: Speeded responses are made following one rule, like discriminating between high and low digits, and then abruptly shift to a different rule, like discriminating between odd and even digits
S-R compatibility
Compatibility between a display location or movement and the location or movement of the associated operator response.
In June 1989, the pilots of a commercial aircraft flying over the United Kingdom detected a burning engine but mistakenly shut down the good engine instead. When their remaining engine (the burning one) eventually lost power, leaving the plane with no engines, it crashed, with a large loss of life. Why? Analysis suggests that a violation of stimulus-response compatibility in the display control relation may have been a contributing factor
S-R compatibility has both static elements (where response devices should be located to control their respective displays) and dynamic elements (how response devices should move in order to control items in the workplace, and their associated dynamic displays). We refer to these as locational and movement compatibility, respectively.
Location compatibility
The foundations of location compatibility are provided in part by the human’s intrinsic tendency to move or orient toward the source of stimulation (Simon, 1969). Given the predominance of this effect, it is not surprising that compatible relations are those in which controls are located next to the relevant displays, a characteristic that defines the colocation principle. The touch-screen CRT display is an example of designs that maximize S-R compatibility through colocation
Point and click cursor controls achieve colocation somewhat indirectly, to the extent that the cursor is viewed as a direct extension of the hand. However, many systems in the real world often fail to adhere to the colocation principle, for example, the location of stove burner controls
. Controls colocated beside their respective burners (Figure 9.5a) are compatible and will of course eliminate the possible confusions caused by arrays shown in Figure 9.5 (b and c), which are more typical.
Unfortunately the principle of colocation is not always possible to achieve. Even the colocation of Figure 9.5a may require the chef to reach across an active (hot) burner to adjust a control. Where colocation cannot be obtained, two important compatibility principles are congruence and rules.
The general principle of congruence is based on the idea that the spatial array of controls should be congruent with the spatial array of displays. This principle was illustrated in a study by Fitts and Seeger (1953), who evaluated RT performance when each of the three patterns of light stimuli on the left in Figure 9.6 was assigned to one of the three response mappings (moving a lever) indicated across the top. In each case an eight-choice RT task was imposed. In stimulus array Sa, any one of the eight lights could illuminate (and for Ra the eight lever positions could be occupied). In Sb, the same eight angular positions could be defined by the four single lights and the four combinations of adjacent lights. In Rb, the eight shaded lever positions could be occupied. In Sc, the eight stimuli were defined by the four single lights and four pairwise combinations of one light from each panel. In Rc, each or both levers could be moved to either side. Fitts and Seeger found that the best performance for each stimulus array was obtained from the spatially congruent response array: Sa to Ra, Sb to Rb, and Sc to Rc. This advantage is indicated by both faster responses and greater accuracy
A stove-top array such as that shown in Figure 9.5c would also achieve this congruence (Hoffman & Chan, 2011). Notice in b and d that there is no possible congruent mapping of the linear array of controls to the square array of burners (displays). The only way to bypass this lack of compatibility is through the drawn links as shown in Figure 9.5d
MOVEMENT COMPATIBILITY
Spatial movements can be represented in terms of either world referenced (north-south, east-west) or ego-referenced (leftright, front-back, up-down) spatial coordinates. Conceptual movements involve the increase or decrease in a quantity, such as risk or money or energy; while these are not directly mapped onto space, we typically think of “more” as higher, and so there is a natural or compatible mapping.
An important design solution that can resolve any potential mapping ambiguity is to put a slight cant, or angling, of one array in a direction that is congruent with the other, as shown in Figure 9.7. If this cant is as great as 45o , then reaction time can be as fast as if the control and display axes are parallel
Population stereotypes. There is a very strong stereotype for moving a control upward to increase.
Congruence of display movement. Most dynamic controls are (or should be) coupled by feedback displays that indicate that the movement was accomplished in the direction intended. Alternatively, in many tracking tasks, the display movement may signal a movement of the controlled agent that requires a compensatory or pursuit control action (see Chapter 5). Here as discussed with regards to FORT in Chapter 5, maximum compatibility is achieved when the display moves in a direction congruent with the control. For example, a linear moving vertical control (e.g., joystick or slider) should be coupled with a vertical display such that upward movement of the control produces upward movement of the displayed element.
Mismatching dimensions. Sometimes physical constraints may limit application of perfect congruence. For example, a rotary control may be more stable to adjust than a linear one in a dynamic, vibrating, or unsupported environment, even as the feedback display is a linear one.
Constrained versus unconstrained controls. When controls are constrained or “channeled” to only move along pure X, Y, or Z axes, it is easy to control along one axis at a time. However, when controls and displays are free to move along any combination of axes, then such pure mapping becomes more difficult. As an example of an unconstrained control, try to control a mouse cursor when the mouse is oriented at an angle to the mouse pad.
Frame of Reference modifications. When analyzing movement in a display with regard to the controlled element in the world, a critical distinction is whether the display depicts the moving element against a stable display frame or depicts a stable element within a moving frame
Compensatory status displays versus pursuit command displays. The distinction between inside-out and outside-in is closely related to the distinction between compensatory and pursuit displays in tracking. In a compensatory display, an increase in error, signaled by a leftward movement of the error cursor, should trigger a rightward (compensatory) movement of the control. In a pursuit display, a leftward movement of the target to be followed should trigger a leftward movement of the control.
The Warrick principle relates co-location with movement and is satisfied whenever the control moves in the same direction as the closest moving element of the display (Hoffman, 1990, 2009; Warrick, 1947). This is illustrated in Figure 9.8a, where the Warrick principle is satisfied by placing the rotary control on the right side of the vertical linear display, but violated by placing it on the left side (9.8b).
What would be the cost of keeping the control on the left side (9b), but now reversing the direction, so that a clockwise-to-decrease mapping was in effect? In such circumstances one would expect the two principles to continue to offset each other, now violating the direction of motion stereotype, but conforming to the Warrick principle. Guidance of course, is to configure control (or display) placement in such a way to maximize all principles (right side), or at least not violate any, which might be the case if the rotary dial were placed below the linear scale (Figure 9.8a).
Movement in different planes. Much of our discussion has focused on controlling a display in the frontal plane, the display mounted vertically in front of the controller. But suppose the display plane is rotated by 90 degrees a tabletop display or one mounted to the right, left, or above.
• The visual field compatibilty (Worringham & Beringer, 1989; Chan & Hoffman, 2010) is dominant. Here, consider an operator viewing a 3D display mounted parallel to the right window, depicting an element that she/he wishes to move to the left on the display. To accomplish this, she should move a front-mounted control to the left (so that left on the control is left on the display when the display is viewed by a 90 degree rightward head rotation).
SERIAL RESPONSE
Many tasks in the real world however, call for not just one but a series of repetitive actions. Typing and assembly line work are two examples.
Psychological refractory period
Describes a situation in which two RT tasks are presented close together in time. The separation in time between the two stimuli is called the interstimulus interval or ISI. The general finding is that the response to the second stimulus is delayed by the processing of the first when the ISI is short. Suppose, for example, a subject is to press a key (R1) as soon as a tone (S1) is heard, and is to speak (R2) as soon as a light (S2) is seen. If the light is presented a fifth of a second or so after the tone, the subject will be slowed in responding to the light (RT2) because of processing the tone. However, RT to the tone (RT1) will not be affected by the presence of the light response task. The PRP delay in RT2 is typically measured with respect to a single-task control condition, in which S2 is responded to without any requirement to respond to S1.
The most plausible account of the PRP is a model that proposes the human being to be a single-channel processor of information
single-channel theory assumes that the processing of S1 temporarily “captures” the single-channel bottleneck of the decision-making/response-selection stage. Thus, until R1 has been released (the single channel has finished processing S1), the processor cannot begin to deal with S2. The second stimulus S2 must therefore wait at the “gates” of this single-channel bottleneck until they open. This waiting time is what prolongs RT2. The sooner S2 arrives, the longer it must wait, just like arriving at a queue of fixed length waiting for the service provider (store owner) to open. According to this view, anything that prolongs the processing of S1 will increase the PRP delay of RT2. Reynolds (1966), for example, found that the PRP delay in RT2 was lengthened if the task of RT1 involved a choice rather than a simple response.
When ISI is long (much greater than RT1), RT2 is not delayed at all. When ISI is shortened to about the length of RT1, some temporal overlap will occur and RT2 will be prolonged because of a waiting period. This waiting time will then increase linearly as ISI is shortened further. The relationship between ISI and RT
Decision complexity advantage
This finding suggests that there is some fundamental limit to the central-processing or decision-making rate, independent of decision complexity, that limits the speed of other stages of processing. This limit appears to be about 2.5 decisions/sec for decisions of even the simplest possible kind
Advantage of a few complex decisions over several simple ones- People are better able to process information delivered in the format of one six-bit decision per second than in the format of six one-bit decisions per second
The most general implication of the decision complexity advantage is that greater gains in information transmission may be achieved by calling for a few complex decisions than by calling for many simple decisions.
The decision complexity advantage also has implications for any data-entry task, such as keyboarding. For example, Seibel (1972) concluded that making text more redundant (less information per key stroke) will increase the rate at which key responses can be made (decisions per second) but will decrease the overall information transmission rate (bits per second). It follows from these data that processing efficiency could be increased by allowing each key press to convey more information than the 1.5 bits provided on the average by each letter
One obvious solution to this motor limitation is to allow chording, such as found in courtroom proceedings transcribers in which simultaneous rather than sequential key presses are required (Baber, 1997). This approach would increase the number of possible strokes without imposing a proportional increase in the number of keys. Thus, with only a five-finger keyboard, it is possible to produce 25 1, or 31, possible chords without requiring any finger movement to different keys. With ten fingers resting on ten keys the possibilities are 210 1, or 1023. C
Decision complexity advantage
Besides capitalizing on the decision complexity advantage, chording keyboards are also useful because they can be easily operated while vision is fixated elsewhere. A major problem with chording keyboards, however, is that the sometimes arbitrary finger assignments take a long time to learn (Richardson, Telson, et al., 1987). One solution is to capitalize on visual imagery, assigning the chording fingers in a way that “looks” like the image of the letters. Such a chording keyboard was designed by Sidorsky (1974), following the scheme in Figure 9.12. Using three fingers, the operator presses twice for each letter, “painting” it from the top row to the bottom. In the figure, the dots represent keys that are not pressed. Once the operator remembers the particular idiosyncratic shapes of the letters, little learning is required, and Sidorsky found that subjects were able to type from 60 percent to 110 percent as fast with this as they could with the conventional keyboard (see also Gopher & Raij, 1988). Because only one hand is required, the chording keyboard can work in harmony with a mouse, controlled by the other hand.
Feedback and Gain
Tracking and Control Tasks
Stability and Error Correction
Applications to Interface Design
Motor Programs
Sequential Action
Evidence for Motor Programming
Capture and Anticipation Errors
Cognitive Control of Motor Sequences
Automation of Action Sequences
Power Law of Practice
Controlled to Automatic Transition
Feedback and Reinforcement
Skill Retention and Transfer
Design for Learning Support
Slips, Lapses, and Mistakes
Error Detection and Recovery
Design for Error Prevention
Design Principles for Action Selection
Human–Automation Interaction
Integrating Cognitive and Motor Perspectives
Summary