Eye movements are fundamental to the operation of the visual system. Eye movements can provide insight into cognitive processes such as language comprehension, memory, mental imagery, and decision making. Eye movement research is of great interest in the study of neuroscience and psychiatry, as well as ergonomics, advertising, and design. Since eye movements can be controlled volitionally, to some degree, and tracked by modern technology with great speed and precision, they can now be used as a powerful input device, and they have many practical applications in human-computer interactions.
Eye movements are fundamental to the operation of the visual system. The term eye movement research relates to a patchwork of fields more diverse than the study of perceptual systems, however. Due to their close relation to attentional mechanisms, eye movements can provide insight into cognitive processes such as language comprehension, memory, mental imagery, and decision making. Eye movement research is of great interest in the study of neuroscience and psychiatry, as well as ergonomics, advertising, and design. Since eye movements can be controlled volitionally, to some degree, and tracked by modern technology with great speed and precision, they can now be used as a powerful input device, and they have many practical applications in human–computer interactions.
Eye movements are arguably the most frequent of all human movements.[ 1 ] Large ballistic scanning movements called saccades typically occur 3–4 times every second. As one early researcher put it, “there seems to be an almost ceaseless twitching, as if rest for more than an instant were the one thing not to be endured.”[ 2 ] Indeed, virtually all animals with developed visual systems actively control their gaze using eye or head movements.[ 3 ] This frenetic movement is a consequence of the enormous amount of visual information that is available to an organism. Rather than devote resources to processing it all in detail, evolution appears to have selected a solution whereby small portions of the visual world are inspected in a rapid sequence.[ 4 ] Consequently, the human eye monitors a visual field of about 200 degrees, but receives detailed information from only 2 degrees.[ 5 ] This region, about the size of a thumbnail at arm's length, is called the fovea. It is jerked around at speeds of up to 500° a second, during which its sensitivity drops to near blindness levels.[6,7] During the 200–300 milliseconds it is at rest, however, over 30,000 densely packed photoreceptors in the fovea provide high‐acuity color vision.
The first data regarding eye movements were obtained either through introspection or by the experimenter observing a subject's eye, using a mirror, telescope, or peephole. These methods were dubious, of course, because any feature of the eye being studied could be obscured by the eye doing the studying. The first significant advance, therefore, was the invention of mechanical devices that would translate the eye's movements into permanent, objective records of its motion.[ 8 ]
At the end of the nineteenth century, eye movement research served a pressing theoretical need. As Delabarre wrote, “[m]any problems suggest themselves to the psychologist whose solution would be greatly furthered by an accurate method of recording the movements of the eye.”[ 9 ] Phenomena such as visual illusions and aesthetic preferences were commonly explained away in terms of eye movements, and yet there were few data beyond introspection to support these hypotheses. In some of the first empirical studies, Javal[ 10 ] used mirrors to observe the eye movements of subjects while reading, and was the first to note that the eyes moved in a series of “jerks.” These fixations were counted by placing a microphone on a closed eyelid while the subject read monocularly. Each time the bulge of the moving cornea bumped the microphone, a saccade could be counted, according to Lamare and Javal.[ 11 ] An approximate measure of the location of these fixations could be obtained by inducing an afterimage in the subject's eye with a bright light, and then asking subjects to report the location of the afterimage as they read.[ 12 ]
Such observational methods, however, were limited by the accuracy and memory of the person making the observations.[ 13 ] An objective record of the motion of the eyes was required. Since “plaster of Paris will attach itself firmly and immovably to any moist surface,” Delabarre[ 9 ] was able to fix a small, moulded cap to a sufficiently “cocainized eye.” A wire ran from the cap to a lever, which drew the horizontal movements of the eye on the smoked surface of a kymograph cylinder. With his lids propped open, the subject (usually Delabarre himself) could read text through a hole drilled in the cap. It was reported that the cap “will not detach itself until it becomes thoroughly soaked with tears.” Delabarre writes, “As to whether there is any danger to the eye to be feared from using it in this manner, I cannot say with assurance.” He reports that he was able to record his own eye movements for up to an hour, and suffered no ill effects after a week's recuperation.
Huey[ 14 ] simultaneously developed a similar mechanical method to study the behavior of subjects reading an article in the magazine Cosmopolitan. By varying the distance between the subject and the text, he found that “the number of jerks was shown to be a function of the matter read rather than of the arc described by the eye's rotation.” Although the temporal resolution of his apparatus was too poor to measure the speed of the eye movements, he conjectured that they may be so fast that “we really do not see foveally what we read except at the few points on the ordinary line where the eye pauses.”
Huey and Delabarre were able to gain valuable first insights into the function and characteristics of eye movements, yet their mechanical devices were criticized for impeding motion and straining the eye. To overcome these flaws, Dodge and Cline[ 15 ] invented a device to produce “a group of what we may justly claim to be the first accurate measurements of the angle of velocity of the eye movements under normal conditions.” Dodge's method, as it became known, used photography to record the movements of the eye accurately and noninvasively, and the same basic technique continued to be used into the 1970s.[ 8 ]
If the eye were a perfect sphere and rotated about its center, a light ray would be reflected at a constant angle despite rotations. Given that the eye has neither of these characteristics, however, the reflection of a ray of light bouncing off the bulge of the cornea will move as the eyes move. In Dodge's first device, a vertical line of light was bounced off the cornea and fell on a horizontal slit. Behind this slit was a photographic plate that moved vertically, regulated by the escape of air from a cylinder. When developed, this plate showed time on the y‐axis and horizontal motion of the eye on the x‐axis.
Further developments in the 1920s, in labs at Chicago and Stanford, allowed two photographic recordings to be made simultaneously. In this way, head position could be recorded by reflecting a light off a bead on a pair of spectacles,[ 16 ] or the horizontal eye movements of one eye could be recorded with the vertical movements of another, producing the first two‐dimensional eye movement records.[ 17 ] Later technology allowed the reflection beam from a single eye to be split, its vertical and horizontal components measured and recombined in the form of a fixation dot recorded on a film reel. This methodology allowed researchers, including Buswell,[ 18 ] to produce some of the first two‐dimensional scan paths of subjects inspecting images. Although a thread of research continued for a couple of decades investigating the relationship between mental imagery and eye movements,[19,20,21,22,23] the vast bulk of eye movement research in the first half of the century investigated the processes, habits, and individual and cultural differences involved in reading.[11,24,25,26,27]
The 1960s saw a renaissance of the turn‐of‐the‐century, invasive techniques of Delabarre[ 9 ] and Huey[ 14 ] for recording eye movements. Researchers found that rather than using sticky plaster of Paris, a device could be tightly clamped to the eye using suction. Yarbus[ 28 ] used a tiny valve to withdraw fluid from under a contact lens, and Fender[ 29 ] found that sodium bicarbonate would osmose through the tissue of the eye and create negative pressure. A tiny plane mirror could be attached to the surface of the contact lens, and its reflection could be recorded in much the same way as a corneal reflection. To avoid a tear film clouding the lens, researchers also mounted the mirrors on stalks that protruded out from the eye.[ 29 ] As well as reflecting with mirrors, these stalks could produce their own light source if fitted with small lamps[ 30 ] or glowing radioactive tritrium.[ 31 ] Finally, a nonoptical method employed a scleral search coil: a contact lens embedded with two orthogonal wire coils that would perturb a magnetic field surrounding the subject's head.[ 32 ] Due to the discomfort produced by these contact lens methods, all have dropped out of use except the last, and that is primarily used in animal research.
In the early 1970s, a host of techniques were developed in which the eye was scanned with a television camera, and certain distinct features were electronically detected and localized. These methods are most sensitive to high contrast, and so one technique scanned the image of the eye for the limbus, the boundary between the white sclera and the coloured iris. If small electronic photoreceptors are aligned near the limbus, their output will vary according to the amount of white sclera exposed. This method will give a very rapid measure of horizontal eye movement.[ 33 ] Unfortunately, the iris is large and often obscured by the eyelid, and vertical eye movements in particular are difficult to track with this method. An alternative is to scan for the lack of reflectance from the pupil (dark‐pupil tracking), although there can be low contrast between this black circle and dark‐brown irises. If the pupil is lit directly from the front, the light will bounce off the back of the retina and appear very bright (bright‐pupil tracking), as it does in poorly taken flash photographs. This bright circle can then be more easily detected by a scanning technique.[ 34 ]
All the methods described so far for recording eye movements are more precisely stated as methods for recording movements of the eye in relation to the head. In order to infer where the subject was looking in the world, researchers needed to ensure that the head was absolutely stationary, by employing severe methods of restraint. These draconian means were obviated by the innovation, in the 1970s, of simultaneously measuring two optical characteristics of the moving eye. Since these features behaved differently under head movement and eye rotation, their differential could be used to calculate the point of regard, the place in the world where the subjects was actually looking (Fig. 1). Although such devices still needed to restrain the head with a bitebar or chinrest, they allowed slight movements of the head to be deconfounded from eye movements, and so produced more accurate gaze tracking.
Fig. 1 The corneal reflections produced by different eye–head positions. The corneal reflection appears as a bright white dot, just to the side of the pupil (A). The relative positions of the pupil and the corneal reflection change when the eye rotates around its vertical (B) and horizontal (C) axes. This relationship does not change, however, when the head moves and the eye is stable (compare A and D).
As described above, Merchant and Morrisette[ 34 ] employed a scanning method to detect the center of a brightly lit pupil. The same technique was also used to find the smaller, brighter corneal reflection. Since the position of the corneal reflection in relation to the center of the pupil remains constant during head translation, but moves with eye rotation, point of regard can be extrapolated. Like contemporary systems,[ 35 ] the Honeywell oculometer was able to calculate what location on a screen was being fixated, while being so noninvasive that the subject was often not aware of its presence in the room.
A similar logic lay behind the development of the dual Purkinje image eye tracker.[ 36 ] Light bouncing off the eye produces a series of reflections. The first, and the brightest, is the corneal reflection. A second image is reflected off the rear surface of the cornea, and another two by the front and rear of the lens. These four Purkinje images all have different motions in relation to eye rotation. The dual Purkinje image eye tracker measures the disparity between the first and the fourth images by adjusting a series of mirrors with servomotors until the two images are superimposed upon electronic photoreceptors. The degree to which the mirrors have to be moved is directly related to eye rotation and is independent of head translation, although the head still must be held in place with a chinrest or bitebar so that the eye can be detected by the equipment. The advantage of the Purkinje eye tracker is that since it is limited only by the speed of the servomotors, it is remarkably fast and accurate. The continuous analog signal was sampled at a rate of 300 Hz in initial incarnations, and modern computers can sample at up to 1000 Hz.
The balance between obtaining a high‐precision record of an observer's point‐of‐regard and allowing natural head and body movements is where much of the technological advancement in eye‐tracking takes place in the current state of the art. Whereas systems like the dual Purkinje eye trackers require that a subject's head be immobilized by a chinrest or bitebar, new headband‐mounted eyetrackers point an additional scene camera at the subject's field of view from the subject's perspective, thus allowing point‐of‐regard to be superimposed on the scene camera's image regardless of where or how the subject moves[37,38,39] (Fig. 2). Recent table‐mounted remote optical eye trackers have also been developed that allow some natural head movement while the subject sits in front of a computer screen for two‐dimensional stimulus presentation.
Fig. 2 A subject wearing a light, head‐mounted eye tracker. The eye monitor, with crosshairs on the pupil and corneal reflection visible, is shown at the top right. The view from the scene camera, with the subject's point of gaze superimposed, can be seen at the bottom right.
One particularly important, and frequently used, method in eye tracking concerns not just the eye tracker itself but also the yoking of the eye‐position signal with real‐time stimulus presentation: gaze‐contingent display paradigms. The visual system receives highly detailed information from the small part of the visual field that the eye is fixating at that moment, and yet the target for each detailed fixation is planned on the basis of information gathered from low‐resolution peripheral vision. This relationship between peripheral information and saccades has intrigued researchers. Its influence could only be assessed, however, if subjects were instructed to hold their eyes still, or if stimuli were presented tachistoscopically before an eye movement could occur. The next advance in eye movement research came when a cluster of researchers in the 1970s were able to overcome these limitations by coupling the system that was measuring eye movements with the system that was presenting stimuli to the subjects. [40,41,42] During the 30–50 milliseconds that the eyes take to saccade to a new location, the visual system's sensitivity is greatly reduced. To trigger a change in the visual stimulus during that brief period, the saccade must be detected within a few milliseconds of onset, the appropriate stimulus change calculated by the computer, and the screen display refreshed. Using a limbus reflection system that sampled eye position every millisecond, McConkie and colleagues[41,43] were able to produce saccade‐contingent display control. In such experiments, the characters of a line of text could be changed while a subject was reading, and hence the amount of information that a subject detected peripherally could be assessed.
The advent of gaze‐contingent paradigms was a highly significant advance in eye movement research. It has had a far greater impact, perhaps, in its practical applications. The technology provides not merely a device a scientist can use to study a subject's eye movement behavior, but also a way for the subject's eye movement behavior to manipulate a device.
Dodge and Cline's[ 15 ] revolutionary invention allowed researchers, for the very first time, to make a permanent, objective record of a subject's eye movements using a noninvasive method. Although eye tracking techniques were refined in the following decades, there were no comparable breakthroughs until the advent of digital technology and image processing in the 1970s. Such advances have placed fewer constraints on experimental subjects, with the result that modern researchers are able to record the eye movements of freely moving subjects carrying out everyday tasks. We speculate that further developments will produce smaller, cheaper eye trackers that may well populate a range of human–computer interfaces. The potential ubiquity of such devices underscores the need for a solid understanding of the cognitive and perceptual processes that govern human eye movements.