Tad Brunyé - Eye tracking measures of uncertainty during perceptual decision making

Prévia do material em texto

Contents lists available at ScienceDirect
International Journal of Psychophysiology
journal homepage: www.elsevier.com/locate/ijpsycho
Eye tracking measures of uncertainty during perceptual decision making
Tad T. Brunyéa,b,c,⁎, Aaron L. Gardonya,b
a Center for Applied Brain and Cognitive Sciences, Medford, MA, United States
b U.S. Army Natick Soldier Research, Development, and Engineering Center, Natick, MA, United States
c Tufts University, Department of Psychology, Medford, MA, United States
A R T I C L E I N F O
Keywords:
Perceptual decision making
Eye tracking
Uncertainty
Pupil diameter
A B S T R A C T
Perceptual decision making involves gathering and interpreting sensory information to effectively categorize the
world and inform behavior. For instance, a radiologist distinguishing the presence versus absence of a tumor, or
a luggage screener categorizing objects as threatening or non-threatening. In many cases, sensory information is
not sufficient to reliably disambiguate the nature of a stimulus, and resulting decisions are done under conditions
of uncertainty. The present study asked whether several oculomotor metrics might prove sensitive to transient
states of uncertainty during perceptual decision making. Participants viewed images with varying visual clarity
and were asked to categorize them as faces or houses, and rate the certainty of their decisions, while we used eye
tracking to monitor fixations, saccades, blinks, and pupil diameter. Results demonstrated that decision certainty
influenced several oculomotor variables, including fixation frequency and duration, the frequency, peak velo-
city, and amplitude of saccades, and phasic pupil diameter. Whereas most measures tended to change linearly
along with decision certainty, pupil diameter revealed more nuanced and dynamic information about the time
course of perceptual decision making. Together, results demonstrate robust alterations in eye movement be-
havior as a function of decision certainty and attention demands, and suggest that monitoring oculomotor
variables during applied task performance may prove valuable for identifying and remediating transient states of
uncertainty.
1. Introduction
Perceptual decision making describes the process of accumulating
sensory evidence and using it to influence how we categorize, under-
stand, and behave within the world (Green and Heekeren, 2009;
Heekeren et al., 2008; Shadlen and Kiani, 2013). This process is ex-
ceedingly common in daily life; airport luggage screeners categorize
objects as threatening or non-threatening, pathologists categorize his-
tological features as normal or abnormal, and law enforcement officers
categorize handheld objects as weapons or non-weapons (Brunyé et al.,
2017; McCarley et al., 2004; Payne, 2001). In many cases, decisions are
made under conditions of uncertainty, which can arise due to occlusion
or distortion of the stimulus itself, or due to down-stream impacts of
attention, memory, emotion, and/or decision criteria (Heekeren et al.,
2008). Despite the ubiquity and importance of perceptual decision
making, and the potential impact of uncertainty on task performance,
surprisingly few studies have attempted to identify quantitative mea-
sures of decision uncertainty. The present study explores whether sev-
eral measures derived from eye tracking might be sensitive to varying
levels of uncertainty during a perceptual decision task.
1.1. Perceptual decision making and uncertainty
Gathering, combining, and interpreting information from the sen-
sory systems is critical for understanding the world and motivating
behavior. Several theories attempt to characterize the sensory, per-
ceptual, cognitive, and behavioral processes involved in perceptual
decision making. In one theory, decision making is considered a con-
tinuous process that transforms sensory information into categories
(e.g., face/house) (Opris and Bruce, 2005). Evidence accumulates from
sensory inputs, is integrated and compared with expectations and
knowledge, and then a behavioral response is selected. This proposed
serial progression from perception to action relies on diverse neural
circuits including the visual system, reward system, cognitive system,
and oculomotor system. Visual information, processed and interpreted
by the cognitive and reward systems, guides the oculomotor system to
shift gaze and gather information as needed. More recently, a relatively
dynamic account of perceptual decision making has emerged (Heekeren
et al., 2008). In this theory, four complementary and interactive sys-
tems are engaged. First, sensory systems, such as visual and tactile,
gather and compare information. Second, perceptual uncertainty or
http://dx.doi.org/10.1016/j.ijpsycho.2017.07.008
Received 10 April 2017; Received in revised form 6 July 2017
⁎ Corresponding author at: Center for Applied Brain and Cognitive Sciences, 200 Boston Ave., Suite 3000, Medford, MA 02155, United States.
E-mail address: tbruny01@tufts.edu (T.T. Brunyé).
International Journal of Psychophysiology 120 (2017) 60–68
Available online 18 July 2017
0167-8760/ Published by Elsevier B.V.
MARK
difficulty is detected, motivating and constraining attention toward
additional information gathering. Third, a cognitive system is used to
compare accumulated information against knowledge, and prepare or
execute a motor response. Finally, a performance monitoring system
assesses outcomes and adjusts behavior accordingly. These four pro-
cesses and their underlying neural circuitry are proposed to occur at
least partially in parallel, and interactively over time.
Relevant to the current research, there are a few important points to
realize about extant theories of perceptual decision making. First, cur-
rent theories consider the importance of oculomotor processes to stra-
tegically shift attention and gather information from a scene; this is
usually formalized through a role of the saccadic eye movement system
and its underlying neural substrates. Thus, there is some suggestion that
the perceptual decision making process engages and guides visual at-
tention to accumulate information relevant to a decision. Second, these
visual search processes appear to be contingent upon conditions of
uncertainty, which should exert some reliable influence over oculo-
motor behavior as information is gathered from a stimulus. Third, while
these theories tend to emphasize neural mechanisms underlying stages
of the perceptual decision process, using tools such as functional
magnetic resonance imaging, these tools may not be tractable for im-
plementation in applied settings intending to monitor perceptual deci-
sion uncertainty during task performance. This is in contrast to eye
tracking technology, which is increasingly available in lightweight,
mobile, and even wireless form factors (Weibel et al., 2012).
1.2. Measuring decision uncertainty
Given the breadth and complexity of sensory, cognitive, and motor
systems dynamically engaged during decision making, it is not sur-
prising that researchers have examined it using a wide range of beha-
vioral, physiological, and neurophysiological techniques. Using func-
tional magnetic resonance imaging (fMRI), research demonstrates that
particular portions of the left dorsolateral prefrontal cortex (DLPFC) are
involved in perceptual decision making (Heekeren et al., 2004). For
instance, when participants are tasked to distinguish faces versus
houses, their ability to do so correlates with activity in the left DLPFC.
The authors proposed that this brain region is engaged in perceptual
decisions by computing differences between activation in face- and
house-specific brain regions. Other research, using event-related elec-
troencephalography (EEG), has demonstrated reliable changes in early
and late event-related brain potentials related to discrimination of faceand car stimuli (Philiastides et al., 2006; Philiastides and Sajda, 2006).
Following stimulus onset, there was an early face-selective N170
component and a later component around 300 milliseconds; the authors
proposed that the early component reflected the early perception of the
stimulus, but the later component reflected the cognitive decision
process. Interestingly, the late component was highly sensitive to var-
iations in task difficulty, and correlated strongly with task accuracy and
response times. These results suggest that EEG may hold promise for
discriminating uncertainty states during perceptual decision making.
There are also fMRI data examining the impact of difficulty during
perceptual decision making. For instance, one study used fMRI to
monitor brain responses to the difficulty of a phonetic discrimination
task (Binder et al., 2004). They found that low-level sensory informa-
tion processing in the auditory cortex was related to decision accuracy,
whereas frontal brain regions showed activity correlated with response
uncertainty. Thus, both EEG and fMRI data suggest reliable spatio-
temporal dissociations of brain activity related to sensory discrimina-
tion and decision-related processes. Overall, task difficulty seems to
modulate both early and late phases of processing, and according to
some theories may promote attention deployment to resolve transient
states of uncertainty (Heekeren et al., 2008).
Regarding eye tracking, surprisingly few studies have explored how
eye movements might be modulated by uncertainty while participants
categorize a visual stimulus. As noted by Krajbich, this could be due to
an assumption that eye movements are relatively restricted and low
variability during single-stimulus perceptual categorization tasks
(Krajbich et al., 2010). Indeed many studies and computational models
have been devoted to characterizing eye movements during choice
decisions, which involve the comparison of multiple stimuli along a
variety of dimensions (Fiedler and Glockner, 2012; Krajbich and
Rangel, 2011; Orquin and Mueller Loose, 2013). However, several de-
cision-making theories make explicit mention of eye movements as an
important contributor to evidence accumulation during decision
making, without clearly differentiating between decisions made re-
garding single versus multiple stimuli. For instance, the drift diffusion
model proposes that eye fixations are used to sample a stimulus to
promote evidence accumulation (Krajbich and Rangel, 2011), and that
eye movements reflect the active deployment and control of attention.
This research, however, is largely constrained to tasks involving the
comparison of two images, which necessitates relatively large saccadic
eye movements between the stimuli. Thus, it is unknown whether si-
milar oculomotor dependence on uncertainty states will emerge with a
single and relatively constrained visual stimulus.
1.3. Oculomotor metrics of decision uncertainty
Though there is a paucity of research examining eye tracking during
the perceptual decision making with a single stimulus, more generally
several candidate eye tracking measures have been linked to decision
making, uncertainty, and task difficulty. These include oculomotor
metrics, such as fixation and saccade parameters, blinks, and alterations
in phasic pupil diameter. We review each of these below.
1.3.1. Fixations
Eye fixations are momentary pauses of eye movements within a
particular location that extend for a minimum duration (Duchowski,
2007). Fixations are thought to reflect the process of directing visual
attention toward a stimulus to bring it into foveal vision, which is the
highest resolution region of the retina and permits the greatest possible
visual detail. During more difficult decisions, participants tend to show
a higher number of fixations (Fiedler and Glockner, 2012; Krajbich
et al., 2010). This research, however, is limited to comparing two or
more visual alternatives. In contrast, when visually searching a single
scene for a stimulus, increasing search task difficulty tends to decrease
fixation frequency and increase fixation durations (Hooge and Erkelens,
1996; Jacobs and O'Regan, 1987), and also increase the duration of the
first fixation on the scene (Zelinsky and Sheinberg, 1997). Thus, re-
search is equivocal regarding the impact of difficulty on fixations:
during choice tasks fixations increase in frequency with more difficult
trials. In contrast, during single stimulus tasks fixation frequency de-
creases and duration increases during more difficult trials, perhaps to
provide more time accumulating evidence from certain regions of a
scene.
1.3.2. Saccades
Saccades describe the ballistic movements of the eyes between
successive fixations, which produces a continually changing sequence
of information projected onto the fovea (Liversedge and Findlay, 2000).
Saccades can be characterized in a few ways, such as the distance be-
tween successive saccades, or saccade amplitude in degrees, and the
speed of saccades in terms of average or peak velocity (in °/s). Saccadic
amplitude and peak velocity tend to be related in power-law function,
with relative saccade magnitude increases matched proportionately to
relative peak velocity increases (Bahill et al., 1975; Di Stasi et al.,
2013). In visual search contexts, saccadic amplitude tends to decrease
when a search task becomes more difficult (Jacobs and O'Regan, 1987;
Phillips and Edelman, 2008), such as when targets are more visually
similar to distractors. Furthermore, an emerging body of research
suggests that the peak velocity of saccades is a valuable index of diffi-
culty-evoked arousal during visual tasks (for a review, see (Di Stasi
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
61
et al., 2013), with lower peak velocity when search or decision diffi-
culty is increased.
1.3.3. Eye blinks
Eye blinks have also received considerable attention as a metric for
decision making, uncertainty, and task difficulty. In general, eye blinks
tend to be inhibited during the decision process, and those that do occur
tend to be shorter in duration (Boehm-Davis et al., 2000). Eye blinks
have also recently been considered as a metric of decision latency by
assessing the latency of blinking onset during or after a decision
(Hackley and Valle-Inclán, 1999; Siegle et al., 2008).
1.3.4. Pupil diameter
Dynamic variation in pupil diameter is elicited by both environ-
mental lighting conditions and decision uncertainty, though only a few
studies have examined it in the context of perceptual decision making.
Studies tracking pupil dynamics generally demonstrate an increased
phasic diameter in response to more difficult task conditions (Beatty,
1982; Brunyé et al., 2016; Geng et al., 2015; Laeng et al., 2012; Nassar
et al., 2012; Payzan-LeNestour and Bossaerts, 2012), such as when vi-
sual stimuli are difficult or ambiguous to discriminate, problem sets are
computationally demanding, or decisions are made under conditions of
uncertainty. When viewing stimuli evoking perceptual rivalry (e.g.,
Necker cube), phasic increases in pupil diameter tend to occur im-
mediately prior to a reported perceptual switch (Einhäuser et al., 2008).
Pupil diameter is also modulated during protracted decision-making
processes (de Gee et al., 2014), predicts subsequent choices (Urai et al.,
2017), is associated with reduced decision bias (de Gee et al., 2017;
Krishnamurthy et al., 2017), and may reflect variability in the rate of
evidence accumulation during decision making (Murphy et al., 2014).
These relationships are thought to at least partially reflect modulation
of the locus coeruleus-norepinephrine (LC-NE) system, with pupil dia-
meter changes reflecting cognitive control states and shifting between
exploring and exploiting visual information (Aston-Jones and Cohen,
2005; Gilzenrat et al., 2010).
1.4. Thepresent study
The present study provides a first integrated examination of several
eye tracking metrics and their dependency on uncertainty during per-
ceptual decision making. Participants viewed face and house images
parametrically varying in visual clarity (from low to high), and were
asked to decide whether each image depicted a face or house, and then
rated their certainty in their decision. During image viewing we used an
eye tracker to monitor fixations, saccades, blinks, and pupil diameter.
We expected to find that conditions of lower visual clarity would in-
crease rated decision uncertainty. We also expected that conditions of
increasing uncertainty would be related to alterations in fixation fre-
quency and duration; and given research concerning the search of a
single stimulus, we expected fixation duration to increase in more un-
certain conditions, perhaps correlating with a lower fixation frequency.
Saccades were expected to show lower amplitudes and possibly lower
peak velocities during more uncertain conditions, and blink frequency
was also expected to decline. We also expected several of the fixation-,
saccade-, and blink-related measures to be intercorrelated. Finally, we
expected phasic pupil diameter to increase during trials eliciting rela-
tively high uncertainty.
2. Method
2.1. Participants and design
Twenty male (n = 6) and female (n = 14) adults (Mage = 20.9,
SDage = 4.9) participated for monetary compensation ($20 USD).
Participants provided written informed consent in accordance with
human use approvals issued by Tufts University and the U.S. Army, in
conformity with the ethical standards of the Declaration of Helsinki. In
a within-participants design, all participants were exposed to a 2 × 8
design: image category was manipulated across two levels (2: face,
house), and image clarity was manipulated across eight levels (8: 0.275,
0.3, 0.325, 0.35, 0.4, 0.45, 0.5, 0.55). Three participants showed ex-
ceedingly poor eye tracking quality with> 40% dropped samples, and
were removed from analysis.
2.2. Materials
Materials consisted of face and house images varying in clarity.
Forty faces were selected from the Max Planck Institute for Biological
Cybernetics (http://faces.kyb.tuebingen.mpg.de/), face database, and
40 house photos were generated by Filimon and colleagues (Filimon
et al., 2013). Each face and house image was manipulated to vary in
phase coherence, with higher coherence indicating a clearer image
(e.g., 0.55), and lower meaning a noisier image (e.g., 0.275); all images
were provided to us by Filimon and colleagues, and details for the phase
coherence manipulation process can be found in their previous research
(Filimon et al., 2013). This process resulted in a total of 320 face images
(40 faces, 8 clarity levels) and 320 house images (40 houses, 8 clarity
levels). Each image was grayscale, 252 × 300 pixels, and clarity levels
were equated for spatial frequency, contrast, and luminance (Filimon
et al., 2013). We confirmed the latter by measuring luminance levels
across the 8 image clarity levels in MATLAB (MathWorks, Inc., Natick,
MA, USA); analysis demonstrated that luminance did not differ statis-
tically by clarity level (p > 0.05). Sample face and house images are
depicted in Fig. 1.
2.3. Equipment
Eye tracking was done using the Remote Eye Tracking Device (RED-
m) by SensoMotoric Instruments (SMI; Boston, MA, USA), sampling
binocular gaze position at 60 Hz. This system uses a 5-point calibration
to achieve 0.4° gaze position accuracy and 0.03° spatial resolution
(RMS). The device was attached to the bottom of a 24″ LCD monitor
running at 1920 × 1080 resolution. Image presentation and data col-
lection were done using the SMI Experiment Center software.
Fig. 1. Sample face and house images varying in
image clarity, from 0.275 (low clarity) to 0.55 (high
clarity).
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
62
2.4. Procedure
Participants were individually seated at a workstation, and the task
was presented using the SMI Experiment Center software. A total of 640
image were presented to participants in random order, one image at a
time in the center of the computer monitor. Each image was shown for
3 s at center against a black background, then followed by a response
screen showing the words FACE and HOUSE vertically aligned in the
center of the screen for 1.5 s. To respond, participants directed their
gaze toward FACE or HOUSE. Note that half of the trials showed the
word FACE above HOUSE, and vice-versa, to reduce the chance of
anticipatory eye movements during image viewing. We chose to use eye
movements as the response mechanism for compatibility with our ex-
perimental software.
After deciding, participants rated their certainty in their response,
ranging from 1 (not at all) to 7 (extremely). A fixation cross centered on
a black background (500 ms) appeared immediately after a rating was
entered. Sixteen of the 640 images (8 faces, 8 houses) were used for a
brief practice session to increase familiar with the task, resulting in 624
images viewed during the primary phase. At 60 cm seating distance,
images subtended 6.4° horizontal, and 7.7° vertical visual angle. The
entire task took approximately 1 h to complete.
2.5. Data processing
To evaluate decision accuracy, we used the position of the final
three fixations to determine whether gaze was directed at FACE or
HOUSE. To evaluate eye movements, we used the eye tracker's software
(BeGaze; SMI; Boston, MA) to parse data into fixations, saccades, and
blinks. Fixations thresholds were> 99 ms and maximum dispersion
100 px. Saccades thresholds were a minimum duration of 22 ms, and
peak velocity threshold of 40°/s, with a start and end of 20% and 80%
of saccade length, respectively. To calculate pupil diameter, we used
raw data logs and removed and interpolated (linear) periods of
blinking; pupil diameter data were referenced (via mean subtraction) to
a 200 ms baseline preceding stimulus onset, and smoothed using a 5-
point moving average filter. These techniques were adopted from ear-
lier research (Van Der Meer et al., 2010).
3. Results
3.1. Accuracy
Overall accuracy was significantly above chance (M= 0.82,
SD= 0.17), t(16) = 18.71, p < 0.001, Cohen's d = 4.5. A 2(image
category: face, house) × 8(image clarity: 0.275, 0.3, 0.325, 0.35, 0.4,
0.45, 0.5, 0.55) repeated-measures ANOVA demonstrated main effects
of image category, F(1, 16) = 6.18, p= 0.024, ƞ2 = 0.05, image
clarity, F(7, 112) = 73.64, p < 0.001, ƞ2 = 0.45, and an interaction, F
(7, 112) = 6.32, p < 0.001, ƞ2 = 0.08. Accuracy increased as a
function of image clarity for both faces and houses, but for houses ac-
curacy was higher at relatively low clarity levels. The psychometric
function relating image category, image clarity, and the probability of
responding “House” is plotted in Fig. 2a. There was a slight response
bias toward responding house, which was chosen during 51.7% of
trials, in comparison to face which was chosen during 48.3% of trials;
recall that exactly 50% of trials had faces and houses. This pattern was
verified in an exact binomial test (z= 4.93, p < 0.001).
3.2. Certainty ratings
Overall certainty ratings were moderately high on the 7-point scale
(M= 4.99, SD= 1.52). A 2 × 2 repeated-measures ANOVA demon-
strated main effects of image category, F(1, 16) = 4.7, p= 0.046,
ƞ2 < 0.01, image clarity, F(7, 112) = 79.57, p < 0.001, ƞ2 = 0.80,
and an interaction, F(7, 112) = 6.94, p < 0.001, ƞ2 < 0.01. Certainty
ratings increased as a function of image clarity for both faces and
houses; however, face ratings were higher at relatively low clarity levels
(Fig.2b). A more detailed examination of confidence ratings as a func-
tion of trial type and accuracy revealed a tendency to incorrectly judge
face trials of low-to-medium clarity as “house”(as seen in Fig. 2b), but
with high certainty. This resulted in overall higher ratings during face
trials, but even when responses were incorrect. In contrast, incorrectly
answering “face” during house trials consistently resulted in lower
certainty ratings, as would be expected. These data are detailed in
Table 1.
Certainty ratings were also associated with accuracy, as verified in
an ANOVA evaluating accuracy as a function of certainty rating (1–7), F
(6, 96) = 23.35, p < 0.001, ƞ2 < 0.59. In general, for each one-digit
certainty increase, there was a relative accuracy increase of about 11%.
Mean accuracy for the 7 ratings were: 0.49, 0.61, 0.72, 0.72, 0.85, 0.86,
and 0.92, respectively.
3.3. Data reduction
Given our primary interest in understanding the dependence of
oculomotor metrics on perceptual decision uncertainty, remaining
analyses were guided by certainty ratings. Given a severely skewed
distribution of certainty responses (Fig. 3), there were relatively few
trials in lower ratings categories. To ensure a similar number of trials
included in each certainty condition, we parsed trials into three cate-
gories: low certainty (ratings 1–4), medium certainty (ratings 5–6), and
high certainty (rating 7). This resulted in 3551, 3469, and 3566 trials in
each of the three conditions, respectively; this data reduction com-
pensated for the skewed distribution and further allowed us to analyze
Fig. 2. a. Psychometric function relating image category (faces, houses) and clarity, to the
probability of responding “House.”
b. Mean certainty ratings (and standard error) for the two Image Categories and eight
image clarity conditions.
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
63
using ANOVAs. Note that with this method, accuracy remained sig-
nificantly different as a function of certainty condition (Mlow = 0.65,
Mmed = 0.85, Mhigh = 0.92), F(2, 32) = 31.11, p < 0.001, ƞ2 = 0.66
(all pairwise comparisons p < 0.001).
3.4. Fixations
To examine fixation frequency and duration, we conducted a series
of 2(Image Type: face, house) × 3(Certainty: low, medium, high)
ANOVAs; data are depicted in Fig. 4. Note there were never any main or
interactive effects of Image Type (face versus house).
Fixation frequency showed only a main effect of Certainty, F(2, 32)
= 3.83, p= 0.032, ƞ2 = 0.17. Planned comparisons using paired t-
tests demonstrated significantly lower fixation rates in the medium
versus high, t(16) = 2.25, p= 0.019, Cohen's D = 0.55, and low versus
high, t(16) = 2.12, p= 0.25, Cohen's d = 0.51, conditions.
Fixation duration also only showed a main effect of Certainty, F(2,
32) = 8.81, p= 0.001, ƞ2 = 0.24. Planned comparisons using paired t-
tests demonstrated significantly longer mean fixation duration in the
low versus medium condition, t(16) = 3.16, p= 0.006, Cohen's
D = 0.77, the medium versus high condition, t(16) = 8.78, p < 0.001,
Cohen's D = 2.15, and the low versus high condition, t(16) = 8.88,
p < 0.001, Cohen's D = 2.13.
First fixation duration showed a similar pattern, with only a main
effect of Certainty, F(2, 32) = 3.76, p= 0.034, ƞ2 = 0.11. Planned
comparisons demonstrated significantly longer mean first fixation
duration in the low (M= 268.5, SE= 51.2) versus high (M= 244.9,
SE= 45.1) condition, t(16) = 2.41, p= 0.02, Cohen's D = 0.58, and
the medium (M= 255.9, SE= 43.3) versus high condition, t(16)
= 2.0, p= 0.031, Cohen's D = 0.49.
3.5. Saccades
We used a 2 × 3 ANOVA to examine saccadic frequency, amplitude,
average and peak velocity, and average and peak acceleration
(Table 2). Note there were never any main or interactive effects of
Image Type (face versus house).
Saccadic frequency showed only a main effect of Certainty, F(2, 32)
= 5.1, p= 0.012, ƞ2 = 0.21; planned comparisons demonstrated sig-
nificantly fewer saccades in the low versus medium, t(16) = 1.85,
p= 0.042, Cohen's D = 0.45, low versus high, t(16) = 2.39,
p= 0.015, Cohen's D = 0.58, and medium versus high, t(16) = 2.28,
p= 0.015, Cohen's D = 0.58, conditions.
Saccadic amplitude showed only a marginal main effect of
Certainty, F(2, 32) = 2.99, p= 0.064, ƞ2 = 0.06, with numerically
higher amplitude with increasing certainty conditions.
Saccadic average velocity showed only a main effect of Certainty, F
(2, 32) = 4.65, p= 0.017, ƞ2 = 0.11; planned comparisons demon-
strated significantly lower saccadic average velocity in the low versus
high condition, t(16) = 2.73, p= 0.007, Cohen's D = 0.66. No other
comparisons reached significance (all p > 0.05).
Saccadic peak velocity also showed a main effect of Certainty, F(2,
32) = 4.02, p= 0.028, ƞ2 = 0.10; comparisons demonstrated sig-
nificantly lower peak velocity in the low versus high condition, t(16)
= 2.37, p= 0.015, Cohen's D = 0.57 (Fig. 5). No other comparisons
Table 1
Mean (and standard error) certainty ratings for correct and incorrect trials, for the eight
image clarity levels, and two image types (faces, houses).
Image clarity and type Decision accuracy
Correct Incorrect
0.275 Faces 3.64 (0.88) 3.53 (0.33)
Houses 4.04 (0.98) 3.08 (0.32)
0.3 Faces 3.79 (0.92) 3.74 (0.31)
Houses 4.18 (1.0) 3.14 (0.35)
0.325 Faces 4.11 (1.0) 3.72 (0.30)
Houses 4.20 (1.0) 2.84 (0.23)
0.35 Faces 4.55 (1.1) 4.20 (0.34)
Houses 4.37 (1.1) 3.14 (0.31)
0.4 Faces 5.50 (1.3) 4.43 (0.28)
Houses 4.98 (1.2) 3.22 (0.39)
0.45 Faces 6.19 (1.5) 5.44 (0.31)
Houses 5.84 (1.4) 5.09 (0.29)
0.5 Faces 6.62 (1.6) 6.40 (0.25)
Houses 6.61 (1.6) 5.83 (0.38)
0.55 Faces 6.75 (1.6) 6.10 (0.31)
Houses 6.70 (1.6) 5.54 (0.49)
Fig. 3. Histogram of cumulative response frequency as a function of certainty rating.
Dimension reduction allowed us to compensate for the negative skew.
Fig. 4. Mean change in fixation duration (bars, left axis) and
fixation frequency (line, right axis) as a function of certainty
condition. Error bars depict standard error. Change is calculated
as difference from each participant's overall mean fixation
duration and frequency (across all conditions).
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
64
reached significance (all p > 0.05).
3.6. Blinks
We used a 2 × 3 ANOVA to examine blink frequency, duration, and
first blink latency (Table 2). Note there were never any main or inter-
active effects of Image Type (face versus house).
Blink frequency showed only a main effect of Certainty, F(2, 32)
= 6.87, p= 0.003, ƞ2 = 0.16; comparisons demonstrated significantly
lower blink rates in the low versus medium, t(16) = 2.88, p= 0.011,
Cohen's D = 0.69, and low versus high conditions, t(16) = 3.84,
p= 0.001, Cohen's D = 0.93. Medium versus high did not reach sig-
nificance, and blink duration did not show significant main or inter-
active effects of Certainty (all p > 0.05).
3.7. Correlations between measures
A correlation matrix relating our nine oculomotor measures of in-
terest is detailed in Table 3. Most measures were interrelated. Some
relationships were not especially surprising, such as the negative re-
lationship between fixation frequency and duration, and positive re-
lationships between fixation duration and first fixation duration, and
between saccadic average and peak velocity. However, some novel in-
sights were derived from this correlational analysis. Specifically, sac-
cadic amplitude and saccadic average and peak velocity were all posi-
tively correlated with blink frequency. This finding suggests that
common mechanisms may underlie the impact of decision uncertainty
on saccadic and blink parameters. Not only are participants more likely
to keep their eyes open during conditions of uncertainty, but during
these periods they are more likely to move their eyes more slowly and
across shorter distances.
3.8. Pupil diameter
Mean relative pupil diameter responses were analyzed by examining
responses time-locked to the onsetof a trial stimulus. Following prior
research, we analyzed pupil dilation relative to baseline in each of six
500-millisecond epochs (Van Der Meer et al., 2010).
A repeated-measures ANOVA was conducted, using a 2(image ca-
tegory: face, house) × 3(certainty: low, med, high) × 6(time: 500 ms
bins) design. There were no main or interactive effects of image cate-
gory, so this variable was removed from further analysis. The 3 × 6
ANOVA demonstrated an interaction between Certainty and Time, F
(10, 160) = 6.01, p < 0.001, ƞ2 = 0.03. As depicted in Fig. 6, relative
to the high certainty condition there was an early and protracted in-
crease in the medium certainty condition, and a relatively latent in-
crease in the low certainty condition.
These patterns were confirmed in a series of simple effects ANOVAs
in each of the six time bins. Within bins 1 (0–500 ms) and 2
(500–1000 ms), there were no effects of Certainty, F(2, 32) = 0.98,
p= 0.39, ƞ2 = 0.06, and F(2, 32) = 1.02, p= 0.37, ƞ2 = 0.06, re-
spectively. Within bins 3 (1000–1500 ms) and 4 (1500–2000 ms),
though there was no main effect of Certainty (p's > 0.05), pupil dia-
meter was higher in the medium versus high certainty conditions, t(16)
= 2.22, p= 0.021, Cohen's D = 0.54, and t(16) = 2.32, p= 0.017,
Cohen's D = 0.56, respectively (other comparisons p > 0.05). Within
bin 5 (2000–2500 ms), there was a main effect of Certainty, F(2, 32)
= 3.92, p= 0.03, ƞ2 = 0.19; pupil diameter was higher in the low
versus high certainty conditions, t(16) = 2.28, p= 0.018, Cohen's
D = 0.55, and the medium versus high certainty conditions, t(16)
= 2.96, p < 0.01, Cohen's D = 0.72. The same pattern was found in
bin 6 (2500-3000 ms), with a main effect of Certainty, F(2, 32) = 5.37,
p= 0.01, ƞ2 = 0.25; there were higher pupil diameters in the low
versus high certainty conditions, t(16) = 2.74, p < 0.01, Cohen's
D = 0.55, and the medium versus high certainty conditions, t(16)
= 2.88, p < 0.01, Cohen's D = 0.7. Note that within each bin, t-tests
remain significant with a more conservative Bonferroni correction
term.
Given that large changes in eye position on a screen (e.g., > 2–4°)
can modulate pupil diameter measurement (Gagl et al., 2011), we re-
analyzed pupil data after eliminating data points wherein the eye's
point of regard was not within the image coordinate space. We thank an
anonymous reviewer for this suggestion. This process resulted in the
removal of approximately 12% of all data, primarily driven by eccentric
point of regard during the first several samples of trials. The overall
pattern of pupil responses, and all significant statistical results, main-
tained after this correction; interaction term: F(10, 160) = 5.92,
p < 0.001, ƞ2 = 0.03.
4. General discussion
The present experiment used a perceptual decision making task and
eye tracking to explore the sensitivity of oculomotor measures to mo-
mentary states of uncertainty. Participants attempted to judge whether
stimuli of varying clarity were faces or houses, and then rated the
certainty of their judgments. Overall accuracy tended to be above
chance, and as the clarity of the stimuli increased, we found robust
increases in discrimination accuracy and certainty ratings. These pat-
terns match earlier research using this same perceptual decision making
task (Filimon et al., 2013). Participants also showed higher accuracy
with house versus face judgments, which appeared to be due to a
modest response bias toward houses during relatively ambiguous trials.
Table 2
Mean and standard deviation values for fixation, saccade, and blink measures (not
otherwise included in figures), for the two image categories, and three certainty condi-
tions.
Decision certainty
Low Medium High
1st fixation duration (ms) Faces 275.9,
219.9
257.3, 180.6 241.5,
181.2
Houses 261.6,
204.7
253.3, 178.5 249.7,
190.7
Saccade count Faces 5.2, 1.2 5.5, 1.3 5.7, 1.3
Houses 5.2, 1.2 5.5, 1.3 5.8, 1.2
Saccade amplitude (°) Faces 5.5, 3.8 5.6, 3.5 6.1, 4.1
Houses 5.5, 3.1 5.7, 3.5 5.8, 2.8
Average saccade velocity
(°/s)
Faces 128.5, 48.4 132, 47.1 140.3, 46.9
Houses 131.5, 45.3 134.5, 46.6 140.1, 37.9
Blink frequency Faces 1.0, 0.6 1.13, 0.62 1.17, 0.63
Houses 1.1, 0.61 1.11, 0.58 1.14, 0.59
Blink duration Faces 238.9,
156.4
218.4, 83.9 212.8, 76.4
Houses 211.9, 80.5 206.7, 69.8 193.7, 46.9
Fig. 5. Mean change in saccadic peak velocity (in °/sec), as a function of certainty con-
dition. Error bars depict standard error. Change is calculated as difference from each
participant's overall mean saccade peak velocity (across all conditions).
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
65
Certainty ratings were higher during face versus house trials, though
this pattern appeared to be driven by a tendency to confidently but
incorrectly perceive a house during ambiguous face trials.
Several eye tracking measures proved sensitive to varying states of
decision certainty. Overall, during conditions of relative uncertainty we
found that there were fewer but longer fixations, fewer saccades, and
saccades were slower and lower in amplitude. In other words, eye
movements were relatively constrained and focused on particular re-
gions of the stimulus during conditions of relative uncertainty. The
most robust effect, in terms of effect size, was found with fixation
durations. This pattern matches previous visual search research
showing longer fixation durations during more difficult search tasks,
and extends it to the domain of perceptual decision making (Hooge and
Erkelens, 1996; Jacobs and O'Regan, 1987). The pattern of smaller
saccade amplitudes is also similar to patterns seen with more difficult
visual search tasks (Jacobs and O'Regan, 1987; Phillips and Edelman,
2008). Finally, the lower peak saccade velocity supports research sug-
gesting that peak saccade velocity is a valuable indicator of arousal,
which tends to decrease in response to increasing task difficulty (Di
Stasi et al., 2013). However, we note that the effect sizes relating un-
certainty to saccade-based measures were relatively small, and confined
predominantly to comparing the two more extreme conditions (low-
high). Overall, our results support relatively disparate research areas,
demonstrating that several oculomotor measures are modulated by
uncertainty during perceptual decision making. Interestingly, these
measures were sensitive to uncertainty-based variation in a task with a
single, and relatively small, stimulus, complementing a larger body of
literature with multi-stimulus choice tasks (Fiedler and Glockner, 2012;
Krajbich and Rangel, 2011; Orquin and Mueller Loose, 2013).
Our fixation and saccade data support recent perceptual decision
making theory that posits a dynamic adaptive attention deployment in
response to detected uncertainty or difficulty (Heekeren et al., 2008).
The present study demonstrates that strategically shifting attention to
gather information from a scene is evident in oculomotor processes,
including longer fixations, slower eye movements, and examining re-
latively restricted areas of a scene. We also saw some evidence that
there were fewer blinks during relatively low certainty conditions, and
interestingly this pattern was strongly correlated with saccade para-
meters. Overall, the pattern of fixations, saccades, and blinks, suggests a
somewhat linear modulation of oculomotor behavior in response to
decision uncertainty.
A more nuanced understanding of the time-course of perceptual
decision making was afforded by examining pupil diameter over time,
under varying conditions of uncertainty. Current theories of pupil
diameter modulation in response to perceptual and cognitive tasks
suggest that diameter changes reflect shifting between exploration and
exploitation cognitive states (Aston-Jones and Cohen, 2005; Gilzenrat
et al., 2010; Privitera et al.,2010; Usher et al., 1999). Specifically,
current theory suggests that the LC-NE system drives behavioral se-
lection by switching between exploration (continue searching) and
exploitation (decide and select a behavior). Though this theory is lar-
gely based on behavioral decision making, the pattern has also been
demonstrated under conditions of perceptual ambiguity, with in-
creasing pupil diameters immediately prior to a participant reporting a
switch during perceptual rivalry tasks (Einhäuser et al., 2008).
The present research finds a similar pattern of pupil diameter in-
creases prior to decision making, particularly during conditions of un-
certainty. Specifically, pupil responses during high certainty trials tend
to be relatively flat over time, without any notable rise during the time
course of exploring the stimulus and deciding. In contrast, pupil re-
sponses during medium certainty trials tend to show a relatively rapid
dilation response beginning approximately 1 s following stimulus onset.
During low certainty trials, there is a pronounced but relatively latent
dilation response beginning at approximately 2 s following stimulus
onset. This is an interesting distinction, providing some support for
current theory (Aston-Jones and Cohen, 2005; Gilzenrat et al., 2010).
Specifically, it appears that participants might switch from an ex-
ploration to exploitation state sooner after stimulus onset when the
perceptual signal is difficult but possible to derive. In contrast, in more
uncertain conditions the switch to exploitation occurs immediately
prior to the required response, possibly reflecting the eventual arrival at
a decision in the absence of a clear perceptual signal. These mid-to-late
pupil dynamics may also reflect alterations in cognitive effort im-
mediately prior to a response (i.e., arriving at a difficult decision in the
relative absence of evidence); we believe it is unlikely to be due to
motor preparation given that any pupil response due to motor pre-
paration should be consistent across conditions. This relatively late
process may be similar to the pupil dilation response seen during per-
ceptual rivalry tasks, wherein one perceptual interpretation “pops out”
even in the absence of a clear perceptual signal (Einhäuser et al., 2008).
This finding is inconsistent with some recent research suggesting rela-
tively monotonic increases in pupil diameter as a function of increased
Table 3
Correlation matrix relating the nine oculomotor measures of interest. **Indicates p < 0.01, *indicates p < 0.05 (two-tailed Pearson coefficient).
Fixation
frequency
Fixation
duration
First fixation
duration
Saccade
frequency
Saccade
amplitude
Saccade average
velocity
Saccade peak
velocity
Blink
frequency
Blink
duration
Fixation frequency 1 −0.857⁎⁎ −0.771⁎⁎ 0.975⁎⁎ 0.333⁎ 0.316⁎ 0.293⁎ 0.378⁎⁎ −0.181
Fixation duration 1 0.969⁎⁎ −0.895⁎⁎ −0.473⁎⁎ −0.493⁎⁎ −0.496⁎⁎ −0.546⁎⁎ −0.043
First fixation
duration
1 −0.807⁎⁎ −0.392⁎⁎ −0.479⁎⁎ −0.473⁎⁎ −0.481⁎⁎ −0.039
Saccade frequency 1 0.494⁎⁎ 0.409⁎⁎ 0.417⁎⁎ 0.509⁎⁎ −0.117
Saccade amplitude 1 0.663⁎⁎ 0.793⁎⁎ 0.822⁎⁎ 0.120
Saccade average
velocity
1 0.975⁎⁎ 0.801⁎⁎ −0.296⁎
Saccade peak
velocity
1 0.870⁎⁎ −0.212
Blink frequency 1 −0.080
Blink duration 1
Fig. 6. Mean change in pupil diameter (in millimeters) over time while viewing stimulus
images, as a function of the three certainty levels (low, medium, high).
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
66
uncertainty (Urai et al., 2017). However, we note that this prior re-
search measures pupil diameter after a decision and immediately prior
to decision feedback, rather than during stimulus viewing. To our
knowledge, ours is the first evidence of a shifted pupil dilation latency
while participants view a visual stimulus eliciting variable certainty,
possibly signaling the differential time course of the perceptual decision
process as a function of uncertainty. Future research will assess the
reliability and generality of this pattern.
Although we used a restricted set of variably ambiguous stimuli, we
note that many of the present patterns have also been found with re-
latively real-world visual stimuli. For instance, the pupil dilates when
pathologists fixate on and correctly interpret diagnostically relevant
regions of a medical image, but only when the diagnosis is relatively
challenging (Brunyé et al., 2016). Pupils also dilate when radar op-
erators detect threats in increasingly dense displays (Van Orden et al.,
2001), and when drivers face ambiguous road conditions (Pedrotti
et al., 2014). Furthermore, reductions in peak saccade velocity have
also been found in response to workload demands in simulated air
traffic control and driving simulations (Di Stasi et al., 2013). Given the
ubiquity of uncertainty during real-world perceptual tasks, we present
research informs not only theory but also the development of closed-
loop systems to monitor and respond to transient states of user un-
certainty. Continuing research will consider the relative and interactive
value of multiple oculomotor measures for the real-time classification
of uncertainty states during the performance of relatively real-world
tasks.
There are a few limitations worth considering. First, because cer-
tainty ratings were highly skewed toward positive responses, our ana-
lyses used discretized categories of certainty ratings to increase stability
and power. Future research may consider using a scale with a relatively
restricted range of responses (e.g., 1–4) to avoid dimension reduction.
Second, our eye tracker used a 60 Hz sampling frequency, though
higher frequency rates may be necessary to identify relatively granular
patterns of saccade behavior. Third, to increase our statistical power,
each participant viewed 640 trials, which likely induced some fatigue
over the course of the experiment. To examine this possibility, we re-
analyzed our saccade, fixation, and blink data with the inclusion of time
(first vs. second half) as an independent variable. Though time tended
to impact saccadic peak and average velocity, there were no interac-
tions with certainty conditions across any of our dependent variables. In
other words, time on task does impact some saccadic measures, as
previously reported (Di Stasi et al., 2011; Di Stasi et al., 2010), though
the influence of certainty on oculomotor metrics remains unchanged.
Further, we also note that we cannot be certain that uncertainty in the
absence of our stimulus manipulation (i.e., clarity) would modulate
oculomotor and pupil outcomes in a similar manner to that described
herein; in other words, uncertainty is driven by the perceptual ambi-
guity of our stimuli, though uncertainty in the absence of our stimuli
may modulate pupil diameter differently.
To our knowledge, this is the first study to demonstrate a wide range
of eye tracking measures dependent on uncertainty during perceptual
decision making. During transient states of uncertainty, we find that the
eye makes fewer and longer fixations, and slower and shorter saccades.
This process is complemented by dynamic shifts of pupil dilation re-
sponses, suggestive of variable-latency shifts between explore-exploit
cognitive states as a function of uncertainty.
Competing interests
The authors have no actual or potential conflicts of interest.
Acknowledgements
We thank Lindsay Houck for assistance with data collection. This
work was supported by the U.S. Army Natick Soldier Research,
Development and Engineering Center under grant number W911-QY-
13-C-0012.
References
Aston-Jones, G., Cohen, J.D., 2005. An integrative theory of locus coeruleus-nor-
epinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci.
28, 403–450. http://dx.doi.org/10.1146/annurev.neuro.28.061604.135709.
Bahill, A.T., Clark,M.R., Stark, L., 1975. The main sequence, a tool for studying human
eye movements. Math. Biosci. 24, 191–204. http://dx.doi.org/10.1016/0025-
5564(75)90075-9.
Beatty, J., 1982. Task-evoked pupillary responses, processing load, and the structure of
processing resources. Psychol. Bull. 91, 276–292. http://dx.doi.org/10.1037/0033-
2909.91.2.276.
Binder, J.R., Liebenthal, E., Possing, E.T., Medler, D.A., Ward, B.D., 2004. Neural cor-
relates of sensory and decision processes in auditory object identification. Nat.
Neurosci. 7 (3), 295–301. http://dx.doi.org/10.1038/nn1198.
Boehm-Davis, D.A., Gray, W.D., Schoelles, M.J., 2000. The eye blink as a physiological
indicator of cognitive workload. Proceedings of the Human Factors and Ergonomics
Society Annual Meeting 44, 6-116–6-119. http://dx.doi.org/10.1177/
154193120004403309.
Brunyé, T.T., Eddy, M.D., Mercan, E., Allison, K.H., Weaver, D.L., Elmore, J.G., 2016.
Pupil diameter changes reflect difficulty and diagnostic accuracy during medical
image interpretation. BMC Medical Informatics and Decision Making 16, 77.
Brunyé, T.T., Mercan, E., Weaver, D.L., Elmore, J.G., 2017. Accuracy is in the eyes of the
pathologist: the visual interpretive process and diagnostic accuracy with digital
whole slide images. J. Biomed. Inform. 66, 171–179.
Di Stasi, L.L., Marchitto, M., Antolí, A., Baccino, T., Cañas, J.J., 2010. Approximation of
on-line mental workload index in ATC simulated multitasks. J. Air Transp. Manag. 16
(6), 330–333. http://dx.doi.org/10.1016/j.jairtraman.2010.02.004.
Di Stasi, L.L., Antoli, A., Canas, J.J., 2011. Main sequence: an index for detecting mental
workload variation in complex tasks. Appl. Ergon. 42 (6), 807–813. http://dx.doi.
org/10.1016/j.apergo.2011.01.003.
Di Stasi, L.L., Catena, A., Cañas, J.J., Macknik, S.L., Martinez-Conde, S., 2013. Saccadic
velocity as an arousal index in naturalistic tasks. Neurosci. Biobehav. Rev. http://dx.
doi.org/10.1016/j.neubiorev.2013.03.011.
Duchowski, A.T., 2007. Eye Tracking Methodology: Theory and Practice. Book. Springer-
Verlag, New York, NY. http://dx.doi.org/10.1007/978-1-84628-609-4.
Einhäuser, W., Stout, J., Koch, C., Carter, O., 2008. Pupil dilation reflects perceptual
selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci.
U. S. A. 105 (5), 1704–1709. http://dx.doi.org/10.1073/pnas.0707727105.
Fiedler, S., Glockner, A., 2012. The dynamics of decision making in risky choice: an eye-
tracking analysis. Front. Psychol. 3 (OCT). http://dx.doi.org/10.3389/fpsyg.2012.
00335.
Filimon, F., Philiastides, M.G., Nelson, J.D., Kloosterman, N.A., Heekeren, H.R., 2013.
How embodied is perceptual decision making? Evidence for separate processing of
perceptual and motor decisions. J. Neurosci. 33 (5), 2121–2136. http://dx.doi.org/
10.1523/JNEUROSCI.2334-12.2013.
Gagl, B., Hawelka, S., Hutzler, F., 2011. Systematic influence of gaze position on pupil
size measurement: analysis and correction. Behav. Res. Methods 43 (4), 1171–1181.
http://dx.doi.org/10.3758/s13428-011-0109-5.
de Gee, J.W., Knapen, T., Donner, T.H., 2014. Decision-related pupil dilation reflects
upcoming choice and individual bias. Proc. Natl. Acad. Sci. 111 (5), E618–E625.
http://dx.doi.org/10.1073/pnas.1317557111.
de Gee, J.W., Colizoli, O., Kloosterman, N.A., Knapen, T., Nieuwenhuis, S., Donner, T.H.,
2017. Dynamic modulation of decision biases by brainstem arousal systems. elife 6,
e23232.
Geng, J.J., Blumenfeld, Z., Tyson, T.L., Minzenberg, M.J., 2015. Pupil diameter reflects
uncertainty in attentional selection during visual search. Front. Hum. Neurosci. 9
(August), 435. http://dx.doi.org/10.3389/fnhum.2015.00435.
Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M., Cohen, J.D., 2010. Pupil diameter tracks
changes in control state predicted by the adaptive gain theory of locus coeruleus
function. Cogn. Affec. Behav. Neurosci. 10 (2), 252–269. http://dx.doi.org/10.3758/
CABN.10.2.252.
Green, N., Heekeren, H.R., 2009. Perceptual decision making: a bidirectional link be-
tween mind and motion. Prog. Brain Res. http://dx.doi.org/10.1016/S0079-
6123(09)01317-X.
Hackley, S.A., Valle-Inclán, F., 1999. Accessory stimulus effects on response selection:
does arousal speed decision making? J. Cogn. Neurosci. 11 (3), 321–329. http://dx.
doi.org/10.1162/089892999563427.
Heekeren, H.R., Marrett, S., Bandettini, P.A., Ungerleider, L.G., 2004. A general me-
chanism for perceptual decision-making in the human brain. Nature 431, 859–862.
Heekeren, H.R., Marrett, S., Ungerleider, L.G., 2008. The neural systems that mediate
human perceptual decision making. Nat. Rev. Neurosci. 9 (6), 467–479. http://dx.
doi.org/10.1038/nrn2374.
Hooge, I.T., Erkelens, C.J., 1996. Control of fixation duration in a simple search task.
Percept. Psychophys. 58 (7), 969–976. http://dx.doi.org/10.1017/
CBO9781107415324.004.
Jacobs, A.M., O'Regan, J.K., 1987. Spatial and/or temporal adjustments of scanning be-
havior to visibility changes. Acta Psychol. 65 (2), 133–146. http://dx.doi.org/10.
1016/0001-6918(87)90023-0.
Krajbich, I., Rangel, A., 2011. Multialternative drift-diffusion model predicts the re-
lationship between visual fixations and choice in value-based decisions. Proc. Natl.
Acad. Sci. U. S. A. 108 (33), 13852–13857. http://dx.doi.org/10.1073/pnas.
1101328108.
Krajbich, I., Armel, C., Rangel, A., 2010. Visual fixations and the computation and
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
67
comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298.
Krishnamurthy, K., Nassar, M.R., Sarode, S., Gold, J.I., 2017. Arousal-related adjustments
of perceptual biases optimize perception in dynamic environments. Nat. Hum. Behav.
http://dx.doi.org/10.1038/s41562-017-0107.
Laeng, B., Sirois, S., Gredeback, G., 2012. Pupillometry: a window to the preconscious?
Perspect. Psychol. Sci. 7 (1), 18–27. http://dx.doi.org/10.1177/1745691611427305.
Liversedge, S., Findlay, J., 2000. Saccadic eye movements and cognition. Trends Cogn.
Sci. 4 (1), 6–14. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10637617.
McCarley, J.S., Kramer, A.F., Wickens, C.D., Vidoni, E.D., Boot, W.R., 2004. Visual skills
in airport-security screening. Psychol. Sci. 15 (5), 302–306. http://dx.doi.org/10.
1111/j.0956-7976.2004.00673.x.
Murphy, P.R., Vandekerckhove, J., Nieuwenhuis, S., 2014. Pupil-linked arousal de-
termines variability in perceptual decision making. PLoS Comput. Biol. 10 (9). http://
dx.doi.org/10.1371/journal.pcbi.1003854.
Nassar, M.R., Rumsey, K.M., Wilson, R.C., Parikh, K., Heasly, B., Gold, J.I., 2012. Rational
regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15
(7), 1040–1046. http://dx.doi.org/10.1038/nn.3130.
Opris, I., Bruce, C.J., 2005. Neural circuitry of judgment and decision mechanisms. Brain
Res. Rev. 48, 509–526.
Orquin, J.L., Mueller Loose, S., 2013. Attention and choice: a review on eye movements in
decision making. Acta Psychol. 144 (1), 190–206. http://dx.doi.org/10.1016/j.
actpsy.2013.06.003.
Payne, B.K., 2001. Prejudice and perception: the role of automatic and controlled pro-
cesses in misperceiving a weapon. J. Pers. Soc. Psychol. 81, 181–192.
Payzan-LeNestour, É., Bossaerts, P., 2012. Do not bet on the unknown versus try to find
out more: estimation uncertainty and “unexpected uncertainty” both modulate ex-
ploration. Front. Neurosci.(OCT). http://dx.doi.org/10.3389/fnins.2012.00150.
Pedrotti, M., Mirzaei, M.A., Tedesco, A., Chardonnet, J., Mérienne, F., Benedetto, S.,
Baccino, T., 2014. Automatic stress classification with pupil diameter analysis. Int.
Journal of Human-Computer Interaction 30 (March), 1–17. http://dx.doi.org/10.
1080/10447318.2013.848320.
Philiastides, M.G., Sajda, P., 2006. Temporalcharacterization of the neural correlates of
perceptual decision making in the human brain. Cereb. Cortex 16 (4), 509–518.
http://dx.doi.org/10.1093/cercor/bhi130.
Philiastides, M.G., Ratcliff, R., Sajda, P., 2006. Neural representation of task difficulty and
decision making during perceptual categorization: a timing diagram. J. Neurosci. 26
(35), 8965–8975. http://dx.doi.org/10.1523/JNEUROSCI.1655-06.2006.
Phillips, M.H., Edelman, J.A., 2008. The dependence of visual scanning performance on
search direction and difficulty. Vis. Res. 48 (21), 2184–2192. http://dx.doi.org/10.
1016/j.visres.2008.06.025.
Privitera, C.M., Renninger, L.W., Carney, T., Klein, S., Aguilar, M., 2010. Pupil dilation
during visual target detection. J. Vis. 10 (10), 3. http://dx.doi.org/10.1167/10.10.3.
Shadlen, M.N., Kiani, R., 2013. Decision making as a window on cognition. Neuron.
http://dx.doi.org/10.1016/j.neuron.2013.10.047.
Siegle, G.J., Ichikawa, N., Steinhauer, S., 2008. Blink before and after you think: blinks
occur prior to and following cognitive load indexed by pupillary responses.
Psychophysiology 45 (5), 679–687. http://dx.doi.org/10.1111/j.1469-8986.2008.
00681.x.
Urai, A.E., Braun, A., Donner, T.H., 2017. Pupil-linked arousal is driven by decision un-
certainty and alters serial choice bias. Nat. Commun. 8, 14637. http://dx.doi.org/10.
1038/ncomms14637.
Usher, M., Cohen, J.D., Servan-Schrieber, D., Rajkowski, J., Aston-Jones, G., 1999. The
role of locus coeruleus in the regulation of cognitive performance. Science 283
(5401), 549–554. http://dx.doi.org/10.1126/science.283.5401.549.
Van Der Meer, E., Beyer, R., Horn, J., Foth, M., Bornemann, B., Ries, J., ... Wartenburger,
I., 2010. Resource allocation and fluid intelligence: insights from pupillometry.
Psychophysiology 47 (1), 158–169. http://dx.doi.org/10.1111/j.1469-8986.2009.
00884.x.
Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.P., 2001. Eye activity correlates of
workload during a visuospatial memory task. Hum. Factors 43 (1), 111–121. http://
dx.doi.org/10.1518/001872001775992570.
Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E., 2012. Let's look at the
cockpit: exploring mobile eye-tracking for observational research on the flight deck.
Proceedings of the Symposium on Eye Tracking Research and Applications ETRA 12 1
(212), 107–114. http://dx.doi.org/10.1145/2168556.2168573.
Zelinsky, G.J., Sheinberg, D.L., 1997. Eye movements during parallel-serial visual search.
J. Exp. Psychol. Hum. Percept. Perform. 23 (1), 244–262. http://dx.doi.org/10.1037/
0096-1523.23.1.244.
T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68
68
	Eye tracking measures of uncertainty during perceptual decision making
	Introduction
	Perceptual decision making and uncertainty
	Measuring decision uncertainty
	Oculomotor metrics of decision uncertainty
	Fixations
	Saccades
	Eye blinks
	Pupil diameter
	The present study
	Method
	Participants and design
	Materials
	Equipment
	Procedure
	Data processing
	Results
	Accuracy
	Certainty ratings
	Data reduction
	Fixations
	Saccades
	Blinks
	Correlations between measures
	Pupil diameter
	General discussion
	Competing interests
	Acknowledgements
	References