Baixe o app para aproveitar ainda mais
Prévia do material em texto
Contents lists available at ScienceDirect International Journal of Psychophysiology journal homepage: www.elsevier.com/locate/ijpsycho Eye tracking measures of uncertainty during perceptual decision making Tad T. Brunyéa,b,c,⁎, Aaron L. Gardonya,b a Center for Applied Brain and Cognitive Sciences, Medford, MA, United States b U.S. Army Natick Soldier Research, Development, and Engineering Center, Natick, MA, United States c Tufts University, Department of Psychology, Medford, MA, United States A R T I C L E I N F O Keywords: Perceptual decision making Eye tracking Uncertainty Pupil diameter A B S T R A C T Perceptual decision making involves gathering and interpreting sensory information to effectively categorize the world and inform behavior. For instance, a radiologist distinguishing the presence versus absence of a tumor, or a luggage screener categorizing objects as threatening or non-threatening. In many cases, sensory information is not sufficient to reliably disambiguate the nature of a stimulus, and resulting decisions are done under conditions of uncertainty. The present study asked whether several oculomotor metrics might prove sensitive to transient states of uncertainty during perceptual decision making. Participants viewed images with varying visual clarity and were asked to categorize them as faces or houses, and rate the certainty of their decisions, while we used eye tracking to monitor fixations, saccades, blinks, and pupil diameter. Results demonstrated that decision certainty influenced several oculomotor variables, including fixation frequency and duration, the frequency, peak velo- city, and amplitude of saccades, and phasic pupil diameter. Whereas most measures tended to change linearly along with decision certainty, pupil diameter revealed more nuanced and dynamic information about the time course of perceptual decision making. Together, results demonstrate robust alterations in eye movement be- havior as a function of decision certainty and attention demands, and suggest that monitoring oculomotor variables during applied task performance may prove valuable for identifying and remediating transient states of uncertainty. 1. Introduction Perceptual decision making describes the process of accumulating sensory evidence and using it to influence how we categorize, under- stand, and behave within the world (Green and Heekeren, 2009; Heekeren et al., 2008; Shadlen and Kiani, 2013). This process is ex- ceedingly common in daily life; airport luggage screeners categorize objects as threatening or non-threatening, pathologists categorize his- tological features as normal or abnormal, and law enforcement officers categorize handheld objects as weapons or non-weapons (Brunyé et al., 2017; McCarley et al., 2004; Payne, 2001). In many cases, decisions are made under conditions of uncertainty, which can arise due to occlusion or distortion of the stimulus itself, or due to down-stream impacts of attention, memory, emotion, and/or decision criteria (Heekeren et al., 2008). Despite the ubiquity and importance of perceptual decision making, and the potential impact of uncertainty on task performance, surprisingly few studies have attempted to identify quantitative mea- sures of decision uncertainty. The present study explores whether sev- eral measures derived from eye tracking might be sensitive to varying levels of uncertainty during a perceptual decision task. 1.1. Perceptual decision making and uncertainty Gathering, combining, and interpreting information from the sen- sory systems is critical for understanding the world and motivating behavior. Several theories attempt to characterize the sensory, per- ceptual, cognitive, and behavioral processes involved in perceptual decision making. In one theory, decision making is considered a con- tinuous process that transforms sensory information into categories (e.g., face/house) (Opris and Bruce, 2005). Evidence accumulates from sensory inputs, is integrated and compared with expectations and knowledge, and then a behavioral response is selected. This proposed serial progression from perception to action relies on diverse neural circuits including the visual system, reward system, cognitive system, and oculomotor system. Visual information, processed and interpreted by the cognitive and reward systems, guides the oculomotor system to shift gaze and gather information as needed. More recently, a relatively dynamic account of perceptual decision making has emerged (Heekeren et al., 2008). In this theory, four complementary and interactive sys- tems are engaged. First, sensory systems, such as visual and tactile, gather and compare information. Second, perceptual uncertainty or http://dx.doi.org/10.1016/j.ijpsycho.2017.07.008 Received 10 April 2017; Received in revised form 6 July 2017 ⁎ Corresponding author at: Center for Applied Brain and Cognitive Sciences, 200 Boston Ave., Suite 3000, Medford, MA 02155, United States. E-mail address: tbruny01@tufts.edu (T.T. Brunyé). International Journal of Psychophysiology 120 (2017) 60–68 Available online 18 July 2017 0167-8760/ Published by Elsevier B.V. MARK difficulty is detected, motivating and constraining attention toward additional information gathering. Third, a cognitive system is used to compare accumulated information against knowledge, and prepare or execute a motor response. Finally, a performance monitoring system assesses outcomes and adjusts behavior accordingly. These four pro- cesses and their underlying neural circuitry are proposed to occur at least partially in parallel, and interactively over time. Relevant to the current research, there are a few important points to realize about extant theories of perceptual decision making. First, cur- rent theories consider the importance of oculomotor processes to stra- tegically shift attention and gather information from a scene; this is usually formalized through a role of the saccadic eye movement system and its underlying neural substrates. Thus, there is some suggestion that the perceptual decision making process engages and guides visual at- tention to accumulate information relevant to a decision. Second, these visual search processes appear to be contingent upon conditions of uncertainty, which should exert some reliable influence over oculo- motor behavior as information is gathered from a stimulus. Third, while these theories tend to emphasize neural mechanisms underlying stages of the perceptual decision process, using tools such as functional magnetic resonance imaging, these tools may not be tractable for im- plementation in applied settings intending to monitor perceptual deci- sion uncertainty during task performance. This is in contrast to eye tracking technology, which is increasingly available in lightweight, mobile, and even wireless form factors (Weibel et al., 2012). 1.2. Measuring decision uncertainty Given the breadth and complexity of sensory, cognitive, and motor systems dynamically engaged during decision making, it is not sur- prising that researchers have examined it using a wide range of beha- vioral, physiological, and neurophysiological techniques. Using func- tional magnetic resonance imaging (fMRI), research demonstrates that particular portions of the left dorsolateral prefrontal cortex (DLPFC) are involved in perceptual decision making (Heekeren et al., 2004). For instance, when participants are tasked to distinguish faces versus houses, their ability to do so correlates with activity in the left DLPFC. The authors proposed that this brain region is engaged in perceptual decisions by computing differences between activation in face- and house-specific brain regions. Other research, using event-related elec- troencephalography (EEG), has demonstrated reliable changes in early and late event-related brain potentials related to discrimination of faceand car stimuli (Philiastides et al., 2006; Philiastides and Sajda, 2006). Following stimulus onset, there was an early face-selective N170 component and a later component around 300 milliseconds; the authors proposed that the early component reflected the early perception of the stimulus, but the later component reflected the cognitive decision process. Interestingly, the late component was highly sensitive to var- iations in task difficulty, and correlated strongly with task accuracy and response times. These results suggest that EEG may hold promise for discriminating uncertainty states during perceptual decision making. There are also fMRI data examining the impact of difficulty during perceptual decision making. For instance, one study used fMRI to monitor brain responses to the difficulty of a phonetic discrimination task (Binder et al., 2004). They found that low-level sensory informa- tion processing in the auditory cortex was related to decision accuracy, whereas frontal brain regions showed activity correlated with response uncertainty. Thus, both EEG and fMRI data suggest reliable spatio- temporal dissociations of brain activity related to sensory discrimina- tion and decision-related processes. Overall, task difficulty seems to modulate both early and late phases of processing, and according to some theories may promote attention deployment to resolve transient states of uncertainty (Heekeren et al., 2008). Regarding eye tracking, surprisingly few studies have explored how eye movements might be modulated by uncertainty while participants categorize a visual stimulus. As noted by Krajbich, this could be due to an assumption that eye movements are relatively restricted and low variability during single-stimulus perceptual categorization tasks (Krajbich et al., 2010). Indeed many studies and computational models have been devoted to characterizing eye movements during choice decisions, which involve the comparison of multiple stimuli along a variety of dimensions (Fiedler and Glockner, 2012; Krajbich and Rangel, 2011; Orquin and Mueller Loose, 2013). However, several de- cision-making theories make explicit mention of eye movements as an important contributor to evidence accumulation during decision making, without clearly differentiating between decisions made re- garding single versus multiple stimuli. For instance, the drift diffusion model proposes that eye fixations are used to sample a stimulus to promote evidence accumulation (Krajbich and Rangel, 2011), and that eye movements reflect the active deployment and control of attention. This research, however, is largely constrained to tasks involving the comparison of two images, which necessitates relatively large saccadic eye movements between the stimuli. Thus, it is unknown whether si- milar oculomotor dependence on uncertainty states will emerge with a single and relatively constrained visual stimulus. 1.3. Oculomotor metrics of decision uncertainty Though there is a paucity of research examining eye tracking during the perceptual decision making with a single stimulus, more generally several candidate eye tracking measures have been linked to decision making, uncertainty, and task difficulty. These include oculomotor metrics, such as fixation and saccade parameters, blinks, and alterations in phasic pupil diameter. We review each of these below. 1.3.1. Fixations Eye fixations are momentary pauses of eye movements within a particular location that extend for a minimum duration (Duchowski, 2007). Fixations are thought to reflect the process of directing visual attention toward a stimulus to bring it into foveal vision, which is the highest resolution region of the retina and permits the greatest possible visual detail. During more difficult decisions, participants tend to show a higher number of fixations (Fiedler and Glockner, 2012; Krajbich et al., 2010). This research, however, is limited to comparing two or more visual alternatives. In contrast, when visually searching a single scene for a stimulus, increasing search task difficulty tends to decrease fixation frequency and increase fixation durations (Hooge and Erkelens, 1996; Jacobs and O'Regan, 1987), and also increase the duration of the first fixation on the scene (Zelinsky and Sheinberg, 1997). Thus, re- search is equivocal regarding the impact of difficulty on fixations: during choice tasks fixations increase in frequency with more difficult trials. In contrast, during single stimulus tasks fixation frequency de- creases and duration increases during more difficult trials, perhaps to provide more time accumulating evidence from certain regions of a scene. 1.3.2. Saccades Saccades describe the ballistic movements of the eyes between successive fixations, which produces a continually changing sequence of information projected onto the fovea (Liversedge and Findlay, 2000). Saccades can be characterized in a few ways, such as the distance be- tween successive saccades, or saccade amplitude in degrees, and the speed of saccades in terms of average or peak velocity (in °/s). Saccadic amplitude and peak velocity tend to be related in power-law function, with relative saccade magnitude increases matched proportionately to relative peak velocity increases (Bahill et al., 1975; Di Stasi et al., 2013). In visual search contexts, saccadic amplitude tends to decrease when a search task becomes more difficult (Jacobs and O'Regan, 1987; Phillips and Edelman, 2008), such as when targets are more visually similar to distractors. Furthermore, an emerging body of research suggests that the peak velocity of saccades is a valuable index of diffi- culty-evoked arousal during visual tasks (for a review, see (Di Stasi T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 61 et al., 2013), with lower peak velocity when search or decision diffi- culty is increased. 1.3.3. Eye blinks Eye blinks have also received considerable attention as a metric for decision making, uncertainty, and task difficulty. In general, eye blinks tend to be inhibited during the decision process, and those that do occur tend to be shorter in duration (Boehm-Davis et al., 2000). Eye blinks have also recently been considered as a metric of decision latency by assessing the latency of blinking onset during or after a decision (Hackley and Valle-Inclán, 1999; Siegle et al., 2008). 1.3.4. Pupil diameter Dynamic variation in pupil diameter is elicited by both environ- mental lighting conditions and decision uncertainty, though only a few studies have examined it in the context of perceptual decision making. Studies tracking pupil dynamics generally demonstrate an increased phasic diameter in response to more difficult task conditions (Beatty, 1982; Brunyé et al., 2016; Geng et al., 2015; Laeng et al., 2012; Nassar et al., 2012; Payzan-LeNestour and Bossaerts, 2012), such as when vi- sual stimuli are difficult or ambiguous to discriminate, problem sets are computationally demanding, or decisions are made under conditions of uncertainty. When viewing stimuli evoking perceptual rivalry (e.g., Necker cube), phasic increases in pupil diameter tend to occur im- mediately prior to a reported perceptual switch (Einhäuser et al., 2008). Pupil diameter is also modulated during protracted decision-making processes (de Gee et al., 2014), predicts subsequent choices (Urai et al., 2017), is associated with reduced decision bias (de Gee et al., 2017; Krishnamurthy et al., 2017), and may reflect variability in the rate of evidence accumulation during decision making (Murphy et al., 2014). These relationships are thought to at least partially reflect modulation of the locus coeruleus-norepinephrine (LC-NE) system, with pupil dia- meter changes reflecting cognitive control states and shifting between exploring and exploiting visual information (Aston-Jones and Cohen, 2005; Gilzenrat et al., 2010). 1.4. Thepresent study The present study provides a first integrated examination of several eye tracking metrics and their dependency on uncertainty during per- ceptual decision making. Participants viewed face and house images parametrically varying in visual clarity (from low to high), and were asked to decide whether each image depicted a face or house, and then rated their certainty in their decision. During image viewing we used an eye tracker to monitor fixations, saccades, blinks, and pupil diameter. We expected to find that conditions of lower visual clarity would in- crease rated decision uncertainty. We also expected that conditions of increasing uncertainty would be related to alterations in fixation fre- quency and duration; and given research concerning the search of a single stimulus, we expected fixation duration to increase in more un- certain conditions, perhaps correlating with a lower fixation frequency. Saccades were expected to show lower amplitudes and possibly lower peak velocities during more uncertain conditions, and blink frequency was also expected to decline. We also expected several of the fixation-, saccade-, and blink-related measures to be intercorrelated. Finally, we expected phasic pupil diameter to increase during trials eliciting rela- tively high uncertainty. 2. Method 2.1. Participants and design Twenty male (n = 6) and female (n = 14) adults (Mage = 20.9, SDage = 4.9) participated for monetary compensation ($20 USD). Participants provided written informed consent in accordance with human use approvals issued by Tufts University and the U.S. Army, in conformity with the ethical standards of the Declaration of Helsinki. In a within-participants design, all participants were exposed to a 2 × 8 design: image category was manipulated across two levels (2: face, house), and image clarity was manipulated across eight levels (8: 0.275, 0.3, 0.325, 0.35, 0.4, 0.45, 0.5, 0.55). Three participants showed ex- ceedingly poor eye tracking quality with> 40% dropped samples, and were removed from analysis. 2.2. Materials Materials consisted of face and house images varying in clarity. Forty faces were selected from the Max Planck Institute for Biological Cybernetics (http://faces.kyb.tuebingen.mpg.de/), face database, and 40 house photos were generated by Filimon and colleagues (Filimon et al., 2013). Each face and house image was manipulated to vary in phase coherence, with higher coherence indicating a clearer image (e.g., 0.55), and lower meaning a noisier image (e.g., 0.275); all images were provided to us by Filimon and colleagues, and details for the phase coherence manipulation process can be found in their previous research (Filimon et al., 2013). This process resulted in a total of 320 face images (40 faces, 8 clarity levels) and 320 house images (40 houses, 8 clarity levels). Each image was grayscale, 252 × 300 pixels, and clarity levels were equated for spatial frequency, contrast, and luminance (Filimon et al., 2013). We confirmed the latter by measuring luminance levels across the 8 image clarity levels in MATLAB (MathWorks, Inc., Natick, MA, USA); analysis demonstrated that luminance did not differ statis- tically by clarity level (p > 0.05). Sample face and house images are depicted in Fig. 1. 2.3. Equipment Eye tracking was done using the Remote Eye Tracking Device (RED- m) by SensoMotoric Instruments (SMI; Boston, MA, USA), sampling binocular gaze position at 60 Hz. This system uses a 5-point calibration to achieve 0.4° gaze position accuracy and 0.03° spatial resolution (RMS). The device was attached to the bottom of a 24″ LCD monitor running at 1920 × 1080 resolution. Image presentation and data col- lection were done using the SMI Experiment Center software. Fig. 1. Sample face and house images varying in image clarity, from 0.275 (low clarity) to 0.55 (high clarity). T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 62 2.4. Procedure Participants were individually seated at a workstation, and the task was presented using the SMI Experiment Center software. A total of 640 image were presented to participants in random order, one image at a time in the center of the computer monitor. Each image was shown for 3 s at center against a black background, then followed by a response screen showing the words FACE and HOUSE vertically aligned in the center of the screen for 1.5 s. To respond, participants directed their gaze toward FACE or HOUSE. Note that half of the trials showed the word FACE above HOUSE, and vice-versa, to reduce the chance of anticipatory eye movements during image viewing. We chose to use eye movements as the response mechanism for compatibility with our ex- perimental software. After deciding, participants rated their certainty in their response, ranging from 1 (not at all) to 7 (extremely). A fixation cross centered on a black background (500 ms) appeared immediately after a rating was entered. Sixteen of the 640 images (8 faces, 8 houses) were used for a brief practice session to increase familiar with the task, resulting in 624 images viewed during the primary phase. At 60 cm seating distance, images subtended 6.4° horizontal, and 7.7° vertical visual angle. The entire task took approximately 1 h to complete. 2.5. Data processing To evaluate decision accuracy, we used the position of the final three fixations to determine whether gaze was directed at FACE or HOUSE. To evaluate eye movements, we used the eye tracker's software (BeGaze; SMI; Boston, MA) to parse data into fixations, saccades, and blinks. Fixations thresholds were> 99 ms and maximum dispersion 100 px. Saccades thresholds were a minimum duration of 22 ms, and peak velocity threshold of 40°/s, with a start and end of 20% and 80% of saccade length, respectively. To calculate pupil diameter, we used raw data logs and removed and interpolated (linear) periods of blinking; pupil diameter data were referenced (via mean subtraction) to a 200 ms baseline preceding stimulus onset, and smoothed using a 5- point moving average filter. These techniques were adopted from ear- lier research (Van Der Meer et al., 2010). 3. Results 3.1. Accuracy Overall accuracy was significantly above chance (M= 0.82, SD= 0.17), t(16) = 18.71, p < 0.001, Cohen's d = 4.5. A 2(image category: face, house) × 8(image clarity: 0.275, 0.3, 0.325, 0.35, 0.4, 0.45, 0.5, 0.55) repeated-measures ANOVA demonstrated main effects of image category, F(1, 16) = 6.18, p= 0.024, ƞ2 = 0.05, image clarity, F(7, 112) = 73.64, p < 0.001, ƞ2 = 0.45, and an interaction, F (7, 112) = 6.32, p < 0.001, ƞ2 = 0.08. Accuracy increased as a function of image clarity for both faces and houses, but for houses ac- curacy was higher at relatively low clarity levels. The psychometric function relating image category, image clarity, and the probability of responding “House” is plotted in Fig. 2a. There was a slight response bias toward responding house, which was chosen during 51.7% of trials, in comparison to face which was chosen during 48.3% of trials; recall that exactly 50% of trials had faces and houses. This pattern was verified in an exact binomial test (z= 4.93, p < 0.001). 3.2. Certainty ratings Overall certainty ratings were moderately high on the 7-point scale (M= 4.99, SD= 1.52). A 2 × 2 repeated-measures ANOVA demon- strated main effects of image category, F(1, 16) = 4.7, p= 0.046, ƞ2 < 0.01, image clarity, F(7, 112) = 79.57, p < 0.001, ƞ2 = 0.80, and an interaction, F(7, 112) = 6.94, p < 0.001, ƞ2 < 0.01. Certainty ratings increased as a function of image clarity for both faces and houses; however, face ratings were higher at relatively low clarity levels (Fig.2b). A more detailed examination of confidence ratings as a func- tion of trial type and accuracy revealed a tendency to incorrectly judge face trials of low-to-medium clarity as “house”(as seen in Fig. 2b), but with high certainty. This resulted in overall higher ratings during face trials, but even when responses were incorrect. In contrast, incorrectly answering “face” during house trials consistently resulted in lower certainty ratings, as would be expected. These data are detailed in Table 1. Certainty ratings were also associated with accuracy, as verified in an ANOVA evaluating accuracy as a function of certainty rating (1–7), F (6, 96) = 23.35, p < 0.001, ƞ2 < 0.59. In general, for each one-digit certainty increase, there was a relative accuracy increase of about 11%. Mean accuracy for the 7 ratings were: 0.49, 0.61, 0.72, 0.72, 0.85, 0.86, and 0.92, respectively. 3.3. Data reduction Given our primary interest in understanding the dependence of oculomotor metrics on perceptual decision uncertainty, remaining analyses were guided by certainty ratings. Given a severely skewed distribution of certainty responses (Fig. 3), there were relatively few trials in lower ratings categories. To ensure a similar number of trials included in each certainty condition, we parsed trials into three cate- gories: low certainty (ratings 1–4), medium certainty (ratings 5–6), and high certainty (rating 7). This resulted in 3551, 3469, and 3566 trials in each of the three conditions, respectively; this data reduction com- pensated for the skewed distribution and further allowed us to analyze Fig. 2. a. Psychometric function relating image category (faces, houses) and clarity, to the probability of responding “House.” b. Mean certainty ratings (and standard error) for the two Image Categories and eight image clarity conditions. T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 63 using ANOVAs. Note that with this method, accuracy remained sig- nificantly different as a function of certainty condition (Mlow = 0.65, Mmed = 0.85, Mhigh = 0.92), F(2, 32) = 31.11, p < 0.001, ƞ2 = 0.66 (all pairwise comparisons p < 0.001). 3.4. Fixations To examine fixation frequency and duration, we conducted a series of 2(Image Type: face, house) × 3(Certainty: low, medium, high) ANOVAs; data are depicted in Fig. 4. Note there were never any main or interactive effects of Image Type (face versus house). Fixation frequency showed only a main effect of Certainty, F(2, 32) = 3.83, p= 0.032, ƞ2 = 0.17. Planned comparisons using paired t- tests demonstrated significantly lower fixation rates in the medium versus high, t(16) = 2.25, p= 0.019, Cohen's D = 0.55, and low versus high, t(16) = 2.12, p= 0.25, Cohen's d = 0.51, conditions. Fixation duration also only showed a main effect of Certainty, F(2, 32) = 8.81, p= 0.001, ƞ2 = 0.24. Planned comparisons using paired t- tests demonstrated significantly longer mean fixation duration in the low versus medium condition, t(16) = 3.16, p= 0.006, Cohen's D = 0.77, the medium versus high condition, t(16) = 8.78, p < 0.001, Cohen's D = 2.15, and the low versus high condition, t(16) = 8.88, p < 0.001, Cohen's D = 2.13. First fixation duration showed a similar pattern, with only a main effect of Certainty, F(2, 32) = 3.76, p= 0.034, ƞ2 = 0.11. Planned comparisons demonstrated significantly longer mean first fixation duration in the low (M= 268.5, SE= 51.2) versus high (M= 244.9, SE= 45.1) condition, t(16) = 2.41, p= 0.02, Cohen's D = 0.58, and the medium (M= 255.9, SE= 43.3) versus high condition, t(16) = 2.0, p= 0.031, Cohen's D = 0.49. 3.5. Saccades We used a 2 × 3 ANOVA to examine saccadic frequency, amplitude, average and peak velocity, and average and peak acceleration (Table 2). Note there were never any main or interactive effects of Image Type (face versus house). Saccadic frequency showed only a main effect of Certainty, F(2, 32) = 5.1, p= 0.012, ƞ2 = 0.21; planned comparisons demonstrated sig- nificantly fewer saccades in the low versus medium, t(16) = 1.85, p= 0.042, Cohen's D = 0.45, low versus high, t(16) = 2.39, p= 0.015, Cohen's D = 0.58, and medium versus high, t(16) = 2.28, p= 0.015, Cohen's D = 0.58, conditions. Saccadic amplitude showed only a marginal main effect of Certainty, F(2, 32) = 2.99, p= 0.064, ƞ2 = 0.06, with numerically higher amplitude with increasing certainty conditions. Saccadic average velocity showed only a main effect of Certainty, F (2, 32) = 4.65, p= 0.017, ƞ2 = 0.11; planned comparisons demon- strated significantly lower saccadic average velocity in the low versus high condition, t(16) = 2.73, p= 0.007, Cohen's D = 0.66. No other comparisons reached significance (all p > 0.05). Saccadic peak velocity also showed a main effect of Certainty, F(2, 32) = 4.02, p= 0.028, ƞ2 = 0.10; comparisons demonstrated sig- nificantly lower peak velocity in the low versus high condition, t(16) = 2.37, p= 0.015, Cohen's D = 0.57 (Fig. 5). No other comparisons Table 1 Mean (and standard error) certainty ratings for correct and incorrect trials, for the eight image clarity levels, and two image types (faces, houses). Image clarity and type Decision accuracy Correct Incorrect 0.275 Faces 3.64 (0.88) 3.53 (0.33) Houses 4.04 (0.98) 3.08 (0.32) 0.3 Faces 3.79 (0.92) 3.74 (0.31) Houses 4.18 (1.0) 3.14 (0.35) 0.325 Faces 4.11 (1.0) 3.72 (0.30) Houses 4.20 (1.0) 2.84 (0.23) 0.35 Faces 4.55 (1.1) 4.20 (0.34) Houses 4.37 (1.1) 3.14 (0.31) 0.4 Faces 5.50 (1.3) 4.43 (0.28) Houses 4.98 (1.2) 3.22 (0.39) 0.45 Faces 6.19 (1.5) 5.44 (0.31) Houses 5.84 (1.4) 5.09 (0.29) 0.5 Faces 6.62 (1.6) 6.40 (0.25) Houses 6.61 (1.6) 5.83 (0.38) 0.55 Faces 6.75 (1.6) 6.10 (0.31) Houses 6.70 (1.6) 5.54 (0.49) Fig. 3. Histogram of cumulative response frequency as a function of certainty rating. Dimension reduction allowed us to compensate for the negative skew. Fig. 4. Mean change in fixation duration (bars, left axis) and fixation frequency (line, right axis) as a function of certainty condition. Error bars depict standard error. Change is calculated as difference from each participant's overall mean fixation duration and frequency (across all conditions). T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 64 reached significance (all p > 0.05). 3.6. Blinks We used a 2 × 3 ANOVA to examine blink frequency, duration, and first blink latency (Table 2). Note there were never any main or inter- active effects of Image Type (face versus house). Blink frequency showed only a main effect of Certainty, F(2, 32) = 6.87, p= 0.003, ƞ2 = 0.16; comparisons demonstrated significantly lower blink rates in the low versus medium, t(16) = 2.88, p= 0.011, Cohen's D = 0.69, and low versus high conditions, t(16) = 3.84, p= 0.001, Cohen's D = 0.93. Medium versus high did not reach sig- nificance, and blink duration did not show significant main or inter- active effects of Certainty (all p > 0.05). 3.7. Correlations between measures A correlation matrix relating our nine oculomotor measures of in- terest is detailed in Table 3. Most measures were interrelated. Some relationships were not especially surprising, such as the negative re- lationship between fixation frequency and duration, and positive re- lationships between fixation duration and first fixation duration, and between saccadic average and peak velocity. However, some novel in- sights were derived from this correlational analysis. Specifically, sac- cadic amplitude and saccadic average and peak velocity were all posi- tively correlated with blink frequency. This finding suggests that common mechanisms may underlie the impact of decision uncertainty on saccadic and blink parameters. Not only are participants more likely to keep their eyes open during conditions of uncertainty, but during these periods they are more likely to move their eyes more slowly and across shorter distances. 3.8. Pupil diameter Mean relative pupil diameter responses were analyzed by examining responses time-locked to the onsetof a trial stimulus. Following prior research, we analyzed pupil dilation relative to baseline in each of six 500-millisecond epochs (Van Der Meer et al., 2010). A repeated-measures ANOVA was conducted, using a 2(image ca- tegory: face, house) × 3(certainty: low, med, high) × 6(time: 500 ms bins) design. There were no main or interactive effects of image cate- gory, so this variable was removed from further analysis. The 3 × 6 ANOVA demonstrated an interaction between Certainty and Time, F (10, 160) = 6.01, p < 0.001, ƞ2 = 0.03. As depicted in Fig. 6, relative to the high certainty condition there was an early and protracted in- crease in the medium certainty condition, and a relatively latent in- crease in the low certainty condition. These patterns were confirmed in a series of simple effects ANOVAs in each of the six time bins. Within bins 1 (0–500 ms) and 2 (500–1000 ms), there were no effects of Certainty, F(2, 32) = 0.98, p= 0.39, ƞ2 = 0.06, and F(2, 32) = 1.02, p= 0.37, ƞ2 = 0.06, re- spectively. Within bins 3 (1000–1500 ms) and 4 (1500–2000 ms), though there was no main effect of Certainty (p's > 0.05), pupil dia- meter was higher in the medium versus high certainty conditions, t(16) = 2.22, p= 0.021, Cohen's D = 0.54, and t(16) = 2.32, p= 0.017, Cohen's D = 0.56, respectively (other comparisons p > 0.05). Within bin 5 (2000–2500 ms), there was a main effect of Certainty, F(2, 32) = 3.92, p= 0.03, ƞ2 = 0.19; pupil diameter was higher in the low versus high certainty conditions, t(16) = 2.28, p= 0.018, Cohen's D = 0.55, and the medium versus high certainty conditions, t(16) = 2.96, p < 0.01, Cohen's D = 0.72. The same pattern was found in bin 6 (2500-3000 ms), with a main effect of Certainty, F(2, 32) = 5.37, p= 0.01, ƞ2 = 0.25; there were higher pupil diameters in the low versus high certainty conditions, t(16) = 2.74, p < 0.01, Cohen's D = 0.55, and the medium versus high certainty conditions, t(16) = 2.88, p < 0.01, Cohen's D = 0.7. Note that within each bin, t-tests remain significant with a more conservative Bonferroni correction term. Given that large changes in eye position on a screen (e.g., > 2–4°) can modulate pupil diameter measurement (Gagl et al., 2011), we re- analyzed pupil data after eliminating data points wherein the eye's point of regard was not within the image coordinate space. We thank an anonymous reviewer for this suggestion. This process resulted in the removal of approximately 12% of all data, primarily driven by eccentric point of regard during the first several samples of trials. The overall pattern of pupil responses, and all significant statistical results, main- tained after this correction; interaction term: F(10, 160) = 5.92, p < 0.001, ƞ2 = 0.03. 4. General discussion The present experiment used a perceptual decision making task and eye tracking to explore the sensitivity of oculomotor measures to mo- mentary states of uncertainty. Participants attempted to judge whether stimuli of varying clarity were faces or houses, and then rated the certainty of their judgments. Overall accuracy tended to be above chance, and as the clarity of the stimuli increased, we found robust increases in discrimination accuracy and certainty ratings. These pat- terns match earlier research using this same perceptual decision making task (Filimon et al., 2013). Participants also showed higher accuracy with house versus face judgments, which appeared to be due to a modest response bias toward houses during relatively ambiguous trials. Table 2 Mean and standard deviation values for fixation, saccade, and blink measures (not otherwise included in figures), for the two image categories, and three certainty condi- tions. Decision certainty Low Medium High 1st fixation duration (ms) Faces 275.9, 219.9 257.3, 180.6 241.5, 181.2 Houses 261.6, 204.7 253.3, 178.5 249.7, 190.7 Saccade count Faces 5.2, 1.2 5.5, 1.3 5.7, 1.3 Houses 5.2, 1.2 5.5, 1.3 5.8, 1.2 Saccade amplitude (°) Faces 5.5, 3.8 5.6, 3.5 6.1, 4.1 Houses 5.5, 3.1 5.7, 3.5 5.8, 2.8 Average saccade velocity (°/s) Faces 128.5, 48.4 132, 47.1 140.3, 46.9 Houses 131.5, 45.3 134.5, 46.6 140.1, 37.9 Blink frequency Faces 1.0, 0.6 1.13, 0.62 1.17, 0.63 Houses 1.1, 0.61 1.11, 0.58 1.14, 0.59 Blink duration Faces 238.9, 156.4 218.4, 83.9 212.8, 76.4 Houses 211.9, 80.5 206.7, 69.8 193.7, 46.9 Fig. 5. Mean change in saccadic peak velocity (in °/sec), as a function of certainty con- dition. Error bars depict standard error. Change is calculated as difference from each participant's overall mean saccade peak velocity (across all conditions). T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 65 Certainty ratings were higher during face versus house trials, though this pattern appeared to be driven by a tendency to confidently but incorrectly perceive a house during ambiguous face trials. Several eye tracking measures proved sensitive to varying states of decision certainty. Overall, during conditions of relative uncertainty we found that there were fewer but longer fixations, fewer saccades, and saccades were slower and lower in amplitude. In other words, eye movements were relatively constrained and focused on particular re- gions of the stimulus during conditions of relative uncertainty. The most robust effect, in terms of effect size, was found with fixation durations. This pattern matches previous visual search research showing longer fixation durations during more difficult search tasks, and extends it to the domain of perceptual decision making (Hooge and Erkelens, 1996; Jacobs and O'Regan, 1987). The pattern of smaller saccade amplitudes is also similar to patterns seen with more difficult visual search tasks (Jacobs and O'Regan, 1987; Phillips and Edelman, 2008). Finally, the lower peak saccade velocity supports research sug- gesting that peak saccade velocity is a valuable indicator of arousal, which tends to decrease in response to increasing task difficulty (Di Stasi et al., 2013). However, we note that the effect sizes relating un- certainty to saccade-based measures were relatively small, and confined predominantly to comparing the two more extreme conditions (low- high). Overall, our results support relatively disparate research areas, demonstrating that several oculomotor measures are modulated by uncertainty during perceptual decision making. Interestingly, these measures were sensitive to uncertainty-based variation in a task with a single, and relatively small, stimulus, complementing a larger body of literature with multi-stimulus choice tasks (Fiedler and Glockner, 2012; Krajbich and Rangel, 2011; Orquin and Mueller Loose, 2013). Our fixation and saccade data support recent perceptual decision making theory that posits a dynamic adaptive attention deployment in response to detected uncertainty or difficulty (Heekeren et al., 2008). The present study demonstrates that strategically shifting attention to gather information from a scene is evident in oculomotor processes, including longer fixations, slower eye movements, and examining re- latively restricted areas of a scene. We also saw some evidence that there were fewer blinks during relatively low certainty conditions, and interestingly this pattern was strongly correlated with saccade para- meters. Overall, the pattern of fixations, saccades, and blinks, suggests a somewhat linear modulation of oculomotor behavior in response to decision uncertainty. A more nuanced understanding of the time-course of perceptual decision making was afforded by examining pupil diameter over time, under varying conditions of uncertainty. Current theories of pupil diameter modulation in response to perceptual and cognitive tasks suggest that diameter changes reflect shifting between exploration and exploitation cognitive states (Aston-Jones and Cohen, 2005; Gilzenrat et al., 2010; Privitera et al.,2010; Usher et al., 1999). Specifically, current theory suggests that the LC-NE system drives behavioral se- lection by switching between exploration (continue searching) and exploitation (decide and select a behavior). Though this theory is lar- gely based on behavioral decision making, the pattern has also been demonstrated under conditions of perceptual ambiguity, with in- creasing pupil diameters immediately prior to a participant reporting a switch during perceptual rivalry tasks (Einhäuser et al., 2008). The present research finds a similar pattern of pupil diameter in- creases prior to decision making, particularly during conditions of un- certainty. Specifically, pupil responses during high certainty trials tend to be relatively flat over time, without any notable rise during the time course of exploring the stimulus and deciding. In contrast, pupil re- sponses during medium certainty trials tend to show a relatively rapid dilation response beginning approximately 1 s following stimulus onset. During low certainty trials, there is a pronounced but relatively latent dilation response beginning at approximately 2 s following stimulus onset. This is an interesting distinction, providing some support for current theory (Aston-Jones and Cohen, 2005; Gilzenrat et al., 2010). Specifically, it appears that participants might switch from an ex- ploration to exploitation state sooner after stimulus onset when the perceptual signal is difficult but possible to derive. In contrast, in more uncertain conditions the switch to exploitation occurs immediately prior to the required response, possibly reflecting the eventual arrival at a decision in the absence of a clear perceptual signal. These mid-to-late pupil dynamics may also reflect alterations in cognitive effort im- mediately prior to a response (i.e., arriving at a difficult decision in the relative absence of evidence); we believe it is unlikely to be due to motor preparation given that any pupil response due to motor pre- paration should be consistent across conditions. This relatively late process may be similar to the pupil dilation response seen during per- ceptual rivalry tasks, wherein one perceptual interpretation “pops out” even in the absence of a clear perceptual signal (Einhäuser et al., 2008). This finding is inconsistent with some recent research suggesting rela- tively monotonic increases in pupil diameter as a function of increased Table 3 Correlation matrix relating the nine oculomotor measures of interest. **Indicates p < 0.01, *indicates p < 0.05 (two-tailed Pearson coefficient). Fixation frequency Fixation duration First fixation duration Saccade frequency Saccade amplitude Saccade average velocity Saccade peak velocity Blink frequency Blink duration Fixation frequency 1 −0.857⁎⁎ −0.771⁎⁎ 0.975⁎⁎ 0.333⁎ 0.316⁎ 0.293⁎ 0.378⁎⁎ −0.181 Fixation duration 1 0.969⁎⁎ −0.895⁎⁎ −0.473⁎⁎ −0.493⁎⁎ −0.496⁎⁎ −0.546⁎⁎ −0.043 First fixation duration 1 −0.807⁎⁎ −0.392⁎⁎ −0.479⁎⁎ −0.473⁎⁎ −0.481⁎⁎ −0.039 Saccade frequency 1 0.494⁎⁎ 0.409⁎⁎ 0.417⁎⁎ 0.509⁎⁎ −0.117 Saccade amplitude 1 0.663⁎⁎ 0.793⁎⁎ 0.822⁎⁎ 0.120 Saccade average velocity 1 0.975⁎⁎ 0.801⁎⁎ −0.296⁎ Saccade peak velocity 1 0.870⁎⁎ −0.212 Blink frequency 1 −0.080 Blink duration 1 Fig. 6. Mean change in pupil diameter (in millimeters) over time while viewing stimulus images, as a function of the three certainty levels (low, medium, high). T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 66 uncertainty (Urai et al., 2017). However, we note that this prior re- search measures pupil diameter after a decision and immediately prior to decision feedback, rather than during stimulus viewing. To our knowledge, ours is the first evidence of a shifted pupil dilation latency while participants view a visual stimulus eliciting variable certainty, possibly signaling the differential time course of the perceptual decision process as a function of uncertainty. Future research will assess the reliability and generality of this pattern. Although we used a restricted set of variably ambiguous stimuli, we note that many of the present patterns have also been found with re- latively real-world visual stimuli. For instance, the pupil dilates when pathologists fixate on and correctly interpret diagnostically relevant regions of a medical image, but only when the diagnosis is relatively challenging (Brunyé et al., 2016). Pupils also dilate when radar op- erators detect threats in increasingly dense displays (Van Orden et al., 2001), and when drivers face ambiguous road conditions (Pedrotti et al., 2014). Furthermore, reductions in peak saccade velocity have also been found in response to workload demands in simulated air traffic control and driving simulations (Di Stasi et al., 2013). Given the ubiquity of uncertainty during real-world perceptual tasks, we present research informs not only theory but also the development of closed- loop systems to monitor and respond to transient states of user un- certainty. Continuing research will consider the relative and interactive value of multiple oculomotor measures for the real-time classification of uncertainty states during the performance of relatively real-world tasks. There are a few limitations worth considering. First, because cer- tainty ratings were highly skewed toward positive responses, our ana- lyses used discretized categories of certainty ratings to increase stability and power. Future research may consider using a scale with a relatively restricted range of responses (e.g., 1–4) to avoid dimension reduction. Second, our eye tracker used a 60 Hz sampling frequency, though higher frequency rates may be necessary to identify relatively granular patterns of saccade behavior. Third, to increase our statistical power, each participant viewed 640 trials, which likely induced some fatigue over the course of the experiment. To examine this possibility, we re- analyzed our saccade, fixation, and blink data with the inclusion of time (first vs. second half) as an independent variable. Though time tended to impact saccadic peak and average velocity, there were no interac- tions with certainty conditions across any of our dependent variables. In other words, time on task does impact some saccadic measures, as previously reported (Di Stasi et al., 2011; Di Stasi et al., 2010), though the influence of certainty on oculomotor metrics remains unchanged. Further, we also note that we cannot be certain that uncertainty in the absence of our stimulus manipulation (i.e., clarity) would modulate oculomotor and pupil outcomes in a similar manner to that described herein; in other words, uncertainty is driven by the perceptual ambi- guity of our stimuli, though uncertainty in the absence of our stimuli may modulate pupil diameter differently. To our knowledge, this is the first study to demonstrate a wide range of eye tracking measures dependent on uncertainty during perceptual decision making. During transient states of uncertainty, we find that the eye makes fewer and longer fixations, and slower and shorter saccades. This process is complemented by dynamic shifts of pupil dilation re- sponses, suggestive of variable-latency shifts between explore-exploit cognitive states as a function of uncertainty. Competing interests The authors have no actual or potential conflicts of interest. Acknowledgements We thank Lindsay Houck for assistance with data collection. This work was supported by the U.S. Army Natick Soldier Research, Development and Engineering Center under grant number W911-QY- 13-C-0012. References Aston-Jones, G., Cohen, J.D., 2005. An integrative theory of locus coeruleus-nor- epinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. http://dx.doi.org/10.1146/annurev.neuro.28.061604.135709. Bahill, A.T., Clark,M.R., Stark, L., 1975. The main sequence, a tool for studying human eye movements. Math. Biosci. 24, 191–204. http://dx.doi.org/10.1016/0025- 5564(75)90075-9. Beatty, J., 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol. Bull. 91, 276–292. http://dx.doi.org/10.1037/0033- 2909.91.2.276. Binder, J.R., Liebenthal, E., Possing, E.T., Medler, D.A., Ward, B.D., 2004. Neural cor- relates of sensory and decision processes in auditory object identification. Nat. Neurosci. 7 (3), 295–301. http://dx.doi.org/10.1038/nn1198. Boehm-Davis, D.A., Gray, W.D., Schoelles, M.J., 2000. The eye blink as a physiological indicator of cognitive workload. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 44, 6-116–6-119. http://dx.doi.org/10.1177/ 154193120004403309. Brunyé, T.T., Eddy, M.D., Mercan, E., Allison, K.H., Weaver, D.L., Elmore, J.G., 2016. Pupil diameter changes reflect difficulty and diagnostic accuracy during medical image interpretation. BMC Medical Informatics and Decision Making 16, 77. Brunyé, T.T., Mercan, E., Weaver, D.L., Elmore, J.G., 2017. Accuracy is in the eyes of the pathologist: the visual interpretive process and diagnostic accuracy with digital whole slide images. J. Biomed. Inform. 66, 171–179. Di Stasi, L.L., Marchitto, M., Antolí, A., Baccino, T., Cañas, J.J., 2010. Approximation of on-line mental workload index in ATC simulated multitasks. J. Air Transp. Manag. 16 (6), 330–333. http://dx.doi.org/10.1016/j.jairtraman.2010.02.004. Di Stasi, L.L., Antoli, A., Canas, J.J., 2011. Main sequence: an index for detecting mental workload variation in complex tasks. Appl. Ergon. 42 (6), 807–813. http://dx.doi. org/10.1016/j.apergo.2011.01.003. Di Stasi, L.L., Catena, A., Cañas, J.J., Macknik, S.L., Martinez-Conde, S., 2013. Saccadic velocity as an arousal index in naturalistic tasks. Neurosci. Biobehav. Rev. http://dx. doi.org/10.1016/j.neubiorev.2013.03.011. Duchowski, A.T., 2007. Eye Tracking Methodology: Theory and Practice. Book. Springer- Verlag, New York, NY. http://dx.doi.org/10.1007/978-1-84628-609-4. Einhäuser, W., Stout, J., Koch, C., Carter, O., 2008. Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. U. S. A. 105 (5), 1704–1709. http://dx.doi.org/10.1073/pnas.0707727105. Fiedler, S., Glockner, A., 2012. The dynamics of decision making in risky choice: an eye- tracking analysis. Front. Psychol. 3 (OCT). http://dx.doi.org/10.3389/fpsyg.2012. 00335. Filimon, F., Philiastides, M.G., Nelson, J.D., Kloosterman, N.A., Heekeren, H.R., 2013. How embodied is perceptual decision making? Evidence for separate processing of perceptual and motor decisions. J. Neurosci. 33 (5), 2121–2136. http://dx.doi.org/ 10.1523/JNEUROSCI.2334-12.2013. Gagl, B., Hawelka, S., Hutzler, F., 2011. Systematic influence of gaze position on pupil size measurement: analysis and correction. Behav. Res. Methods 43 (4), 1171–1181. http://dx.doi.org/10.3758/s13428-011-0109-5. de Gee, J.W., Knapen, T., Donner, T.H., 2014. Decision-related pupil dilation reflects upcoming choice and individual bias. Proc. Natl. Acad. Sci. 111 (5), E618–E625. http://dx.doi.org/10.1073/pnas.1317557111. de Gee, J.W., Colizoli, O., Kloosterman, N.A., Knapen, T., Nieuwenhuis, S., Donner, T.H., 2017. Dynamic modulation of decision biases by brainstem arousal systems. elife 6, e23232. Geng, J.J., Blumenfeld, Z., Tyson, T.L., Minzenberg, M.J., 2015. Pupil diameter reflects uncertainty in attentional selection during visual search. Front. Hum. Neurosci. 9 (August), 435. http://dx.doi.org/10.3389/fnhum.2015.00435. Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M., Cohen, J.D., 2010. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affec. Behav. Neurosci. 10 (2), 252–269. http://dx.doi.org/10.3758/ CABN.10.2.252. Green, N., Heekeren, H.R., 2009. Perceptual decision making: a bidirectional link be- tween mind and motion. Prog. Brain Res. http://dx.doi.org/10.1016/S0079- 6123(09)01317-X. Hackley, S.A., Valle-Inclán, F., 1999. Accessory stimulus effects on response selection: does arousal speed decision making? J. Cogn. Neurosci. 11 (3), 321–329. http://dx. doi.org/10.1162/089892999563427. Heekeren, H.R., Marrett, S., Bandettini, P.A., Ungerleider, L.G., 2004. A general me- chanism for perceptual decision-making in the human brain. Nature 431, 859–862. Heekeren, H.R., Marrett, S., Ungerleider, L.G., 2008. The neural systems that mediate human perceptual decision making. Nat. Rev. Neurosci. 9 (6), 467–479. http://dx. doi.org/10.1038/nrn2374. Hooge, I.T., Erkelens, C.J., 1996. Control of fixation duration in a simple search task. Percept. Psychophys. 58 (7), 969–976. http://dx.doi.org/10.1017/ CBO9781107415324.004. Jacobs, A.M., O'Regan, J.K., 1987. Spatial and/or temporal adjustments of scanning be- havior to visibility changes. Acta Psychol. 65 (2), 133–146. http://dx.doi.org/10. 1016/0001-6918(87)90023-0. Krajbich, I., Rangel, A., 2011. Multialternative drift-diffusion model predicts the re- lationship between visual fixations and choice in value-based decisions. Proc. Natl. Acad. Sci. U. S. A. 108 (33), 13852–13857. http://dx.doi.org/10.1073/pnas. 1101328108. Krajbich, I., Armel, C., Rangel, A., 2010. Visual fixations and the computation and T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 67 comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298. Krishnamurthy, K., Nassar, M.R., Sarode, S., Gold, J.I., 2017. Arousal-related adjustments of perceptual biases optimize perception in dynamic environments. Nat. Hum. Behav. http://dx.doi.org/10.1038/s41562-017-0107. Laeng, B., Sirois, S., Gredeback, G., 2012. Pupillometry: a window to the preconscious? Perspect. Psychol. Sci. 7 (1), 18–27. http://dx.doi.org/10.1177/1745691611427305. Liversedge, S., Findlay, J., 2000. Saccadic eye movements and cognition. Trends Cogn. Sci. 4 (1), 6–14. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10637617. McCarley, J.S., Kramer, A.F., Wickens, C.D., Vidoni, E.D., Boot, W.R., 2004. Visual skills in airport-security screening. Psychol. Sci. 15 (5), 302–306. http://dx.doi.org/10. 1111/j.0956-7976.2004.00673.x. Murphy, P.R., Vandekerckhove, J., Nieuwenhuis, S., 2014. Pupil-linked arousal de- termines variability in perceptual decision making. PLoS Comput. Biol. 10 (9). http:// dx.doi.org/10.1371/journal.pcbi.1003854. Nassar, M.R., Rumsey, K.M., Wilson, R.C., Parikh, K., Heasly, B., Gold, J.I., 2012. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15 (7), 1040–1046. http://dx.doi.org/10.1038/nn.3130. Opris, I., Bruce, C.J., 2005. Neural circuitry of judgment and decision mechanisms. Brain Res. Rev. 48, 509–526. Orquin, J.L., Mueller Loose, S., 2013. Attention and choice: a review on eye movements in decision making. Acta Psychol. 144 (1), 190–206. http://dx.doi.org/10.1016/j. actpsy.2013.06.003. Payne, B.K., 2001. Prejudice and perception: the role of automatic and controlled pro- cesses in misperceiving a weapon. J. Pers. Soc. Psychol. 81, 181–192. Payzan-LeNestour, É., Bossaerts, P., 2012. Do not bet on the unknown versus try to find out more: estimation uncertainty and “unexpected uncertainty” both modulate ex- ploration. Front. Neurosci.(OCT). http://dx.doi.org/10.3389/fnins.2012.00150. Pedrotti, M., Mirzaei, M.A., Tedesco, A., Chardonnet, J., Mérienne, F., Benedetto, S., Baccino, T., 2014. Automatic stress classification with pupil diameter analysis. Int. Journal of Human-Computer Interaction 30 (March), 1–17. http://dx.doi.org/10. 1080/10447318.2013.848320. Philiastides, M.G., Sajda, P., 2006. Temporalcharacterization of the neural correlates of perceptual decision making in the human brain. Cereb. Cortex 16 (4), 509–518. http://dx.doi.org/10.1093/cercor/bhi130. Philiastides, M.G., Ratcliff, R., Sajda, P., 2006. Neural representation of task difficulty and decision making during perceptual categorization: a timing diagram. J. Neurosci. 26 (35), 8965–8975. http://dx.doi.org/10.1523/JNEUROSCI.1655-06.2006. Phillips, M.H., Edelman, J.A., 2008. The dependence of visual scanning performance on search direction and difficulty. Vis. Res. 48 (21), 2184–2192. http://dx.doi.org/10. 1016/j.visres.2008.06.025. Privitera, C.M., Renninger, L.W., Carney, T., Klein, S., Aguilar, M., 2010. Pupil dilation during visual target detection. J. Vis. 10 (10), 3. http://dx.doi.org/10.1167/10.10.3. Shadlen, M.N., Kiani, R., 2013. Decision making as a window on cognition. Neuron. http://dx.doi.org/10.1016/j.neuron.2013.10.047. Siegle, G.J., Ichikawa, N., Steinhauer, S., 2008. Blink before and after you think: blinks occur prior to and following cognitive load indexed by pupillary responses. Psychophysiology 45 (5), 679–687. http://dx.doi.org/10.1111/j.1469-8986.2008. 00681.x. Urai, A.E., Braun, A., Donner, T.H., 2017. Pupil-linked arousal is driven by decision un- certainty and alters serial choice bias. Nat. Commun. 8, 14637. http://dx.doi.org/10. 1038/ncomms14637. Usher, M., Cohen, J.D., Servan-Schrieber, D., Rajkowski, J., Aston-Jones, G., 1999. The role of locus coeruleus in the regulation of cognitive performance. Science 283 (5401), 549–554. http://dx.doi.org/10.1126/science.283.5401.549. Van Der Meer, E., Beyer, R., Horn, J., Foth, M., Bornemann, B., Ries, J., ... Wartenburger, I., 2010. Resource allocation and fluid intelligence: insights from pupillometry. Psychophysiology 47 (1), 158–169. http://dx.doi.org/10.1111/j.1469-8986.2009. 00884.x. Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.P., 2001. Eye activity correlates of workload during a visuospatial memory task. Hum. Factors 43 (1), 111–121. http:// dx.doi.org/10.1518/001872001775992570. Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E., 2012. Let's look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. Proceedings of the Symposium on Eye Tracking Research and Applications ETRA 12 1 (212), 107–114. http://dx.doi.org/10.1145/2168556.2168573. Zelinsky, G.J., Sheinberg, D.L., 1997. Eye movements during parallel-serial visual search. J. Exp. Psychol. Hum. Percept. Perform. 23 (1), 244–262. http://dx.doi.org/10.1037/ 0096-1523.23.1.244. T.T. Brunyé, A.L. Gardony International Journal of Psychophysiology 120 (2017) 60–68 68 Eye tracking measures of uncertainty during perceptual decision making Introduction Perceptual decision making and uncertainty Measuring decision uncertainty Oculomotor metrics of decision uncertainty Fixations Saccades Eye blinks Pupil diameter The present study Method Participants and design Materials Equipment Procedure Data processing Results Accuracy Certainty ratings Data reduction Fixations Saccades Blinks Correlations between measures Pupil diameter General discussion Competing interests Acknowledgements References
Compartilhar