Feature-Based Attention and Feature-Based Expectation

recent study, Yao and colleagues [9] pro- Acknowledgments vide the first neural evidence for a transfer M.R. is supported by the Emmy Noether Program of of attentional states across saccades. the Deutsche Forschungsgemeinschaft, DFG (RO While recording the activity of neurons 3579/2-1). M.S. is supported by DFG temporary position for principal investigator (SZ 343/1). in the motion-sensitive area MT (feature map), they presented a spatial attention 1Department of Psychology and Bernstein Center for cue either in the future (post-saccadic) Computational Neuroscience, Humboldt Universität zu Berlin, 10099 Berlin, Germany location of the RF of a neuron or in a 2Allgemeine und Experimentelle Psychologie, Ludwigcontrol location. Following that cue, two Maximilians-Universität München, Munich, 80802, patches of moving dots appeared, with Germany 3 Equal contribution of the authors. one in the post-saccadic RF – thus either coinciding with the cued (attended) or the *Correspondence: [email protected] (M. Rolfs). control location (unattended). In their crit- http://dx.doi.org/10.1016/j.tics.2016.04.003 ical condition, the monkey prepared a saccade and the motion patch disap- References 1. Wurtz, R.H. et al. (2011) Thalamic pathways for active peared before the eyes started moving. vision. Trends Cogn. Sci. 15, 177–184 Although the stimulus was purely pre- 2. Cavanagh, P. et al. (2010) Visual stability based on remapping of attention pointers. Trends Cogn. Sci. 14, 147–153 saccadic and never appeared in the RF, 3. Rolfs, M. et al. (2011) Predictive remapping of attention MT neurons showed a clear remapping across eye movements. Nat. Neurosci. 14, 252–256 response. Importantly, this memory trace 4. Jonikaitis, D. et al. (2013) Allocation of attention across saccades. J. Neurophysiol. 109, 1425–1434 of remapping was enhanced by a top- 5. Szinte, M. et al. (2015) Attentional trade-offs maintain the down attentional modulation established tracking of moving objects across saccades. J. Neurophysiol. 113, 2220–2231 before the saccade. Moreover, this effect 6. Zirnsak, M. et al. (2014) Visual space is compressed in did not require a match between the prefrontal cortex before eye movements. Nature 507, 504–507 direction of motion in the pre-saccadic 7. Zirnsak, M. and Moore, T. (2014) Saccades and shifting stimulus and the direction preference of receptive fields: anticipating consequences or selecting the MT neuron. These results support key targets? Trends Cogn. Sci. 18, 621–628 predictions of the theory of remapping of 8. Neupane, S. et al. (2016) Two distinct types of remapping in primate cortical area V4. Nat. Commun. 7, 10402 attention pointers: the existence of hori- 9. Yao, T. et al. (2016) An attention-sensitive memory trace in zontal transfer of attentional states that Macaque MT following saccadic eye movements. PLoS Biol. 14, e1002390 are selective for location. The signals driv10. Marino, A.C. and Mazer, J.A. (2016) Perisaccadic updating ing these effects are likely to originate in of visual representations and attentional states: linking behavior and neurophysiology. Front. Syst. Neurosci. 10, 3 priority maps that have little selectivity for features. Together, these studies reestablish ‘remapping’ as a mechanism for visual stability and suggest a key role of attentional top-down processes. Importantly, they support a link between neural and behavioral evidence of remapping through a simple attentional mechanism (Figure 1): horizontal transfer of activity in priority maps (LIP, FEF, and SC) increases sensitivity at the remapped locations of attended stimuli in feature maps, enabling trans-saccadic tracking of attended targets [4]. This exciting work provides key insights into the link between remapping and attention, taking us two steps further in our endeavor to understand visual stability.

Forum

Feature-Based Attention and Feature-Based Expectation Christopher Summerfield1 and Tobias Egner2 Foreknowledge of target stimulus features improves visual search performance as a result of ‘feature-based attention’ (FBA). Recent

studies have reported that ‘featurebased expectation’ (FBE) also heightens decision sensitivity. Superficially, it appears that the latter work has simply rediscovered (and relabeled) the effects of FBA. However, this is not the case. Here we explain why. Attention can prioritize the processing of stimulus features (e.g., red) or dimensions (e.g., color). This ‘feature-based attention’ (FBA) has been most intensively investigated using visual search paradigms. Consider a search task in which observers view several dot motion patches, and are asked to detect which one is moving coherently (Figure 1A). Feature-based cues providing valid foreknowledge of the target motion direction (e.g., 458) facilitate detection performance relative to neutral or invalid cues [1,2]. A distinct line of research has investigated how expectations about features influence behavior and modulate brain activity [3]. Consider a discrimination task in which observers view two dot motion patches, and are asked to report whether the motion direction in one patch (e.g., right of fixation) is clockwise (+458) or counterclockwise ( 458) of vertical (Figure 1B). When cues signal the expected direction of dot motion (e.g., +458), observers can combine this prior knowledge with visual feature information. This leads to an overall increase in accuracy. This advantage for expected features on the discrimination task seems wholly consistent with FBA, exactly as facilitation in search tasks seems to follow naturally from expectations about the target feature. Superficially, it may thus appear that these two manipulations (which, in our example, both cue an expectation of +458 motion) simply index the same attentional process. Here, however, we argue that this is not the case. Instead, we draw a distinction between manipulations (i) that provide information about the relevance of

Trends in Cognitive Sciences, June 2016, Vol. 20, No. 6

401

(A)

(C)

Which is coherent?

Invalid cue

Valid cue

Invalid cue

Valid cue

(D)

(B)

Valid cue Which direcon is the right smulus moving?

Criterion Invalid cue Criterion

–90

–45

0

45

90

Moon direcon Figure 1. (A) Visual search task. Observers detect which stimulus contains coherent motion (top left). The motion can occur in any direction. (B) Discrimination task. Observers are cued to report the motion direction (CW vs CCW) in one of the patches (rightmost). The motion direction varies independently between patches. (C) Priors and evidence in the visual search task. Simulated motion energy for each stimulus (black lines). The peak at 458 shows motion energy in the target stimulus. Black bars show the sum of motion energy for each stimulus. On valid trials, a prior belief over motion energy is present (upper red trace). Red bars: motion energy after filtering by the prior, equivalent to marginalizing over a posterior computed via Bayes’ rule. Evidence in the target stimulus is stronger. (D) Priors and evidence in the discrimination task. Valid and invalid cues lead to opposing priors (upper blue lines). When filtered by the prior, the overall level of evidence for CW and CCW is different (blue bars), as is the optimal criterion (blue broken line; shift shown by the blue arrow). However, the difference in evidence between blue bars is the same, and an ideal observer should therefore perform equivalently in the two conditions.

Consider first the search task (Figure 1A, C). In the absence of a motion direction cue, the natural strategy is to sum noisy motion energy across all possible directions, and compare the resulting maxima between patches. However, in the presence of a cue, one motion direction becomes more relevant than others, and thus can be given more weight in the decision. This should lead to more sensitive detection judgConsider these tasks from the perspective ments, as described in past studies of of a theoretical agent who makes the best FBA [1,2,4]. possible decisions given uncertainty in sensory signals. This ‘ideal observer’ will By contrast, an ideal observer discriminatweight sources of information according ing motion direction at a single location to their relevance to the task at hand. should add up and compare noisy motion perceptual signals for a decision (FBA), and (ii) that offer information about signal probability, but not relevance. We call the latter ‘feature-based expectation’ (FBE). In recent years the distinction between FBA and FBE has attracted growing interest but also provoked confusion. We discuss here a principled way of distinguishing between these two sources of foreknowledge.

402


evidence in favor of the two categories (e. g., clockwise vs counterclockwise of vertical). In this case both the expected (e.g., +458) and unexpected ( 458) motion signals are equally relevant, and should be given equivalent weight, irrespective of whether an FBE cue is available or not (Figure 1D). Thus, according to the ideal observer framework, knowledge of stimulus probabilities should not enhance discrimination sensitivity. Instead, performance increases because the ideal observer adjusts the decision criterion to respond consistently with the cue more often, leading to a response bias towards the expected category.

(A)

(B)

1 Detecon rate

Energy sensivity (a.u)

0.2

0.1

0

0

–30

30 –30

0

30

ns ns

Low

high

Tilt relave to vercal

Low

high

Signal energy

Energy sensivity (a.u)

(C)

1

1

0.5

0.5

0

0

–0.5

–0.5 –1

–1 –35

–25 –15

–5

5

15

25

35

–35 –25

–15

–5

5

15

25

35

Tilt relave to vercal Key: Valid aenon cues

Valid expectaon cues

Invalid aenon cues

Invalid expectaon cues

Figure 2. (A) Left panel: energy sensitivity (i.e., sensitivity to signal-like fluctuations in noise) for attended (green line) and unattended (red line) vertical gratings in a detection task similar to that described in Figure 1B (a.u., arbitrary units). Right panel: energy sensitivity for expected (blue line) and unexpected (pink line) gratings, where expectations are guided by feature-based expectation (FBE) cues as defined in the text. (B) Detection rate (i.e., false alarm rate, for low-energy trials; hit rate, for highenergy trials) as a function of signal energy at vertical orientation for attended and unattended gratings (left panel), and expected and unexpected gratings (right panel). The relative sensitivity advantage for attended gratings (slope of green vs red line) is strongest for high signal energy trials, whereas the advantage for expected gratings (slope of blue vs pink line) is greatest for low signal energy. Based on [7]. (C) Energy sensitivity for attended (green line) and unattended (red line) gratings in a fine discrimination task (left panel) and for expected (blue line) and unexpected (pink line) gratings. Observers were more sensitive to expected information at more tilted orientations (lateral shift in blue sensitivity curve), where information is most diagnostic for choices (based on [9]). In all panels, black bars indicate significant differences in energy sensitivity between conditions. Error bars and shading show SEM.

The predictions of this normative framework – that FBA will increase decision sensitivity, whereas FBE will increase bias but not sensitivity – are supported by a longstanding psychophysical literature that has computed sensitivity (d’) and bias (b) from the relative proportions of correct and error trials. Interestingly, however, recent studies have cast doubt on the

classical perspective, suggesting instead that FBE and FBA may both facilitate detection sensitivity. We evaluate these claims in the light of the ideal observer framework outlined above.

locations. The authors report that valid expectations about the category membership (e.g., ‘car’) of the target object facilitated detection. In this case, however, the cues offered observers the opportunity to weight visual information according to its In one recent study [5], observers relevance in the high-dimensional space of detected a degraded visual object that object classes (e.g., to weight ‘car’ more could occur at one of four spatial heavily than ‘non-car’ features). Thus, a


403

facilitation in performance is expected under the ideal observer framework. A similar logic applies to demonstrations that degraded objects are identified more readily when preceded by a valid written name [6]. In the terms proposed above, these interesting findings may be attributable to FBA, rather than to FBE. In another recent study [7], observers judged the presence or absence of a vertical grating embedded in noise, while orthogonal FBE and FBA cues provided information about the probability (cue present vs absence) and relevance of the visual signals, respectively. Instead of using conventional approaches to estimating decision-theoretic statistics, the authors used psychophysical reverse correlation methods, a more sensitive approach that measures the influence that signal-like fluctuations in noise (‘noise energy’) in a psychophysical stimulus have on decisions [8]. Choices were better predicted by noise energy in the presence of valid expectation cues (Figure 2A) and, unlike the influence of FBA cues, this effect was strongest when sensory signals were weak (Figure 2B). This finding was explained by a computational simulation in which FBE cues facilitate detection sensitivity via an early gain-control mechanism, whereas FBA acts to reduce noise in the decision process. In another study [9], observers discriminated the tilt (clockwise vs counterclockwise) of a noisy grating, and psychophysical reverse correlation (PRC) was used to assess the influence of sensory signals on choices, under FBA and FBE cues. In this case, FBE cues increased sensitivity to more diagnostic, ‘off-channel’ features, whereas FBA had a more general multiplicative influence on decision sensitivity (Figure 2C).

404

The latter studies suggest that human behavior deviates from that of an ideal observer. FBE cues (that offer probabilistic information alone) can enhance decision sensitivity. However, they do so in a fashion that is distinct from classical manipulations of FBA. The precise computational mechanisms underlying FBE and FBA remain unclear, but one possibility is that FBE cues engage an adaptive gain-control mechanism that allows limited resources to be allocated efficiently, in other words to those sensory features that are most likely to occur [3]. Consistent with this view, gratings with expected orientation elicit lower-amplitude BOLD signals in early visual cortex, and they can be decoded more accurately from multivoxel activity, suggestive of sharper population tuning for the expected orientation [10]. More generally, FBA has been studied using a range of experimental approaches. Some researchers have employed a variant of the search paradigm that asks observers to respond when a specific feature occurs in an ongoing stimulation stream, reporting a neural advantage on ‘match’ trials. Others have manipulated the similarity of features in a primary and secondary decision task, and measured the benefits when they overlap, prompting the ‘feature-similarity gain hypothesis’, a prominent model of FBA [11]. Because FBA effects are strongest when targets defined by a common feature are repeated between or within trials, others have suggested that FBA may be related to bottom-up processes such as repetition priming [12]. Given this diversity of experimental approaches, it is perhaps unsurprising that the mechanisms underlying FBA remain controversial. We suggest that it will be crucial for future research to


explicitly orthogonalize FBE and FBA – guided by the normative distinction we highlight here – and to measure their separate influences at both the neural and the computational levels. Acknowledgments This work was supported by the National Institute of Mental Health, National Institutes of Health (Grant R01 MH097965 to T.E.) and by the European Research Council (Starter Award 281628 to C.S.). 1 Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, UK 2 Center for Cognitive Neuroscience, Duke University,

Durham NC, USA *Correspondence: christopher.summerfi[email protected] (C. Summerfield). http://dx.doi.org/10.1016/j.tics.2016.03.008 References 1. Ho, T.C. et al. (2012) Perceptual consequences of featurebased attentional enhancement and suppression. J. Vis. 12, 15 2. Ling, S. et al. (2009) How spatial and feature-based attention affect the gain and tuning of population responses. Vis. Res. 49, 1194–1204 3. Summerfield, C. and de Lange, F.P. (2014) Expectation in perceptual decision making: neural and computational mechanisms. Nat. Rev. Neurosci. 15, 745–756 4. White, A.L. et al. (2015) Stimulus competition mediates the joint effects of spatial and feature-based attention. J. Vis. 15, 7 5. Stein, T. and Peelen, M.V. (2015) Content-specific expectations enhance stimulus detectability by increasing perceptual sensitivity. J. Exp. Psychol. Gen. 144, 1089–1104 6. Eger, E. et al. (2007) Mechanisms of top-down facilitation in perception of visual objects studied by FMRI. Cereb. Cortex 17, 2123–2133 7. Wyart, V. et al. (2012) Dissociable prior influences of signal probability and relevance on visual contrast sensitivity. Proc. Natl. Acad. Sci. U.S.A. 109, 3593–3598 8. Solomon, J.A. (2002) Noise reveals visual mechanisms of detection and discrimination. J. Vis. 2, 105–120 9. Cheadle, S. et al. (2015) Feature expectation heightens visual sensitivity during fine orientation discrimination. J. Vis. 15, 14 10. Kok, P. et al. (2012) Less is more: expectation sharpens representations in the primary visual cortex. Neuron 75, 265–270 11. Maunsell, J.H. and Treue, S. (2006) Feature-based attention in visual cortex. Trends Neurosci. 29, 317–322 12. Theeuwes, J. (2013) Feature-based attention: it is all bottom-up priming. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 368, 20130055