Attention to Detail - MIT Press Journals

Attention to Detail: Why Considering Task Demands Is Essential for Single-Trial Analysis of BOLD Correlates of the Visual P1 and N1 Tracy Warbrick1, Jorge Arrubla1, Franks Boers1, Irene Neuner1,2,3, and N. Jon Shah1,2,3

Abstract ■ Single-trial fluctuations in the EEG signal have been shown

to temporally correlate with the fMRI BOLD response and are valuable for modeling trial-to-trial fluctuations in responses. The P1 and N1 components of the visual ERP are sensitive to different attentional modulations, suggesting that different aspects of stimulus processing can be modeled with these ERP parameters. As such, different patterns of BOLD covariation for P1 and N1 informed regressors would be expected; however, current findings are equivocal. We investigate the effects of variations in attention on P1 and N1 informed BOLD activation in a visual oddball task. Simultaneous EEG-fMRI data were recorded from 13 healthy participants during three conditions of a visual oddball task: Passive, Count, and Respond. We show that the P1 and N1 components

INTRODUCTION It has been well documented that simultaneous EEG-fMRI is a useful tool for investigating brain function (see Huster, Debener, Eichele, & Herrmann, 2012, for a review) because of the complementary nature of the high temporal resolution of EEG data and high spatial resolution of fMRI data. Furthermore, the fMRI and EEG signals share common neural substrates (Mussall, von Pfostl, Rauch, Logothetis, & Whittingstall, 2012; Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001), meaning that this approach can optimize the measurement of perceptual and cognitive processes by taking multiple measures of the same processes or responses at the same time. A number of approaches have been taken for data integration (see Biessmann, Plis, Meinecke, Eichele, & Muller, 2011, for a review), a common method being “integration-by-prediction” whereby single-trial ERP parameters are included in the analysis of fMRI data. Integration-by-prediction is based on the assumption that the hemodynamic response is linearly related to local changes in neuronal activity, in particular

1

Institute of Neuroscience and Medicine 4, Forschungszentrum Jülich, 2RWTH Aachen University, 3JARA-BRAIN, Translational Medicine, Jülich/Aachen Germany © 2014 Massachusetts Institute of Technology

of the visual ERP can be used in the integration-by-prediction method of EEG-fMRI data integration to highlight brain regions related to target detection and response production. Our data suggest that the P1 component of the ERP reflects changes in sensory encoding of stimulus features and is more informative for the Passive and Count conditions. The N1, on the other hand, was more informative for the Respond condition, suggesting that it can be used to model the processing of stimulus, meaning specifically discriminating one type of stimulus from another, and processes involved in integrating sensory information with response selection. Our results show that an understanding of the underlying electrophysiology is necessary for a thorough interpretation of EEG-informed fMRI analysis. ■

local field potentials (Lauritzen & Gold, 2003; Heeger & Ress, 2002; Logothetis et al., 2001). Single-trial fluctuations in the EEG have been shown to temporally correlate with the fMRI BOLD response (Warbrick et al., 2009; Debener et al., 2005; Eichele et al., 2005) and are useful for modeling the trial-to-trial fluctuations in responses that are usually lost in a standard averaging approach (Bagshaw & Warbrick, 2007; Fell, 2007). However, only a small fraction of the ongoing brain activity can be used for an EEG-informed fMRI analysis at any one time, and the selection of the feature of interest is usually determined by the research question addressed (Huster et al., 2012). Careful selection of the EEG parameter to be used is therefore paramount. The aim of EEG-informed analysis is to model perceptual/ cognitive effects reflected in the ERP components; therefore, consideration of the underlying processes represented by the ERP parameters chosen is necessary. Particularly given that previous research has shown that different ERP peaks can isolate different patterns of BOLD activation (Karch et al., 2010; Eichele et al., 2005). However, this crucial aspect of the data is given little attention in some (but not all) articles (e.g., Fuglø, Pedersen, Rostrup, Hansen, & Larsson, 2012). The current article investigates the BOLD correlates of the P1 and N1 components of the visual ERP. The current Journal of Cognitive Neuroscience 26:3, pp. 529–542 doi:10.1162/jocn_a_00490

findings for BOLD correlates of these components are equivocal. There are reports that the P1 and N1 components covary with BOLD activation in the same regions (Novitskiy et al., 2011) and that they covary with different regions (Fuglø et al., 2012). The regions implicated in the relationship between these ERP components and the fMRI BOLD activation also differ across studies; Fuglø et al. (2012) focus on whether P1 and N2 (equivalent to our N1) correlate with known generators of these ERP components, finding that P1 correlated with BOLD activation in primary visual areas and N2 correlated with BOLD in the middle occipital gyrus ( V3/ V4), supporting the generation of P1 and N1 in difference regions. This is further supported by EEG source localization studies that have shown P1 to be localized in occipital regions whereas N1 is localized in more parietal regions (Di Russo, Martinez, Sereno, Pitzalis, & Hillyard, 2002). It is important to note that EEG-fMRI does not only focus on identifying generators of the ERP signal. It is possible that BOLD activation in other regions also covaries with the amplitude of the ERPs. Fuglø et al. (2012) suggest that the correlation between ERP parameters and BOLD activation in the visual cortex illustrates that EEG can be used to model fluctuations in attention. However, they do not consider factors that influence P1 and N1 in terms of attention. Activation in the primary visual areas is likely to be influenced by attention because of its influence on sensory encoding, but it is also likely that other brain regions are influenced by fluctuations in attention and this could also be modeled using appropriate ERP parameters. For example, Novitskiy et al. (2011) found both P1 and N1 to be correlated with dorsal stream areas at the temporo-occipital junction. These disparate findings mean that it is not clear what P1 and N1 informed regressors actually model—generators of visual ERPs, attention effects, or both—given that there may be some interaction between attention and the regions generating ERPs. It is worth noting that the task designs for studies by Fuglø et al. (2012) and Novitskiy et al. (2011) differ slightly in that one involved a response and the other did not, and Novitskiy and colleagues used a spatial detection task looking at the laterality of responses. This is important because it has been shown that response modality can influence interpretation of neuroimaging data ( Warbrick, Reske, & Shah, 2013). In addition to the effects on BOLD activation, the differences in task could also influence what is reflected in the ERP parameters and thus the EEG-informed analysis. In the current study, we focus on P1 and N1 components, both of which are known to reflect attentional changes. These attentional modulations can occur in one component and be absent in the other ( Vogel & Luck, 2000), suggesting that the components are somewhat independent and can reflect different processes. The P1 component is sensitive to stimulus properties, whereas the N1 reflects attentional modulations associated with general discrimination processes (Luck, Woodman, & 530

Journal of Cognitive Neuroscience

Vogel, 2000). The P1 and N1 are sensitive to different attentional modulations suggesting that different aspects of stimulus processing can be modeled with these ERP parameters. As such, different patterns of BOLD covariation for P1 and N1 informed regressors would be expected. It would also be expected that we can model single-trial fluctuations or task manipulation effects on these different processes by using the relevant ERP component. The equivocal findings across studies investigating P1 and N1 BOLD covariation make it difficult to draw conclusions about these issues; this is further hindered by the differences in task requirements across studies. In an attempt to resolve these equivocal findings and determine what integration-by-prediction using P1 and N1 can tell us about attention, we will answer the following questions: 1. Do P1 and N1 covary with different patterns of BOLD activation? 2. Does this vary with task requirements? 3. Can we use these peaks to model effects of attention across task manipulations? To do this, we use three versions of a visual oddball task. The stimuli and analysis methods remain the same across task versions; however, the task requirements change. A Passive version of the task requires no response to any stimuli. A silent Count version of the task requires the participant to count the target stimuli, thus directing attention toward these stimuli and requiring a discrimination process. A Respond version requires the participant to push a response button when they see a target stimulus, thus requiring attention to the stimuli, the process of discriminating target from frequent stimuli, and the selection/production of a response to target stimuli. These task manipulations should modulate both the ERP and fMRI responses and allow us to investigate the differences in ERP-fMRI coupling across these conditions.

METHODS The study protocol was approved by the local human subjects review board at RWTH Aachen University and was carried out in accordance with the Declaration of Helsinki. Participants were recruited via flyers, word of mouth, and newsletter alerts. Participants Sixteen healthy right-handed participants (11 men) were recruited for the study; data for three participants were excluded because of poor EEG data quality. Data from 13 participants (eight men, mean age = 28.7 years, SD = 7.2 years) were included in the analysis. Handedness was assessed using the Edinburgh Handedness Scale (Oldfield, 1971). Prior, written informed consent was obtained from all participants. Volume 26, Number 3

Task Participants completed three runs of a visual oddball task using yellow circles as the frequent stimuli and blue circles as the target stimuli. Each run consisted of a total of 200 trials: 160 frequent and 40 target stimuli. The stimuli were 6.8° in diameter with a viewing distance of 2540 mm and were presented on a black background for 500 msec with a variable ISI of 500-10000 msec. Stimuli were generated by a stimulus generator board ( ViSaGe MKII, Cambridge Research System Ltd.) and were displayed on a TFT monitor behind the scanner, viewed via a mirror placed on the head coil. The stimuli were the same across all three runs with only the order changing; three trial sequences were generated using optseq (surfer.nmr.mgh. harvard.edu/optseq). Each run lasted for 10 min 8 sec. The three runs, referred to as Passive, Count, and Respond, varied in the instructions given to the participants and the response required from the participant. In the Passive condition, participants were required to view the stimuli only. In the Count condition, they were asked to silently count the blue circles (target stimuli) and report the number at the end of the run. In the Respond condition, they were required to press a button (Lumitouch, Photon Control, Inc., Burnaby, BC, Canada) with their right index finger in response to target stimuli. The Passive condition was always completed first to ensure that the condition was truly passive viewing and not influenced by the knowledge that the blue circles were targets. The purpose of the Passive condition was to assess the participantsʼ response to stimuli when no cognitive processing was required, thus allowing us to assess the contribution of the cognitive and motor responses over and above that elicited by Passive viewing. We thought this was best achieved by having all participants perform the Passive version of the task before any additional information about counting or responding to the blue circle was given. The order of the Count and Respond conditions was counterbalanced across participants. EEG Data Acquisition EEG data were recorded using a 64-channel, MR-compatible EEG system (Brain Products, Gilching, Germany). The EEG cap (BrainCap MR, EasyCap GmbH, Breitbrunn, Germany) consisted of 63 scalp electrodes distributed according to the 10–20 system and one additional electrode, which was attached to the participantsʼ back for recording the electrocardiogram. Data were recorded relative to an FCz reference, and a ground electrode was located at Iz (10-5 electrode system; Oostenveld & Praamstra, 2001). Data were sampled at 5000 Hz, with a bandpass of 0.016–250 Hz. Impedance at all recording electrodes was less than 10 kΩ. fMRI Data Acquisition Functional MR images were acquired using a 3T scanner (TIM-Trio, Siemens, Erlangen, Germany). To avoid head

movements, the head of each participant was secured using sponge pads. Using EPI, 304 volumes were obtained per run applying the following EPI parameters: 33 slices, slice thickness = 3 mm, field of view = 200 × 200 mm, 64 × 64 matrix, repetition time = 2000 msec, echo time = 30 msec, flip angle = 79°. To facilitate localization and coregistration of functional data, structural scans were acquired using a T1-weighted MRI sequence (MP-RAGE: repetition time/echo time = 2250/3.03 msec, flip angle = 9°, 176 sagittal slices, field of view = 256 × 256 mm, 64 × 64 matrix, voxel size = 1 × 1 × 1 mm). EEG Data Analysis Raw EEG data were initially processed off-line using BrainVision Analyzer 2 (Brain Products, Gilching, Germany). Gradient artifact correction was performed using modified versions of the algorithms proposed by Allen, Josephs, & Turner, (2000), where a gradient artifact template is subtracted from the EEG using a baseline-corrected sliding average of 20 MR volumes. Data were then down-sampled to 256 Hz. Following gradient artifact correction, the data were corrected for cardioballistic artifacts. The initial step involved the semiautomatic detection of R-peaks using Brain Vision Analyzer; the data were then exported to EEGLAB (sccn.ucsd.edu/eeglab/; Delorme & Makeig, 2004) where cardioballistic artifact correction was performed using the FMRIB plug-in (Niazy, Beckmann, Iannetti, Brady, & Smith, 2005). Once corrected for gradient and cardioballistic artifacts, the data were then further processed using the FASTER (Fully Automated Statistical Thresholding for EEG Artifact Rejection; Nolan, Whelan, & Reilly, 2010) plug-in for EEGLAB to perform independent component analysis (ICA) and remove residual artifact. Data were bandpass-filtered between 1 and 95 Hz, notch-filtered at 50 Hz, re-referenced to an average reference, segmented into 600 msec epochs (−200 to 400 msec), and baselinecorrected. Epochs with large artifacts (e.g., muscle activity) are removed by FASTER (95% of all epochs across all participants and conditions were retained). FASTER automatically removes artifactual independent components (ICs), the remaining ICs were used for further analyses. An average of 2.9 (SD = 1.4) ICs were rejected across all participants and conditions. The average initial variance was .69 μV2 (SD = .50), the average final variance was .39 μV2 (SD = .38) resulting in an average of .30 μV2 (SD = 33) variance being removed. ICs representing the P1 and N1 components of the ERP were selected for each participant based on waveform morphology, peak latency and topography and back projected to channel level. For each participant a minimum of 1 and a maximum of 2 ICs represented each component (P1 and N1). All selected ICs were back-projected to channel level (so two to four components in total for each participant). An example of the back-projected ERP can be seen in Figure 1. P1 and N1 were most prominent at parieto-occipital electrodes (PO7 and PO8). The electrode with the largest Warbrick et al.

531

Figure 1. (A) Grand average of ERPs at PO7 and PO8 with the P1 and N1 peaks marked. (B) Stacked plots of single-trial responses at PO7 and PO8 for a representative subject, illustrating a consistent positivity for the P1 component and a consistent negativity for the N1 component. (C) Topography of the P1 (top) and N1 (bottom) components.

P1–N1 amplitude signal was chosen for each participant (Bagshaw & Warbrick, 2007); PO7 was selected for three participants, whereas PO8 was selected for the rest). Single trial N1 and P1 parameters were extracted from responses to target stimuli for each participant by identifying the peak positive amplitude in a window from 70 to 130 msec poststimulus for the P1 and 150–250 msec poststimulus for N1. Single-trial values were then demeaned before inclusion as additional regressors in the general linear model (GLM; see below for further details).

fMRI Data Analysis fMRI analysis was performed with FSL (FMRIBʼs Software Library, www.fmrib.ox.ac.uk/fsl) by employing different modules of the FSL software package. Motion correction was conducted using MCFLIRT ( Jenkinson, Bannister, Brady, & Smith, 2002), nonbrain removal using BET (Smith, 2002), spatial smoothing using a Gaussian kernel of FWHM = 5 mm, mean-based intensity normalization of all volumes by the same factor, and high-pass temporal filtering (125 sec). GLM time-series statistical analysis of individual data sets was carried out using FILM with local autocorrelation correction (Woolrich, Ripley, Brady, & Smith, 2001). Registration of functional images to high-resolution 532


structural images was performed with FLIRT ( Jenkinson et al., 2002; Jenkinson & Smith, 2001).

Combined EEG and fMRI Analysis For statistical analysis, six regressors were created for each participant, one for the target stimuli, one for the frequent stimuli (representing the mean fMRI response), and the other four regressors for the single-trial ERP information P1 target, P1 frequent, N1 target, and N1 frequent. All regressors were convolved with a double gamma hemodynamic response function. A positive contrast was set on each regressor: target stimuli (1 0 0 0 0 0), frequent stimuli (0 1 0 0 0 0), P1 target (0 0 1 0 0 0), P1 frequent (0 0 0 1 0 0), N1 target (0 0 0 0 1 0), N1 frequent (0 0 0 0 0 1). An additional regressor of no interest was added to model out trials where no single-trial EEG data were available. Group level mixed-effect analyses were conducted using FLAME with spatial normalization to MNI (Montreal Neurological Institute) space and applying a cluster significance threshold of Z > 2.0, p = .05 (Forman et al., 1995; Friston, Worsley, Frackowiak, Mazziotta, & Evans, 1993; Worsley, Evans, Marrett, & Neelin, 1992). For the EEG-informed analyses, an uncorrected threshold was used ( p = .001), which is common for EEG-informed Volume 26, Number 3

GLM analysis (Fuglø et al., 2012; Novitskiy et al., 2011; Karch et al., 2010). Group level analysis was carried out for the Passive, Count, and Respond conditions separately to create a mean for each first-level contrast. To test differences between the three conditions, paired sample t tests were performed incorporating the following contrasts: Passive > Count, Passive > Respond, Count > Respond, Respond > Passive, Count > Passive, Respond > Count. The aim was to show the differences in the cognitive processes associated with the different response modalities. For these paired sample t test, a cluster-corrected threshold of z = 2.0, p < .05 was used. To test whether P1 and N1 informed analyses were different from each other, a second-level analysis was performed to directly contrast P1 and N1 (P1 > N1 and N1 > P1); for each condition, a cluster-corrected threshold of z = 2.0, p < .05 was used. Functional data were imported to MRIcron (Rorden, Karnath, & Bonilha, 2007) for visual display purposes.

RESULTS

Table 1. t Statistics for Comparisons between Conditions on ERP Parameters Variable

T

p

Passive vs. Count

0.68

.51

Passive vs. Respond

0.36

.72

Count vs. Respond

0.99

.34

Passive vs. Count

0.67

.51

Passive vs. Respond

0.19

.86

Count vs. Respond

0.59

.56

Passive vs. Count

0.32

.75

Passive vs. Respond

1.1

.28

Count vs. Respond

1.5

.15

Passive vs. Count

0.98

.35

Passive vs. Respond

1.24

.24

Count vs. Respond

0.15

.42

P1 Amplitude

P1 Single-Trial Standard Deviation

N1 Amplitude

Behavioral Data Mean RT in the respond condition was 464 msec (SD = 57) with participants correctly responding to 38–40 (mean = 39.7, SD = 0.6) of the target stimuli. Only 0–2 (mean = 0.54, SD = 0.9) false alarms were made in the respond condition. In the count condition, participants reported between 38 and 41 (mean = 39.7, SD = 0.7) of the 40 target stimuli demonstrating very good compliance with the task. EEG Data All three conditions yielded ERPs with the expected morphology: P1 peaking around 100 msec poststimulus and N1 peaking around 180 msec poststimulus for all conditions and both stimulus types (Figure 1). Mean P1 and N1 amplitudes did not differ between conditions. The standard deviation of single-trial amplitudes for both the P1 and N1 components did not differ between conditions. All statistics for these comparisons can be seen in Table 1; mean amplitudes and mean single trial standard deviation can be seen in Figure 2. P1 and N1 were related to each other for all conditions (Passive: r(12) = −0.84, p < .001; Count: r(12) = −0.62, p = .02; Respond: r(12) = −.68, p = .01, respectively). Neither P1 or N1 were related to RT in the Respond condition (r(12) = −0.25, p = .42 and r(12) = −.40, p = .17, respectively). Conventional fMRI Analysis All three conditions elicited activation in regions associated with a response to visual stimulation (occipital cortex) for the target stimuli; the Count and Respond conditions also showed target detection-related activity (Figure 3).

N1 Single-Trial Standard Deviation

Paired samples t tests. df = 12.

To assess the contribution of response-related processing over and above that elicited by passive viewing of a stimulus, the differences between conditions for the target stimuli were investigated using paired samples t tests (described in the Methods section). Both the Count and Respond conditions differed significantly from the Passive condition in a number of brain regions including the precentral and postcentral gyri, regions of the parietal cortex, and the middle and inferior frontal gyri (Figure 4, top and bottom rows). The Respond > Count contrast yielded significant differences in the parietal operculum, inferior parietal lobule, insula, ACC, and the posterior cingulate cortex (Figure 4, bottom row). The following contrasts revealed no significant differences between Passive > Count, Passive > Respond, and Count > Respond. EEG-informed fMRI Analysis Using a cluster-corrected threshold, only P1 informed analysis for the Passive condition revealed significant covariation between P1 amplitude and the BOLD response (Figure 5; Table 2) in the precentral gyrus, superior parietal lobule, SMA, superior frontal gyrus, central operculum, and insula cortex. At a less conservative threshold (uncorrected p < .001), P1 amplitude covaried with BOLD activation Warbrick et al.

533

Figure 2. Mean amplitude of the P1 and N1 components for all conditions (left side) n.b. the polarity of the N1 amplitude has inverted for display purposes. The right side of the figure shows mean single-trial standard deviation for all conditions for both the P1 and N1. Error bars represent standard deviation on all plots.

Figure 3. BOLD activation (z scores) in response to target stimuli in the Passive, Count, and Respond conditions (second-level mixed-effects FLAME, n = 13, Cluster-corrected threshold Z = 2.0, p = .05).

534


in all conditions and N1 covaried with BOLD activation in the Passive and Respond conditions. This can be seen in Figure 5; the regions involved for each condition and ERP component are given in Table 2. P1 and N1 appear to covary with overlapping brain regions and also independent brain regions. To test this P1 and N1 covariation with BOLD response was directly contrasted (see Methods) for all conditions. For the Passive condition, P1 informed analysis revealed more covariation with BOLD response than the N1 informed analysis in the SMA and precentral gyrus. The N1 > P1 contrast revealed no covariation with BOLD response for the Passive condition. For the Count condition, P1 informed analysis revealed more covariation with BOLD response than the N1 informed analysis in the precentral gyrus, SMA, superior parietal lobule, superior frontal gyrus, middle frontal gyrus, precuneus inferior frontal gyrus, lateral occipital cortex, parietal operculum, and supramarginal gyrus. The N1 > P1 contrast revealed no difference in covariation with BOLD response for the Count condition. For the Respond condition, the P1 > N1 contrast revealed no difference in covariation with BOLD response, but the N1 > P1 contrast revealed more covariation with BOLD response in the lateral occipital cortex, angular gyrus, superior parietal lobule, and middle frontal gyrus. These findings are summarized in Figure 6 and Table 3.

Volume 26, Number 3

frontal gyrus, and middle frontal gyrus. For the Respond condition, N1 informed analysis covariation with BOLD response than the Count condition in lateral occipital cortex, precuneus, occipital pole, and intracalcarine cortex. These findings are summarized in Figure 7 and Table 4.

DISCUSSION

Figure 4. BOLD activation (z scores) for the Count and Respond conditions against the Passive condition (top and bottom rows, respectively) and the Respond condition against the Count condition (bottom row) for the target stimuli for the conventional analysis (second-level mixed-effects FLAME, n = 13, Cluster-corrected threshold Z = 2.0, p = .05).

The difference between conditions on the P1 and N1 informed analysis was also of interest. For the Passive condition, P1 informed analysis covaried with more BOLD activation than the Count condition and the Respond condition in the postcentral gyrus and precentral gyrus. For the P1 informed analysis, the Count condition also differed from the Respond condition in the angular gyrus, supramarginal gyrus, lateral occipital cortex, superior

We investigated the effects of task demands on the correlation between single-trial ERP parameters and the fMRI BOLD response. Using three versions of the visual oddball task (Passive, Count, and Respond), we aimed to show that attentional variations could be modeled using singletrial P1 and N1 amplitudes. All three conditions elicited ERPs and fMRI BOLD activation in regions associated with a response to visual stimulation (occipital cortex) for the target stimuli; the Count and Respond conditions also showed target detection-related activity in parietal and prefrontal regions. The fMRI results are discussed in detail elsewhere (Warbrick et al., 2013); the current manuscript focuses on how EEG-informed fMRI can be used to investigate single-trial variability in task performance. Single-trial P1 and N1 amplitude covaried with BOLD activation in different brain regions. Furthermore, this varied as a function of task demands. Our first aim was to investigate whether P1 and N1 model BOLD activation in different regions. Our EEG-informed fMRI analyses revealed different patterns of activation to covary with the single-trial P1 and N1 amplitudes. Previous research has shown that different ERP peaks can isolate different patterns of BOLD activation (Karch et al., 2010; Eichele et al., 2005); however, the findings for the visual P1 and N1 are equivocal. Novitskiy et al. (2011) found both P1 and N1 to be correlated with dorsal stream areas at the temporo-occipital junction and also the left medial frontal and precentral gyri, whereas Fuglø et al. (2012) found that P1 and N2 (equivalent to our N1) single-trial amplitudes correlated with activation in the primary visual regions. In this study, we found that BOLD activation in frontal,

Figure 5. P1 and N1 informed analysis for all three conditions. The figure shows the mean BOLD response (z scores) for each set of analyses for all conditions separately, illustrating the different patterns of activation for the P1 and N1 informed analyses and the differences between the conditions. The differences between conditions are illustrated in Figures 6 and 7 (second-level mixed-effects FLAME, n = 13, uncorrected, p = .001).

Warbrick et al.

535

Table 2. BOLD Activation that Covaries with the P1 and N1 Components for All Conditions Region (Harvard-Oxford, Max. Prob.)

MNI Coordinates of Local Max. (x, y, z)

Max. Z

Passive P1

Region (Harvard-Oxford, Max. Prob.)

MNI Coordinates of Local Max. (x, y, z)

Max. Z

Passive N1

Precentral gyrus (L)

−26

−18

54

4.0

Cerebelluma

−14

−58

−30

Sup. parietal lob (L)

−22

−46

70

3.7

Frontal pole (L)

−54

38

4

2.8

Postcentral gyrus (L)

−44

−28

66

3.5

Frontal pole (R)

52

46

0

2.7

SMA (L)

−10

−8

56

3.9

Supramarginal gyrus (L)

−54

−36

42

3.0

Sup. frontal gyrus (L)

−12

2

64

3.3

Lat. occ. Cortex (L)

−58

−62

8

2.4

Cent. Op. cortex (L)

−46

20

24

3.6

Insula (L)

−34

−14

10

2.9

Count P1

Count N1

Sup. frontal gyrus (L)

−22

−4

60

2.6

Middle frontal gyrus (L)

−32

26

56

2.4

Middle frontal gyrus (R)

30

34

50

2.8

−50

−70

40

2.3

6

−10

62

2.9

68

−18

−4

2.4

Lat. occ. Cortex (L) SMA (R) Sup. Temp. gyrus (R)

3.12

Respond P1

Respond N1

Fr. orbital cortex (L)

−20

8

−22

3.2

Inf. temporal gyrus (L)

−56

−48

−20

3.42

Cent. Op. cortex (R)

48

−8

6

2.4

Inf. temporal gyrus (R)

52

−38

−20

2.6

Lat. occ. Cortex (L)

−44

−58

56

2.6

Lat. occ. Cortex (R)

26

−58

64

2.3


−32

34

34

3.1


28

34

50

2.4

SMA (R)

4

−6

64

2.7

SMA (L)

−10

−6

42

2.4

−8

34

46

2.6

Lat. occ. Cortex (R)

Sup. Frontal gyrus (R) Uncorrected, p < .001, n = 13. L = left, R = right. a

MNI label.

parietal, and temporal cortical regions correlated with P1 and N1 amplitude, suggesting that we are able to capture activation related to higher-order processing of the stimulus with the P1 and N1 components of the ERP. Furthermore, in contrast to Novitskiy et al. (2011), we found P1 and N1 to be correlated with BOLD activation in different regions, suggesting that they are differentially informative. Broadly, P1 was shown to covary with BOLD activation in the SMA, precentral gyrus, middle and superior frontal gyri, lateral occipital cortex, and superior parietal lobule. N1 amplitude was shown to covary with the frontal pole, middle and superior frontal gyri, supramarginal gyrus, 536


lateral occipital cortex, SMA, and inferior temporal gyrus. There was some degree of overlap between the P1 and N1 informed analysis, which is to be expected given that they are responses to the same stimulus and are, to some extent, related; however, P1 and N1 informed analyses did reveal different patterns of covariation with BOLD response across the different task manipulations. Where Is P1 More Informative than N1? P1 should model “attended to” versus “not attended to” stimuli because it is sensitive to such attentional modulation Volume 26, Number 3

Figure 6. Comparison of the P1 and N1 informed analysis. (A) The P1 > N1 contrast reveals significant areas of covariation between EEG parameters and BOLD response (z scores) for the Passive and Count conditions, but not the Respond condition. (B) The N1 > P1 contrast reveals significant areas of covariation for the Respond condition only (second-level mixed-effects FLAME, n = 13, cluster-corrected z = 2.0, p = .05).

(Luck et al., 2000). It was therefore expected that P1related BOLD activation in the Count and Respond conditions would be different to the Passive condition. For the respond condition, P1 did not covary with any BOLD activation over and above the N1. For the Passive and

Count conditions, P1 emphasized covariation with BOLD response in the SMA and precentral gyrus. This is an interesting result, given that these regions are associated with planning and production of a motor response, but no response is required in these two conditions. However,

Table 3. A Comparison between P1 and N1 Informed Analysis of Responses to Target Stimuli for All Conditions MNI Coordinates of Local Maxima x

y

z

Maximum Z Value

Overall Cluster Size (# Voxels)

−10

−8

56

3.9

413

−4

−20

62

3.2

SMA (R)

6

0

52

4.1

Precentral gyrus (R)

4

−14

58

4.4

Postcentral gyrus (R)

28

−36

48

3.9

Superior parietal lobule (R)

22

−42

68

3.5


36

4

64

3.5

Superior frontal gyrus (R)

6

−52

66

3.4

Superior frontal gyrus (L)

30

0

70

3.8

Precuneus (R)

−22

−4

70

3.5

309

Inferior frontal gyrus (L)

−58

18

24

3.1

294

Lateral occipital cortex (L)

−58

−60

40

3.9

278


−44

−60

54

4.2

455

Angular gyrus (L)

−50

−52

54

3.7

Superior parietal lobule (L)

−36

−52

38

3.23


−30

30

34

5.6

Region (Harvard-Oxford, Maximum Probability) Passive P1 > N1 SMA (L) Precentral gyrus (L) Count P1 > N1

780

522

384

Respond N1 > P1

258

Cluster-corrected, z = 2.0, p = .05, n = 13. L = left, R = right.

Warbrick et al.

537

Figure 7. Comparison of conditions for the P1 and N1 informed analysis. For the P1 informed analysis, the Passive condition shows more covariation between EEG parameters and BOLD response than the Count and Respond conditions, and the Count condition shows more covariation than the Respond condition. For the N1 analysis, the Respond condition shows more covariation than the Count condition (second-level mixed-effects FLAME, n = 13, cluster-corrected z = 2.0, p = .05).

Table 4. A Comparison between Conditions for P1 and N1 Informed Analysis of Responses to Target Stimuli MNI Coordinates of Local Maxima x

y

z

Maximum Z Value

Overall Cluster Size (# Voxels)


−34

−26

70

3.5

459


−26

−18

54

3.1

−8

−12

76

2.1


−34

−28

68

3.2


−28

−22

54

3.5

Angular gyrus (L)

−46

−58

54

3.4

Supramarginal gyrus (L)

−54

−50

54

3.3


−32

−70

54

3.2

Superior frontal gyrus (L)

−20

26

60

3.6


−28

18

56

3.3

12

−66

54

3.3

Precuneus (R)

4

−62

54

3.1

Occipital pole (R)

8

−96

2

2.9

Intracalcarine (R)

2

−86

−4

2.8

Region (Harvard-Oxford, Maximum Probability) Passive > Count P1

Superior frontal gyrus Passive > Respond P1

391

Count > Respond P1 526

433

Respond > Count N1 Lateral occipital cortex (R)

480

384

Cluster-corrected, z = 2.0, p = .05, n = 13. L = left, R = right.

538


Volume 26, Number 3

the SMA is also involved in the integration of sensory information for further processing (Heekeren, Marrett, & Ungerleider, 2008), so its involvement here can be partially explained by this role. Furthermore, the fMRI data alone showed target-related SMA activation for the Count condition (Warbrick et al., 2013), suggesting that it is involved in processing a stimulus even when no response is required. However, what remains unclear is why SMA activation should correlate with P1 amplitude in the Passive condition. In addition, the fact that P1 variability covaried with activation in the precentral gyrus (primary motor region) is also difficult to explain, because there was no motor response required in the Passive and Count conditions. Novitskiy et al. (2011) found P1 and N1 to be correlated with BOLD activation in the precentral gyri; however, their task did involve a motor response. It is logical to assume that the covariation in the study by Novitskiy and colleagues can be explained by the motor response but our findings suggest that P1 covaries with BOLD activation in these regions in the absence of a motor response, suggesting that this region plays a different role here. Further investigation of task requirements and P1-BOLD covariation is required. For the Count condition, P1 additionally covaried with BOLD activation in the superior and middle frontal gyri and lateral occipital cortex when compared with N1. These frontal and posterior parietal regions are involved in the top–down influence of the dorsal attention network (Corbetta & Shulman, 2002) and could influence attention to the target stimuli. Although this network has often been associated with orienting attention to the spatial location of stimuli (Corbetta & Shulman, 2002), it has recently been shown that frontal and parietal cortical regions can also respond to nonspatial features (Liu, Hospadaruk, Zhu, & Gardner, 2011). We propose that attention to the target stimuli in the Count condition is captured in the P1 amplitude and that this activity is related to the top–down control exerted by these frontoparietal regions.

Where Is N1 More Informative than P1? N1 amplitude was not more informative than P1 amplitude for the Passive and Count conditions. However, for the Respond condition, N1 amplitude covaried with more BOLD activation than P1 in the middle frontal gyrus, angular gyrus, superior parietal lobule, and lateral occipital cortex. N1 should model activation associated with discrimination processes. Given that discrimination processes are involved in the Count and Respond conditions, it was expected that they should differ from the Passive condition. Count and Respond should also differ from each other given that N1 modulation can vary between response and working memory type paradigms (Luck et al., 2000). We only found support for the latter. The regions involved are associated with the integration of sensory information,

making a decision about a stimulus and selecting the correct response. The middle frontal gyrus has been shown to be involved in mapping of stimuli to responses (Huettel & McCarthy, 2004), supporting the presence of this activation in the Respond condition. The superior parietal regions and the lateral occipital cortex are associated with target detection, attentional allocation, and integrating sensory and decision information (Warbrick et al., 2013; Goldman et al., 2009; Weissman, Roberts, Visscher, & Woldorff, 2006). Given that a number of processes are taking place as we reach the latency of the N1 component, it is likely that its amplitude covaries with a number of brain regions involved in the detection of a target stimulus and making the appropriate response to that stimulus. This can be clearly seen in our findings for the Respond condition in comparison with the other conditions. This suggests that the capability of N1 to model cognitive processes changes with task demands.

BOLD Covariation with P1 and N1 across Task Conditions Our second aim was to investigate whether P1/N1 and BOLD covariation varied with task requirements. Our data show that P1 and N1 appear to be differentially informative across task conditions. When comparing the conditions with each other, we can see that P1 is more informative in the precentral gyrus and SMA for the passive condition compared with both the count and respond conditions. P1 is also more informative in the count condition compared with the respond condition in the middle and superior frontal gyri, supramarginal gyrus, and the lateral occipital cortex. The N1, on the other hand, only shows more covariation with BOLD response for the Respond condition compared with the Count condition. It is possible that the P1 is more informative for the Passive condition compared with the other conditions because attention is not focused on a task in the Passive condition. With the single-trial informed regressors, we are modeling single-trial variability; it is possible that this variability, in both the ERP and BOLD data, is higher when there are no specific task instructions to follow. This fluctuation in attention could influence how much attention is allocated to the task and thus influences the sensory encoding process. P1 is also more informative for the Count condition compared with the Respond condition in brain regions involved in integrating sensory information into an appropriate decision and response. This dorsal frontoparietal network has been implicated in the top– down modulation of attentional control in that this network contributes to the encoding of priority information for visual features (Hou & Liu, 2012). The correlation between P1 amplitude and activity in these regions could reflect the effects of attention on the encoding of stimulus features because P1 is sensitive to stimulus properties and interpretation of stimulus properties is required to make a Warbrick et al.

539

decision about whether a target or nontarget stimulus has been presented. It is not clear why this would be more so in the Count condition compared with the Respond condition, but one potential explanation is that, in addition to a decision about stimulus type, the Count condition also requires the number of target stimuli to be updated and held in ongoing working memory, thus placing an increased demand on these regions. The N1 explains more variance for the Respond condition than in the Count condition. This could be explained by the nature of the task because the N1 is known to behave differently for working memory compared with response-type tasks (Luck et al., 2000). N1 can be modulated by stimulus novelty and frequency (at least in the auditory domain; Herrmann & Knight, 2001), and the nature of our oddball task could have influenced the N1 amplitude in this respect. Single-trial N1 amplitude covaries with BOLD activation in frontal and temporal regions that are likely to be involved in the ventral stream of visual processing and are associated with stimulus saliency (Corbetta & Shulman, 2002). The fact that this is more apparent in the Respond condition compared with the other conditions suggests that the N1 is able to model BOLD activation related to identification of the target stimuli and executing the appropriate response to those stimuli. The N1 informed analysis shows more covariation with BOLD response in the lateral occipital cortex and the precuneus for the Respond condition compared with the Count condition. The lateral occipital cortex is not only indicative of processing visual stimuli but has also been implicated in attention allocation (Goldman et al., 2009; Weissman et al., 2006). In addition, the precuneus has shown increased activation when participants are engaged in target detection (Corbetta, Kincade, Ollinger, McAvoy, & Shulman, 2000). The N1 component in the Respond condition also showed more covariation with BOLD response in the inferior temporal gyrus that might reflect attention to the saliency of the target stimuli. In summary, the regions where BOLD activation covaries with N1 amplitude in the Respond condition are involved in the integration of sensory information and response selection and production, further supporting the idea that these processes can be modeled with N1. Limitations The absence of effects in the ERP data can be considered a limitation of the study. It could be expected that, if single-trial fluctuations in attention are important and measurable, then they would be reflected in the variance in the single-trial measures for each condition. We did not find differences in SD between any of the condition. Also, if our conclusions regarding the differences in P1 and N1 informed analyses are “true” in terms of differences between conditions, we would expect the average ERP data to also reflect these differences. However, we found no differences between any of the conditions in the peak 540


amplitudes of the P1 and N1 components. It is possible that the effects in the single-trial informed fMRI analysis are only visible when the data are combined in this way on a single-trial level, which would support our approach and highlight the advantage of using combined EEG-fMRI data. An additional potential limitation of the study is the small sample size; however, n = 12 is not unreasonable for an fMRI study, particularly given that we employed a within-subject design. We also performed a number of whole-brain contrasts, which may increase the overall possibility of type I error. The whole-brain analyses for the main findings (P1 vs. N1 and the condition comparisons) are corrected for multiple comparison across voxels for each analyses, but this does not account for the multiple tests performed. Additional support for our findings through replication and perhaps larger sample sizes in future work would be beneficial. However, despite this, our findings are plausible in the context of the study and contribute to the understanding of covariation between fMRI BOLD response and the early ERP components. Conclusions In summary and in response to our third question, we can conclude that it is indeed possible to use the P1 and N1 components of the ERP to model task-related modulation of brain activation between conditions We have shown that the P1 and N1 components of the visual ERP can be used in the integration-by-prediction method of EEG-fMRI data integration to highlight brain regions related to target detection and response production. We have also shown that EEG-informed analysis using these peaks is sensitive to task manipulations and can be used to model attentional and response differences across conditions. Our data suggest that the P1 component of the ERP can reflect changes in sensory encoding of stimulus features (as a result of attentional fluctuations) and is more informative for the Passive and Count conditions. The N1, on the other hand, was more informative for the Respond condition, suggesting that it can be used to model the processing of stimulus, meaning specifically discriminating one type of stimulus from another, and processes involved in the early stages of integrating sensory information with response selection and production. All too often studies attempt to assess how BOLD activation covaries with ERP components without any real consideration of what these components actually reflect. EEG-fMRI analysis should go beyond the methodological considerations of how to perform the analyses and the simple question “does this component correlate with any BOLD activation” and consider what aspects of cognitive function are being measured. Our results show that an understanding of the underlying electrophysiology is necessary for a thorough interpretation of results. Volume 26, Number 3

Acknowledgments We would like to thank Christina Regenbogen and Andrew Bagshaw for assistance with single-trial ERP data analysis. Reprint requests should be sent to Tracy Warbrick, Institute of Neuroscience and Medicine, Forschungszentrum Jülich, Leo-Brandt Strasse, 52425 Juelich, Germany, or via e-mail: [email protected].

REFERENCES Allen, P. J., Josephs, O., & Turner, R. (2000). A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage, 12, 230–239. Bagshaw, A. P., & Warbrick, T. (2007). Single trial variability of EEG and fMRI responses to visual stimuli. Neuroimage, 38, 280–292. Biessmann, F., Plis, S., Meinecke, F. C., Eichele, T., & Muller, K. R. (2011). Analysis of multimodal neuroimaging data. IEEE Reviews in Biomedical Engineering, 4, 26–58. Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, M. P., & Shulman, G. L. (2000). Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nature Neuroscience, 3, 292–297. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215. Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D. Y., & Engel, A. K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. Journal of Neuroscience, 25, 11730–11737. Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134, 9–21. Di Russo, F., Martinez, A., Sereno, M. I., Pitzalis, S., & Hillyard, S. A. (2002). Cortical sources of the early components of the visual evoked potential. Human Brain Mapping, 15, 95–111. Eichele, T., Specht, K., Moosmann, M., Jongsma, M. L., Quiroga, R. Q., Nordby, H., et al. (2005). Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proceedings of the National Academy of Sciences, U.S.A., 102, 17798–17803. Fell, J. (2007). Cognitive neurophysiology: Beyond averaging. Neuroimage, 37, 1069–1072. Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995). Improved assessment of significant activation in functional magnetic-resonance-imaging (fMRI)—Use of a cluster-size threshold. Magnetic Resonance in Medicine, 33, 636–647. Friston, K. J., Worsley, K. J., Frackowiak, R. S. J., Mazziotta, J. C., & Evans, A. C. (1993). Assessing the significance of focal activations using their spatial extent. Human Brain Mapping, 1, 210–220. Fuglø, D., Pedersen, H., Rostrup, E., Hansen, A. E., & Larsson, H. B. (2012). Correlation between single-trial visual evoked potentials and the blood oxygenation level dependent response in simultaneously recorded electroencephalography-functional magnetic resonance imaging. Magnetic Resonance in Medicine, 68, 252–260. Goldman, R. I., Wei, C. Y., Philiastides, M. G., Gerson, A. D., Friedman, D., Brown, T. R., et al. (2009). Single-trial discrimination for integrating simultaneous EEG and

fMRI: Identifying cortical areas contributing to trial-to-trial variability in the auditory oddball task. Neuroimage, 47, 136–147. Heeger, D. J., & Ress, D. (2002). What does fMRI tell us about neuronal activity? Nature Reviews Neuroscience, 3, 142–151. Heekeren, H. R., Marrett, S., & Ungerleider, L. G. (2008). The neural systems that mediate human perceptual decision making. Nature Reviews Neuroscience, 9, 467–479. Herrmann, C. S., & Knight, R. T. (2001). Mechanisms of human attention: Event-related potentials and oscillations. Neuroscience and Biobehavioral Reviews, 25, 465–476. Hou, Y., & Liu, T. (2012). Neural correlates of object-based attentional selection in human cortex. Neuropsychologia, 50, 2916–2925. Huettel, S. A., & McCarthy, G. (2004). What is odd in the oddball task? Prefrontal cortex is activated by dynamic changes in response strategy. Neuropsychologia, 42, 379–386. Huster, R. J., Debener, S., Eichele, T., & Herrmann, C. S. (2012). Methods for simultaneous EEG-fMRI: An introductory review. Journal of Neuroscience, 32, 6053–6060. Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002). Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage, 17, 825–841. Jenkinson, M., & Smith, S. (2001). A global optimisation method for robust affine registration of brain images. Medical Image Analysis, 5, 143–156. Karch, S., Feuerecker, R., Leicht, G., Meindl, T., Hantschk, I., Kirsch, V., et al. (2010). Separating distinct aspects of the voluntary selection between response alternatives: N2- and P3-related BOLD responses. Neuroimage, 51, 356–364. Lauritzen, M., & Gold, L. (2003). Brain function and neurophysiological correlates of signals used in functional neuroimaging. Journal of Neuroscience, 23, 3972–3980. Liu, T., Hospadaruk, L., Zhu, D. C., & Gardner, J. L. (2011). Feature-specific attentional priority signals in human cortex. Journal of Neuroscience, 31, 4484–4495. Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature, 412, 150–157. Luck, S. J., Woodman, G. F., & Vogel, E. K. (2000). Event-related potential studies of attention. Trends in Cognitive Sciences, 4, 432–440. Musall, S., von Pfostl, V., Rauch, A., Logothetis, N. K., & Whittingstall, K. (2012). Effects of neural synchrony on surface EEG. Cerebral Cortex. doi:101093/cercor/bhs389. Niazy, R. K., Beckmann, C. F., Iannetti, G. D., Brady, J. M., & Smith, S. M. (2005). Removal of fMRI environment artifacts from EEG data using optimal basis sets. Neuroimage, 28, 720–737. Nolan, H., Whelan, R., & Reilly, R. B. (2010). FASTER: Fully automated statistical thresholding for EEG artifact rejection. Journal of Neuroscience Methods, 192, 152–162. Novitskiy, N., Ramautar, J. R., Vanderperren, K., De Vos, M., Mennes, M., Mijovic, B., et al. (2011). The BOLD correlates of the visual P1 and N1 in single-trial analysis of simultaneous EEG-fMRI recordings during a spatial detection task. Neuroimage, 54, 824–835. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97–113. Oostenveld, R., & Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology, 112, 713–719. Rorden, C., Karnath, H. O., & Bonilha, L. (2007). Improving lesion-symptom mapping. Journal of Cognitive Neuroscience, 19, 1081–1088.

Warbrick et al.

541

Smith, S. M. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17, 143–155. Vogel, E. K., & Luck, S. J. (2000). The visual N1 component as an index of a discrimination process. Psychophysiology, 37, 190–203. Warbrick, T., Mobascher, A., Brinkmeyer, J., Musso, F., Richter, N., Stoecker, T., et al. (2009). Single-trial P3 amplitude and latency informed event-related fMRI models yield different BOLD response patterns to a target detection task. Neuroimage, 47, 1532–1544. Warbrick, T., Reske, M., & Shah, J. (2013). Do EEG paradigms work in fMRI? Varying task demands in the visual oddball

542


paradigm: Implications for task design and results interpretation. Neuroimage, 77, 177–185. Weissman, D. H., Roberts, K. C., Visscher, K. M., & Woldorff, M. G. (2006). The neural bases of momentary lapses in attention. Nature Neuroscience, 9, 971–978. Woolrich, M. W., Ripley, B. D., Brady, M., & Smith, S. M. (2001). Temporal autocorrelation in univariate linear modeling of fMRI data. Neuroimage, 14, 1370–1386. Worsley, K. J., Evans, A. C., Marrett, S., & Neelin, P. (1992). A three-dimensional statistical analysis for CBF activation studies in human brain. Journal of Cerebral Blood Flow and Metabolism, 12, 900–918.

Volume 26, Number 3