Temporal integration - CiteSeerX

Clinical Neurophysiology 113 (2002) 1909–1920 www.elsevier.com/locate/clinph

Temporal integration: intentional sound discrimination does not modulate stimulus-driven processes in auditory event synthesis Elyse Sussman a,b,*, Istva´n Winkler c, Judith Kreuzer a, Marieke Saher d, Risto Na¨a¨ta¨nen d, Walter Ritter e a Department of Neuroscience, Albert Einstein College of Medicine, 1410 Pelham Parkway S, Bronx, NY 10461, USA Department of Otolaryngology, Albert Einstein College of Medicine, 1410 Pelham Parkway S, Bronx, NY 10461, USA c Institute of Psychology, Hungarian Academy of Sciences, H-1394, Szondi u. 83/85, P.O. Box 398, Budapest, Hungary d Cognitive Brain Research Unit, P.O. Box 13 University of Helsinki, Helsinki, FIN-00014, Finland e Cognitive Neuroscience and Schizophrenia Program, Nathan Kline Institute for Psychiatric Research, 140 Old Orangeburg Rd., Orangeburg, NY, USA b

Accepted 3 September 2002

Abstract Objective: Our previous study showed that the auditory context could influence whether two successive acoustic changes occurring within the temporal integration window (approximately 200 ms) were pre-attentively encoded as a single auditory event or as two discrete events (Cogn Brain Res 12 (2001) 431). The aim of the current study was to assess whether top-down processes could influence the stimulus-driven processes in determining what constitutes an auditory event. Methods: Electroencepholagram (EEG) was recorded from 11 scalp electrodes to frequently occurring standard and infrequently occurring deviant sounds. Within the stimulus blocks, deviants either occurred only in pairs (successive feature changes) or both singly and in pairs. Event-related potential indices of change and target detection, the mismatch negativity (MMN) and the N2b component, respectively, were compared with the simultaneously measured performance in discriminating the deviants. Results: Even though subjects could voluntarily distinguish the two successive auditory feature changes from each other, which was also indicated by the elicitation of the N2b target-detection response, top-down processes did not modify the event organization reflected by the MMN response. Conclusions: Top-down processes can extract elemental auditory information from a single integrated acoustic event, but the extraction occurs at a later processing stage than the one whose outcome is indexed by MMN. Significance: Initial processes of auditory event-formation are fully governed by the context within which the sounds occur. Perception of the deviants as two separate sound events (the top-down effects) did not change the initial neural representation of the same deviants as one event (indexed by the MMN), without a corresponding change in the stimulus-driven sound organization. q 2002 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Auditory perception; Auditory event formation; Event-related potentials; Mismatch negativity; Stimulus-driven; Temporal integration

1. Introduction The auditory system tends to integrate the incoming sensory information that arrives within a 200 ms period (see, e.g. loudness summation (Zwislocki, 1960) and auditory recognition masking (Massaro, 1975)). However, not all processes related to the 200 ms long integration window results in loss of information, otherwise it would be impossible to understand normal speech, in which fast transient changes, such as most consonant sounds, are interleaved

with somewhat longer and more stable vowel sounds (Cowan, 1984). Discriminating rapid changes in sound parameters, such as frequency or intensity, may therefore be dependent upon the auditory context. Our previous study showed that the auditory context could determine whether a succession of changes was pre-attentively encoded as a single auditory event or as two discrete events (Sussman and Winkler, 2001). The current study compares the pre-attentive encoding of successive changes that occur within this integration period with the perceptual representation of these changes.

* Corresponding author. Tel.: 11-718-430-3313; fax: 11-718-430-8821. E-mail address: [email protected] (E. Sussman). 1388-2457/02/$ - see front matter q 2002 Elsevier Science Ireland Ltd. All rights reserved. PII: S 1388-245 7(02)00300-0

CLINPH 2002064

1910

E. Sussman et al. / Clinical Neurophysiology 113 (2002) 1909–1920

1.1. Mismatch negativity (MMN) component An infrequent auditory stimulus (a deviant sound) occurring within a repetitive sequence of a frequent sound (the standard), elicits the mismatch negativity (MMN) component of event-related potentials (ERPs), irrespective of the direction of attention (for a recent review, see Picton et al., 2000). The most common way to elicit MMN is to present an infrequent stimulus that deviates from a frequently repeating (standard) stimulus. Simple deviations in stimulus features, such as frequency, intensity, or spatial location, as well as deviations in complex temporal patterns of sounds, elicit the MMN (Na¨ a¨ ta¨ nen, 1992; Na¨ a¨ ta¨ nen et al., 2001). The main neural generators associated with MMN are located bilaterally in the supratemporal plane (also the location of primary auditory cortex), as determined by dipolemodeling of electric (Scherg et al., 1989) and magnetic responses (Sams and Hari, 1991; Woldorff et al., 1998), scalp-current density maps of scalp-recorded ERPs (Giard et al., 1990), functional magnetic resonance imaging (fMRI; Opitz et al., 1999), and intracortical ERP recordings (Halgren et al., 1995; Kropotov et al., 1995). The scalp distribution of the MMNs elicited by changes in stimulus frequency, intensity, and duration differ slightly, indicating that the MMNs elicited by deviations in the different features arise from slightly different neural populations in auditory cortex (Giard et al., 1995; Paavilainen et al., 1991). The location of the generators in auditory cortex accounts for the observed scalp topography of the waveform, which is maximally negative over the fronto-central scalp locations and inverts in polarity below the Sylvian fissure. The component typically peaks between 100 and 200 ms from the detection (by the brain) of stimulus deviance. The deviance-detection process, reflected by the MMN, is based upon cortical sound representations extracted from the acoustic input. Evidence that activation of NMDA receptors plays a role in the MMN process is compatible with the notion that the underlying mechanisms of this cortical auditory information processing network involve sensory memory (Javitt et al., 1996). The MMN was used, in the current study, to investigate sound change detection because the elicitation of MMN in a given auditory context provides information about the pre-attentive encoding of the sound change (the deviant event) and also about the preattentive encoding of the previous stimulus events (the neural representation of the regular or standard sounds). The pre-attentive encoding of successive auditory changes occurring within the temporal integration period (as indicated by the MMN component) will then be compared with the perceptual representation of these events (as measured by the participants’ performance in distinguishing the successive sounds changes from each other). 1.2. N2b component Another ERP component assessed in the present study

was the N2b response, which indexes attentional deviance detection. N2b is evoked when an infrequent stimulus is attentively detected as being different from the regular stimuli, whether or not a decision about the deviant sounds is required by the task (Na¨ a¨ ta¨ nen et al., 1982; Novak et al., 1990; Renault and Lesevre, 1979). The N2b ERP component peaks about 200–300 ms from stimulus onset, following the MMN, and can be further distinguished from MMN by its scalp topography (showing its maximum in centroparietal sites with no polarity reversal at the mastoid sites; Alho et al, 1986; Novak et al., 1992a,b, 1990), as its generators are widespread and located outside the auditory cortex. In our recent ERP studies (Czigler and Winkler, 1996; Sussman et al., 1999; Sussman and Winkler, 2001; Winkler et al., 1998), we investigated the stimulus factors that govern how two successive change events are processed when subjects have no task related to the auditory stimuli (e.g. they read a book). These studies showed that when two different stimulus deviations (always presented together in the same order within a 150 ms interval: ‘double deviant’) occurred within a sequence of a repetitive standard stimulus, only one MMN was elicited by them (this context for the double deviants will be called a ‘blocked condition’), whether the two deviations were carried on a single stimulus (Czigler and Winkler, 1996; Winkler et al., 1998) or across two discrete stimuli (Sussman et al., 1999; Sussman and Winkler, 2001). Pre-attentive auditory processes treated the two successive changes as a single change from the repetitive sound. However, when double deviants were presented within a stimulus sequence that also contained single deviants (one or the other deviant sound that comprised the double deviant sequence; this will be called ‘mixed condition’), the double deviants then elicited two discrete MMNs, which were separated in latency by the temporal distance between the successive deviations (Sussman and Winkler, 2001 and unpublished data). We reasoned (Sussman and Winkler, 2001) that in the blocked condition, the two successive deviations were integrated into a single deviant event because the second of the two deviants provided no new information about ‘deviancy’ in that particular auditory context (see also, Winkler and Czigler, 1998). In the mixed condition, however, the second of the double deviants distinguishes the double deviant event from the single deviants appearing in the same block. Therefore, the second successive deviant carries new information, which signals that it should be processed separately from the first deviant. The fact that in both the blocked and mixed conditions of these studies the timing between the successive stimuli was the same (150 ms onset to onset) precludes explanations based on backward masking or refractoriness of the MMN generator. The results demonstrate that the characteristics of the stimulus sequence, that is, the context within which the two consecutive deviants were presented (a blocked or mixed condition), governed how the series of auditory change events were processed pre-attentively.


Thus, stimulus-driven processes determined whether the same acoustic events elicited one or two MMN responses. In the current study, we addressed the question of whether the interaction between stimulus-driven and top-down processes can determine what constitutes an auditory event. We asked: (1) whether two successive deviations, that are integrated into a single event during stimulus-driven sound organization, can still be perceived as two separate auditory events; (2) whether perception of the double deviants as two separate events would result in two MMNs being elicited by them; and (3) whether distinguishing the two successive deviant events from each other would then induce longer-term changes affecting the pre-attentive processing of these events (determined in a subsequent ‘ignore’ condition by the number of MMNs elicited by the double deviants when the tones were not attended). The results of this study will thus help us to determine the extent of top-down control over the construction of auditory stimulus events and the locus of interaction between top-down control and stimulus driven processes in the path leading to perception. The question is whether the central processes involved in voluntarily discriminating the two successive deviant events affect how these events are represented (i.e. as one integrated event or two separate events) in the memory underlying the generation of MMN. A finding of attentional effects on this memory would place the locus of interaction between top-down and stimulus-driven processes to an earlier processing stage (thus suggesting a more far-reaching top-down control over auditory stimulus processing) than if the memory representations underlying MMN generation proved to be insensitive to voluntary discrimination of the successive deviant events. 2. Methods and materials 2.1. Participants Nine adults (3 males, ranging in age from 28 to 42 years, mean age 34 years) with reportedly normal hearing were paid to participate in the first session of the study. Eight adults (3 males, ranging in age from 21 to 39 years, mean age 27 years), who did not participate in the first session of the study, were paid to participate in a second session of the study (the post-training mixed-attend condition, see below). Subjects were treated in accordance with the ethical guidelines for human subjects treatment at Albert Einstein College of Medicine, where the study was conducted. The data from one subject who participated in the first session was excluded from analysis for the mixed-ignore condition only, due to excessive non-cortical artifacts contaminating the EEG recorded in this condition. 2.2. Stimuli Stimuli were pure tones (50 ms in duration including 5 ms rise/fall times) presented binaurally through insert

1911

earphones with a uniform onset-to-onset interval of 150 ms. The standard stimuli were 440 Hz with an intensity level of 75 dB; the frequency deviants were 494 Hz with the same intensity level as the standards (75 dB); and the intensity deviants had the frequency of the standards (440 Hz) with an intensity level of 65 dB. 2.3. Procedures Data collection was carried out in an acoustically dampened room. In the first session of the study, there were 4 different phases: a pre-training ignore, discrimination training, attend, and a post-training ignore. In each of the 3 experimental conditions (pre-training blocked ignore, post-training blocked attend, post-training blocked ignore), a modified auditory oddball paradigm was used, in which deviants were always two successive changes (a frequencydeviant followed by an intensity-deviant [see Fig. 1]; we shall call these double deviants ‘FI-deviants’) occurred randomly in the block 7.5% of the time (thus the overall deviant-stimulus probability was 15%). A mixed-ignore condition was presented once during the experiment, either before or after phase 1, or before or after phase 3 of the experiment. Its position in the experiment was counterbalanced across subjects. In the mixed ignore condition, FIdeviants (2.5%) co-occurred with F-deviants (5%) and Ideviants (5%) in the same block, with standard sounds making up the rest of the sound sequence (85%). Four thousand stimuli were presented for each blocked condition, delivered in two stimulus blocks of 2000 each. Eight thousand stimuli were presented for the mixed ignore condition, in 4 stimulus blocks of 2000 stimuli each. In all of the ignore conditions, subjects were instructed to read a book and disregard the sounds. In the first phase of the first session, the pre-training blocked ignore condition was presented. In the training phase, subjects were presented with a variable length (,3 s) sequence of the repeating standard tone, within which, one of 3 types of changes occurred at a random place within the trial. The order of the trials was randomized. The change was either a single frequency deviant (F-deviant), a single intensity deviant (I-deviant), or an FI-deviant. The purpose of the training was to allow the subjects to learn the difference among the 3 types of changes and be able to detect the FI deviant in the following phase of the study. A diagram of the 3 different target types was used to visually illustrate the task. Subjects were instructed to listen to the sequences and then give a verbal response (EEG was not recorded in the training phase) indicating which of the 3 changes occurred in each trial (F, I, or FI). Subjects were provided with verbal feedback as to the correctness of their response after each trial and continued with the training until a stable 80% or better accuracy was achieved. The average training time for subjects was 13 min (average of 40 trials) with an average of 86% correct (SD ¼ 10) responses. In the attend phase (posttraining blocked attend condition), subjects were instructed

1912


Fig. 1. Schematic diagram of the stimulus paradigm. In the blocked sequences, each frequency deviant (denoted by a black triangle) was followed by an intensity deviant (denoted by a black circle; double deviants). The double deviants were randomly interspersed within the repetitive sequence of the standards (denoted by white squares). In the mixed sequences, single frequency deviants, single intensity deviants, and double deviants were randomly interspersed among the standards. The proportion of deviants to standards was equal in the mixed and blocked sequences.

to listen for the FI change in the stimulus block and to press the response key after they detected the second of the two successive deviants. Only FI deviants occurred in the blocked attend FI condition, however, subjects were not informed of the make-up of the stimulus condition. In the final phase of the session, subjects were presented with a blocked FI condition and they were again instructed to read a book and disregard the sounds (post-training blocked ignore condition). In a separate session (and with a different group of subjects, see above), another mixed condition was conducted; the stimulus sequence was described above for the mixed ignore condition. Before presenting the mixed stimulus sequences, training was administered to subjects, in the same manner as described above. The training phase was followed by the post-training mixed attend condition, in which subjects were instructed to attend to the sound sequence and press the response key only when they heard the double (FI) deviants. 2.4. EEG recording and data analysis EEG was recorded from the following electrode sites: Fz, Cz, Pz, F3, F4, C3, C4, P3, P4, and LM, and RM (left and right mastoids, respectively) using an electrode cap. Horizontal eye movements were measured by recording the electrooculogram (EOG) between F7 and F8. Vertical EOG was monitored with FP1 and an electrode placed below the left eye. The common reference electrode was attached to the tip of the nose. EEG was digitized (Synamps amplifiers) at a 250 Hz rate (0.05–100 Hz bandpass) and then filtered off-

line (1 and 20 Hz). Epochs were 600 ms in duration, starting 100 ms before and ending 500 ms after the onset of the standard and deviant tones. Thus, the responses displayed in the figures include the ERPs elicited by two successive tones (either two consecutive standards, two successive deviants, or a single deviant followed by a standard (in the mixed conditions only). Epochs containing electric activity exceeding ^100 mV at any recording site were rejected from subsequent processing. On average, 10% of the trials were excluded using this criterion, which resulted in over 250 double deviant events being averaged in the blocked and over 150 double deviant events in the mixed conditions for each subject. One-way ANOVA was performed to determine whether there was a significant difference in the N1 amplitude of the standard as a function of the condition in which it was elicited (mixed vs. blocked). ERPs were averaged separately for each stimulus type and condition. Difference waves were calculated by subtracting the ERP elicited by the standard of a given condition from those elicited by the deviant stimuli in the same condition. Amplitude measurements were referred to the mean voltage of the 100 ms pre-stimulus interval. The mean MMN amplitudes were measured for each subject using a 40 ms window centered on the grand-average MMN peak. The ranges used to measure the MMN amplitudes were determined at Fz from the deviant-minus-standard difference curves of the mixed ignore condition, in which the FI deviants elicited two discrete MMNs. Range 1 was measured in the 128–168 ms interval (peak was 148 ms) and Range 2 was measured in the 328–368 ms


interval (peak was 348 ms, which would be a peak of about 198 ms from the onset of the second [intensity] deviant) in all conditions. For both of the mixed conditions, the ranges used to measure the MMNs elicited by the single frequency and single intensity deviants were also determined from the grand mean peaks obtained for single deviants in the mixed ignore condition. These ranges were 128–168 ms (peak 148 ms) for the frequency MMN and 148–188 (peak 168 ms) for the intensity MMN. Because the predictions

1913

for the MMNs were a priori, one-sample one-tailed Student’s t tests were used to verify the presence of the MMN response, separately for each condition/range, conducted on the mean voltage, in the MMN range, of the individual subjects. The N2b was measured in the 392–432 interval (the peak was about 412 ms), which was determined from the deviantminus-standard difference curves at Cz in the mixed attended condition. Two-way repeated-measures analysis

Fig. 2. Grand-averaged ERPs elicited at Fz by the FI-deviants (solid line) overlain with the ERPs elicited by the standard tones (dashed line) for the blocked (left column) and mixed (right column) sequences, separately for each condition. The N1s elicited by successive tones can be seen in the epoch. Arrows placed beneath the time scale point out stimulus onsets.

1914


Table 1 Mean amplitude (in mV) and mean latency (in ms) measured in range 1 (128–168 ms) and range 2 (328–368 ms) at Fz from the grand mean difference waveforms for FI-deviants a Condition

Pre-training ignore Post-training attend Post-training ignore Mixed-ignore Mixed-attend a

Amplitude

Latency

Range 1

Range 2

Range 1

Range 2

22.41** (1.7) 23.39** (1.2) 21.93** (1.1) 22.61** (1.1) 23.53** (1.8)

20.61 (0.9) 20.12 (2.0) 0.01 (0.5) 20.95* (1.1) 21.7* (1.9)

163 (^18) 156 (^13) 148 (^13) 149 (^7) 151 (^23)

NA NA NA 349 (^33) 364 (^28)

Standard deviations are in parentheses. Results of the one-sample, one-tailed Student t tests: **P , 0:01, *P , 0:05.

of variance (ANOVA) with factors of stimulus type (standard and deviant) and electrode (Cz, C3, C4) were used to verify the presence of N2b. One-way mixed-mode and grouped ANOVAs were used to compare amplitudes and latencies across conditions. Two-way mixed-mode ANOVA was used to assess effects of condition (mixed vs. block) on MMN elicitation. The Greenhouse–Geisser correction was applied when appropriate. The alpha level criterion was set at P , 0:05. A behavioral response was considered correct if it fell within a 200–1200 ms window from the onset of the intensity deviant (the second of the two successive deviants). Subjects who performed below 70% correct were not included in further analysis. Applying this criterion, two subjects from the first session and one subject from the second session were excluded from further analysis. Thus, 7 subjects’ data were included in the first session and 7 subjects’ in the second session (except for the mixed ignore condition, for which 6 subjects data was retained – see Section 2.1). Hit rates and reaction times were compared between the two attended conditions with a one-way between-groups ANOVA. 3. Results 3.1. Behavioral data After the training, subjects were able to perceive the FIdeviants as separate successive events, and they responded correctly, on average, on 85% (SD ¼ 10) of the trials (RT 413 ms [^72]) in the post-training blocked condition and on 85% (SD ¼ 6) of the trials (RT 473 ms [^90]) in the posttraining mixed condition. There was no significant difference in behavioral performance for responding to the second of the double (FI) deviants between the post-training blocked attend and the post-training mixed attend conditions, measured by RT: F1;6 ¼ 1:9, P . 0:19 or hit rate: F1;6 , 1, P . 0:97. 3.2. ERP data Fig. 2 displays the ERPs at Fz elicited by the standards (dashed line) and the deviants (solid line), separately for

each condition. In all conditions, multiple N1 waveforms can be clearly seen within the epoch, occurring roughly 150 ms apart (the spacing of tone onsets). In the pre-training blocked ignore condition, one MMN was elicited by the FIdeviants (t6 ¼ 5:5, P , 0:01 and t6 ¼ 1:9, P . 0:1, in ranges 1 and 2, respectively; see Table 1), which can be seen in the difference waveforms (Fig. 3) at Fz (solid line) overlain with the traces from LM (the left mastoid lead; dashed line). In the post-training blocked attend condition, when subjects distinguished the FI-deviants as separate change events, one MMN was nevertheless elicited by them (t6 ¼ 7:5, P , 0:01 and t6 , 1, P . 0:88, in ranges 1 and 2, respectively; see Table 1). The late negativity appearing after the second MMN latency-range will be discussed below. In the post-training blocked ignore condition, one MMN was again elicited by the FI-deviants (t6 ¼ 4:5, P , 0:01 and t6 , 1, P . 0:85, in ranges 1 and 2, respectively; see Table 1). No ERP components associated with attentionrelated deviance detection (e.g. N2–P3) were elicited when subjects ignored the sounds (see Fig. 4). In the mixed-ignore condition, two discrete MMNs were elicited by the FI-deviants (t5 ¼ 5:6, P , 0:01 and t5 ¼ 2:1, P , 0:05, in ranges 1 and 2, respectively; see Table 1). They can be seen in Figs. 2 and 3, top right column. In the post-training mixed attend condition, two MMNs were again elicited by the FI-deviants (t6 ¼ 5:3, P , 0:01 and t6 ¼ 2:2, P . 0:05, in ranges 1 and 2, respectively; see Table 1). A late negativity can be seen for both attend conditions (post-training blocked and post-training mixed) peaking about 412 ms from stimulus onset (F1;6 ¼ 12:9, P , 0:01 and F1;6 ¼ 6:9, P , 0:04, respectively). In the post-training mixed attend condition, this late negativity followed the second MMN elicited by the FI-deviants (and has a concurrent negativity at the mastoid site, see Fig. 5), whereas in the post-training blocked attend condition (also with a concurrent negativity at the mastoid site, see Fig. 5) there was no MMN immediately preceding it. The topography of these negative waveforms is consistent with that of N2b, being largest at the central–parietal scalp sites (see Fig. 5), whereas the MMN is largest at the fronto-central sites and


1915

Fig. 3. Difference waveforms. The deviant-minus-standard difference waveforms are displayed at the frontal site (Fz) overlain with the left mastoid (LM) for the blocked (left column) and mixed (right column) sequences, separately for each condition. Arrows point to the MMNs in each condition. One MMN was elicited by the double deviants in the blocked conditions and two MMNs (which can be seen successively in the epoch) were elicited by the double deviants in the mixed conditions. Arrows placed beneath the time scale point out stimulus onsets.

is absent at the parietal sites. Therefore, these components can be identified as target-related N2b responses, which were elicited by the target double deviants. There was no effect of condition (mixed vs. blocked) on the N1 amplitude of the standards (F4;29 ¼ 1:6, P . 0:2). MMNs were also elicited by the single frequency and single intensity deviants in the mixed ignore (t6 ¼ 4:02, P ,

0:01 and t6 ¼ 2:34, P , 0:04, respectively) and post-training mixed attend conditions (t6 ¼ 5:95, P , 0:01 and t6 ¼ 2:57, P , 0:04, respectively; see Fig. 6). The MMN amplitude appears to be larger between the attend and ignore conditions, however, this apparent difference is due to overlap of N2b with MMN when the sounds were attended.

1916


Fig. 4. Scalp topography (all 11 electrode sites) of the ERP difference waveforms (deviant-minus-standard responses) for the ignore conditions. The MMN can be clearly seen for both the pre-training ignore (solid line) and the post-training ignore (dashed line) conditions at the fronto-central electrode sites, with a clear polarity inversion at the mastoid (LM and RM) leads. Also note that attention-related ERP responses (N2b, P3) are absent from these waveforms.

Two-way mixed-mode ANOVA with factors of stimulus type (standard vs. deviant) and condition (blocked vs. mixed) revealed a main effect of stimulus type (F1;5 ¼ 56:23, P , 0:01), no main effect of condition (F2;4 ¼ 1:6, P . 0:41) and no interaction (F2;4 ¼ 1:6, P . 0:15), supporting the conclusion of no top-down effect on MMN elicitation. There was no significant difference in amplitude (F ¼ 1:8, P . 0:14) of the MMNs elicited in range 1 (compared across all 5 conditions). Neither was the amplitude (F , 1, P . 0:45) or latency (F , 1, P . 0:35) of the second MMNs (those elicited by the intensity deviant of the FI-deviants in the mixed ignore and post-training mixed attend conditions) elicited in range 2, significantly different from each other. The mean amplitude of N2bs was similar between the two attend conditions (t , 1, P . 0:80). The MMN elicited in the pre-training ignore condition peaked significantly later than the post-training attend (t ¼ 3:2, df ¼ 5, P , 0:03) and post-training ignore (t ¼ 4.1, df ¼ 5, P , 0:01) conditions. The shorter latency for the post-training conditions may reflect easier discrimination that resulted from training and that had longer-term

effects in the memory representations formed in the subsequent ignore condition.

4. Discussion It has previously been suggested that for pure tones, if you could perceive a deviant, MMN would be elicited by it. Therefore, we expected that the double deviants in the blocked condition would elicit two MMNs when subjects actively distinguished them as successive events (compared to when they ignored them). However, this was not the case; perceiving the double deviants separately did not change the MMN response. The FI-deviants in the post-training blocked attend condition still elicited only one MMN. However, the independent variable that resulted in a change of the MMN response to the double FI deviants was the context within which the FI deviants occurred (mixed vs. blocked conditions). In the current study, we observed this effect of context whether the sounds were ignored or attended: one MMN was elicited in the pre-training and post-training blocked ignore conditions, as we expected


1917

Fig. 5. Scalp topography (all 11 electrode sites) of the ERP difference waveforms (deviant-minus-standard responses) for the attend conditions. One MMN and one N2b can be seen in the post-training blocked attend (solid line), whereas two MMNs and two N2bs can be seen in the post-training mixed attend condition waveforms (dashed line). MMN and N2b can be dissociated by comparing the frontal waveforms with those obtained at the central and parietal scalp locations, because the N2b amplitude increases and the MMN amplitude diminishes when moving from the front to the back of the scalp.

(e.g. Sussman et al., 1999), whereas two MMNs were elicited in the mixed ignore and post-training mixed attend conditions. Observing a stimulus-driven change in the MMN response based on the auditory context, such as that occurring between the blocked vs. mixed ignore conditions of the current study, is consistent with our previous findings (Sussman and Winkler, 2001 and unpublished data). These results suggest that the processes separating the two successive deviant events in perception affected the analysis of the sounds after the processing stage at which MMN operated. The shorter latency of the MMN elicited by the FI-deviants in the post-training, compared to the pre-training conditions (both ignore and attend), suggests that there may have been longer-term effects of training on the processing of the FI deviants. Actively discriminating the successive deviant sounds appears to have made the stimulus-driven discrimination process easier. This is consistent with previous findings showing that the quality of the representation of sensory input can improve as a result of discrimination training (Na¨ a¨ ta¨ nen et al., 1993; Tremblay et al., 1997). In contrast to the effect on the MMN peak latency, training did not interfere with the role of the context (i.e. the blocked or

mixed sequence) in determining the number of MMNs elicited by the double deviants. That is, MMN elicitation was governed by the context within which the sounds occurred irrespective of the various experimental manipulations (the training, or active discrimination, of the sounds as separate events). Thus the elicitation of one vs. two MMNs in the ignore and attend conditions resulted from stimulusdriven changes in the sound sequence and not from topdown processes. Thus, the current results strongly support the view that stimulus-driven processes govern how the MMN is elicited; top-down control did not affect the outcome of the eventrepresentation and change-detection processes reflected by MMN. This conclusion is consistent with those reached in our previous studies, which showed that the MMN process itself is not affected by top-down control (Ritter et al., 1999; Rinne et al., 2001; Sussman et al., 2002). These studies demonstrated that knowing about deviance, in and of itself, does not change the automatic or stimulus-driven change detection process. In addition, the current results indicate that top-down processes affect later processing stages, enabling one to extract elements of the integrated informa-

1918


tion, which suggests that, despite temporal integration, the details are not lost. Although there is probably not a general (unspecific)

effect of attention on the MMN-generating process (Alain and Woods, 1997; Alho et al.,1992; Na¨ a¨ ta¨ nen, 1992; Dittmann-Balcar et al., 1999; Na¨ a¨ ta¨ nen et al., 1993; Paavilainen

Fig. 6. Grand averaged ERPs (left column) elicited at Fz by the single frequency (top panel) and single intensity (bottom panel) deviants (solid line) overlain with the ERPs elicited by the standard tones (dashed line) for the mixed ignore and post-training mixed attend conditions. The deviant-minus-standard difference waveforms (right column) are displayed at the frontal site (Fz) overlain with the left mastoid (LM), separately for each single deviant type and condition. The overlap of N2b with MMN can clearly be seen at the Fz electrode site when the sounds were attended. Arrows placed beneath the time scale point out stimulus onsets.


et al., 1993; Sussman et al., 2002; Szymanski et al., 1999; Trejo et al., 1995; Woldorff et al., 1991, 1998) we have shown instances in which top-down control has affected the elicitation of MMN (Sussman et al., 1998a, 2002). The common aspect of the studies finding top-down effects on MMN elicitation was that the sound sequences were voluntarily reorganized, in one case by attending to a subset of the sounds (Sussman et al., 1998a), and in the other by listening for a repeating pattern that was not noticed when the sounds were ignored (Sussman et al., 2002). In these studies, top-down effects acted on processes preceding MMN generation, showing top-down modulation of the neural representations (of the acoustic regularities) that underlie the MMN process. In contrast, in the current study, top-down processes were used to extract additional information about the change events, having no effect on the organization of the sounds or on the context within which the deviants occurred. That is, examining the double deviants in more detail did not alter their role in the blockedsequence conditions and thus did not change MMN elicitation. Our previous results obtained in blocked vs. the mixed conditions (Sussman and Winkler, 2001 and unpublished data) showed that the information carried by the double deviants (their role in the sequence) determined how MMN was elicited. When the second deviant of the double deviants carried distinct event information (distinguishing the different types of deviant events), it was registered separately, eliciting its own MMN (for a related effect, see Winkler and Czigler, 1998). We have previously suggested that the auditory system maintains a neural model of the auditory environment by encoding the regularities extracted from the acoustic input (Winkler et al., 1996; Winkler and Czigler, 1998). According to the auditory model hypothesis, the processes underlying MMN generation are involved in updating the regularity representations when the extrapolations based on them mismatch the actual input; that is, when new information arrives that should then be incorporated into the model. The effects obtained in the present study as well as those discussed in the previous paragraph are consistent with the notion of the regularity-representation maintenance function of MMN. The top-down effects separating the two successive deviant events in perception (in the current experiment) did not change the MMN response because distinguishing the two deviants did not alter the information these events carried with respect to the existing neural regularity representations. Thus, the model-updating processes reflected in the MMN response did not change. However, when top-down control was used to change the regularities maintained in the auditory model (such as in Sussman et al., 1998a, 2002), it altered the information represented in the brain, which was reflected by changes in the set of events that would elicit the MMN response. Similarly, changing the stimulus characteristics of a sound sequence (such as turning from blocked to mixed stimulus sequences, as was done in the

1919

present study and in Sussman and Winkler, 2001) also alters the regularities maintained in the auditory model, thus affecting MMN elicitation in a stimulus-driven manner (i.e. irrespective of the subject’s task; for other compatible stimulus-driven effects, Sussman et al., 1998b, 1999; Winkler et al., 1998, 2001). The results of the current study demonstrated that a change in the perception of the deviant without a corresponding change in the underlying regularity representations does not affect the MMN process. The results show that attention can enable one to extract temporally integrated information, which suggests that the integrated information is not lost. This has implications for how we process the rapidly occurring sounds in speech, which may mask each other, without losing the information in the speech signal that distinguish one phenomenon from another. Acknowledgements This research was supported by the National Institutes of Health grants (R01 DC04263 and R01 NS30029), the Hungarian National Research Fund (OTKA T034112), and the Academy of Finland. We thank Lela Giannaris for research assistance. References Alain C, Woods D. Attention modulates auditory pattern memory as indexed by event-related potentials. Psychophysiology 1997;34:534– 546. Alho K, Paavilainen P, Reinikainen K, Sams M, Na¨ a¨ ta¨ nen R. Separability of different negative components of the event-related potentials associated with auditory stimulus processing. Psychophysiology 1986;23:613–623. Alho K, Woods DL, Algazi A, Na¨ a¨ ta¨ nen R. Intermodal selective attention. Electroenceph clin Neurophysiol 1992;82:356–368. Cowan N. On short and long auditory stores. Psychol Bull 1984;96:341– 370. Czigler I, Winkler I. Pre-attentive auditory change detection relies on unitary sensory memory representations. NeuroReport 1996;7:2413– 2417. Dittmann-Balcar A, Thienel R, Schall U. Attention-dependent allocation of auditory processing resources as measured by mismatch negativity. NeuroReport 1999;10:3749–3753. Giard MH, Lavikainen J, Reinikainen K, Perrin F, Bertrand O, Pernier J, Na¨ a¨ ta¨ nen R. Separate representation of stimulus frequency, intensity, and duration in auditory sensory memory: an event-related potential and dipole model study. J Cogn Neurosci 1995;7:133–143. Giard MH, Perrin F, Pernier J, Bouchet P. Brain generators implicated in processing of auditory stimulus deviance: a topographic event-related potential study. Psychophysiology 1990;27:627–640. Halgren E, Baudena P, Clarke JM, Heit G, Liegeois C, Chauvel P, Musolino A. Electroencephalography and Clinical Neurophysiology 1995;94:191– 220. Javitt DC, Steinschneider M, Schroeder CE, Arezzo JC. Role of cortical Nmethyl-d-aspartate receptors in auditory sensory memory and mismatch negativity generation: implications for schizophrenia. Proc Natl Acad Sci 1996;93:11962–11967. Kropotov JD, Na¨ a¨ ta¨ nen R, Sevostianov AV, Alho K, Reinkainen K, Kropotova OV. Mismatch negativity to auditory stimulus change recorded

1920


directly from the human temporal cortex. Psychophysiology 1995;32:418–422. Massaro DW. Experimental psychology and information processing, Chicago, IL: Rand McNally, 1975. Na¨ a¨ ta¨ nen R. Attention and brain function, New York, NY: Erlbaum, 1992. Na¨ a¨ ta¨ nen R, Paavilainen P, Tiitinen H, Jiang D, Alho K. Attention and mismatch negativity. Psychophysiology 1993;30:436–450. Na¨ a¨ ta¨ nen R, Schro¨ ger E, Karakas S, Tervaniemi M, Paavilainen P. Development of a memory trace for a complex sound in the human brain. NeuroReport 1993;4:503–506. Na¨ a¨ ta¨ nen R, Simpson M, Loveless NE. Stimulus deviance and evoked potentials. Biol Psychol 1982;14:53–98. Na¨ a¨ ta¨ nen R, Tervaniemi M, Sussman E, Paavilainen P, Winkler I. Preattentive cognitive processing (‘primitive intelligence’) in the auditory cortex as revealed by the mismatch negativity (MMN). Trends Neurosci 2001;24:283–288. Novak G, Ritter W, Vaughan Jr HG, Wiznitzer ML. Differentiation of negative event-related potentials in an auditory discrimination task. Electroenceph clin Neurophysiol 1990;75:255–275. Novak G, Ritter W, Vaughan Jr HG. Mismatch detection and the latency of temporal judgments. Psychophysiology 1992a;29:398–411. Novak G, Ritter W, Vaughan Jr HG. The chronometry of attention-modulated processing and automatic mismatch detection. Psychophysiology 1992b;29:412–430. Opitz B, Mecklinger A, Von Kramon DY, Kruggel F. Combining electrophysiological and hemodynamic measures of the auditory oddball. Psychophysiology 1999;36:142–147. Paavilainen P, Alho K, Reinikainen K, Sams M, Na¨ a¨ ta¨ nen R. Right hemisphere dominance of different mismatch negativities. Electroenceph clin Neurophysiol 1991;78:466–479. Paavilainen P, Tiitinen H, Alho K, Na¨ a¨ ta¨ nen R. Mismatch negativity to slight pitch changes outside strong attentional focus. Biol Psychol 1993;37:23–41. Picton TW, Alain C, Otten L, Ritter W, Achim A. Mismatch negativity: different water in the same river. Audiol Neurootol 2000;5:111–139. Renault B, Lesevre N. A trial-by-trial study of the visual omission response in reaction time situations. In: Lehmann D, Callaway E, editors. Human evoked potentials, New York, NY: Plenum Press, 1979. pp. 317–329. Rinne T, Antila S, Winkler I. MMN is unaffected by top-down predictive information. NeuroReport 2001;12:2209–2213. Ritter W, Sussman E, Deacon D, Cowan N, Vaughan Jr HG. Two cognitive systems simultaneously prepared for opposite events. Psychophysiology 1999;36:835–838. Sams M, Hari R. Magnetoencephalography in the study of human auditory information processing. Ann NY Acad Sci 1991;620:102–117. Scherg M, Vajsar J, Picton TW. A source analysis of the late human auditory evoked potentials. J Cogn Neurosci 1989;1:336–355. Sussman E, Ritter W, Vaughan Jr HG. Attention affects the organization of

auditory input associated with the mismatch negativity system. Brain Res 1998a;789:130–138. Sussman E, Ritter W, Vaughan Jr HG. Stimulus predictability and the mismatch negativity system. NeuroReport 1998b;9:4167–4170. Sussman E, Ritter W, Vaughan Jr HG. An investigation of the auditory streaming effect using event-related brain potentials. Psychophysiology 1999;36:22–34. Sussman E, Winkler I. Dynamic process of sensory updating in the auditory system. Cogn Brain Res 2001;12:431–439. Sussman E, Winkler I, Huotilainen M, Ritter W, Na¨ a¨ ta¨ nen R. Top-down effects on the initially stimulus-driven auditory organization. Cogn Brain Res 2002;13:393–405. Sussman E, Winkler I, Ritter W, Alho K, Na¨ a¨ ta¨ nen R. Temporal integration of auditory stimulus deviance as reflected by the mismatch negativity. Neurosci Lett 1999;264:161–164. Sussman E, Winkler I, Schro¨ ger E. Top-down control over involuntary attention-switching in the auditory modality. Psychon Bull Rev 2002 (in press). Sussman E, Winkler I, Wang WJ. MMN and attention: competition for deviance detection, submitted for publication. Szymanski M, Yund EW, Woods DL. Phonemes, intensity and attention: differential effects on the mismatch negativity (MMN). J Acoust Soc Am 1999;106:3492–3505. Treijo LJ, Ryan-Jones DL, Kramer AF. Attentional modulation of the mismatch negativity elicited by frequency differences between binaurally presented tone bursts. Psychophysiology 1995;32:319–328. Tremblay K, Kraus N, Carrell TD, McGee T. Central auditory system plasticity: generalization to novel stimuli following listening training. J Acoust Soc Am 1997;102:3762–3773. Winkler I, Czigler I. Mismatch negativity: deviance detection or the maintenance of the ‘standard’. NeuroReport 1998;9(17):3809–3813. Winkler I, Czigler I, Jaramillo M, Paavilainen P, Na¨ a¨ ta¨ nen R. Temporal constraints of auditory event synthesis: evidence from ERPs. NeuroReport 1998;9:495–499. Winkler I, Karmos G, Na¨ a¨ ta¨ nen R. Adaptive modeling of the unattended acoustic environment reflected in the mismatch negativity event-related potential. Brain Res 1996;742:239–252. Winkler I, Schro¨ ger E, Cowan N. The role of large-scale memory organization in the mismatch negativity event-related brain potential. J Cogn Neurosci 2001;13:59–71. Woldorff MG, Hackley SA, Hillyard SA. The effects of channel-selective attention on the mismatch negativity wave elicited by deviant tones. Psychophysiology 1991;28:30–42. Woldorff M, Hillyard S, Gallen CC, Hampson SR, Bloom FE. Magnetoencephalographic recordings demonstrate attentional modulation of mismatch-related neural activity in human auditory cortex. Psychophysiology 1998;35:283–292. Zwislocki JJ. Theory of temporal auditory summation. J Acoust Soc Am 1960;32:1046–1060.