Alma Mater Studiorum University of Bologna, August 22-26 2006
Temporal attention in the processing of short melodies: Evidence from event-related potentials Kathrin Lange
Martin Heil
Department of Experimental Psychology, Heinrich-Heine-University Duesseldorf
Department of Experimental Psychology, Heinrich-Heine-University Duesseldorf
Duesseldorf, Germany
Duesseldorf, Germany
[email protected] melodies (the target tone) was identical or not. When the temporal structure of the standard and the comparison melody was identical, the allocation of temporal attention should improve the processing of the target tone. This should be indicated by faster and / or more accurate responding and by an enhancement of early negativities in the ERPs.
ABSTRACT Focusing attention on a point in time can improve stimulus processing, as indicated by faster responses to attended as compared to unattended stimuli. Eventrelated potential (ERP) studies provide evidence that temporal attention operates on early processing stages, presumably related to perceptual analyses.
As expected, participants responded faster to isochronous melodies if the target tone was presented at an attended rather than at an unattended point in time. Moreover, the ERPs to attended target tones of isochronous melodies were associated with relatively more negative amplitudes between 140 and 220 ms. Responding to the nonisochronous melodies was not influenced by temporal attention. Here, ERPs to unattended target tones were more negative, possibly due to a kind of mismatch response.
It has been speculated that temporal attention is also involved in the perception of melodies within a complex musical context. Goal of the present study was therefore, to analyze the influence of temporal attention on the processing of single tones within a melody with behavioral and ERP measures. In each trial, participants heard two melodies (a standard and a comparison). The temporal structure of the melodies of a trial was either identical (e.g. both isochronous) or differed in the timing of the third and fourth tone (e.g. isochronous standard and non-isochronous comparison). Participants had to indicate whether the pitch of the third tone of both
Keywords temporal attention, rhythm, timing, auditory, ERP
INTRODUCTION In: M. Baroni, A. R. Addessi, R. Caterina, M. Costa (2006) Proceedings of the 9th International Conference on Music Perception & Cognition (ICMPC9), Bologna/Italy, August 22-26 2006.©2006 The Society for Music Perception & Cognition (SMPC) and European Society for the Cognitive Sciences of Music (ESCOM). Copyright of the content of an individual paper is held by the primary (first-named) author of that paper. All rights reserved. No paper from this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval systems, without permission in writing from the paper's primary author. No other part of this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval system, without permission in writing from SMPC and ESCOM.
ISBN 88-7395-155-4 © 2006 ICMPC
A number of recent studies provide evidence that attention can be flexibly allocated to points in time, similar as has been known for spatial attention (for reviews see Correa, Lupiánez, Madrid, & Tudela, 2006; Nobre, 2001). Coull and Nobre (1998) were the first to describe faster responding to temporally attended than unattended stimuli with a visual temporal cuing task similar to the spatial cuing task introduced by Posner (1980). Several studies have replicated this temporal
1433
ICMPC9 Proceedings attention effect using visual stimuli (Correa, Lupiánez et al., 2006; Correa, Lupiánez, Milliken, & Tudela, 2004; Doherty, Rao, Mesulam, & Nobre, 2005; Griffin, Miniussi, & Nobre, 2002; Miniussi, Wilding, Coull, & Nobre, 1999). Moreover, Lange and Röder (2006) observed faster responses to temporally attended than unattended auditory and tactile stimuli when manipulating attention by varying stimulus probability.
used by Dowling and colleagues. Participants heard two melodies which had either an identical temporal structure (both isochronous or both non-isochronous) or differed in the timing of the third and fourth tone (one isochronous and one non-isochronous, see also Figure 1). They had to evaluate the pitch of the third tone (the target tone), which could be identical for both melodies or differ by one whole step. To encourage participant to focus their attention on the points in time indicated by the first melody (the standard melody), distracter tones were presented, temporally interleaved between the tones of the second melody. When the temporal structure of the first and second melody was identical (target tone presented at an attended point in time, T+), the processing of the third tone should be improved as compared to when the temporal structure of the first and the second melody was different (target tone presented at an unattended point in time, T-). This improvement should be evident in faster and / or more accurate responding and in an enhancement of early negativities in the ERPs to attended as compared to unattended target tones.
A central question in attention research concerns the level(s), at which stimulus processing can be influenced by attention. Because of their excellent time resolution, ERP measures provide a useful tool to answer this question. Several studies using ERPs as dependent measure have been conducted to investigate the locus of temporal attention selection. Evidence for modulations of early, perceptual processing levels was obtained using visual (Griffin et al., 2002, experiment 1; Correa et al., 2006; Doherty et al. 2005; but see Miniussi et al., 1999; Griffin et al., 2002, experiment 2), auditory (e.g. Lange & Röder, 2006; Lange, Rösler, & Röder, 2003) and tactile (Lange & Röder, 2006) stimuli. It can be concluded, that information about the point in time, when a stimulus is expected to occur, can be used to optimize the allocation of processing resources, leading to a faster processing of attended as compared to unattended stimuli. ERP measures indicate, that these effects might be mediated by early, possibly perceptual, processing stages. In everyday life, the use of temporal information to enhance stimulus salience is evident e.g. in the perception of music. When listening to a piece of music, one often has to extract the melody out of a complex musical context - sometimes even without additional cues provided by loudness, timbre or frequency range. Experiments by Dowling (1973; Dowling, Lung, & Herrbold, 1987), using the "interleaved melodies" paradigm, showed that people are able to recognize a melody interleaved with tones that were of the same timbre and fell in the same pitch range as the melody tones. Dowling concluded that prior knowledge of a melody is used to form "expectancy windows" defined by pitch and time. These expectancy windows are sufficient to follow a musical event (see also Neisser, 1979). For example, subjects heard a cue melody and a comparison pattern, in which the melody was interspersed with distracter tones. The melody could be either the same as in the cue or one tone changed its pitch (higher or lower than expected) and / or its temporal position (earlier or later than expected). Participants were significantly better at judging the pitch of the tones when the tones were presented at the expected time (Dowling et al., 1987, Experiments 5 and 6). Thus, temporal information provided by the temporal structure of a melody can be used to enhance processing of single tones within a complex stimulus sequence.
Figure 1. Outline of the experimental conditions, exemplarily for the d-f-a-d standard melody. Melody and distracter tones are indicated by black and gray dots, respectively. Participants had to indicate whether the pitch of the target tone of the standard and the comparison melody was identical or not (here: same or higher in pitch). Note that reaction times, error rates and ERPs were compared between physically identical isochronous (middle) and nonisochronous (right) comparison melodies as a function of temporal attention.
METHODS Participants Twenty-eight participants (19 - 51 years old, mean age: 26.4, 12 male) took part in the experiment. All had normal hearing (self report). Twenty-two had a formal musical education (lessons in an instrument or in singing) with a mean of 8.9 years (range: 1 to 30 years). Participants were naive with regard to the purpose of the experiment. They received course credits or a monetary compensation for
In the present study, we used event-related potentials to investigate which levels of stimulus processing are modulated in a temporal attention task, similar to that ISBN 88-7395-155-4 © 2006 ICMPC
1434
ICMPC9 Proceedings or at 10 kΩ or below (EOG electrodes). The EEG was recorded continuously with a sampling rate of 250 Hz. The bandpass of the amplifiers (NuAmps, Neuroscan; Inc., Herndon, VA) was set from DC to 100 Hz.
taking part. Written informed consent was obtained by all participants. The data of six participants (four male) were excluded from further analyses (five: less than 67% correct responses, one: eye movement artifacts in the EEG). The final sample comprised data of twenty-two participants (19 and 51 years old; mean age 27.3 years; 7 male). Eighteen participants had a formal musical education (2 30 years; mean: 9.1 years).
Data Analyses Data from the two standard melodies (i.e. d-f-a-d or d-f-hd) were collapsed for the analyses of behavioral and ERP data. Target tones of isochronous and non-isochronous comparison melodies were classified as attended or unattended, depending on the preceding standard melody.
Stimuli and Apparatus
Behavioral Data
Two different standard melodies (d-f-a-d or d-f-h-d; relative to a frequency of a = 440 Hz), consisting of computer generated tones (70 ms duration, triangle wave; Adobe Audition 1.5), were used. Melodies were presented via earphones (KOSS 35) at a comfortable listening level.
The influence of temporal attention on reaction times (delay between the question mark and the response) and on the relative proportion of errors was analyzed using repeated measures ANOVAs with factors Temporal Attention (attended/unattended) and Temporal Structure (isochronous/non-isochronous) of the comparison melody.
Stimulus onset asynchrony (SOA) between melody tones was 400 ms for the isochronous melodies. For the nonisochronous melodies, SOAs between the first and second and between the third and fourth tone were 400 ms and the SOA between the second and third tone was 300 ms. Distracter tones were presented between the tones of the comparison melody. Distracter tones were chosen randomly in each trial from a set of tones for each position (between 7 and 10 semitones higher or lower than the preceding tone).
Event-Related Potentials Event-related potentials to target tones were averaged separately for the isochronous and non-isochronous melodies and the attended and unattended conditions. Segments were 800 ms long and related to a 200 ms prestimulus baseline. Only trials with a correct response were considered for the ERP analysis. To eliminate high frequency noise, the raw data of all participants were digitally low-pass filtered using a Butterworth Zero Phase Filter (High Cutoff: 40 Hz, 24 dB/oct), implemented in the Vision Analyzer Software (Brain Products, München, Germany). Trials were removed from analysis whenever (1) the voltage difference between two consecutive sampling points exceeded 50 µV, (2) the absolute difference of two values in the segment exceeded 120 µV, or (3) activity was less than 0.01 µV for a time epoch longer than 100 ms (amplifier saturation).
Procedure In the EEG experiment, participants first received an exercise block to familiarize them with the experimental task. The experiment proper consisted of 12 blocks with 48 trials each. In each trial, one of the four possible melodies (i.e. d-f-a-d and d-f-h-d, either isochronous or non-isochronous; Figure 1) was presented as the standard melody and was paired with one of the four melodies as the comparison melody. Each melody was used equally often as standard and comparison melody. While the standard melody was presented, the word "Standard" (standard) was presented on the computer screen. After a delay of 1500 ms, the comparison pattern was presented and the word "Vergleich" (comparison) appeared on the screen. A question mark, which indicated that the participants had to respond, was presented 100 ms after the last tone of the pattern. The participants responded by pressing one of two response keys. The assignment of the "same/different"-response to the left or right key was counterbalanced across participants.
Attention effects were assessed separately for isochronous and non-isochronous melodies, because visual inspection of the grand average waveforms had revealed substantial differences. For ERPs to targets of isochronous melodies, mean amplitudes 90 - 130 ms (N1 time range) and 140 220 ms were calculated. For ERPs to targets of nonisochronous melodies, mean amplitudes 100 - 200 ms were calculated. Mean amplitudes were submitted separately to a repeated measures ANOVA with factors Temporal Attention (attended/unattended) and Electrode (7 levels).
ERP Recording
RESULTS
The Electroencephalogram (EEG) was recorded from seven electrodes (Fz, Cz, C3, C4, Pz, P3, P4). The Electrooculogram (EOG) was recorded using two electrodes at the outer canthi of the eyes (horizontal EOG) and one electrode above and one below the left eye (vertical EOG). An additional electrode was placed at the left earlobe. All electrodes were referenced to the right earlobe and re-referenced off-line to linked earlobes. Impedances were kept at 5 kΩ or below (scalp electrodes)
ISBN 88-7395-155-4 © 2006 ICMPC
Behavioral Data Pitch judgments were equally accurate for attended and unattended targets, irrespective of the temporal structure of the melody (all p > .19; Figure 2).
1435
ICMPC9 Proceedings
Event-related Potentials Isochronous Melodies Time epoch 90-130 ms. No significant effects of Temporal attention were observed in this time epoch (all p > .22). Time epoch 140-220 ms. Attended target tones elicited more negative ERPs than non-attended target tones over the right central scalp (Temporal Attention × Electrode, F(6, 126) = 2.99, p = .04, C4: Temporal Attention, F(1, 21) = 5.06, p = .04; Cz: Temporal Attention, F(1, 21) = 3.37, p = .08; Figure 4).
Non-isochronous Melodies
Figure 2. Mean proportions of errors in the pitch judgment task. for targets at attended (T+, light gray) and unattended (T-, dark gray) points in time. Error bars represent the standard error of the mean.
Time epoch 100-200 ms. Unattended target tones elicited more negative ERPs than attended target tones (Temporal Attention, F(1, 21) = 8.54, p = .008; Figure 5). This effect was significant for the frontal and central electrodes (all p < .01) and for P3 (p < .05).
In the analysis of reaction times, the Temporal Attention by Temporal Structure interaction just failed to reach significance level (F(1, 21) = 3.82, p = .06): Participants responded faster to isochronous (F(1, 21) = 6.05, p = .02) but not to non-isochronous melodies (F < 1) if the target tone was attended rather than unattended (Figure 3).
Figure 3. Mean reaction times in the pitch judgment task for targets at attended (T+, light gray) and unattended (T-, dark gray) points in time. Error bars represent the standard error of the mean.
ISBN 88-7395-155-4 © 2006 ICMPC
1436
ICMPC9 Proceedings
Figure 4. Grand average ERPs to target tones from isochronous melodies when attended (dashed line) and unattended (solid line). The traces are aligned with respect to a 200-msec-long pre-stimulus baseline. Here and in the following figure, negativity is up. The onsets of the target and of the adjacent distracters (D2 and D3) are indicated by arrows. The time windows used for statistical analyses are shaded gray. Lange et al. (2003). However, the latency of the present effect was longer than that of temporal attention effects from earlier studies, where the earliest effects were observed in the auditory N1 around 100 ms (Lange & Röder, 2006; Lange et al., 2003).
DISCUSSION Goal of the present study was to investigate the effects of temporal attention on the processing of single tones within a melody. Therefore, we compared the processing of identical target tones in identical melodies when preceded by a melody with either the same (target attended) or a different temporal structure (target unattended). Participants responded faster but equally accurate to isochronous melodies when the target was attended rather than unattended. This was accompanied by an enhanced negativity in the ERPs to attended as compared to unattended target tones between 140 and 220 ms over the central scalp, particularly over the right hemisphere. The behavioral responses to attended and unattended non-isochronous melodies did not differ. However, a relative negativity to unattended target tones of non-isochronous melodies was observed between 100 and 200 ms over central and frontal electrodes.
ERPs to unattended target tones from non-isochronous melodies were more negative than ERPs to attended target tones between about 100 and 200 ms at central and frontal electrodes. While this effect is contrary to the attentionrelated negativity observed for targets from isochronous melodies, it may be speculated that it reflects a kind of mismatch response to target tones that appear earlier than expected. It has been reported that ERPs to premature but not late tones (with respect to a regular ISI) are characterized by a fronto-central negativity between about 100 and 200 ms (Ford & Hillyard, 1981, see also Nordby, Roth, & Pfefferbaum, 1988), similar to a mismatch negativity (for a recent review see Näätänen, Tervaniemi, Sussman, Paavilainen, & Winkler, 2001; Picton, Alain, Otten, Ritter, & Achim, 2000). Consistent with these findings, we observed a relative negativity for unattended targets from non-isochronous (earlier than expected) but not isochronous melodies (later than expected).
The temporal attention negativity to targets of isochronous melodies was observed particularly over central electrodes and lateralized to the right. This resembles the topography of the temporal attention N1 effect to standard stimuli of short intervals in the study by
ISBN 88-7395-155-4 © 2006 ICMPC
1437
ICMPC9 Proceedings
Figure 5. Grand average ERPs to target tones from non-isochronous melodies when attended (dashed line) and unattended (solid line). The traces are aligned with respect to a 200-msec-long pre-stimulus baseline. The onsets of the target and of the adjacent distracters (D2 and D3) are indicated by arrows. The time window used for statistical analyses is shaded gray. allocation of attention, based on extrapolating the SOA between the first and second tone of the comparison melody. It might be speculated that the latency differences between the present and earlier studies (e.g. Lange et al., 2003) are due to a different involvement of voluntary and stimulus-driven attentional processes.
In addition to voluntary, endogenous temporal attention, as described above (for reviews see Correa, Lupiánez et al., 2006; Nobre, 2001), it has been suggested that there is also a kind of stimulus-driven or exogenous allocation of temporal attention (Jones, Moynihan, MacKenzie, & Puente, 2002). It has been assumed that temporal attention is an oscillatory process that entrains to regularities in stimulus sequences (dynamic attention model, Large & Jones, 1999). According to this model, the time interval between two successive stimuli is used for targeting temporal expectancies to a future point in time. If the focus of temporal attention coincides with the onset of the next stimulus, the corresponding temporal expectancy is strengthened. If not, the period and/or the phase of the oscillator process is adapted.
Using a different, more natural task than earlier studies, the present study provides additional evidence that temporal attention can modulate the processing of auditory stimuli at relatively early stages. It would be interesting for future studies to disentangle the relative contributions of voluntary and automatic processes of orienting attention in time.
REFERENCES
Both voluntary and stimulus-driven attention might have been involved in the present task. Voluntary attention refers to the use of temporal information provided by the standard melody as a template for separating the comparison melody from the distracters. Additionally, successive melody tones might trigger an automatic allocation of attention on future points in time. Thus, the processing of target tones in the unattended isochronous melodies might have been affected by such an automatic
ISBN 88-7395-155-4 © 2006 ICMPC
Correa, A., Lupiánez, J., Madrid, E., & Tudela, P. (2006). Temporal attention enhances early visual processing: A review and new evidence from event-related potentials. Brain Research, 1076, 116-128. Correa, A., Lupiánez, J., Milliken, B., & Tudela, P. (2004). Endogenous temporal orienting of attention in detection and discrimination tasks. Perception & Psychophysics, 66, 264-278.
1438
ICMPC9 Proceedings Picton, T., Alain, C., Otten, L., Ritter, W., & Achim, A. (2000). Mismatch negativity: Different water in the same river. Audiology and Neuro-Otology, 5, 111-139.
Coull, J. T., & Nobre, A. C. (1998). Where and when to pay attention: The neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. The Journal of Neuroscience, 18, 7426-7435.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25.
Doherty, J. R., Rao, A., Mesulam, M. M., & Nobre, A. C. (2005). Synergistic effect of combined temporal and spatial expectations on visual attention. The Journal of Neuroscience, 25, 8259-8266. Dowling, W. J. (1973). The perception of interleaved melodies. Cognitive Psychology, 5, 322-337. Dowling, W. J., Lung, K. M.-T., & Herrbold, S. (1987). Aiming attention in pitch and time in the perception of interleaved melodies. Perception & Psychophysics, 41, 642-456. Ford, J. M., & Hillyard, S. A. (1981). Event-related potentials (ERPs) to interruptions of a steady rhythm. Psychophysiology, 18, 322-330. Griffin, I. C., Miniussi, C., & Nobre, A. C. (2002). Multiple mechanisms of selective attention: differential modulation of stimulus processing by attention to space or time. Neuropsychologia, 40, 2325-2340. Jones, M. R., Moynihan, H., MacKenzie, N., & Puente, J. (2002). Temporal aspects of stimulus-driven attending in dynamic arrays. Psychological Science, 13, 313-319. Lange, K., & Röder, B. (2006). Orienting attention to points in time improves stimulus processing both within and across modalities. Journal of Cognitive Neuroscience, 18, 715-729. Lange, K., Rösler, F., & Röder, B. (2003). Early processing stages are modulated when auditory stimuli are presented at an attended moment in time: An eventrelated potential study. Psychophysiology, 40, 806-817. Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review, 106, 119-159. Miniussi, C., Wilding, E. L., Coull, J. T., & Nobre, A. C. (1999). Orienting attention in time. Modulation of brain potentials. Brain, 122, 1507-1518. Näätänen, R., Tervaniemi, M., Sussman, E., Paavilainen, P., & Winkler, I. (2001). "Primitive intelligence" in the auditory cortex. Trends in Neurosciences, 24, 283-288. Neisser, U. (1979). The control of information pickup in selective looking. In A. D. Pick (Ed.), Perception and its development. A tribute to Eleanor J. Gibson (pp. 201219). Hillsdale, N. J.: Lawrence Erlbaum Associates. Nobre, A. C. (2001). Orienting attention to instants in time. Neuropsychologia, 39, 1317-1328. Nordby, H., Roth, W. T., & Pfefferbaum, A. (1988). Event-related potentials to time-deviant and pitch-deviant tones. Psychophysiology, 25, 249-261.
ISBN 88-7395-155-4 © 2006 ICMPC
1439
