Alma Mater Studiorum University of Bologna, August 22-26 2006
Notational strategies as representational tools for sense-making in music listening tasks: limitations and possibilities Mark Reybrouck University of Leuven Section of Musicology Center for Instructional Psychology and Technology Belgium
[email protected]
cal grounding and from previous and ongoing empirical research.
ABSTRACT This paper is about musical semantics. It aims at providing tools for sense-making in music-listening tasks in bringing together theoretical grounding and empirical evidence. A central focus is the semiotic claim of knowledge construction and symbolizing as a process of sense-making. Rather than relying on symbolic notations which are out-of-time and post-hoc abstractions of the sounding music, it conceives of notational strategies which are self-constructed with as major aim the development of better listening and processing strategies. As such listeners are invited to make a selection as to what they hear and how they represent it. Three major questions can be addressed here: (i) what do listeners actually represent and how do they represent the elements they consider to be relevant? (ii) how can we categorize these representations, both their contents and expression forms? and (iii) can the level of sophistication of the representation be improved? All these questions are approached from the background of theoreti-
Keywords Graphic notation, musical semantics, sense-making, representation, listening experience.
1 INRODUCTION: ONTOLOGICAL AND EPISTEMOLOGICAL CLAIMS What is music and how is it represented in the listener’s mind? Does it mean the written score or audio file, the performance or the listening experience? Is there a difference between music as a conceptual category and music-as-heard? Can we rely on music as a sounding artifact in order to give a full description of its meaning, or should we rely on strategies of sense-making by the listener? The distinction is critical and raises theoretical claims concerning the conception of music as an “ontological category”. Two possibilities can be considered: music is an object that can be objectified or reified as an abstract entity that exists out of time, or music is a process that unfolds in real time (see Reybrouck, 2004, 2005a, b). Besides, there are “epistemological” claims which are related to what the listener actually hears and makes sense of while listening to the music. This latter question is the central topic of this paper which aims at providing theoretical grounding and empirical evidence for some strategies of sensemaking and their assessment. It focusses mainly on the listener’s representational strategies with a special emphasis on selfinvented graphical notations.
In: M. Baroni, A. R. Addessi, R. Caterina, M. Costa (2006) Proceedings of the 9th International Conference on Music Perception & Cognition (ICMPC9), Bologna/Italy, August 22-26 2006.©2006 The Society for Music Perception & Cognition (SMPC) and European Society for the Cognitive Sciences of Music (ESCOM). Copyright of the content of an individual paper is held by the primary (first-named) author of that paper. All rights reserved. No paper from this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval systems, without permission in writing from the paper's primary author. No other part of this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval system, without permission in writing from SMPC and ESCOM.
ISBN 88-7395-155-4 © 2006 ICMPC
1727
ICMPC9 Proceedings
Graphic notations are important windows to music cognition (Barrett, 2000): they represent the perception of the music-asheard and the listener’s disposition towards abstraction with respect to the sounding music (Elkoshi, 2002, see also Bamberger, 1991; Barrett, 1997, 2000; Davidson & Scripp, 1988; Demorest & Serlin, 1995; Hargreaves, 1978; Sloboda, 1988; Smith, Cuddy & Upitis, 1994; Upitis, 1985). They go beyond verbal reports and the constraints of discrete-symbolic labeling, allowing continuous mapping of data sets with several levels of resolution.
2 SENSE-MAKING AND THE LISTENING EXPERIENCE Dealing with music is a processual experience that calls forth epistemic interactions with the sounding world. Several options are possible: the interactions can be coperceptual or post-hoc abstractions from the sounding flux; they can focus on temporal windows with a rather small size—there is a difference between auditory events of say 30 msec to several seconds—or take distance from the perceptual flux in a kind of synoptic overview; and they can induce analog-continuous or discrete-symbolic representations. The distinctions are really important as they call forth different strategies of sense-making which span the whole continuum between “global” and “differentiated” responses to the music. It is argued, however, that representations which are finegrained offer a better level of “resolution” than the global and less differentiated ones which refer rather loosely to the sounding music. As such, it is possible to conceive of continuous representations that tap the moment-to-moment history of the sonorous unfolding in time. Representations, however, can use time intervals rather than discrete points in time in an attempt to provide a more global overview. As such, there is simply no best way of representing music: what listeners use are distinct strategies which can be considered as tools for making sense out of the sounding music.
Graphic notations, further, have a number of possibilities: they are information-carrying representations which provide means for visualizing music; they can be considered as tools for sensemaking which focus on the sounding music rather than on formal analysis which is the product of reflection after-the-fact; and they can be considered as an addition to existing notational systems which are related to music, such as conventional staff notation, piano roll notation, tablature, chord names, sequencer charts, MIDI files, and waveforms. There is, nonetheless, a difference between self-produced or invented graphic notations and more sophisticated notations that are the outcome of more elaborate procedures such as computer graphics and visualization techniques. The latter, especially, are very promising in making music visible in a way that goes beyond intuitive approaches and in providing ways of visualization that are iconic rather than symbolic. These techniques, however, are not the topic of this paper (see Reybrouck, 2003 for an overview), which focusses on ways of construction of meaning rather than on the decoding of existing representations. What really matters are active attentional processes of selection—the elements to be depicted—and translation—how to depict the elements visually, which expression forms—, rather than processes of perceiving and recognizing. Active construction, as a rule, is more demanding than understanding and interpretation of preexisting notations.
Representational strategies, further, can be continuous or intermittent, they can occur in real-time—as in perception—or out-oftime—as in rehearsal from memory or imagination—, and they can provide local or global representations. All these options are viable ways of dealing with the sound, but there is a major distinction between the fulness of the sensory rich experience and the conceptual abstraction that tries to generalize from the discrete particulars. Listeners can focus at will and deliberately choose the level of resolution and perspective (see Godøy, 1997, Reybrouck, 2004) in an attempt to make sense out of the music. There is, however, a critical distinction between a motivated choice between different strategies as against the quasi obligatorily reliance on diffuse and syncretic ways of listening which are exemplary of lack of sophistication of the listening experience.
4 NOTATIONS AND NOTATIONAL STRATEGIES: AN OVERVIEW Studies on the nature of self-invended graphic representations are related to research on symbolic representation of music and visual representation of music perception. They are among the common tools for studying the process of sense-making while dealing with music and a lot of empirical data has been gathered that is representative for the way listeners—mostly children—actually make sense of music and for the way they graphically represent it. Symbolizing, further, can be conventional or idiosyncratic, but it deals mostly with different levels of specification, abstraction, formalization and reflection with regard to music as a sounding stimulus. To the extent that we can generalize from these notations, it is possible to reduce them to some major “categories”. As such, there have been several attempts to propose classifications and typologies of children’s graphic representations. Most of them are grounded in cognitive theories of development—in an attempt to describe the developmental paths of children’s mental representations and their strategies for making them visual—, in theories about perceptual learning, or in recent work on symbolizing and knowledge construction in general. The empirical findings, on the other hand, point in some well-defined directions: the notational strategies seem to be influenced by age and previous exposure to
A further distinction, finally, concerns the relation between the elicited sense-making and the music as a stimulus: is music to be considered as a direct and causal stimulus for sense-making or does it function only as an indirect means for evocations, associations, moods and emotions? The question is challenging as it covers a whole research program which is directed at improving the level of sophistication in music-listening tasks (Reybrouck, Verschaffel & Lauwerier, submitted). Sense-making, on this view, starts from the sensory experience. This bottom-up approach, which is data-driven, can be complemented with top-down conceptual processing as well. The aim, however, is to set up experimental designs that are able to assess the strategies of sensemaking and it is argued that graphical notations are likely to be suited for this task.
3 GRAPHIC NOTATIONS: WHY AND WHAT FOR?
ISBN 88-7395-155-4 © 2006 ICMPC
1728
ICMPC9 Proceedings
(reflecting the sounding quality of the music as unfolding over time), “global compound “ (the combination of several global categories), “differentiated compound” (the combination of several differentiated categories), and “global/differentiated” (the combination of global and differentiated categories).
exposure to music and some transitions become manifest as children grow older, such as (i) an increase in sophistication with notations becoming more detailed and reflective, (ii) more accurate symbol-meaning relationships, (iii) an emancipation from context-bound recordings with an increased concentration on musical elements and (iv) an increasing range of musical dimensions to be represented.
The main results were rather amazing, at least for the first study. Most representations were of the global type with little or no evidence of recording of any of the musical parameters beyond instruments as a global reference. All other categories were really negligible. The vast majority of children represented the musical excerpts with global categories. Actually 93,4 % of all notations were of that type, with differences between age groups and between musical excerpts being negligible. The most prominent simple categories were “instrument” (about 49,4% for the younger and 36,3% for the older children) and “evocation” (42,3% for the younger and 47,5% for the older children), with the category of instrument being more important than evocation for the younger, and evocation being more important than instrument for the older children. None of the other categories seemed to be important. To summarize: elementary school children do not seem to differentiate much when asked to listen and graphically respond to existing musical excerpts and they rely on very basic global listening and representational strategies when producing graphic notations.
5 NOTATIONAL STRATEGIES: EMPIRICAL FINDINGS In an attempt to test these theoretical claims,we conducted a number of empirical experiments. In a first experiment we set up two empirical studies with the aim to classify the graphic expressions generated by children in a music-listening task (see Reybrouck, Verschaffel & Lauwerier, submitted). In the first study, two age groups (third and sixth grade) of elementary school children (n = 89) with or without formal musical training were exposed, in the context of a whole-class test, to five fragments from existing musical compositions, and were asked to make graphic notations “so that someone else could re-construct (or 'hear' internally) the music afterwards”. This first experiment was conceived as a pilot study aimed at revealing the variety of children’s notations of these fragments, as well as the impact of some subject and task variables on these notations. Another major aim of this study was to develop a “classification system” for children’s notations taking into account the available empirical data as well as the stateof-the-art of the research literature. Based on the results of this first investigation, we set up a second, larger study with four groups of subjects, namely younger and older subjects with and without musical training (n = 331) which were exposed to 6 selfcomposed fragments with the aim to maximally 'isolate' one musical parameter (melody, rhythm, dynamics, structure, voice leading, and timbre).
The second study which used self-composed and artificial fragments yielded very different results. Actually 60,5% of all notations were of a differentiated type. Most of the notations depicted simple categories (91,3%): either global (37,4%) or differentiated (53,9%). Only a minority of the notations were compound (7,3%). Of these latter, 0.7% involved only global elements, 1,1 % involved only differentiated elements, and 5,5 % showed a mixture of global and differentiated elements.
Our “classification grid” was constructed in terms of welldefined, research-based criteria and comprised 12 subcategories that were subsumed under additional categories that reflected the difference between strategies that capture the music in a global or more differentiated way, and simple vs compound categories, which consist of only one type of graphical notation (simple) or many of them (compound). As such, there were 12 categories to which the notations were assigned according to the following criteria: “no reaction”, “instrument (global)” (depiction of an instrument in an all-or-none fashion as a global label), “evocation” (everything that is evoked by the music, ranging from drawings of concrete objects or scenes that are globally evoked by the musical fragment to drawings of particular feelings or emotions, “floating notes” (depiction of one or more music notes without any specific reference to pitch or duration), “sounding object/action” (a representation of an object that is actually sounding with at least some temporal unfolding or of a sound-producing action that refers to (a part or an aspect of) the musical fragment, “analog image” (depiction of the unfolding of a musical dimension over time (pitch, loudness, duration) in a pictorial, nonformal way), “non-formal graphic notation” (depiction of the unfolding of music in time, not in a pictorial but in a more abstract way, relying on non-conventional graphical notation with the intention to depict the musical unfolding in a rather precise way), “formal-conventional music notation” (depiction of standard musical notation of notes (on a staff or without staff) and other conventional musical symbols), “instrument (parameter)”
ISBN 88-7395-155-4 © 2006 ICMPC
As to the influence of the two “subject variables”—age and formal training—on the distribution over the main categories of notations, and more particularly, on the percentage of differentiated notations, the results can be summarized as follows: (i) The influence of age was obvious, with the older children obtaining a much higher percentage of (simple or compound) differentiated categories (77%) than the younger ones (43,5%). (ii) The influence of formal musical training was obvious as well, be it to a lesser extent than age. The children who received at least one year of training generated 69,6% simple or compound notations that were classified in a differentiated category, against 54,3% for children without training. Moreover, the notations of the children with training were more commonly spread over the distinct categories. (iii) There seemed to be an interaction effect between age and formal musical training in the sense that in the youngest age group formal musical training resulted in more differentiated notations (63,3% for the trained children versus only 30,7% for those without training), while the oldest age group showed almost no difference (76,6% for the trained children versus 77,5% for those without training). (iv) With respect to the influence of the kind of musical fragment on the overall results, finally, we also observed a difference in the percentage of notations scored in the differentiated categories between the distinct fragments (both simple and compound).
1729
ICMPC9 Proceedings
tempt to capture the actual listening strategies that are at work in listening to “real music”. This means that the interpretation of the findings of “reductionist” experiments—working with simplified and rather short musical fragments—should be complemented with the outcomes from research involving more realistic and more complex musical material.”
Both experimental studies yielded a number of interesting results: they showed the rich variety in children's spontaneous ways of perceiving and representing music; they showed that with age and music education, children’s recordings become more concerned with genuine musical ideas and concepts (beyond instrumentation); and they showed, finally, the very strong impact of the musical characteristics of the fragments—both in the sense of the length and complexity of the fragments and in the sense of the type of musical parameter that dominates in the fragment, on the nature of the representations. There were, however, some questions which were not answered up to now: (i) Are children’s invented notations true ‘windows’ on their musical perception and sense-making? (ii) How to solve the big gap between fragmented and contextual perception and notation of music? (iii) Can we rely exclusively on graphic notations or must we rely on the children’s verbal descriptions and explanations in conjunction with the analysis of their notations products? (iv) Is it possible to conceive of design experiments which aim at improving music education by helping children to build up more conventional formal notations out of more intuitive and informal ways of representing music?
The third study, finally, is an intervention study on the effects of focusing on student’s informal graphic representations. It aims at going beyond an ascertaining study, in an attempt to describe and analyze the development of children’s musical notations under given instructional conditions. As such, there is an urgent need of design experiments which aim at improving music education by helping children to gradually and actively build up more conventional formal notations out of their more intuitive and informal ways of representing music. The rigid conventions of standard notation, in fact, are not the most ideal device to learn to graphically encode musical entities.
6 LIMITATIONS AND POSSIBILITIES Graphic notations are likely to be important “windows” to music cognition. They differ from standard notations in being more intuitive and being probably more akin to children’s actual understanding of the music. This means that children have more freedom in choosing and organizing the musical dimensions they wish to record, but above all, these notations allow them to represent dimensions that are extramusical as well (associations, evocations, sound producing actions...). As such, graphic notation is a child-centered measure that enables children to convey their comprehension of music through a symbolic representation or notation of their musical experience (see Barrett, 1997, for an overview).
As such, we set up further ongoing research which aims at elaborating on these questions with three main topics: (i) what are students criteria for representations in music listening and understanding? (ii) a qualitative study of student’s graphic representations of authentic and artificial musical fragments, and (iii) an intervention study on the effects of focusing on student’s informal graphic representations. The first study aims at uncovering students’ criteria for representations. As such, it is related to the topic of meta-representational knowledge. To quote Disessa: «Meta-representational competence (MRC) refers to the full complex of abilities to deal with representational issues. MRC includes, centrally, the ability to design new representations, including both creating representations and judging their adequacy for particular purposes. It also includes understanding how representations work, how to work with representations for different purposes, and, indeed, what the purposes of representations are. Finally, it is also useful to include within the scope of MRC knowledge that allows students to learn to use new representations quickly and the ability to explain representations and their properties» (Disessa, 2002, p. 105). Several criteria can be used in order to categorize the student’s representations: “make-centered” (does the criterion concern the making of the representation?), “use-centered“ (does the criterion concern the use or interpretation of the representation?), “epistemic fidelity” (does the criterion concern correctness or completeness?), “formal criteria” (such as systematicity, consistency, redundancy, conventionality…), and “aesthetic” (is the criterion based on pleasant visual effects?). The design of the study is a collective paper and pencil test asking children to compare the quality of two contrasting representations of a simple musical fragment which systematically differ on one representational criterion. The children are asked for a choice and an explanation, eventually to be followed by individual interviews with another, smaller group of children.
Graphic notations, however, have shortcomings as well. There is, first of all, the difficulty to depict graphically what has been attended to, and there is also the speed of “notational resolution”, which means that is is not always possible to depict graphically events that unfold at a speed that is above the time needed for making notational marks of them. Many of these difficulties are related to the actual “construction” of graphic notations. As such, there is a need of graphic interfaces that can help in providing interfaces between the process of listening and the process of making sense of what has been heard. There is, further, a major difference between making notations in real-time—keeping step with the sonorous unfolding in time—and making notations afterthe-fact—relying on memory. What really matters here is the tension between “in time” and “out-of-time”. Notations that are produced simultaneously with the listening experience are coperceptual and are likely to be induced by the sounding music; notations that are produced after-the-fact are less constrained. They allow the conscious mind to interfere with the process of sensemaking with many degrees of freedom that make it difficult to trace any causal relation between the sounding stimulus and the reported reactions. There is, therefore, a need of methodologies that provide continuous measurements of reported reactions to the sounding stimuli. Much has already been done in the domain of emotional reactions to music (Schubert, 2001), but more encompassing procedures must be developed that measure continuous responses to music which go beyond the mere emotional responses as well. Graphic notations can be helpful here, but much
The second study is a qualitative study of student’s graphic representations of authentic and artificial musical fragments in an at-
ISBN 88-7395-155-4 © 2006 ICMPC
1730
ICMPC9 Proceedings
more theoretical and empirical grounding is needed in order to find ways of continuous modelling of real-time epistemic interactions with the sounds.
Reybrouck, M. (2003). Deixis, Indexicality and Pointing as Heuristic Guides for Enactive Listening. Route Description, Cue Abstraction and Cognitive Maps. Proceedings of the 5th Triennial ESCOM Conference. Hanover, 8-13 September 2003, pp. 298301.
7 CONCLUSIONS AND PERSPECTIVES The assessment of self-produced music representation is a challenging area of research. Rather than providing ready-made and static representations—as in a musical score or other visualisations of music—it aims at uncovering the underlying strategies for sense-making. The area of research raises theoretical, methodological and educational issues. Besides a better understanding of actual representational competence by means of further ascertaining studies, there is an urgent need of design experiments aimed at improving music education by helping children to gradually and actively build up music cognition out of their more intuitive and informal ways of representing music.
Reybrouck, M. (2004). Music Cognition, Semiotics and the Experience of Time. Ontosemantical and Epistemological Claims. Journal of New Music Research, 33, 4, pp. 411-428.
REFERENCES
Reybrouck, M. (2006). Musical Creativity between Symbolic Modelling and Perceptual Constraints: the Role of Adaptive Behaviour and Epistemic Autonomy. In: I.Deliège é G.Wiggins (Eds.). Musical Creativity: Multidisciplinary Research in Theory and Practice (pp. 42-59). Oxford: Psychology Press.
Reybrouck, M. (2005a). A Biosemiotic and Ecological Approach to Music Cognition: Event Perception between Auditory Listening and Cognitive Economy. Axiomathes. An International Journal in Ontology and Cognitive Systems, 15, 2, pp. 229-266. Reybrouck, M. (2005b). Body, mind and music: musical semantics between experiential cognition and cognitive economy. Trans. Transcultural Music Review, 9 . 46 pp.
Bamberger, J. (1991). The Mind behind the Musical Ear. How Children develop Musical Intelligence. Cambridge, MA - London: Harvard University Press. Barrett, M. (1997). Invented Notation: a View of Young Children’s Musical Thinking. Research Studies in Music Education, 8, 1-14.
Reybrouck, M. (2006). Music Cognition and the Bodily Approach: Musical Instruments as Tools for Musical Semantics. Contemporary Music Review, 25, 1/2, pp. 59-68.
Barrett, M. (2000). Windows, Mirrors, and Reflections: A Case Study of Adult Constructions of Children’s Musical Thinking. Bulletin of the Council for Research in Music Education, 145, 4361.
Reybrouck, M., L.Verschaffel & S.Lauwerier. Children’s drawings as representational tools for musical semantics: an investigation into the assessment of sense-making in a music-listening task. Musicae Scientiae (submitted).
Davidson, L., & Scripp, L. (1988). Young children’s musical representations: Windows on music cognition. In J. Sloboda (Ed.), Generative processes in music (pp. 195-230). Oxford: Oxford University Press.
Schubert, E. (2001). Continuous measurement of self-report emotional response to music. In: Juslin, P.N. & J.Sloboda (Eds.). ) Music and Emotion: Theory and Research. (pp. 393-414). Oxford: Oxford University Press.
Demorest, S.M., & Serlin, R.C. (1995). The integration of pitch and rhythm in musical judgment: testing age-related trends in novice listeners. Journal of Research in Music Education, 45, 5166.
Sloboda, J. (1988). Generative Processes in Music. The Psychology of Performance, Improvization, and Compostition. Oxford: Clarendon Press. Smith, K.C., Cuddy, L.L., & Upitis, R. (1994). Figural and metric understanding of rhythm. Psychology of Music, 22, 117-135.
diSessa, A. (2002). Student’s criteria for representational adequacy. In: Gravemeijer, K.; R.Lehrer; B. van Oers, Symbolizing, Modeling and Tool Use in Mathematics Education. Dordrecht Boston - London: Kluwer Academic Publishers, pp. 105-129.
Upitis, R. (1985). Children’s understanding of rhythm: the relationship between development and musical training. Unpublished doctoral dissertation (Cambridge, Massachussets, Harvard University).
Elkoshi, R. (2002). An Investigation into Children’s Responses through Drawing, to Short Musical Fragments and Complete Compositions. Music Education Research, 4, 2, 199-211. Hargreaves, D.J. (1978). Psychological studies of children’s drawings. Educational Review, 30, 247-254. Godøy, R.I. (1997), Formalization and Epistemology, Oslo: Scandinavian University Press.
ISBN 88-7395-155-4 © 2006 ICMPC
1731
