Music perception of hearing-impaired persons with focus on one test subject Hiraga Rumi⇤ , Kjetil F. Hansen† , Naoya Kano‡ , Masaki Matsubara§ , Hiroko Terasawa§¶ ,and Keiji Tabuchik ⇤ Dept.
of Industrial Technology, Tsukuba University of Technology, Japan Email:
[email protected] † Royal Institute of Technology, Sweden ‡ School of Informatics, University of Tsukuba, Japan § Faculty of Library, Information and Media Science, University of Tsukuba, Japan ¶ JST PRESTO, Japan k Faculty of Medicine, University of Tsukuba, Japan
Abstract—We previously investigated how hearing-impaired people perceive music in several types of musical experiments. By following-up on the results of some of these experiments with a single test subject Sd , we found that the ability of the subject to perceive music was high and that she appreciated music in a way that was different from that of hearing people. In this paper, we describe three musical experiments with hearing-impaired subjects, their results, and Sd ’s music perception through the experiments. The three experiments involved the Music Puzzle game, the appreciation of harmony, and tempo perception. Music Puzzle is a music game we made that is played on a tablet and is intended to be used by hearing-impaired persons as a serious game with which they can improve their hearing ability by continuously playing it. The experiment on appreciation of harmony was conducted with three subject groups, and the result showed that experience with music affected the appreciation of music accompanied with the tonal code. Tempo perception was investigated with a simple game in which the subjects tapped along with the tempo of the music. By examining the subjects’ hearing acuity in standard medical hearing tests and crossing over the results of these musical experiments, we observed that hearing acuity is not necessarily related to the perception and understanding of music. Index Terms—hearing-impaired, music perception
I. I NTRODUCTION In this paper, we describe the cross-examined results of three experiments by focusing on one test subject, Sd , because her results showed a case that a profoundly deaf person listened to and understood music. With our experiences with hearingimpaired college students, we know that many of them like to listen to music and have music related activities. So far we conducted several experiments on music perception and music understanding. Though some of the results showed no differences, results showed there were differences between subjects with and without hearing impairment in most of the experiments. The experiments had no specific relationships each other, because they were based on musical elements. Therefore, we have not looked over and compared the results of the past experiments. The past experiments we conducted include the emotion recognition in improvised percussion performances by a pro-
fessional player, perception of timbre, appreciating harmony, and using the Music Puzzle and Tapping Game that we developed. Among these experiments, we found that there were no differences between subjects with hearing impairment and without it in recognizing emotion expressed in percussion performances by a professional player [7]. However, there were differences between subject groups in other experiments [6][8][9]. Although hearing-impaired people in general recognized timbre differences less, did not mind much about harmony, and had difficulty in completing Music Puzzle when making hearing-impaired subjects into one group, there are individual differences between participants who attended the experiments. Therefore, it is necessary to investigate what the differences are, why the differences are derived, and how much the differences are, depending on individual participant. It is known that there are amusia in hearing people in spite of their listening to music intentionally or involuntarily [1]. Also, understanding some elements of music, such as music structure or cadence (the progression of harmony), are acquired and improved by experiences with music [21]. With the variety of music perception not only by hearing people and with our experiences with hearing-impaired people, we came to think that the way of listening to music is not directly related to hearing level, which is measured with pure tone audiometry. To know how hearing-impaired people listen to music, we used the results of the past experiments and physical examinations, and we reached the conclusion that hearing-impaired people listen to music independent from their hearing acuity. This requires further investigations such as using music battery tests [10] and performing a questionnaire study of everyday listening [11]. II. R ELATED W ORKS Our research relates to the music perception and experiences of hearing-impaired people. Limb describes the difficulty that cochlear implant users have understanding music. Because the music characteristics of frequency range, dynamic range, resolution on time, and
complicated timbre in musical sound are different from those of speech voices, what cochlear implant users listen to in music is not the same as the original sound [16]. Chasin focused on hearing-impaired people who use hearing aids and described the difficulty they have in perceiving music [3]. Darrow [4], as a music therapist, showed the music perception of hearingimpaired children. Recent studies discuss the effect of music training on language ability. Patel proposed the “OPERA” hypothesis on the effectiveness of music training on speech processes in terms of neural plasticity [19]. Moreno et al. showed that music training for eight-year old children enhanced reading and pitch discrimination abilities in speech [18]. The effectiveness of music on people has been investigated in not only hearingimpaired people but for the life-span [15]. Brandt et al. challenged that spoken language is a special type of music [2]. The effectiveness of music is not limited to language ability. Acquiring cognitive skills [21] and social/cooperative skills [14] is also discussed. Thaut introduced music therapy for various types of disorders such as aphasia and apraxia [22]. The effectiveness of music training for hearing-impaired children was also reported by several groups. Torppa et al. [23] found that musical experiences are strongly linked to the perception of focus in experience with music. Mitani et al. [17] reported the effectiveness of this experiences on children with cochlear implants acquiring language ability. Yucel et al. [24] also showed the long term music training for children with cochlear implants enhanced music and other auditory domains. III. E XPERIMENTS We briefly describe three experiments conducted in the past and focus on one test subject and her results with music perception and game activities. Experiments were done with Music Puzzle, on harmony appreciation, and with Tapping Game. We gathered participants in the three experiments independently to each other. All the subjects with hearing impairment participated the experiments with the condition which they felt comfortable to listen to music. A. Subject We describe the results by focusing on a hearing-impaired subject Sd by crossing over the three experiments. We chose Sd by looking at the results of the three experiments and Sd ’s physical examination by an otolaryngologist. The physical examinations Sd took were pure-tone audiometry, a free field audiometry test, otoacoustic emissions (OAE), and speech audiogram. Sd is a congenitally deaf and sensorineural hearing loss whose hearing acuity was measured over 100 dB on both sides of the ear. When Sd put a hearing aid on her right ear, the hearing acuity was around 40 dB in the frequency range from 125 to 2000 Hz. The OAE on both ears showed no functions in the cochlear and Sd ’s speech audiogram was not excellent. Sd is a bright college student, has many friends, once was a member of a dance circle, played Japanese drum and Japanese koto, and listened to piano performances when she was a kid.
Fig. 1.
Music Puzzle
Sd listens to music from three to five hours a day and more than six hours on weekend. Her favorite is popular music. B. Music Puzzle Music Puzzle (MP) is a computer game for tablets. The original sound is divided into sound objects of the same length that are represented each by a circle, as shown in Figure 1. The goal is to reconstruct a complete original sound from sound objects like a traditional jigsaw puzzle [5][6]. In addition to the horizontal order of circles being randomized, each sound object is modified in timbre by using band-pass filters and pitch shift. Thus, a player listens to each sound object, corrects the sound in terms of timbre and pitch, and moves it in the correct order. When a player uses a cheat button to automatically correct the timbre or pitch (“Eq. Cheat” button or “Pitch Cheat” button in the left bottom corner in Fig. 1), he/she can concentrate on arranging the order of the sound objects. The puzzle is accomplished when both the order of particles and the timbre/pitch are corrected. We used three types of sound: music, voice (reading poems), and a mix of music and voice. To successfully complete a game session, a subject must actively listen to the sound and understand the meaning of the sound. If the sound is music, then a subject needs to understand the passage and structure of the music. Altogether, 48 subjects participated. They were four groups of hearing-impaired Japanese, musically not experienced Japanese, musically not experienced Swedish, and musically experienced Japanese people. All the subjects were either undergraduate or master course students. We prepared four sets of sound. Each set consisted of three types of sound. Though the first set was used for practice, which all participants used in common, other sets were placed randomly. Told to play the puzzle in order of sets, the participants did the puzzle for 25 minutes, and we used a log to analyze their way of playing. The most notable results reported were the differences in the rate of accomplished game sessions and the preferred sound type. Hearing-impaired people preferred the music sound the most, while hearing people preferred the mixed sound.
b6 œ œ œ™ œ œ œ™ & b 8 œ™ œ œ œ™ œ™ œ œ œ™ J J ? bb 68 Œ ‰ œ ™ œ œ œ ™ œ œ ™ œ œ œ™ œ œ œ œ œ œ œ ™ œ œ œ ™ œ œ
{
(a) Original b6 œ œ œ™ œ œ œ™ & b 8 œ™ œ œ œ™ œ™ œ œ œ™ J J ? b 68 Œ ™ œ™ œ œ œ™ œ b œ™ œ œ n œ ™ œ œ œ œ œ œ œ™ œ œ œ™ œ œ
{
(b) Dominant b6 œ œ œ™ œ œ œ™ & b 8 œ™ œ œ œ™ œ™ œ œ œ™ J J ? b 68 Œ ™ b œ™ œ œ œ™ œ œ ™ œ œ œ™ œ œ bœ œ œ œ œ ™ œ œ œ ™ bœ œ
{
(c) Subdominant b6 œ œ œ™ œ œ œ™ & b 8 œ™ œ œ œ™ œ™ œ œ œ™ J J ™ b œ ? b 68 Œ ™ œ œ bœ ™ œ b œ ™ œ œ œ™ œ œ œbœ œ œ bœ ™ œ œ œ ™ œ œ
{
(d) Parallel b6 œ œ œ™ œ œ œ™ & b 8 œ™ œ œ œ™ œ™ œ œ œ™ J J ? b 68 Œ ™ ™ œ œ œ œ™ b œ ™ œ œ #œ ™ œ œ œ œ œ œ œ œ ™ œ œ œ ™ œ œ
{
(e) Relative Fig. 2. Five accompaniments for “Silent Night.” Accompaniment keys are in parentheses. (a) Original (B flat major). (b) Dominant (F major). (c) Subdominant (E flat major). (d) Parallel (B flat minor). (e) Relative (G minor).
C. Appreciating Harmony In the experiment on appreciating harmony, we presented subjects with music pieces for a single melody with five types of accompaniment with different related keys and then asked them to score their appreciation of each music piece [8]. Altogether, 30 Japanese subjects participated. They were in three groups of hearing-impaired, musically not experienced, and musically experienced. Each group consisted of ten members. They listened to eight pieces music, each of which were accompanied by five accompaniments of different keys, and they gave their preference scores with five levels. The duration of music was between 16 to 27 seconds. The order of harmonies was randomly decided. Figure 2 shows an example of music with five different accompaniment keys. The most notable results were that musically experienced people showed a strong preference for the original key accompaniment and musically not experienced people showed a preference in-between that of hearing-impaired people and musically experienced people. D. Tapping Game With Tapping Game (TG), subjects either only listen to music or look at a visual tempo-guide accompanying the music
Fig. 3.
The display of TG
and tap the tempo of music [12][13]. The purpose of the TG experiment was to find how hearing-impaired people recognize tempo and the possibility of improving hearing ability in the recognition of tempo by the repeated use of the TG. When the visual guide was not used, no objects were displayed. The visual guide was a bar that drops from the top of the display, and the timing of a tap is indicated when the bar reaches some point that makes the ladder like figure. Six hearing-impaired people participated. Figure 3 shows the display when a game was completed either with a great or good performance. In the experiment, a subject played ten game sessions. Each session used a single piece of music and was categorized either in the music condition (MC) or visual condition (VC). In both conditions, a piece of music was played three times, and the subject followed the tempo. In VC, the second time of music played was presented with the visual guide. The conditions were randomly assigned to ten sessions. Among the ten pieces of music, one was a noise sequence of 140 beats per minute (BPM), and the other was a drum-only sequence of 120 BPM. The observed results were that the subjects were found to follow tempo in a syncopated way more than we expected, the music that subjects identified as “easy music” was played with regular beats, and the recognition of tempo improved with the visual information for easy music. The TG experiment was the start of our longitudinal observation in improving hearing ability with music game playing. IV. R ESULTS The detailed results of the each experiment are in the past papers [6][8][12][13]. We describe Sd ’s results by comparing them with those of hearing subjects and other hearing-impaired subjects with the purpose of explaining that hearing acuity is not necessarily related to appreciating music. Hereafter, HI, NE, and EXP denote the subject groups of hearing-impaired people, Japanese hearing people with little musical experience, and hearing people with a lot of musical experience.
1.5" 1" A"
TABLE I S OME RESULTS FROM MP ( AVERAGE VALUES )
Number of game sessions Ratio of completed sessions (%) Duration (sec) Speed (clicks/sec)
HI 6.8 32.1 271.7 0.18
NE 6.6 91.1 216.2 0.43
EXP 8.7 98.2 179.3 0.41
B"
C"
D"
E"
F"
G"
H"
TABLE III R ATIO OF USING CHEAT BUTTONS AND RESULT TYPES BY HEARING - IMPAIRED SUBJECTS (%) Sd 8 100 177.1 0.17
Use cheat buttons Not use cheat buttons
Accomplished 30.9 3.7
Give Up 21.0 44.4
5"
TABLE II
R ATIO OF USING CHEAT BUTTONS (%)
4.5" 4"
cheat Pitch cheat Eq.
HI 51.9 51.9
NE 10.1 17.7
EXP 3.8 1.3
Sd 100 100
HI"
3.5"
NE"
3"
EXP"
2.5"
Sd"
2"
A. Music Puzzle Table I shows the average values by each subject of the three groups and Sd . The values are the number of game sessions that a subject performed, the ratio of completed game sessions by each subject, the average duration that a subject spent for a game session, and the average speed of playing the game. The speed was calculated by dividing the total number of clicks for either each sound object or buttons by duration. Sd completed all the game sessions she played, while the average of the completed sessions for the hearing-impaired group was 32.1%. That by the musically experienced group was 98.2% which means that some game sessions were not completed even by the musically experienced subjects. Sd spent a shorter amount of time (177.1 seconds per game session) than did other hearing-impaired subjects (271.7 seconds), which is even shorter than the average duration of the musically experienced group (179.3 seconds). The lower speed means that a subject spent a longer amount of time after a clicking on a sound object or button. The way Sd interacted with MP was closer to that by hearing-impaired group. Table II shows the ratio of using cheat buttons by game sessions. The Eq. cheat corrects the timbre automatically. A subject in the hearing-impaired group who used cheat button used both the Pitch and Eq. cheats. Sd used cheat buttons in all game sessions (100%). This means she concentrated on the game to arrange the order of sound objects. However, in many of the sessions with hearing subjects, the subjects corrected the pitch shift and timbre by themselves. Table III shows the relationships between the use of cheat buttons and the result types in the ratio of number of game sessions of each category to all game sessions. Even when using the cheat buttons, 40.5% of game sessions ended in the subjects giving up. B. Harmony In the experiment on harmony, we asked for the preferred harmony by preparing accompaniments with different keys from a melody. Subjects chose 1 when they disliked the music when they listened to the performance and chose 5 when they liked it. Figure 4 shows the average values of preference levels by the three subject groups and Sd . Bars are grouped by the
1.5" 1" Original"
Fig. 4.
Dominant" Subdominant"
Parallel"
Rela:ve"
Preference results of harmony experiment (each accompaniment)
accompaniment key. It is apparent in the figure that the two hearing groups liked the original accompaniment (two bars in the middle of the left most group). Sd ’s harmony preference was not so different from the average of the hearing-impaired group. Figure 5 shows the average values of preference for each piece of original music by three subject groups and the value by Sd . Sd did not show a strong preference for the original music. C. Tapping Game (TG) The tapped data logged in the experiment with TG were analyzed to see how well a subject followed the tempo of each piece of music. A tap was recognized to be correct (following the tempo) if it happened in the range of the duration of a sixteenth note before and after a beat of the music. Namely, for the inter onset interval (IOI) of music, a tap-follow is defined as the relationship with the time of a beat if and only if taptime beattime +IOI/4 and taptime beattime IOI/4, where taptime is a time which a subject tapped and beattime is a beat time of music. In this experiment, the beat time is the onset of the quarter note, and the IOI is the length of a quarter note. A tap-follow is counted when two consecutive 5" 4.5" 4" 3.5"
HI"
3"
NE" EXP"
2.5"
Sd"
2" 1.5" 1" A"
Fig. 5.
B"
C"
D"
E"
F"
G"
H"
Preference results for each piece of original music
Snare
° 4 /4V
Ó V
V
V
V j
Tenors
4 /4‰ V
V
V
V
j V V
V
V
V
V j
V
V
V
j V V
V ‰ Ó
Sd rated the difficulties of TG to be easier compared with other hearing-impaired subjects. V. D ISCUSSION A. Distinctive music listening
Bass
4 /4
∑
Fig. 6.
Syncopation∑
∑
TABLE IV
Cymbals
4 ∑AND ITS INCREASED VALUE .∑M USIC CONDITION (MC, ∑ ¢ / 4C ATCH UP RATE LEFT ) AND VISUAL CONDITION (VC, RIGHT ).
Xylophone
° 4 &4
C1 ∑ 0.02 0.15 0.32 0.82 ∑ 0.82
4 &4
∑
raphone 1
4 ockenspiel & 4 {
raphone 2
Marimba 1
Marimba 2
Marimba 3
Timpani
Aux. 1
Aux. 2
Aux. 3
MC Cincrease 0.20 0.07 -0.05 0.18 0.09
C1 0.00 0.00 0.13 0.88 0.65
VC ∑Cincrease 0.06 0.06 0.63 0.08 ∑ 0.31
∑
∑
∑ ∑
tap-follows happen. the data of the TG experiment, we noticed 4 When analyzing ∑ could not be grouped∑ into either following∑ 4 the log data &that well or bad. For example, there were times when following 4 tempo was∑bad if the tap occurs at ∑the regular syncopated∑ 4 &the quarter notes. As shown in Figure 6, the notes in the second stair with the first stair but lagged behind 4 did not synchronize ∑ ∑ ∑ 4 in the first &those stair regularly by eighth notes. Thus, we categorized the tapped log data into the following 4 classes. In∑ all the classes, we categorized ∑ five a session if we∑ &4 could find that the longest part matched the category, even if 4 parts were ?other ∑ not done well in a part. ∑ ∑ ¢ 4• Well following (WF). Log is categorized into WF once the beat was followed for a certain duration. • Syncopated (SP). Syncopation is a method in music in ° 4 ∑ point of non-beat. ∑ / 4 which the∑rhythm is changed to the • Double (D). Taps were at double tempo (slow). 4• Half (H). ∑Taps were at half tempo∑ (fast). ∑ / 4• Not following (NF). When taps could not be categorized one of the above, the session was categorized as NF. 4 ∑ SP, D, and H were∑ 4 / Among the ∑five categories above, WF, ¢ in a sense following the tempo. Sd ’s results were 40% of WF, 30% of SP, 20% of D, and 10% of NF. Among the six participants, SP was found in three participants, though the ratio of SP to the two others was 10%. We defined the catch up rate C as C = T /B, where T was the number of correct taps and B was the whole number of beats. Then, we could determine an improvement in the catch up rate Cincrease with Cincrease = C3 C1 , where Ci is the C of the ith trial in a session (1 i 3). Table IV shows Sd ’s Cincrease and C1 for five pieces of music in the music condition (MC) and five in the visual condition (VC) . D. Subjective description Sd described her understanding of music in the interview in TG as below. • Could recognize lower sounds. • Did not notice syncopated beat tapping. • Used the piano sound as a clue for beat tapping.
Among the results with the three musical experiments, Sd was good at the Music Puzzle and Tapping Game, while she showed the typical results of hearing-impaired people in listening to harmony. We do not think she listens to music in the typical way that hearing-impaired people do. We will investigate other hearing-impaired participants to see how they perform in these experiments, making it possible to clarify that there are many ways to appreciate music. From the results, it is apparent that Sd was very aware of what she listened to in music. To accomplish an MP session, a player must understand the music. Sd used cheat buttons to correct the timbre and the pitch, and completely understood the music. Though she said that she recognizes the piano timbre but not its pitch, she somehow recognized the melody (MP used no piano sound). She listens to music every day for long time and listened to the acoustic piano when she was a child. Her way of listening to music can be a good case to show that experience with music gives a person a kind of effectiveness. Though we are not so conscious on how we listen to music as hearing people, we found a lot of interesting ways in O. Sacks book [20]. In his book, he also stated that “even profoundly deaf people may have innate musicality.” Namely, music listening and music appreciation are very personally independent from hearing acuity. B. Music perception of hearing-impaired people Today, many people have relationships with music whether actively, spontaneously, voluntarily, or passively. These include listening to, playing, and creating music, whether a person is a professional musician or not. In fact, the description above on experience with music is no different between hearingimpaired people and hearing people. If the differences in listening to music between hearing-impaired people and not hearing-impaired people lie, then it might be the opportunities of touching music in everyday life. It is quite true that establishing a communication method for hearing-impaired people is important. This communication usually means “to communicate through a language.” Although the ability of communication through music is recognized, this way of non-verbal communication is indirect in terms of conveying information. Therefore, language acquisition is thought to be unavoidable and necessary in everyday life and language training is given to hearing-impaired children. Though the communication training given to hearing-impaired children has been the training of language acquisition, some speech-language-hearing therapists, such as Torppa [23], introduce music into the language training and derive better results. If hearing-impaired people are given the opportunity to listen to music from an early age, their understanding of sound could be different from those without music in their life.
Chasin [3] pointed out the three differences between music and speech: spectral shape, intensities, and crest factor (the difference between the peak level and average level). Different from the speech spectrum, which is defined by the long-term average speech spectrum with a peak around 500 Hz, music can have peaks in any frequency range, depending on each musical piece. The intensities of the acoustical instruments are actually high. A clarinet played at a distance of 3 m is around 68 - 82 dBs, and a trumpet is 88 - 108 dBs. The noise of a bowling alley and a jet plane 200 m away is 80 dBs and 120 dBs, respectively, and the common communication is 60 dBs. The crest factor is smaller in speech than in music. As hearing aids are adjusted to hearing speech, listening to music through hearing aids causes some distortion. In spite of the differences of sound properties and disadvantages of using hearing aids, hearing-impaired people recognize music and like to listen to music. C. Future Work Many questions arise from the above discussions, especially, since a hearing-impaired person could perform well in the experiments. • •
•
How is it possible to achieve a game session of Music Puzzle without perceiving the pitch? It is known that those who obtained music training for some terms understand music closure. Did Sd recognize the closure when she listened to music in Music Puzzle? What makes a person understand tempo? What makes a person understand tempo in a syncopated way?
There is a lot more that we can find by closely investigating the TG results. Though we put Sd ’s catch up rate C in Table IV, we can look at it by difficulty levels of music, the existence of vocals, and differences between music sound and noise/drum sequences. We would also like to find out how hearing-impaired people recognize music; is it completely individual or common in some way? We will continue with longitudinal music training, either with MP or TG, and see whether experience with music has an effect in some way. To measure effectiveness, it is important to establish a way to understand what hearing-impaired individual listen to in music. For this purpose, we will develop several kinds of music battery tests and questionnaires. Furthermore, experiments on brain response of perception of music should be conducted in terms of neurosciences in order to acquire evidences of the effectiveness of music training for hearingimpaired people. ACKNOWLEDGMENTS We would like to express our gratitude to Dr. Furuhashi, an otolaryngologist, for his sincere discussion on the relationship between music and hearing-impaired college students. This work was supported by JSPS KAKENHI Grant Numbers 23611005 and 26282001.
R EFERENCES [1] J. Ayotte, I. Peretz, and K. Hyde, Congenital amusia, A group study of adults afflicted with a music-specific disorder, Brain, 125, pp. 238–251, 2002. [2] A. Brandt, M. Gebrian, and L. R. Slevc, Music and Early language acquisition, Frontiers in Psychology, 3, 2012. [3] M. Chasin and N. S. Hockley, Some characteristics of amplified music through hearing aids, Hearing Research, 308, pp. 2–12, 2014. Frontiers in Psychology, vol. 3, Article 327, 2012. [4] A.-A. Darrow, The role of music in deaf culture: Deaf students’ perception of emotion in music, Journal of Music Therapy, XLIII(1):2–15, 2006. [5] K. F. Hansen, R. Hiraga, Z. Li, and H. Wang, Music Puzzle: an AudioBased Computer Game That Inspires to Train Listening Abilities, Creative Showcase of ACE 2013, pp. 540–543, LNCS, Springer International Publishing, 2013. [6] R. Hiraga and K. F. Hansen, Sound preferences of persons with hearing loss playing an audio-based computer game, Workshop of ACM Multimedia 2013, IMMPD, pp. 25-30, 2013. [7] R. Hiraga, N. Kato, and N. Matsuda, Effect of visual representation in recognizing emotion expressed in a musical performance, Proc. IEEE SMC, pp. 131–136, 2008. [8] R. Hiraga and M. Matsubara, Appreciating Harmony –differences between the hearing-impaired, musically inexperienced, and musically experienced–, Proc. IEEE SMC, pp. 3464–3469, 2014. [9] R. Hiraga and K. Otsuka, On the recognition of Timbre, A first step toward understanding how hearing-impaired people perceive timbre, Proc. IEEE SMC, pp. 2013–2018, 2012. [10] J. R. Iversen and A. D. Patel, The Beat Alignment Test (BAT): Surveying beat processing abilities in the general population, Proc. ICMPC, 2008. [11] P. N. Juslin and P. Laukka, Expression, Perception and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening, Journal of New Music Research, 33:3, pp. 217–238, 2010. [12] N. Kano, M. Matsubara, H. Terasawa, and R. Hiraga, Developing a beat tapping game for hearing-impaired college students and its feasibility test, IPSJ, 2014-MUS-104(4), 2014 (in Japanese). [13] N. Kano, Development and evaluation of Tapping Game for hearingimpaired students–its short-term effect for hearing capability, the graduation thesis for School of Informatics, University of Tsukuba, 2015 (in Japanese). [14] S. Kirschner and M. Tomasello, Joint music making promotes prosocial behavior in 4-year-old children, Evolution and Human Behavior, 31, pp. 354–364, 2010. [15] N. Kraus and B. Chandrasekaran, Music training for the development of auditory skills, Nature Reviews Neuroscience 11, pp. 599–605, 2010. [16] C. J. Limb and A. T. Roy, Technological, biological, and acoustical constraints to music perception in cochlear implant users, Hearing Research, 308, pp. 13–26, 2014. [17] C. Mitani, T. Nakata, S. E. Trehub, Y. Kanda, H. Kumagami, K. Takahashi, I. Miyamoto, and H. Takahashi, Music Recognition, Music Listening, and Word Recognition by Deaf Children with Cochlear Implants, pp. 29S–33S, Ear and Hearing 28, 2007. [18] S. Moreno, C. Marques, A. Santos, M. Santos, and S. L. Castro, Musical Training Influences Linguistic Abilities in 8-Year-Old Children: More Evidence for Brain Plasticity, Cereb. Cortex, 19(3), pp. 712–723, 20009. [19] A. D. Patel, Can nonlinguistic musical training change the way the brain processes speech? The expanded OPERA hypothesis, Hearing Research, 308, pp. 98–108, 2014. [20] O. Sacks, Musicophilia: Tales of Music and the Brain, Revised and Expanded Edition, Vintage; Revised and enlarged edition, 2008. [21] E. G. Shellenberg and M. W. Weiss, Music and cognitive abilities, in D. Deutch Ed., The psychology of music, 3rd ed., pp. 499-550, Academic Press, 2012. [22] M. H. Thaut, Biomedical Research in Music in Rhythm, Music, and the Brain: Scientifc Foundation and Clinical applications (Studies on New Music Research), Routledge, 2007. [23] R. Torppa, A. Faulkner, J. J¨arvikivi, and M. Vainio, Acquisition of focus by normal hearing and cochlear implanted children: The role of musical experience, Proc. 5th International Conference on Speech Prosody, 2010. [24] E. Yucel, G. Sennaroglu, and E. Belgin, The family oriented musical training for children with cochlear implantes: speech and musical perception results of two year follow-up, Int’l J. Pediatr Otorhinolaryngol, 73(3), pp. 1043–1052, 2009.
