Novel Tools for Quantification of Brain Responses to Music Stimuli O. Sourina1, V.V. Kulish 2 and A. Sourin3 1 2
Nanyang Technological University/School of Electrical and Electronic Engineering, Singapore Nanyang Technological University/School of Mechanical and Aerospace Engineering, Singapore 3 Nanyang Technological University/School of Computer Engineering, Singapore
Abstract — Therapeutic effect of music is known for long time. Music therapy can be used to improve the current patient’s state or even prevent some deceases of psychosomatic origin. For decades, the basic research methods of music psychology mainly relied on survey with questionnaire or on observations of subjects. Investigating the effect of music through electroencephalograms (EEG) could be more precise and objective. In this paper, we describe fractal dimension model for quantification of brain responses to external stimuli. Human EEG evoked by music stimuli are analysed with the developed software. The proposed technique of processing EEG is based on the Rényi entropy – the concept of generalized entropy of a given probability distribution. It allows defining generalized fractal dimension of EEG. The experiments involved ten subjects, both male and female twenty years old university students. The EEGs were processed by the fractal dimension analysis tools. Analysis of fractal dimension values demonstrates that the method can be used to quantitatively distinguish between different states of the brain. Correlation between music, and subject’s emotion and concentration was studied. Few hypotheses were validated. Keywords — EEG, music therapy, brain response quantification, fractal dimension, emotions.
I. INTRODUCTION For decades, the basic research methods of music psychology is no different from other disciplines in social science domain, mainly relying on survey with questionnaire and attaining results through statistical approach, or otherwise through the observation of subjects. Investigating the effect of music through EEG was still new to the research community until recent times. Now, combination of the accomplishment of past study of music psychology with EEG-based research is already bearing fruit, as therapeutic effect of music is now widely recognized and music is used for treatment of neural abnormality or mental disorder. Modeling of brain activity with EEG signals is more precise and objective than survey, because the subject being surveyed might not be fully aware of his/her psychological and emotional status when answering questions. On the other hand, EEG signals would not be altered by the subject’s own judgment. Therefore this approach would be more used in the coming years.
EEG signals are the records of electrical potential produced by the brain along with its activities. Quantification of responses of human brain to various external stimuli is a research area that can help to solve a mystery of the human brain. Researchers attempt to create tools and models to study and describe the behaviors and nature of EEG signal. The current methodology is to analyze real-time EEG readings in frequency domain. But there is a new trend for EEG signal processing as moving from linear Fast Fourier Transform (FFT) to non-linear Fractal Time Series analysis to study dynamic nature of EEG signals. The analysis of EEG data in fractal dimension domain is based on its geometric pattern, rather than frequency content. In our works [1-3], we proposed to process EEG signals using fractal dimension model and implemented novel algorithm to process EEG signals evoked by odor stimuli. We studied brain responses to six basic olfactory stimuli given to the subjects in our experiments. In this paper, we describe results of experiments with music used as external stimuli. The psychological effect of music measured in terms of EEG responses is studied. In this paper, we describe fractal dimension analysis model and apply it to analyze EEG evoked by music stimuli. We extend fractal dimension model with dynamic component and develop a novel algorithm and software system. The fractal dimension changing over time is visualized dynamically on the chosen EEG channels. The experiments are conducted with music used as the external stimulus, and the EEG response of the subjects, university students, are recorded and subjected to fractal dimension analysis. The analysis results are plotted graphically for intuitive investigation and visualized dynamically with the developed visualization software. Few hypotheses were validated. In Section II, an introduction to fractal dimension analysis is given. Section III describes the experiment setting and analyzes experiments results. Section IV summarizes the outcomes of the project and provides recommendations for the future development.
Chwee Teck Lim, James C.H. Goh (Eds.): ICBME 2008, Proceedings 23, pp. 411–414, 2009 www.springerlink.com
412
O. Sourina, V.V. Kulish and A. Sourin
II. FRACTAL
ANALYSIS MODEL DESCRIPTION
pi
Fractal dimension analysis can be generally applied to non-linear systems. It basically tells how many points lie in the set and how scattered they are or how “strange” is a geometric structure [4]. A common practice to distinguish among possible classes of time series is to determine their so-called correlation dimension. The correlation dimension, however, belongs to an infinite family of fractal dimensions [5]. Hence, there is a hope that the use of the whole family of fractal dimensions may prove to be advantageous in comparison with using only some of these dimensions. The concept of generalized entropy of a probability distribution was introduced by Alfred Rényi [6]. Based on the moments of order q of the probability pi, Rényi obtained the following expression for entropy Sq
1 log q 1
N
¦
piq
i 1
(1)
where q is not necessarily an integer and log denotes log2. Note that for q o 1 , Eq. (1) yields the well-known entropy of a discrete probability distribution [7]
lim
ti
T of T
(5)
where ti is the time spent by the signal in the ith bin during the total time span of measurements T. Further, the generalized fractal dimensions of a given time series with the known probability distribution are defined as N
log Dq
¦p
1 i 1 q 1 log GV
lim
GV o0
q i
(6)
where the parameter q ranges from f to f . Note that for a self-similar fractal time series with equal probD D0 for abilities pi 1 / N , the definition Eq. (6) gives q all values of q [8]. Note also that for a constant signal, all probabilities except one become equal to zero, whereas the remaining probability value equals unity. Consequently, for D D0 0 . The fractal dimension a constant signal, q
log N log GV
D0
(7)
N
S1
¦ p log p i
i
i 1
(2)
The probability distribution of a given time series can be recovered by the following procedure. The total range of the signal is divided into N bins such that Vmax Vmin GV
N
Ni N of N
mension N
(3)
where Vmax and Vmin are the maximum and the minimum values of the signal achieved in the course of measurements, respectively; GV represents the sensitivity (uncertainty) of the measuring device. The probability that the signal falls into the ith bin of size GV is computed as pi
is nothing else but the Hausdorff-Besicovitch dimension [8]. The correlation dimension, mentioned previously, is the fractal dimension with q = 2. As q o 1 , Eq. (6) yields the so-called information di-
D1
i
GV o0
Df
log(1 / GV )
(8)
log p max log GV
(9)
log p min GV o0 log GV
(10)
lim
GV o0
(4)
where N i equals the number of times the signal falls into the ith bin. Alternatively, in the case of a time series, the same probability can be found from the ergodic theorem, that is
i
i 1
where the numerator is Shannon’s entropy given by Eq. (2). Note also that
lim
_______________________________________________________________
lim
¦ p log p
and
IFMBE Proceedings Vol. 23
D f
lim
_________________________________________________________________
Novel Tools for Quantification of Brain Responses to Music Stimuli
such that Df t Df . In general, if a < b, Da t Db , D such that q is a monotone non-increasing function of q D [7]. For a given time series (“signal”), the function q , corresponding to the probability distribution of this signal, is called the fractal spectrum. Such a name is well-justified, because the fractal spectrum provides information about both frequencies and amplitudes of the signal. This definition of fractal dimension covers a very large family corresponding to different q values. When q = 0, it is the Hausdorff-Besicovitch Dimension. When q =1, it is socalled the information dimension; the entropy will become Shannon’s entropy. The correlation dimension is the fractal dimension with q =2. As EEG signals are not stationary traces, but a dynamic and time varying measure, it is necessary to reveal the history of fractal dimension in time. Hence an algorithm is implemented to calculate a time-series dynamic fractal dimension data. Thus, the introduction of fractal dimension analysis is to change the way of EEG signal analysis from frequency perspective to geometric or pattern-based perspective. The described equation for general fractal dimension model is for a sample set of fixed number of data entities N, or can be otherwise called a static model. But in real application, the samples set size would be accumulating all the time to accommodate incoming data. Therefore, we proposed and implemented novel algorithm to handle sample set of growing size or in this sense a dynamic one.
413
“sticks” showing which channels are active and how active they are during listening music. The subjects investigated in our experiments were the university students, both males and females 22-25 years old. The results processing confirmed our hypothesis that it is possible to recognize happiness and sadness emotions by computing fractal dimension values. By processing the EEG during experiments it was also proved that fractal dimension model can be used to differ between concentration/relaxation states of the mind. Hard Rock music showed the biggest fractal dimension values for all subjects throughout the whole experiment. It was also supported by the interview conducted during the experiment that such song gives the most disturbing effect. The subjects were required to answer the survey questionnaires before and after the experiments.
III. EXPERIMENTS AND RESULTS As human brain is processing in a tremendous speed, the real time EEG signal should be sampled in a reasonably high sampling frequency. In our project, the equipment used for EEG signal recording, called MINDSET24, records 256 EEG samples per second for every single channel. There are 24 channels in total covering the entire skeleton sculp. We proposed and conducted experiments using different music stimuli. We had series of experiments spiritual (religious), hard rock music, classical music, and hip-hop music. The questionnaire was used to record emotional state of subjects to discover correlation between fractal dimension values and mental state induced by music. The results of experiments were processed with the implemented dynamic fractal dimension algorithm and were visualized as graph changing over time/frequency. Fractal dimension changing over time can be also visualized with the developed Brain Visualization system. In Fig. 1, fractal dimension values are represented by changing in heights
_______________________________________________________________
Fig. 1 Visualization of fractal dimension in 6 active channels with VisuBrain
The following questions were asked: x x x x
Do you have any musical background? What genre of music do you like to listen? What emotion does the music evoke? Have you heard the song before?
The subjects were given the memory tests during experiments to assess their level of concentration when they were listening different types of music. The following observations were drawn after processing the results. Subjects generally performed better in the memory test while listening to the songs which induce feeling of happiness. Subjects
IFMBE Proceedings Vol. 23
_________________________________________________________________
414
O. Sourina, V.V. Kulish and A. Sourin
have the lowest score when they are listening to songs which they find disturbing or irritating. Jazz song was the most disturbing to the subjects, followed by hard rock song, except for one subject who named hard rock music as favorite. All the subjects indicated that classical songs make them feel happy and comfortable. Music which evoked positive emotions such as happiness, acceptance and pleasantly surprise resulted in subjects scoring better for the memory test. Music which brings negative emotions such as anger and distress generally resulted in the subjects performing their worst in the memory test. After processing both EEG and survey questionnaires the following hypotheses have been confirmed: x x x x x x
The subjects’ concentration levels will show significant differences among different genres of songs. The subjects’ EEG responses do not depend on gender of subjects. The subjects’ EEG responses depend on music familiarity. Music which evokes positive emotions will results in better concentration as compared to music evoking negative emotions. The subjects’ levels of concentration depend on their preferred genre of music. The subjects’ EEG responses depend on music background of subjects. Subjects that took music clases have higher fractal dimension values in all experiments.
a person’s emotional state can influence a person’s level of concentration. In our experiments, it was confirmed that it is possible recognize happiness and sadness emotions by computing fractal dimension values that could be used in development of music therapy applications. We believe that fractal dimension model can be used for quantification of brain responses to develop personally adaptive music therapy program. Further research and experiments should be carried out. The more conclusive results could be achieved in the case of more careful choice of subjects, regarding their musical affinities and cultural background. This field of research is still relatively new, and there is still much to be done to improve on emotions quantification models and algorithms. The list of basic emotions to be studied should be extended as well.
REFERENCES 1.
2.
3.
4.
The following hypotheses have not been confirmed: x x
5.
Music which brings relaxation will result in the highest level of concentration. The subjects perform better on memory test while listening to any music. Some other hypotheses were inconclusive.
7. 8.
IV. CONCLUSIONS Music stimulus could be used for subject’s concentration and to put subjects into different emotional states. In our experiments, the highest level of concentration was evoked by music that evokes positive emotions. Since concentration is relative to emotions, it is therefore reasonable to say that
_______________________________________________________________
6.
Kulish V, Sourin A, Sourina O (2005) Human electroencephalograms seen as fractal time series: mathematical analysis and visualization. Computers in Biology and Medicine 36:3, 291-302 Kulish V, Sourin A, Sourina O (2006) Analysis and visualization of human electroencephalograms seen as fractal time series. Journal of Mechanics in Medicine & Biology 26:2, 175-188 Kulish V, Sourin A, Sourina O (2007) Fractal spectra and visualization of the brain activity evoked by olfactory stimuli, Proc. of the 9th Asian Symposium on Visualization Hong Kong, 4-9 June, pp.37-1 37-8 Moon F C (1992) Chaotic and fractal dynamics: an introduction for applied scientists and engineers. John Wiley & Sons Hentschel H G E., Procaccia I (1983) The infinite number of generalized dimensions of fractals and strange attractors, Physica, 8D, 435444 Rényi A (1955) On a new axiomatic theory of probability, Acta Mathematica Hungarica, 6, 285-335 Shannon C E (1998) The mathematical theory of communication. University of Illinois Press Schroeder M R (1991) Fractals, chaos, power laws. W. H. Freeman & Co Author: Institute: Street: City: Country: Email:
IFMBE Proceedings Vol. 23
Olga Sourina Nanyang Technological University 50 Nanyang Crescent Singapore Singapore
[email protected]
_________________________________________________________________
