lie detection using electromyography and

0 downloads 0 Views 309KB Size Report
expressions, eyes and upper half compared with the lower half. ... the skin around the ear and tip of nose and produces crows feet wrinkles ( eye corners).
LIE DETECTION USING ELECTROMYOGRAPHY AND ELECTROENCEPHALOGRAPHY Anjali Arya1,Dinesh Bhatia2,Mohammed Abdelghani3 1,2

Department of Biomedical Engineering, Deenbandhu Chhotu Ram University of Science and Technology, Murthal,Sonepat, Haryana-131039 3

Biomedical Engineering Department, Florida International University, Miami-33174, USA

Email: [email protected] , [email protected], [email protected]

Abstract When people lie, it leads to changes in their facial expressions, skin conductance values, as well as in brain activity. It is also known that the results of facial expressions are controlled by facial muscles. However, no study has been done which establishes the facial EMG and the EEG signals from the brain to identify lying patterns. In this study, we introduce the concept of measurement of facial EMG and EEG when people lie including the type of electrode selection; their positioning and noise reduction. Extraneous noise can come from embarrassment or anxiety and not be specific to lying and can impact data. Additionally, physiological disorders can cause problems with data or errors in lead fabrication could make the technology unreliable. This paper points out potential of EEG and facial EMG for studying/analyzing the human emotions during lying through EMG and EEG. Such study/analysis may also provide deeper insight into the role of EEG in brain activity, EMG in facial expression in lie detection. Keywords:Electromyogram , brain-computer interaction, electroencephalogram, facial electromyography. I. INTRODUCTION Facial electromyography and electroencephalography signals are largely unresolved issues in brain-computer interface research [1]. fEMG refers to an electromyography technique that measures the muscle activity of the face by detecting and amplifying tiny electrical impulses that are generated by muscle fibers when they contract. The electroencephalography technique measures and records the electrical activity of the brain or detects abnormalities related to electrical movement of the brain or detect lies. This procedure track and records brain wave pattern. Human brain provides an interface for us to exchange the information with the real world [2]. All organs of the human body could sense and communicate to the external world. For example, the eyes can obtain the image of the surroundings, the ears can pick the sound, the tongue tastes the food savor, the nose can smell, and even the skin has the sense of touch [2]. In the same way, facial muscle can express the different emotions when people tend to lie along with changes in the brain activity. In the process of information passing, the facial muscles and brain activity play a dominant role to accomplish the information acquiring and message transmitting. We may therefore, reasonably conclude that measuring the EMG and EEG would provide more useful information.

II. OVERVIEW From the past decade the role of EMG is described in the field of facial expression as fEMG used in speech analysis measurements, emotional expressions, masticatory function evaluation etc [2]. Role of EEG is used in human emotions, truth telling and detecting deceit by recording the electrical conductivity of brain. In the present work a study related to lie detection using facial EMG and EEG is done. As we know EEG and fEMG does not depend upon language and does not require cognitive effort or memory. It is capable of registering the response even when subjects are instructed to inhibit their emotional expression. fEMG can detect or measure the facial deception on zygomaticus muscle regions, corrugator supercilii muscle regions, levator and epicranius. By using these regions EMG activity can be picked up from facial muscles to study lying patterns, whereas activity of brain during lying can be recorded by little metal discs with electrodes are placed on the scalp, and then send these accumulated signals to the computer for recording the results.

A. Indicators of lying There are some physiological indicators of lying that reveals facial clues such as timing, emotion and facial expressions, eyes and upper half compared with the lower half. If facial expressions are not synchronised with what someone is saying, he/she might be hiding something or lying. For example, if the moment calls for a smile and the smile is delayed and the timing is off or when someone is lying, he/she will show emotion when they have none, or mask one emotion with another or show no emotion (poker face). The upper half of a person's face is a much more reliable measure than the lower half. It generates more involuntary clues and is extremely difficult for someone to control [3]. Using smiling as an example, the most revealing clue if a smile is genuine or manufactured is in the eyes. The Zygomaticus major muscle (figure1) running from the cheekbone at an angle to the corner of the lips indicates a sincere smile. The most important facial expression is the smile, contains zygomatic muscles in which one muscle is for enjoyment expression, 1-3 for more complex expressions of enjoyment, moves cheekbones, lip corners, the skin around the ear and tip of nose and produces crows feet wrinkles ( eye corners).

Figure1 Zygomatic muscles [4]

In 2008, James Adwin Mahon stated that, a lie to the another person is, make a believed-false statement (to another person) with the intention that statement be believed to be true (by the other person), or with the intention that it be believed (by the other person) that statement is believed to be true (by the person making the statement), or with both intentions[5].

III. THE 10-20 SYSTEM FOR ELECTRODE PLACEMENT The 10-20 System of Electrode Placement is the method which is used to describe the location of electrodes which are implanted in the scalp as shown in figure2. The numbers 10 and 20 shows that the distance between adjacent electrodes are 10% or 20% of total front or right left distance. Each point on the left indicates an electrode position. Each site has an alphabet which identifies the lobe and a number or another letter which identifies the hemisphere location. The alphabets F, T, C, P, and O abbreviated as Frontal, Temporal, Central, Parietal and Occipital. There is no "central lobe", but it is just used for only identification purposes. Even numbers (2,4,6,8) refers to the right hemisphere and odd numbers (1,3,5,7) refers the left hemisphere. The z refers an electrode which is placed on the median. The smaller in the number, closer is the position to Median.

Figure 2 Electrode placement (The “10” and “ 20” refer to the 10% or 20% interelectrode distance.) [6]

IV. EEG WAVE PATTERNS EEG waveforms are generally classified according to their frequency, amplitude, and shape, as well as the sites on the scalp at which they are recorded. The most familiar classification uses EEG wavefroms frequency are alpha, beta, theta, and delta having frequency range:Delta- 3Hz or below, Theta- 3.5-7.5Hz, Alpha-7.5-13 Hz, Beta - 14 or greater Hz.

Figure 3 EEG Waves [7] According to Danny Oude Bos, the need for computer applications which can detect the current emotional state of the user is ever growing. In an effort to copy human communication, research has already been done into recognizing emotion from face and voice. Humans can recognize emotions from these signals with a 70-98% accuracy, and computers are already pretty successful especially at classifying facial expressions (80-90%). Emotions are not just what is displayed. Facial expression and voice concern the aspect of human emotion: the expression. This can be consciously adapted, and its interpretation is not objective [8]. For this reason, research has been conducted to look at the physiological aspects like the user’s heart rate, skin conductance, and pupil dilation. With the rising interest for brain-computer interaction (BCI), user’s EEGs have been analyzed as well. Whether the EEG just shows a physiological response or also gives insight into the emotion as how it is experienced mentally is still unclear. IV. REVIEW & RECENT DEVELOPMENTS Alan J. Fridlund et. al. (1986) has worked on the guidelines for human electromyographic research for the collection, analysis and description of electromyographic data [9]. Carlo J. De luca (1997) explored various uses of surface electromyography in the field of biomechanics and explained the three groups of applications i.e. the activation timing of muscles, the force and EMG signal relationship, the use of EMG as a fatigue index to provide assistance for the proper detection, analysis and interpretation of theEMG signal and measure force [10]. Carlo J. De luca (2002) explained the concepts related to the electrode geometry, placement and electrical safety concerns during detection and recording of surface electromyography[11]. Cheng-Ning Huang et. al. (2004) proposed many measurements and applications in facial electromyography on masticatory function evaluation , speech analysis and recognition, and emotional expression observation. In addition also introduce the measurement of facial EMG including the electrode selection; electrode position and noise reduction [2]. Dingcheng Wu1et. al (2011) examined the neural correlation of evaluations of both lying and truth-telling in different social contexts using fMRI methodology. The results demonstrated the differentiation between lying and truth-telling and between different types of lying in a network of brain regions. These regions included bilateral superior frontal gyrus (SFG), bilateral

inferior parietal lobule (IPL), bilateral cuneus, right lingual gyrus (LG), right precuneus, and left postcentralgyrus (PoCG) [12]. Paul Root et. al (2005) examined the detection of deception and confirmation of truth telling with conventional polygraphy [13]. Xinyi Yong et. al (2008) concluded that the fEMG contamination of the electroencephalography (EEG) signals is a largely unresolved issue in BCI research [1]. M Murugappan et. al (2010) established a new communication channel for physically immobilized people to interact with the outside world through their brain waves in BCI. Their main objectives holds (i) to review the previous works on human emotion detection using EEG (ii) to design a audiovisual induction based data acquisition protocol for data collection and (iii) to propose the new time-frequency analysis based features for emotion detection [14]. M.C. Kolkman (2012) explained that deception is part of everyday life and the detection of deception has been subject to lots of research [15]. This research looked into the ability of humans to integrate different indicators into one assessment. Not only was looked at how good people can detect a lie, but also on which indicators people would base their decisions. No significant results were found, which indicates most people are just not good at spotting lies, whatever indicators are at their disposal. JoseMiguel Fernandez-Dols (2013) has worked on the advances in the study of facial expressions,this special section of Emotion Review includes reviews on the physical, social, and cultural dynamics of expressions, and on the complex ways in which, throughout the lifespan, facial behavior and emotion are perceived and categorized by primates and humans brain [16]. All these advances are certainly paving the way for new exciting approaches to facial behavior more likely to strike an appropriate balance between description and explanation. V. METHODOLOGY The experiment was performed with 10 subjects, with age group between 18-28 years. The subjects were informed about the purpose of the study. Four facial muscles were selected for performing the experiment. The EMG electrodes were placed over the zygomaticus muscle regions, corrugator supercilious muscle regions, levatorandEpicraniusmajorwith the reference electrodes placed on the hands of the patients. The EEG electrodes were placed according to the 10-20 electrode placement system on the patient's scalp. The questions were asked from the subjects in which the probability of lying was maximum. A four channel, portable BIOPAC MP 100 equipment was used for recording the fEMG which works on Acknowledge 3.9.1 software. The sampling rate used for raw EMG signal recording was set to 2000 samples/second/channel. The sampling rate was kept on the high side to avoid the aliasing effects. The frequency range of EMG signals was in the range 0Hz-500Hz. The male subjects were requested to shave their facial hair to avoid the erroneous signal recording. The required facial area was cleaned with ethanol. The surface electrode (Ag/AgCl) was used for fEMG recording. The electrodes were placed in a bipolar symmetry. The distance between two consecutive electrode was kept 1 cm for the whole experiment to minimize noise from the signal. For recording of EEG, Polysomnogrsphy Hardware System RMS -32 with 32 channels was used for simultaneous data acquisition. The software used was SUPERSEC which consists of two software- Acquire and Analysis. EEG signal of 10 healthy persons was acquired. After the acquisition of EEG signal, different digital signal processing techniques like digital filter processing, fast Fourier transform, autocorrelation, cross correlation were used to process the EEG signal.

Figure 4 Electrode placement for the experiment work.

VI. RESULTS AND DISCUSSION The results of the experiment present the relative activities of brain and selected dominant facial muscles involved in lying. Figure 5 shows the EEG waves during false telling for 10 male subjects. In this figure, it can be seen that delta wave is most efficient during lying in this research study. It covers 67.02% area whereas others like, theta wave covers 15.52%, alpha wave covers 10.55%, beta wave covers 6.79% and gamma wave covers 0.13% of EEG wave patterns.

17.414895

0.336225182

27.0533886

Delta Theta Alpha

39.80666351

Beta 171.9311218

Gamma

Figure 5 Pie chart showing the effect of lie on EEG wave patterns

Figure 6 and 7, shows the autocorrelation of 10 male subjects in between both lying and truth telling for various muscles. It is the cross-correlation of a signal with itself. Informally, it is similarity between observations as a function of the time separation between them. Auto-correlation helps identify signal features buried in noise. In this research study, Autocorrelation of following muscles like zygomatic, levator, corrugators and epicranius muscle has been done, in which zygomatic muscle having maximum autocorrelation value is 0.3 whereas levator muscle having maximum value in between 0.08 to 0.1.

Autocorrelation 0.4 0.3 0.2

Autocorrelation

0.1 0 SUB2

SUB4

SUB6

SUB7

SUB9

SUB10

Figure 6 Plot of Auto correlation values of subjects for the Zygomatic muscle

Autocorrelation 0.1 0.08 0.06 Autocorrelation

0.04 0.02 0 SUB1

SUB2

SUB3

SUB6

SUB7

SUB8

Fig. 7 Plot of Auto correlation values of subjects for the Levator muscle

Table 1. Minimum and maximum correlation values for selected channels CHANNEL F3-C3 F4-C4 T3-T5 T4-T6 T5-O1 T6-O2 F8-T4 F7-T3

MIN 0.28 0.16 0.05 0.12 0.05 0.07 0.16 0.51

MAX 0.96 0.96 0.97 1 0.96 1 0.93 0.96

Table1 shows the minimum and maximum value of a dataset. Power spectral density of continuous analogue EEG signal is a set of approximately 1000 values. Minimum and maximum function gives the minimum and maximum values out of such a large set of correlation data. This indicates that during lying and truth telling experiment from subject to subject, similar brain activities was found. From the large set of correlation data, we defined minimum and maximum correlation values for various channels.

VII. CONCLUSION Facial EMG and EEG both are precise and sensitive methods to measure emotional expression. They have wide scope and there are many areas where this technique has been used. Using fEMG and EEG we may be capable of registering the response even when subjects were instructed to inhibit their emotional expression during lying. It has been studied to assess its utility as a tool for measuring facial emotional reactions and brain mapping activities. This also can improve accuracy and reliability for EMG and EEG in studying facial expression and brain activities. Overall, this research study provided a valuable experience in the field of facial EMG and EEG. REFERENCES [1] Xinyi Yong, Rabab K Ward and Gary E. Birch, “Facial EMG Contamination of EEG Signals: Characteristics and Effects of Spatial Filtering’’,ISCCSP, Malta, March 2008, pp. 12-14. [2] Cheng-Ning Huang, Chun-Han Chen and Hung-Yuan Chung,“The Review of Applications and Measurements in Facial Electromyography”,Journal of Medical and Biological Engineering, 2004, vol.25, pp. 15-20. [3] http://www.selfgrowth.com/articles/3 Ways to Spot a Liar by their Facial Expression.html. [4] http://classconnection.s3.amazonaws.com/657/flashcards/1422657/jpg/zygomaticus1335297574274.jpg [5] James Edwin Mahon, “ Two Definitions of Lying”, International Journal of Applied Philosophy, 2008, pp. 211– 230. [6] http://www.hindawi.com/journals/aai/2011/384169.fig.002.jpg [7] http://www.youcanstaysharp.com/fileadmin/user_upload/eeg_traces.gif [8] http://hmi.ewi.utwente.nl/verslagen/capita-selecta/CS-Oude_Bos-Danny.pdf [9] A.J. Fridlund and J.T. Cacioppo,“Guidelines for Human Electromyography Research”, The Society of Psychophysiological Research inc., 1986, vol.23, no.5,pp.567-589. [10] Carlo J. De Luca, “The Use of Surface Electromyography in Biomechanics”, Journal of Applied biomechanics, 1997, vol.13, no.2, pp. 135-163. [11] http://www.delsys.com/Attachments_pdf/WP_Sampling1-4.pdf [12] Dingcheng Wu1, Ivy Chiu Loke, FenXu, and Kang Lee, “Neural Correlates of Evaluations of Lying and Truth-Telling in Different Social Contexts’’,Brain Res.,2011, 1389: 115–124. [13] Paul Root Wolpe, Kenneth R. Foster and Daniel D. Langleben, “Emerging Neurotechnologies for LieDetection: Promises and Perils’’,The American Journal of Bioethics,2005, 5(2): 39–49. [14] M Murugappan, M Rizon, R Nagarajan, S Yaacob, “Inferring of Human Emotional States using Multichannel EEG’’, European Journal of Scientific Research, 2010, Vol.48 No.2. [15] http://referaat.cs.utwente.nl/conference/16/paper/7299/deception-detection-importance-of-modality.pdf [16] Jose-Miguel Fernandez-Dols,“Advances in the Study of Facail Expression: An Introduction to the Special Section”,Emotion Review, 2013,Vol.5No.1. [17] Gary E. Schwartz,“Facail Expression and Imaginary in Depression: An Electromyographic Study”, Psychosomatic Medicine ,1976,vol.38, No.5.

AUTHORS BIOGRAPHY Anjali Arya is a post graduate student in Biomedical Engineering at Deenbandhu Chhotu Ram University of Science and Technology, Murthal, Sonepat, Haryana. She completed her bachelors in Biomedical Engineering from the same institute in 2010. Her areas of interest are image processing, digital signal processing, electromyography, electroencephalography. Currently she is working on lie detection using facial electromyography and electroencephalography.

Dinesh Bhatia pursued his PhD in Biomechanics and Rehabilitation Engineering from MNNIT, Allahabad, India with bachelor’s and master’s degree in BME. He is employed as an Assistant Professor (Sr. Grade) at the Biomedical Engineering Department, Deenbandhu Chhotu Ram University of

Science and Technology,

Murthal (Sonepat), Haryana, India. He was selected for the young scientist award in (2011-12) by Govt. of India to pursue further research at Adaptive Neural Systems Laboratory, Florida International Univ, Maimi, USA. He has several research papers in reputed journals with teaching and research experience of more than ten research focuses on understanding muscle

mechanics, joint kinematics and dynamics involved in performing

locomotion and routine tasks and undermining it effects during an injury or disease.

Full Address of Authors

1. Anjali Arya P.G Scholar Department of Biomedical Engineering, J.C. Bose Block, Deenbandhu Chhotu Ram University of Science and Technology Murthal (Sonepat) Haryana-131039, India Email-ID: [email protected]

2. Dr. Dinesh Bhatia Assistant Professor Department of Biomedical Engineering, J.C. Bose Block, Deenbandhu Chhotu Ram University of Science and Technology Murthal (Sonepat) Haryana-131039, India Email: [email protected]

years. His