Non-contact heart rate and heart rate variability ...

Biomedical Signal Processing and Control 13 (2014) 102–112

Contents lists available at ScienceDirect

Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc

Review

Non-contact heart rate and heart rate variability measurements: A review J. Kranjec ∗ , S. Beguˇs, G. Gerˇsak, J. Drnovˇsek Faculty of Electrical Engineering, University of Ljubljana, Trˇzaˇska cesta 25, Ljubljana, SI, Slovenia

a r t i c l e

i n f o

Article history: Received 29 May 2013 Received in revised form 10 March 2014 Accepted 20 March 2014 Keywords: Heart rate Heart rate variability Non-contact ECG Radar Doppler Camera

a b s t r a c t The following paper investigates published work on non-contact human physiological parameter measurement, more precisely measurement of the human heart rate (HR) and consequently the heart rate variability (HRV), which is considered to be an important marker of autonomic nervous system activity proven to be predictive of the likelihood of future health related events. The ability to perform measurements of cardiac activity in a non-contact manner could prove to become an important alternative to the conventional methods in the clinical field as well as in the more commercially oriented fields. Some of the published work so far indicates that the measurement of cardiac activity in a non-contact manner is indeed possible and in some cases also very precise, however there are several limitations to the methods which need to be taken into account when performing the measurements. The following paper includes a short description of the two conventional methods, electrocardiogram (ECG) and photoplethysmography (PPG), and later on focuses on the novel methods of non-contact measuring of HR with capacitively coupled ECG, Doppler radar, optical vibrocardiography, thermal imaging, RGB camera and HR from speech. Our study represents a comparative review of these methods while emphasising their advantages and disadvantages. © 2014 Elsevier Ltd. All rights reserved.

Contents 1. 2. 3. 4.

5.

6. 7.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heart rate and heart rate variability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Challenges and limitations in measuring the HRV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring of cardiac activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1. Conventional methods for cardiac activity measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Experimental non-contact methods for cardiac activity measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology of IBI measurement with different methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1. HR from speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Thermal imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3. RGB imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4. HR measurements based on Doppler effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1. Optical vibrocardiography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2. Doppler radar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5. Capacitively coupled ECG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

∗ Corresponding author. Tel.: +386 14768245. E-mail addresses: [email protected] (J. Kranjec), [email protected] (S. Beguˇs). http://dx.doi.org/10.1016/j.bspc.2014.03.004 1746-8094/© 2014 Elsevier Ltd. All rights reserved.

103 103 103 104 104 105 105 105 106 106 107 107 107 108 108 111 111

J. Kranjec et al. / Biomedical Signal Processing and Control 13 (2014) 102–112

103

1. Introduction Measuring of human physiological parameters on a regular basis out of the hospitalisation period could become an important feature in health care, affecting healthcare policies and healthcare economics on the one hand and our daily life on the other. During the past few years a lot has been learned about diseases at a genomic level, creating possibilities of an early detection of illness symptoms and improving the treatment process itself. Amongst other findings, numerous studies have shown a significant relationship between the autonomic nervous system (ANS) and cardiovascular mortality. More precisely, perturbations of the ANS and its imbalance were discovered to indicate impending cardiac diseases, which may lead to a sudden cardiac death, one of the leading causes of cardiovascular mortality [1]. The ANS function is necessary for the maintenance of homeostasis. It operates independently of voluntary control through the sympathetic and the parasympathetic nervous systems which often function in an antagonistic manner. The autonomic processes are involved in the control of many bodily functions, such as thermoregulation, blood pressure, regional blood flow, etc. The status of the ANS can therefore be assessed by observing several physiological parameters which can be obtained and processed with different measuring and analytical methods [2]. One of the markers for ANS assessment that has caught the attention of the profession is called the heart rate variability (HRV). Next to the clinical settings (e.g. diabetic neuropathy, myocardial infarction, sudden cardiac death, etc.) this parameter is also used in several other fields, such as sports science and ergonomics [3,4]. HRV is a measurement of the oscillation between adjacent QRS complex intervals as well as the oscillations between consecutive instantaneous heart rates. Due to the seemingly easy derivation of the parameter, its use has been popularised with many (commercial) measuring devices providing automated HRV measurement [1]. However, the significance and meaning of the parameter analysis is more complex then generally appreciated. Furthermore, incorrect conclusions may lead to excessive extrapolations, which were one of the main reasons for the constitution of the Task Force back in 1996, responsible among other things for defining of measurement standards, result interpretations and identification of areas for future research [1]. The cardiac data used for further HRV analysis are generally obtained with one of the following two methods practised in clinical environment: electrocardiogram (ECG) or photoplethysmography (PPG). Although based on a different concept and measuring different phenomena, both methods provide reliable results when properly executed on the one hand, but are limited by several factors deriving mostly from the need of physical contact with the subject on the other hand. These limitations combined with increasing demands for ubiquitous measuring of human physiological parameters inside and outside of the hospital environment on the one hand and the possibility for use in commercial settings on the other hand has led researchers worldwide to search for a way to optimise the measuring process, freeing it from its limitations. As a result, several promising and innovative methods have been published. They can roughly be grouped into measuring methods using “contact” electrodes, “fixed-in-the-environment” electrodes and “non-contact” electrodes. The acquisition of cardiac activity parameters in a non-contact manner could become a valuable tool in clinical health care applications as well as in the non-clinical environment. In an ideal measuring setting, the subject would not be aware of the measuring process itself, which would result in a decreased psychological factor of the measurement. Next to eliminating several limitations of the contact sensor based methods, such measurements would therefore also result in more objective readings. The main objective

Fig. 1. R–R interval; an interval between adjacent QRS complexes of a normal sinus depolarisation.

of this paper is to review the published novel and experimental non-contact measuring methods for measuring of heart rate (HR) and HRV. Additionally, we present a comparative review of the discussed novel methods, emphasising their advantages and disadvantages. 2. Heart rate and heart rate variability HR is defined as the rate of occurrence of cardiac beats in a specific period of time, usually expressed in beats per minute. Although the occurrence of cardiac beats could be triggered by the electrical pulses generated within the sinoatrial (SA) node, the actual frequency of heart’s electrical and contractile activity is in the most part modulated by the ANS. This neural regulation causes variability in the HR in the active as well as the resting state. The variability should be high in the normal physiological state of an individual and should only erode with age or progression of the disease [1,5]. The temporal variation between sequences of consecutive heart beat intervals is defined as the HRV. The parameter has been recognised as a marker reflecting the activity of the ANS components on the sinus node of the heart. On a normal ECG reading, the standard upwards deflection of a QRS complex is at the peak of the R wave. The duration between two adjacent R waves is defined as the R–R interval (Fig. 1). The HRV is evaluated in time and frequency domain or lately with the help of nonlinear dynamics based on measurements of the N–N (normal–normal) intervals, which is the resulting period between adjacent QRS complexes after the process of removal of all non-sinus node originating beats. The parameter provides a powerful means of observing the interplay between the sympathetic and parasympathetic nervous systems and is thus an extremely useful parameter for assessing and even understanding of the ANS status [1,3,5]. A variable heart rate is considered to be present in a normal physiological state of an individual. The parameter is influenced by many different factors, such as gender, age, physiological and psychological condition, drug interferences, etc. [1,3,5]. As a result, the parameter’s value can differ greatly for specific groups of patients. Nevertheless, a general loss of variability that can be caused by various physiological and pathological processes is considered to be associated with an increased mortality in patients post myocardial infarction (MI). In fact, patients with persisting low HRV have been found to have up to three times greater mortality compared to those that have HRV within normal levels [1,5]. 3. Challenges and limitations in measuring the HRV Despite the vast amount of available literature and many experimental studies on the HRV measurement, its use is still somehow limited to a research technique rather than a clinical tool. There are several reasons contributing to this fact, amongst other the absence of a specific therapy for prognosis improvement, the lack of standardised methodology for parameter assessment due to the variability of parameters (e.g. gender, age, drug interferences, etc.), lack of consensus about the most accurate HRV parameter for clinical use, etc. [3,6].

104


As already mentioned, the HRV analysis is based on the measurement of N-N intervals, which is the period between adjacent QRS complex resulting from sinus node depolarisations [5]. Due to the dynamics of the cardiovascular system and consequent instantaneous changes within the HR and other parameters, locating these complexes is difficult in healthy subjects and especially difficult with subjects suffering from certain heart-related conditions [3,5]. The intervals are typically obtained within a clinical environment by using the ECG or the PPG, which are limited by several factors. The main limitation of both measuring methods is the need for direct contact of the sensor with the subject. Due to this need for physical contact, the measurement process may be annoying, unpleasant or even inappropriate for specific groups of patients (e.g. infants and patients with burns). It may also be time dependent (drying of conductive ECG gel) or cause allergic reactions to skin and is prone to signal noise due to movement artefact. Furthermore, the presence of cables can considerably limit the subject’s mobility for the complete duration of the monitoring period. While the decreased mobility can be overcome by exercising a measurement with a wireless ECG sensor [7,8], the rest of the mentioned drawbacks remain present to a certain level. The data acquisition of the cardiovascular parameters is usually done in a clinical environment and is carried out by a physician. The setting of the measurement process combined with the awareness of measurement itself can have a negative effect on an individual. In many cases this would result in a certain degree of stress and/or anxiety of the subject, which can affect the value of the measured physiological parameters. Some studies report that the autonomic function reflected by the HRV is to a certain degree affected by specific acute stress, e.g. white coat hypertension and masked hypertension [9–11]. The bioelectrical signals are by definition of a non-stationary nature and are very low in amplitude and frequency. In order to be able to perform an effective statistical diagnostics over measurement results and consequently recognise possible pointers of a certain pathological condition, a study of measured signals may have to be carried out on measured data obtained over a longer period of time. In the case of HRV, it is important to know that its total variance is time dependent as the parameter increases with the length of the recordings. Therefore, statistical variables obtained from recordings of same durations must be compared. With this reason, a 5 min short-term recording and a 24 h nominal long-term recording were recognised as appropriate standard options for further data analysis [1,5]. The heart’s electrical activity is easier to obtain compared to other bio-electric signals. The amplitude value of the signals is however still relatively small and as such prone to various interferences (e.g. electromyogram (EMG) signal interferences, power line interferences, baseline wander, etc.). Such noise can in some cases prove to be dominant over the actual signal, thereby causing contamination by altering the original signal and masking the heart signal morphology [12]. Prior to the HRV analysis, these interferences need to be minimised in the data acquisition stage. Furthermore, any noise within the obtained signal must be removed in order for the R-waves of the QRS complex to be accurately located with a robust R-wave detector algorithm. In the process of data acquisition, a good signal quality and elimination of background noise is of essential importance for further HRV analysis. Based on the measuring method, the data acquisition itself can be optimised by correct positioning of the measuring system and by applying several conditioning steps to the acquired raw signal. These conditioning steps include the use of appropriate filter, signal rectifier to prevent errors due to wrong polarity of the sensor, a mechanism for fast adaptation to threshold changes in R wave amplitude, etc. After the process of data acquisition is completed, the R waves are detected and the HRV is analysed by

applying appropriate mathematical algorithms over the obtained data [12–17]. 4. Monitoring of cardiac activity Monitoring of cardiac activity represents one of the most important and commonly observed parameters in vital sign monitoring domain. It represents a routine part of any complete medical evaluation due to the heart’s essential role in human health and disease on the one hand, and the relative ease of recording and analysing on the other hand. Because a healthy heart makes a specific pattern of waves on the recording, a damaged or diseased heart changes that pattern in recognisable ways. By examining the recordings, a physician can detect and analyse possible abnormalities which straightforwardly indicate the condition of the patient’s heart (e.g. atrial fibrillation, arrhythmia, etc.). With an appropriate analysis of the recording, a physician can also obtain other information that carry possible future health-related events (e.g. HRV as the quantitative marker of autonomic activity [1]). The following chapters give a short overview of the two conventional methods for cardiac activity measurement, followed by the introduction to the experimental non-contact methods. 4.1. Conventional methods for cardiac activity measurement In order to detect HRV changes over longer periods of time, a large volume of data needs to be collected and analysed. Suitable data for further analysis is normally obtained with a Holter monitor in out-patients, which is a portable device for continuous monitoring of various electrical activity of the cardiovascular system for at least 24 h. More traditionally, the data is collected with one of the two conventional methods in clinical use, the ECG or the PPG. The ECG on the one hand is considered to be one of the oldest diagnostic tools still used in medicine today with first recordings dating back as early as 1903. It is a clinical tool used in the field of cardiac abnormalities evaluation and is characterised by its high accuracy and easy interpretation. Despite the difficulties, the ECG is considered to be an optimal way of measuring the interbeat intervals (IBI), which are intervals between two adjacent heart beats. The method uses conductive Ag/AgCl electrodes attached to the patient’s body in a predefined and standardised fashion in order to detect and record the difference in the electric potential between different electrodes generated by the electric activity of the cardiac muscular fibres over a period of time. In the past the recording was presented in a graphical way on a standardised paper. Nowadays the data is stored in a digital form and can be displayed on a digital screen or transferred to another digitalised device for further analysis [18,19]. Although fixed-on-body electrodes are reliable and give good signal quality, there are several disadvantages to the method. The largest one is the need for direct contact of sensors with the skin. As such, the method may be annoying or uneasy. It presents movement limitations and is inappropriate for specific group of patients, such as infants or patient with burns. The conductive gel applied to the electrodes may cause allergic reactions on the one hand and time dependent measurement due to its drying on the other hand. Furthermore, the need of standard placement of electrodes requires a competent operator as misplacing an electrode may result in a faulty recording. Since the presence of trained staff is required, the hospitalisation period is extended and treatment costs are increased. Another conventional technique capable of measuring heart’s IBIs is the PPG, which is regularly used in many different clinical settings, including clinical physiological monitoring (blood oxygen saturation, HR, blood pressure and cardiac output), vascular


assessment (arterial disease, arterial compliance and ageing) and autonomic function (vasomotor function and thermoregulation, blood pressure and HRV, orthostatic intolerance) [20]. The PPG is a simple, non-invasive, portable, low cost and above all a biocompatible optical measurement technique used to measure the peripheral pulse, which is indirectly connected to the heart’s electrical activity via blood flow caused by the muscle’s contraction. The measurement is done by illuminating human tissue with a light source (usually red or near infrared (NIR)) and opto-electronically detecting the amount of transmitted or reflected light. Since the tissue is relatively transparent to NIR wavelengths, the amount of absorption level depends on haemoglobin, a protein responsible for oxygen transportation, which strongly absorbs the NIR light. During the cardiac cycle an increase in NIR light absorbance is present at times of high pressure (systole) whereas a decrease in NIR light absorbance is present at times of low pressure (diastole). Thus, the method’s main feature is the ability to provide a quick indication of the cardiac rhythm [20,21]. The recording of blood oxygenation pulsations (PPG) has been suggested as an alternative to ECG for the purpose of HRV analysis. However, the method is less accurate compared to the ECG due to wider peaks in the measured signal, which could also affect the HRV analysis. Furthermore, the method has been proved to be especially vulnerable to motion artefacts when compared to the ECG, consequently making it less reliable for longer term recordings [22,23].

4.2. Experimental non-contact methods for cardiac activity measurement The demand for ubiquitous measuring of human physiological parameters is ever increasing not only in the medical field (e.g. monitoring of hospitalized patients, home health care, rehabilitation, nursing of elderly [23,24]) but also in several commercially oriented fields, such as automotive industry (vital sign monitoring of the driver [25,26]), psychology (measure of stress response [23,27–31]), sports (optimisation of training [23,32]) and even in the field of man–machine relation (emotional communication [33]). In order to be able to conduct measurements in such diverse fields, the existing contact methods for obtaining parameter values with the known limitations would seem inadequate in some cases. A non-contact method (Fig. 2) would present a more appropriate solution for such instances where the goal is to acquire only the IBI and not the exact details concerning cardiac electrical conduction that ECG offers. In the past years several innovative non-contact methods for measuring cardiovascular parameters, particularly the HR and HRV, have in fact been studied world-wide. Some of the published results are promising and thus indicate that the noncontact measurements of certain human physiological parameters are indeed possible and will without a doubt have a great impact on many fields of application in the near future. There are however several limitations to the current non-contact measuring systems that

105

Fig. 2. A representation of the proposed non-contact methods for HR(V) measurements found in the literature. There is an explicit air gap between the measuring sensor and the human body. The size of the air gap varies for different non-contact methods.

need to be thoroughly investigated for optimal interpretation of the results. These limitations may be method dependent. The common limitations are reliance on distance between the measuring probe and the human body, susceptibility to environmental disturbances, movement artefacts, etc. The list of published non-contact methods for measuring of HR (and consequently HRV) that have come to our attention, accompanied by the two conventional methods, is presented in Table 1. 5. Methodology of IBI measurement with different methods Due to the nature of the functioning of the cardiovascular system on the one hand and the specific characteristics of the human body on the other hand, the IBI can be assessed directly or indirectly through specific physiological parameters with several different methods. Within this chapter, a brief overview of the known possibilities is given, substantiated by examples found in the published literature. 5.1. HR from speech Speaking is the most basic form of communication between people. Next to the basic expression and linguistic information, the voice output also includes hidden organic and biological information [32]. In fact, the human heart rates are dynamically related to the variations of vocal cord parameters via the larynx, which is indirectly connected to the human circulatory system [32]. Due to this fact, it should be possible to detect human heart activities by extracting appropriate frequency characteristics from the changes in human speech [32]. Mesleh et al. [32] demonstrated the studying of the frequency characteristics of the vowel speech signal. With the method, the author was able to obtain information of the heart activity, producing a frequency modulation of the vowel speech signal within a certain frequency band, making it possible for the IBI interval to be extracted.

Table 1 The list of novel non-contact measuring methods for HR (and consequently HRV) measurement. Measuring method

Classification

Measured quantity

Measuring sensor

Electrocardiogram Photoplethysmography Headphones Capacitively coupled ECG (CCECG) Microwave distance measurement Ultrasound distance measurement Optical vibrocardiography Thermal imaging RGB camera HR from speech

Contact, conventional Contact, conventional Contact, experimental Non-contact, experimental Non-contact, experimental Non-contact, experimental Non-contact, experimental Non-contact, experimental Non-contact, experimental Non-contact, experimental

Electric potential Absorption of electro-magnetic radiation Induced voltage due to movement artefact, sound Change of surrounding electrical field due to body movement Displacement of the body Displacement of the body Displacement of the body Radiation in infrared range of electromagnetic (EM) spectrum Absorption of electro-magnetic radiation Human speech sound

Conductive electrode Phototransistor Coil within the head-phone Capacitively coupled electrodes Microwave sensor Ultrasound sensor Laser Thermal imaging camera Digital camera Standard microphone

106


The method was tested on a 6 s recording of a vowel against a conventional PPG measuring method (with maximum error estimated to be 2%). Within the study, the average percentage of error was 5%, with lowest accuracy 91.60% and best accuracy 97.82%. The method is supposed to be robust and able to work in noisy environments (discarding machine and side talk noise). The main problem arises from patients with insufficient respiratory lung volume, who were not able to keep the same tone of speech during the recordings [32]. 5.2. Thermal imaging All surfaces with temperatures over 0 K emit electromagnetic radiation. The amount of emitted energy at a particular wavelength depends on object’s or subject’s temperature and emissivity. The human skin is considered to be an excellent emitter/absorber of thermal energy with an emissivity value between 0.95 and 0.98 [34]. At a regular human skin temperature of about 300 K, the emitted radiation is in the far infrared (IR) part of the spectrum, which is not visible to the naked human eye. Thermal imaging is an example of infrared imaging science. It is a passive (does not emit energy), non-contact method for measuring the emitted radiation in the IR range of the electromagnetic spectrum and produces an image of that radiation in the form of a thermograph. The method could be put to use in clinical applications for measuring of various physiological parameters. Such is the case with distant HR measurement. The cardiac pulse measurement with thermal imaging is based on the fact that the human skin temperature in the vicinity of major superficial arteries is directly modulated by the pulse blood flow, however, the exact shape is smoothed, shifted and noisy with respect to the actual pulse [23,35–37]. The measurements can be performed straightforwardly by observing time-varying heat patterns caused by major arteries, which yield information about the cardiac cycle and isolating relevant frequency components related to the heart signal. Since the pulsating heart contribution produced by an artery is of a very low magnitude compared to other heat patterns on the skin surface, the signal has to be enhanced [34]. Our laboratory developed several instruments and methods to support non-contact measurement methods of human parameters [63–67]. The idea of performing physiological measurements on the face was presented by Pavlidis et al. [38] and later also demonstrated through analysis of facial thermal videos [23]. According to the statistical analysis, the overall agreement between the novel approach and the reference method with the piezoelectric transducer reached 98%. There were, however, several limitations to the method: (a) the need for manual intervention by selecting the region of interest (ROI), (b) assumption that the test subject’s HR is within 60–100 bpm, (c) time lapse of approximately 2–3 min needed to make an estimation of the HR, (d) the need for the subject to remain seated in front of the camera. Chekmenev et al. [34] later presented a multi-resolution (MR) approach for non-contact passive measurement of the arterial pulse, based on thermal imaging of the most accessible regions of the human face and neck, which eliminated the limitations above. Using the periodic nature of arterial thermal patterns, the method automatically detects regions of measurement (ROM), which output a descriptive arterial pulse waveform. This is accomplished by selecting a scale representation, on which most of the heat variations is produced by true arterial pulse. Automatic selection of ROM is a complex and challenging task which depends on many anatomical parameters of the subject, including structure of the

artery and surrounding veins, surrounding fat, muscle tissue, etc. [34]. The filtered thermal data within the selected ROM is split into orthogonal low and high scale bands, using continuous wavelet analysis (CWA) to improve the cardiac waveform. The approach was tested on seated subjects up to a metre away from the IR camera (long-wave Phoenix IR from FLIR with thermal sensitivity of 0.025 ◦ C, 14-bit extended dynamic range in a 320 × 256 format). The selected data acquisition time was 20–40 s with a frame rate of 30 fps. For all subjects, the obtained results were reported to be 100% accurate compared to the reference signal, which was obtained with a portable heart rate monitor by Polar [34,35]. Despite the mentioned improvements, the non-contact measuring technique is still influenced by several known limitations. The presented results derive from experimental setting in a controlled environment, where the individual remains still without explicit body movements. Any spontaneous movements, such as small movements of the limbs or even stressed breathing affect the shape of the measured signal dramatically. Furthermore, the method is dependent on unwanted thermal distortions, such as sweating, blushing, external heat radiation, air flow, etc. 5.3. RGB imaging The cardio-vascular pulse wave travelling through the body periodically causes the vessel walls to stretch. The volumetric changes that are a result of fluctuations in the amount of blood or contained air within the human body can be measured by means of a PPG. These fluctuations modulate the absorbance of light passing through a given tissue volume, which is detected by the mentioned measuring method. PPG is performed with a dedicated light source and considers the ambient light as the source of noise. Recent studies have shown that some cardiovascular signals (e.g. HR, IBI) can be acquired remotely from a distance of several metres by processing a video file of a human face obtained with standard cameras with ambient light as the illumination source. Furthermore, the studies show that the method can be extended for simultaneous HR measurements of multiple persons [39,40]. The RGB sensor of the used camera is able to pick up a mixture of the reflected plethysmographic signal with fluctuations in the amount of reflected ambient light. This phenomenon is again caused by volumetric changes in the facial blood vessels during the cardiac cycle and thus indicates the timing of cardiovascular events [39,40]. The novel approach is based on automatic face tracking and localisation of measurement ROI on the one hand and recovery of underlying source signal of interest on the other. In this case, the signal in question is the cardiovascular pulse wave that spreads throughout the body. Its recovery is achieved with Blind Source Separation (BSS) by Independent Component Analysis (ICA). The idea for distant measurement of PPG parameters was presented in several papers [41–43]. However, the efforts lacked rigorous physiological and mathematical models for computation. Furthermore, motion artefact presented noise within the same frequency band as the signal of interest, thus rendering linear filtering ineffective. Poh et al. [39,40] presented a method for motion artefact reduction through efficient and robust image analysis (ICA). In their experiment, a basic webcam from a laptop was used (24-bit RGB with 3 channels × 8 bits/channel, 15 fps, pixel resolution 640 × 480) to record a 1 min video of 12 participants between the ages of 18–31 years seated approximately 0.5 m from the camera. The experiment was conducted indoors with varying amount of sunlight as the only source of illumination. The results were compared to an FDAapproved finger blood volume pulse (BVP) using Bland–Altman and correlation analysis. The robustness of the method was shown in three examples:


(a) Heart rate measurement at rest. Within this experiment the subjects were asked to remain still and stare at the webcam. Using the proposed ICA method to recover the HR, the technique showed high agreement with BVP measurements: – mean bias d = −0.05 bpm ± 2.29 bpm (SD of bias before ICA = 6.01 bpm); – the root mean squared error (RMSE) = 2.29 (before ICA = 6.00); – correlation coefficient r = 0.98 (p < 0.001). (b) Heart rate measurement during slow motion of head or body while remaining seated. Within this experiment the subjects were asked to move naturally as if they were interacting with the laptop, but to avoid large or rapid motions to keep the hand wearing the BVP sensor still. Using the proposed ICA method to recover the HR, the technique showed high agreement with BVP measurements: – mean bias d = −0.64 bpm ± 4.59 bpm (SD of bias before ICA = 17.58 bpm); – RMSE = 4.63 (before ICA = 19.36); – correlation coefficient r = 0.95 (p < 0.001). (c) Simultaneous heart rate measurement of multiple participants. Within this experiment, the authors recorded a single, 1-min recording of three subjects sitting together at rest. – RMSE = 2.23; 2.66 and 4.56 bpm for 3 participants respectively.

107

Fig. 3. Doppler effect: If assumed that the receiving wave comes from a single reflection on the chest surface and the surface movements are small compared to the wavelength, then according to the Doppler effect the phase change (t) of the signal reflected from the subject’s chest is proportional to chest motion and scaled by the wavelength of the signal (t) = 4x(t)/, where x(t) is the chest displacement and is the wavelength of the transmitted signal.

The primary function of the heart muscle is to pump blood through the circulatory system and is fundamentally mechanical. During each cardiac cycle the heart undergoes volumetric changes which are to a certain level transmitted on the chest. Next to the movement due to respiration process (from 4 mm to 12 mm with frequencies between 0.1 Hz and 0.3 Hz [44]), the chest at rest also moves as a result of cardiac activity (from 0.2 mm to 0.5 mm with frequencies between 1 Hz and 2 Hz [44]). The phenomena can be measured by using sensors with satisfactory displacement resolution and based on the Doppler effect (Fig. 3), described as the change of frequency of a periodic event relative to the time-varying position of the target [44,45]. More precisely, the phase of the reflected signal received from a person’s chest is directly proportional to the chest motion and is scaled by the wavelength of the signal [45].

can be done by using a single point laser Doppler vibrometer (LDV) from relatively large distances (tens of metres [46]). The measuring approach is referred to as optical Vibrocardiography (VCG). A LDV is based on extracting the vibration amplitude and frequency from the shift in laser frequency due to motion artefact of the surface of interest. The LDV is generally a two beam laser interferometer, measuring the frequency or phase difference between a reference beam and a test beam, which makes it possible to appreciate even the smallest Doppler shifts. Several studies [46–48] have demonstrated the ability of VCG to measure both the HR and the HRV by comparing it to a reference method (e.g. ECG). The data was computed from the extracted adjacent vibrocardiographic intervals (VV) and compared to adjacent NN intervals from the ECG. Morbiducci et al. [46] carried out short-term recordings from human subjects lying on a bed from a distance of 1.5 m with LDV maximum velocity range 10 m/s, a 0–35 kHz maximum bandwidth, a resolution of about 1 ␮m/s and accuracy in the range of 1–2% of RMS reading. In order to optimise the SNR, a small (2 mm2 ) adhesive retro-reflective tape was placed on the chest of each subject. Analogue inputs were sampled at 1 kHz. In the VCG signal, the first local extreme value was selected as a reference point. Prior to further computing of HRV parameters, R and V peaks were reviewed and manually edited for error correction. HR and HRV indices obtained from the VCG signal agreed with the rate derived from the ECG recordings by mean percent difference 250 Hz). Since the method could be used as a long term monitoring application, the patient would have to remain illuminated throughout the complete duration of the recordings. Also, the patient would need to be present within the recording area throughout the recording. 5.4. HR measurements based on Doppler effect

108


(e.g. microwave range: 24 GHz, ultrasound range: 40 kHz) and different output powers, which are directly related to the sensitivity of the system (higher frequency and power result in higher sensitivity to small displacements, best described by Obeid et al. [44]). Measuring of the human physiological parameters has been explored with continuous-wave (CW), frequency-modulated (FM) and wide band pulsed radar (UWB) configured as a single channel or quadrature radar [45]. A major limitation of the single channel configuration is its sensitivity to target position due to a periodic phase relationship between the received signal and the local oscillator [45]. Due to this limitation, a quadrature radar receiver capable of selecting the better of the two channels (I and Q) has been proposed to overcome detection sensitivity to the position of the subject [45]. Two demodulation methods for the quadrature outputs have been proposed, (a) the linear (complex) method which essentially rotates the data to optimum position and is accurate for low frequency and small displacements, and (b) the non-linear (arctangent) method, which overcomes the sensitivity to the target position and has proven accurate for larger displacements [45]. Achieving a high displacement resolution by applying a sensor with high frequency and output power coincides with high amplitude electromagnetic radiation and therefore high possibility of biological effects on human. The greater the required measuring distance, the higher the power should be [49]. Therefore, the characteristics of a measuring system must be chosen in such a way that the radiation does not present a safety threat. The accepted safe power density level for human exposure at frequencies from 10 to 300 GHz is 10 mW/cm2 [50]. Yamada et al. [51] demonstrated that the subject’s HR can be monitored from the distance of 1 m with transmit power levels from 2 mW ranging to as low as 20 nW, where appreciable degradation of the signal may be observed and thus demonstrating that the radiating power received is within the suggested safe levels. The HR (and HRV) measurements with a non-contact monitoring system based on Doppler radar have been demonstrated in many studies (e.g. [44,45,49–53]). The results show the ability to obtain precise results at different frequencies, output powers, distances between the subject and the measuring system etc. For example, Lu et al. [52] conducted tests at a distance of 3.6 m at frequency 35 GHz, transmission power 10 mW and antenna gain 17 dB. The results were cross referenced with results obtained from a BIOPAC 3-lead ECG. The R–R interval from a 5 min recording in both cases was 0.73 ± 0.02 s (mean ± SD) with mean HR 83 bpm. Despite the promising results there are a few setbacks related to the measuring technique. Next to the already mentioned issues related to the active electromagnetic emission, the Doppler radar is prone to movement artefacts [49,52]. The experimental protocol usually requires immobility of the subjects in order to limit the body movements, which are typically difficult to eliminate, which makes the method rather inappropriate for long-term recordings. The susceptibility to motion artefacts could be improved by using multiple antennas [54]. Furthermore, the recordings have been found out to vary between subjects. While still being able to determine the R peaks, the signal varies for each individual. The phenomenon is caused by different physical constitution, breathing, size and position of the heart muscle, etc. [49]. 5.5. Capacitively coupled ECG The activities of human organs, such as the heart, brain, muscle, etc. result in bioelectric signals. Bioelectric signals accompany all biochemical processes and are defined as electric potentials between points in living cells and can be measured with several techniques, including the ECG. Compared to other signals, the amplitude and the bandwidth range of heart signal (0.1–0.5 mV;

0.5–100 Hz) is amongst the largest and as such appropriate to measure in a non-contact manner [55]. In order to detect such relatively small signals, a sensor with high sensitivity and low susceptibility to ambient interferences needs to be applied. Capacitive type electrodes are able to detect biopotentials with an explicit gap between the sensor and the body, even through hair and clothing [55–60]. Compared to standard conductive type electrodes, the surface of these electrodes is electrically insulated and thus remains stable even in long-term applications. The sensor’s metal electrode and the body surface are capacitively coupled, forming a capacitance CECG . As such, the capacitive type electrodes rely on detecting the so-called displacement current ID , posited by J. C. Maxwell to explain magnetic fields around a capacitor, which is proportional to the rate of change of the electric field associated with the ECG signal [55]. The method is therefore entirely passive as it relies on the measurement of the local electric field as modified by the movement of the body (particularly the respiration and cardiac activity). The capacitance CECG depends on several factors, but usually corresponds to relatively small values 0.1–10 pF [55]. For the low frequency measurements as is the ECG, such weak coupling requires high input impedance of the sensor as finite input resistance would attenuate the input voltage VIN . Very high impedance nodes are very susceptible to any electromagnetic interference from the environment and motion induced artefacts which is why the electrodes need to be actively shielded in order to suppress the interference. Prance et al. [58,59] presented an Electric Potential Sensor (EPS) as a high input impedance measurement (1015 ) and low input capacitance (10 fF) system with an operating bandwidth from 100 MHz. The characteristics are achieved with a positive feedback technique. The measuring system is able to operate in dry contact mode or contactless mode. The measurements were made using a pair of sensors and thus removing the need for an additional reference electrode. Specifically Prance et al. demonstrated the ability of non-contact mode single EPS electrode to detect combined heart rate and respiration up to 100 cm from the surface of the body in an unshielded environment [58,59]. Compared to other non-contact measuring devices, the CCECG method is more susceptible to various environmental noise, especially electromagnetic interferences, such as mains voltage power supply interference and static electricity. Furthermore, the method is affected by the changes in air circulations.

6. Discussion Measuring of human physiological parameters in a non-contact and discrete manner could become an important alternative to standard monitoring systems used in clinical environment on the one hand and also various other more commercially oriented fields on the other hand. Even though the discussed non-contact methods are focused on measuring the cardiovascular activity, it has been reported in majority of the cases that detailed cardiac parameters (e.g. QRS wave) cannot be detected. Nevertheless, the studies indicate that the R peaks are indeed obtainable, making the HR and consequently the HRV analysis possible. All of the discussed non-contact measuring methods are reported to be successful and are capable of providing the wanted information with a certain reliability and accuracy. Each of the discussed methods however has its advantages and disadvantages that are method dependent. Due to this reason, a specific method may prove to perform better in a specific environment and consequently offers optimal results. In terms of HR measurement properties it is important to investigate for each method also uncertainties and

Working principle/method

A HR from speech Frequency modulation of vowel speech

B Thermal imaging Thermal imaging of the human face and neck

C RGB imaging Reflected plethysmographic signal of reflected ambient light

D Optical vibrocardiography Deflections on the observed human body area caused by the cardiac activity

E Doppler Radar Doppler effect on microwave radiation/ultrasound waves reflected from body

F Capacitively coupled ECG Measurement of the local electric field as modified by the movement of the body

Acoustic noise immunity Electrical noise immunity Radio-frequency immunity Multiple person Data processing complexity Equipment price Development state Reliability Active/Passive Distance Environment influence

** ***** ***** * ****

***** ***** ***** * *****

***** ***** ***** ***** *****

***** ***** ***** * *

***** ***** *** * *

***** * ** * **

* **

** *** *** P **** Ambient light variation

***** **** **** A *****

***

P * Acoustic noise

***** *** *** P *** Air flow

***

*****

*

*

13

Min. measurement duration Main advantages

*** ** ** P * Air flow, static electricity, line noise *

Cheap sensor

A passive non-contact method

Low cost sensor, multiple subjects

Long range

Non-contact method

14

Main disadvantages

Complex algorithm, unreliable

Expensive sensor, low temporal resolution

Complex algorithms, low temporal resolution

Detection of small movements, long range, high bandwidth Expensive hardware, complex optical interface

Microwave/ultrasound radiation exposure

Low noise immunity

1 2 3 4 5 6 7 8 9 10 11 12

***** A **** Microwave radiation


Table 2 The table provides an overview of the discussed non-contact measuring methods for cardiac activity, focusing on their advantages and disadvantages. The methods are listed in columns “A” through “G” whereas the comparison area is listed in lines 1 through 14. The impact of the comparison area in question on a specific non-contact measuring method is ranked with a star symbol “*” where “*” represents the minimum value and “*****” represents the maximum value.

109

110


traceability of results [61,62]. The advantages and disadvantages of individual methods are given in Table 2. The comparison of the influencing parameters and features is further discussed hereinafter. The first chosen influencing environmental parameter is the acoustic noise against which the immunity of the discussed noncontact methods is compared. In line with expectations, the most affected is the method extracting the HR recording and analysing human vowel speech signals with a microphone. Despite the reports of the method being robust also in noisy environments with the use of appropriate filtering, it remains the only one potentially susceptible to the environmental acoustic noise. The next environmental parameter which may have an effect on the discussed methods is the electrical noise. Surrounding static electricity can influence a certain type of sensor and thus affect the measurement itself. The possible issue includes sensor overload and corresponding long recovery time. The latter depends on the time constant of the sensor. Furthermore, the surrounding AC field, depending on its frequency, may behave as the measured signal if within the HR range. This can be avoided by appropriate filtering; however, it requires high dynamic range of the measuring sensor and the data acquisition system. The CCECG method needs high sensitivity of the sensor in order to successfully detect biopotentials, making it highly susceptible to surrounding electrical noise. Nowadays we are surrounded with electronic devices and household appliances, which next to electrical noise also emit a lot of radio frequency emissions (e.g. microwave ovens, wireless baby monitors, Wi-Fi computer networks, mobile phones, etc.). Such emissions can indeed interfere with some of the discussed non-contact measuring sensors. In fact, the most susceptible sensors to such noise are the CCECG and the microwave Doppler radar. Typically, a measurement of a physiological parameter is carried out on an individual. This is also the case with all of the discussed measuring methods but one. While the rest of the methods rely on extracting the cardiac activity parameters from a single person, the RGB imaging method has been reported to be able to successfully measure the HR from at least 3 persons simultaneously. The concept could prove to be useful in several fields when applied to a larger audience. Successfully measuring the data in a non-contact manner represents only one part of the process, where the other part is presented by data processing. The complexity of the obtained measurements differs for the discussed methods. The process is most challenging in the non-contact measuring methods marked with “A”, “B” and “C” in Table 2. In the HR from speech, the recording (when done in an out-of-laboratory environment) would contain some environmental noise, which could interfere with the sought signal. In order to be able to extract the HR from speech, several appropriate algorithms and filtering needs to be applied. The methods based on thermal imaging and RGB imaging present similar problem for signal extraction. In both methods, the user is faced with searching for the optimal area of interest, which in some cases needs to be manually carried out. After the appropriate area is selected, the signal also needs to be processed with several mathematical algorithms. Optical vibrocardiography, both radar methods and the CCECG on the other hand provide a more straight forward signal with more distinctive peaks which are easier to process (when the measurement is carried out appropriately). Another important aspect of the measuring device is next to the data processing complexity also its price. The prices of the discussed non-contact measuring methods vary greatly from low cost microphone required for HR from speech and a standard laptop RGB camera, to more expensive thermal cameras and lasers required for optical vibrocardiography. Both radar sensors and the CCECG

can be assembled from electronic components found in standard electronic shops. The development stages of individual measuring methods vary. The HR from speech and the CCECG are considered to be in an early development stage. Despite the fact that the latter has been discussed over several years and in many papers, it is our impression that the method is susceptible to many environmental influences, making it necessary for the process to be carried out within controlled environmental conditions. The method could be made a lot more interesting if it were made more robust. Both imaging techniques are according to our opinion positioned somewhere in the middle with big promises especially in the field of recording several subjects simultaneously. The laser and Doppler techniques present methods which are the most developed since they both acquire straight forward data which is relatively easy to process. Similarly, the most reliable non-contact measuring methods are the laser and the Doppler based methods, which provide signals with highest accuracy amongst the discussed methods and best signal to noise ratio when executed properly. They are, however, probably most affected by uncontrolled minor movement of the subject. Despite the fact that the radiating power of the active methods discussed within this manuscript is within the suggested safe levels, there are circumstantial evidences implying that the exposure to specific radiation may be harmful to human health. Due to this, the EM radiation of some of the methods can be considered a drawback in contrast to the passive measuring methods. Different approaches for the measurement of cardiac parameters can be carried out at different distances, which may also affect the radiating power of the measuring method. The method reported to be able to obtain satisfactory results at the largest distance (>10 m) is the optical vibrocardiography, followed by both radar imaging techniques (approx. 1 m) and HR from speech and the CCECG (