 Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 89 Biosignal –Based Multimodal Biometric System Kuk Won Ko and Sangjoon Lee* School of Mechanical and ICT Convergence Engineering, Sun Moon University, S. Korea. Received 02 May 2016; received in revised form 25 June 2016; accept ed 26 June 2016 Abstract This study concerns personal identification based on electrocardiogra m (ECG) and phot o- plethysmogram (PPG) signals. We manufac- tured a bio-signal measure ment system that can simu ltaneously measure ECG and PPG signals, using which three channels of ECG signal and one channel of PPG signal we re acquired fro m the right-hand index finger of a total of 33 su b- sets for 3 minutes. Lead-I signal of the three-channel ECG signal and the one-channel PPG signal we re selected for recognition. For each subject, 160 heartbeats were automatically separated from the acquired bio-signals, and a total of 21 features, co mprising 15 ECG features, 4 ECG-related PPG features, and 2 features co n- cerning PPG only, were e xtracted fro m each heartbeat. Letting the 21 features form a single data point, heartbeat features of each subset were used as the training data for a support vector mach ine (SVM) c lassifie r, with the number of data points being adjusted from 10 to 80, and the data points (80 – 150) other than the tra ining data were used as the testing data, in order to investigate the recognition performance indices. As a result, the proposed algorithm showed high recognition performance o f 99.28% accuracy, 0.88% FRR, 0.85% FAR, 99.28% sensitivity, and 99.31% specificity, when there are 80 training data points. Moreover, even when there are 10 tra ining data points, the proposed alg o- rith m showed the performance of 92.77% ac - curacy, 7.23% FRR, 6.29% FAR, 92.77% sen- sitivity, and 93.21% s pecific ity, wh ich can be evaluated as an extre me ly high recognition performance considering that there was a total of 4,950 testing data points. Keywor ds : biometric, pattern recognition, per- sonal identification 1. Introduction Following the rapid recent advances in in- formation technology (IT), the demand for b i- ometric recognition technology is increasing for the purpose of information security and the prevention of privacy invasion. Fingerprints, iris, facia l features, dorsal metacarpal veins of the hand, or gait, which are the intrinsic features of human body, are used as the ma in e le ments of e xisting recognition systems. Biologica l features for b io metric recognition can be selected the characteristics that have almost no variation with time, or reflect slow variation with time. This study introduces a mu ltimodal bio metric system and algorithm based on electrocardiogra m (ECG) and photoplethysmogram(PPG) signals. It is known that there are around 80,000 – 100,000 contractions in a healthy heart in o rder to circu- late the blood in at ria to the vessels of the entire body, within 0.1% cases of wh ich e xh ibit ab- norma l ECG waveforms. Moreover, ECG s ig- nals are known to be regular and convenient for pattern classification compared to other bio-signals. PPG signals are bio-signals that are lin ked with ECG signals, and also represent the state of b lood vessels of the subject with heart contraction. Furthermore, while ECG and PPG waveforms of an indiv idual subject vary slightly at each time, these signals have the characteris- tics of unchanging overall pattern. The first study on recognition systems based on bio-signals was introduced by Lena Biel [1] in 2001, followed by fe w researchers around the world. John M. Irvine [ 2] showed the effec - tiveness of ECG for personal identification even when the heart is in a stressed state (excite ment or e xe rcise). In addition, T. W. Shen [3], Steven A. Israel [4], Konstantinos N. and Plataniotis[5] introduced personal identification algorithms based on ECG. The proposed algorithm is based on the detection of feature points fro m ECG and PPG signals obtained simu ltaneously, and a total of 21 features are automatica lly e xt racted, inclu d- ing 15 features fro m the ECG heartbeat signals * Corresponding aut hor, Email: mcp94lee@sunmoon.ac.kr Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 90 Copyright © TAETI separated fro m the overall ECG signal, and 6 features from PPG signal, in order to promote the real-t ime operation of the recognition a lg o- rith m. The proposed algorithm adopts down slope trace waveform (DSTW) method [ 6], which shows high peak detection rate, for the detection of P, Q, R, S, and T points of the ECG signal. For pattern recognition, one-against-one support vector machine (SVM ) a lgorith m was used, which shows excellent recognition results in various fie lds, and is capable of mu lti -class classification. 2. Method 2.1. Bio-Signal Measurement System While the previous ECG recognition and disease detection algorithms can be assessed in terms of validity by using e xisting public dat a- base files such as MIT ECG database for ve ri- fication, there is no public database that includes a simultaneous measurement of PPG s ignals lin ked with ECG signals. There fore, the hard- ware for simu ltaneous measurement of ECG and PPG signals was manufactured for the acquis i- tion of b io-signals for the recognition algorith m in this study. Fig. 1 ECG and PPG measurement system blockdiagram Fig. 1 shows the block diagra m and the photograph of the manufactured ECG and PPG measure ment system hardware . ECG was measured in three channels (LA -RA, LL-LA, and RA-LL) based on the Wille m Einthoven triangle method. Moreover, low-pass filters and high-pass filte rs with cutoff frequencies of 0.16Hz and 160Hz, respectively, were imple mented in hardwa re, with the total a mp lification gain of the ECG signal being 1500, as shown in (a-b) and (a-c) of Fig. 1. For PPG measurement, the output signal fro m the filter and the origina l signal were d iffe rentially a mp lified by imple - menting a high-pass filter with cutoff frequency of 0.1 Hz in o rder to prevent the saturation due to the DC offset fro m the light output during a m- plification, as shown in the PPG measurement hardware in Fig. 1. Moreover, a band -pass filter with cutoff frequencies of 0.1Hz and 8Hz was designed and imple mented. Fig. 2 shows an actual measured signal being displayed and stored on PC. Fig. 2 Measurement software 2.2. Peak Detection from ECG and PPG Signals through DSTW Fig. 3 DSTW peak detection method[6] Peaks are detected fro m the ECG signal by generating DSTW , as shown in Fig. 3, in order to firstly detect the R peaks in ECG, which a re convenient to detect [6]. After the R peak is detected as shown in Fig. 4, Q, S, P, T , and end of T are sequentially detected in accordance with a given detection rule (Eqs. 1- 5). Fig. 4 Results from the detection of Fiducial Points from ECG Signal (P, Q, R, S, T , TE) (Subject 4) Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 91 Copyright © TAETI Fig. 5 Results from the detection of PPG Peaks (Onset, Peak) (Subject 4) The points to be detected fro m the PPG signal we re selected to be the onset points, which are the initia l points, and PPG peaks. Onset peaks are used to detect the fiducial points in accordance with g iven ru les (Eq. 6), a fter the PPG peak has been detected as shown in Fig. 5.    Q k Index( [ ]), min Arg X n for  R k 30 n R(k)   (1)    S k Index( [ ]), min Arg X n for    R k n R k 30   (2)     P k Index( ), max Arg X n for  Q k 60 n Q(k)   (3)     T k Index( ), max Arg X n for    S k n S k 100   (4)     k Index( ), end min T Arg X n for    T k n T k 50   (5)     Onset k Index( ), min Arg P n for    k 100 n k peak peak PPG PPG   (6) where k denotes the indices of each peak fro m one to M (1≤k≤M), and n denotes the indices of the sampled data for X(n ) and P(n). The actual data were samp led at 250 Hz, and n=1 imp lies 4 ms 〖 (T〗 _s=1/F_s ) 2.3. Feature Extraction As shown in Fig. 6, the features for b io- metric recognition can be e xtracted by detecting the six types of ECG peaks and the two types of PPG peaks. For this a lgorith m, a total of 21 features were established by e xtracting 15 types of ECG features, 3 types of PPG features, and 3 types of features relating to both ECG and PPG. 2.4. Support Vector Machine for classification SVM is a new genre of lea rning systems for pattern recognition, devised by Corters and Vapnik [7]. While SVM did not receive much attention initially, it is recently being used in various fie lds including bio informatics, charac - ter recognition, handwriting recognition, fac ia l recognition, and object recognition, and is wide ly used for supervised pattern recognition owing to its excellent performance. Fig. 6 Extraction of 21 features from ECG and PPG peak detection The train ing and classificat ion algorithm for the proposed algorithm was based on one-against-one mu lti-class SVM algorith m, which is not af- fected by the curse of dimensionality. 160 peaks were detected (P, Q, R, S, T , and Tend) fro m the ECG signals simultaneously acquired fro m each of the 33 test subjects, and, likewise, 160 peaks (Onset, and PPGpeak ) we re detected fro m the PPG signals, forming a set of 21 features for each peak. Fro m each of the 34 dataset files, the first (10-80) features were used as the training data, and the other (140 – 80) features were used as the testing data. Moreover, we e xa mined the diffe rence in recognition rate between the case where only ECG features are considered, and the case where both ECG and PPG features are considered. 3. Results and Discussion 3.1. Testing method For the e xperiment, the testing data were not provided with any informat ion for recognition, and the average and standard deviation of the performance evaluation ele ments were invest i- gated, such as accuracy, sensitivity, specific ity, false reject ion rate (FRR), and fa lse acceptance rate (FAR) of the result with varying nu mber of training data points. The formulae for FRR and FAR are shown in Eqs. 13 and 14, respectively. For bio metric recognition systems, it is common to measure the equal error rate (ERR) perfo r- mance inde x, wh ich is where FRR and FA R coincide, and this was achieved by analyzing the FRR and FA R, and the resulting ERR and re - ceiver operating characteristic (ROC) curve while increasing the threshold value fo r the distance measurement algorith m for the decision of recognition fro m a lower value to a higher value [14]. Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 92 Copyright © TAETI FRR = 𝑁𝑤 𝑁𝑡 × 100 (7) FAR = Tr −Tn Tr × 100 (8) where Nw is number of wrong recognition, Nt is number of testing data for target subject, Tr is total recognition number for taget subject, Tn is testing data number. Moreover, the sensitivity and specificity of the proposed algorith m we re investigated using the true positives (TP), the fa lse positives (FP), and the false negatives (FN). For a bio metric recognition algorith m, T P, FP, and FN a re d e- fined as follow:  TP (true positive): Nu mber of accu- rately recognized test subjects.  FP (false positive): Number of falsely rec- ognized test subjects . (The number of data points classified as the subject in question – TP)  FN (false negative): The number of subject data points that should have been recognized, however, we re not reco g- nized. (The number of testing data points from each subject -TP) Sensitivity and specific ity are defined by Eqs. 9 and 10 below, respectively. Sensitivity = TP TP+ FN × 100 (9) Specificity = TP TP+ FP × 100 (10) 3.2 Testing Result Table 1 Recognition Performance Indices with both ECG and PPG signals used P P G and ECG Feat ures analysis and mult i class classifier T raining Number 10 20 30 40 50 60 70 80 Accuracy (%) AVR 92 95 96 97 97 98 99 99 ST D 10 9.0 5.7 4.8 2.6 2.0 1.6 1.6 FRR (%) AVR 7.2 4.7 3.1 2.4 2.6 1.2 0.8 0.8 ST D 10 9.0 5.7 4.8 2.6 2.0 1.5 1.6 FAR (%) AVR 6.7 4.3 2.8 2.3 2.3 1.1 0.8 0.8 ST D 7.8 6.8 4.9 3.6 2.8 2.0 1.3 1.5 Sensit ivit y (%) AVR 93 95. 96 97 98 98 99 99 ST D 10 9.0 5.7 4.8 2.6 2.0 1.6 1.6 Specificit y (%) AVR 93 95 97 97 98 98 99 99 ST D 7.8 6.8 4.9 3.6 2.8 2.0 1.4 1.5 Fig. 7 ROC curve with varying tra ining data size (15 ECG features used only) Fig. 8 ROC curve with varying training data size (21 ECG and PPG features used) Fig. 7 shows the performance indices of recognition using a mult i-c lass SVM classifie r with 15 ECG features detected fro m the actual measure ment data fro m 33 test subjects. 89.92% accuracy, 10.08% FRR, 9.79% FA R, 89.92% s e ns it iv it y , a nd 90. 21% s p e c ific it y we re observed when the classifie r is tra ined with 80 heartbeat data obtained from the ECG s ignals Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 93 Copyright © TAETI fro m each subject. This result corresponds to the re c o gn it io n wit h resp e ct to 2,640 t e s t in g heartbeat data from a total of 33 ECG signals. The proposed algorithm was tested based on a total of 21 feature points, by add ing the 6 PPG feature points measured simu ltaneously fro m the same subject to the 15 ECG feature points. Recognition perfo rmance indices, which a re shown in Table 1, d isplayed the results of 99.28% accuracy, 0.88% FRR, 0.85% FAR, 99.28% sensitivity, and 99.31% specificity. This is the performance when 80 tra ining data points fro m the ECG and PPG signals measured fro m 33 test subjects are used, and 2,970 testing data points were used overall. Moreover, as shown in Tab le 1 and Fig. 8, the accuracy index shows the high performance of 92.7% even when 10 tra ining data points are applied out of the 160 heartbeat data from each subject (4,950 testing heartbeat data). The result of the above test is better than or equal to the p e rfo rma n c e o f t h e e xis t in g c o mmo n re c o g n it io n s ys t e ms , a n d th e re c o g n it io n performance is observed to be e xtre me ly higher compared to the previous test. Therefore, it can be deduced that the consideration of the features of two types of signals for the design of b io- metric recognition systems leads to a higher recognition performance than when a single type of signal is used. 4. Conclusions This paper concerns the imp le mentation of a mu lti-modal bio metric recognition system and algorith m, where in ECG and PPG signals we re used for the recognition. Since there is no pre - vious database where ECG and PPG signals a re lin ked, a bio metric system that is capable of a c t u a l EC G a n d PP G me a s u re me n t s wa s manufactured, and three channels of ECG signal and one channel of PPG signal were acquired fro m a total of 33 subjects for 3 minutes. Lead -I signal of the three channels of ECG signals only and the 1-channel PPG signal were selected for recognition. With the ECG signals acquired fro m e a c h su b je ct , 160 h e a rt be a ts we re separated and 15 ECG features were e xt racted fro m each heartbeat. Moreover, 4 PPG features related to ECG and 2 features of PPG only we re selected, so that a total of 21 features we re used for the recognition. The recognition performance indices of the proposed biometric reco gnition system were investigated by adjusting the number of train ing heartbeat data between 10 and 80. The performance indices of the reco g- nition based on ECG features only were found to be 89.92% accuracy, 10.08% FRR, 9.79% FAR, 89.92% sensitivity, and 90.21% s p e c ific it y . Fu rt h e rmo re , t h e p e rfo r ma n c e indices of the recognition combining PPG were found to be 99.28% accuracy, 0.88% FRR, 0.85% FAR, 99.28% sensitivity, and 99.31% specific ity, when there are 80 heartbeat data for training, which indicate an e xt re me ly high recognition performance. Therefore, it can be deduced that the use of the features of a comb ination of mo re than two bio-signals can lead to a higher reco g- nition performance than the use of a single bio-signal. Moreover, the recognition system based on the combination of the two signals e xhibits the performance of 92.77% accuracy, 7.23% FRR, 6.29% FAR, 92.77% sensitivity , and 93.21% specific ity, even when 10 heartbeat data are used for training, which can be viewed as an extre me ly high recognition performance, considering that there were 4,950 testing heart- beat data. Acknowledgement This work was supported by the Human Re s o u rc e T ra in in g Pro g ra m fo r Re g io n a l Innovation and Creativity through the Ministry of Education and National Research Foundation of Korea (NRF-2014H1C1A1066998). References [1] L. Bie l, et al., “ ECG ana lysis: A new ap- proach in human identification,” IEEE T ra ns a ct io ns on Inst ru me n t at io n an d Measurement, vol. 50, no. 3, pp. 808-812, 2001. [2] J. M . Irv in e, et a l., “ Eig en Pu lse : Robust human identification fro m ca rdiovascular function,” Pattern Recognition, vol. 41, no. 11, pp. 3427-3435, 2008. [3] T. W. Shen and W.J. To mp kins , “Bio metric statistical study of one-lead ECG features and body mass index (BM I) ,” 27th Annual International Conference Engineering in Medicine and Biology Society, IEEE press , September, 2005. [4] S. A. Israel, et al., “ECG to identify individuals,” Pa t t e rn Re c o gn it io n , vo l. 3 8, n o . 1, p p . 133-142, 2005. [5] K. N. Plataniotis, D. Hatzinakos, and J. K. M. Lee, “Analysis of human electrocard iog ra m ECG for b io met ric re cognit ion ,” Proc . o f Advances in Technology Innovation , vol. 2, no. 3, 2016, pp. 89 - 94 94 Copyright © TAETI Bio metrics Symposiums , Baltimore, USA, 2006. [6] J. Kim, et al., “An event detection algorithm in ECG with 60Hz interference and bas e- line wandering,” Proceedings of the 2nd Inte rn at iona l Con fe ren ce on Inte ra ct ion Sciences, Information Technology (ICIS ‘09), Culture and Hu man New York, NY, USA , 2009. [7] C. Cortes, and V. Vapnik, “Support-vector network,” Machine Learn ing, vol. 20, no. 3, pp. 273-297, 1995.