1 American Academic Scientific Research Journal for Engineering, Technology, and Sciences ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 https://asrjetsjournal.org/index.php/American_Scientific_Journal/index Robust Feature Sets for Implementation of Classification Machines Binesh T * Govt. Model Engineering College-APJ Abdul Kalam Technological University, Kochi, India-682021 Email: binesht@mec.ac.in Abstract Classification Machines have evolved over a lot during recent times, in the field of engineering and sciences. Various classification schemes have been developed, taking into account, the aspect that can be optimized to give optimum system performance. The feature set in a classifier system is very significant, since it determines the efficiency and performance of the machine. Three powerful feature sets possessing robust classification capabilities are discussed in this paper. Cepstral coefficient analysis based Kruskal-Wallis H statistic, F-test statistic and Discrete Sine Transform based feature sets are found to be very effective for detection and classification of signals. Simulation results for typical data set are also presented in this paper. Statistical estimators, Neural Network and Hidden Markov Model based classifiers, along with various deep learning algorithms can be incorporated along with these feature sets to implement an efficient classification machine. Typical results based on these feature sets are also presented for different signal sources. Keywords: Classification Machine; Discrete sine transform; Statistical estimators; Hidden Markov Models; H- statistic; F-statistic. 1. Introduction Various class of signals call for specific considerations because of the unique generating mechanisms that are known to create them at the source. The primary requirements of a signal classifier are the capability of extracting and selecting the right feature combinations, efficient processing and generation of unambiguous classification parameters from the source specific features. An efficient underwater target classifier, making use of non-parametric estimators is available in [1]. Speech recognition systems based on different source specific cepstral features are presented in [2, 5]. This work is significant in that the cepstral features possess unique characteristics for identification and classification. An audio classification system based on a biological feature set has been mentioned in [6]. A robust underwater target recognition system based on combined invariant moments of underwater images has been proposed in [7]. ------------------------------------------------------------------------ Received: 2/27/2023 Published: 4/20/2023 ------------------------------------------------------------------------ * Corresponding author. American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 2 In the research work [8], a new Gaussian process classifier capable of accepting probabilistic training targets using an autonomous underwater vehicle is mentioned. The robustness and efficiency of the existing systems, depend on a large degree, the availability of source specific features which include those in non-acoustic domain. Also, in nonlinear environments, typical to that for underwater signals, the performance degrades, and the problem is accentuated by the presence of Gaussian ambient noise. Another class of feature set having significance is the discrete transforms. A study of discrete transforms exploited for prominent applications in signal analysis is presented in [10]. Of these, the Discrete sine transform (DST) which can be effectively utilized in the classification of underwater signal sources, operates on a specific function at a finite number of discrete data points [11]. DST is having properties similar to other transformations but when properly applied, they are capable of highly efficient performance in data enhancement and other signal processing applications. The system performance under nonlinear channel conditions has also been reported to be efficient. Various studies on stochastic classifiers like HMM, under noise conditions can be found in [12, 21]. 2. Robust Feature Sets Three feature sets are presented in this paper, which can give high accuracy for a classification machine. The extraction of these feature set is of prime importance in the design and implementation of a statistical or stochastic classifier. 2.1. Estimation of Transition Probabilities The signal is converted to frames of Ns samples, with adjacent frames being separated by md samples [1]. Denoting the sampled signal by s[n], the l th frame of data by xl[n], and there are L frames, then, (1) Where n = 0, 1, …., Ns -1, and l = 0, 1, ….L-1. Each individual frame is windowed to minimize the signal discontinuities at the boundaries of each frame. If the window is defined as w[n], then the windowed signal xw is: (2) where 0 < n < Ns-1. Hann or Hamming window are typical for classifying machines and Linear Prediction analysis is performed [9]. The Linear Prediction Coefficients are then converted to the required number of Cepstral coefficients, which are weighted by a raised sine window. In the next step, a clustering process is applied to generate a code book which is again utilized in the estimation of transition probability vector. For fixing the centroids of a cluster model, the K-means algorithm has been used. American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 3 The extracted cepstral coefficients of the signal source are being utilized as the data in this vector quantization process of cluster identification. A matrix is defined, which represents the data which is being clustered, in a concatenation of K clusters, with each row corresponding to a vector. The cluster centroids are generated as a vector with the cluster identity. The sum of square error function can be used, and a logarithm of the error values after each iteration can be returned in a variable, with the maximum number of iterations being specified. A vector of transition probabilities can be generated from the vector quantized output, for the estimation of H- Statistics [1]. 2.2. H and F statistics estimation The H and F statistics are estimated with the three-sample set consisting of the previously generated transition probability vector, a down sampled version of the signal and a predefined reference sample vector. A correction for ties can be made by dividing the H-statistic value by a Correction Factor(CF) defined as follows[1]: ∑ (3) where g is the number of groupings of different tied ranks, and ti is the number of tied values within group i that are tied at a particular value. This correction usually makes only negligibly small change in the value of test statistic unless there are large numbers of ties. 2.3. Discrete sine transform (DST) based feature set For a sequence x(n), the DST and the Inverse DST can be defined as: √ ∑ ( ) (4) √ ∑ ( ) (5) where n=1, 2,…N and k=1,2,…N. Ocean signal classification, making use of DST based features has been mentioned in [10]. DST based feature set has been modified by appropriate polynomials which render itself to efficient vector quantization by the algorithmic cluster analysis adopted by the system in connection with the training phase of the Hidden Markov Model. The DST based feature set is very robust and possess specific characteristics suitable for classification machines like Hidden Markov Models [11]. 2.4. Hidden Markov Model based classifier machines A Hidden Markov Model (HMM) is a doubly stochastic process that is hidden but can only be observed through another set of stochastic process that produces the sequence of observed symbols [15]. HMM can be regarded as the simplest dynamic Bayesian network. In a dynamic Bayesian network, the hidden state is represented in terms of a set of random variables, each of which can be discrete or continuous. The observation American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 4 can be represented in terms of another similar set of random variables. In a Hidden Markov model, each state has a probability distribution over the possible output tokens. Therefore, the sequence of tokens generated by a Hidden Markov Model is capable of giving information related to the sequence of states. The HMM consists of a finite set of states and each state is associated with a probability distribution. Transitions among the different states are governed by the parameter called State Transition Probabilities. In any state, an outcome is generated depending on the corresponding probability distribution. The states are hidden from an external perspective and only the outcome is visible, unlike a regular Markov process in which the state is directly visible to the observer. An HMM can be completely described in terms of the number of states, the state transition probabilities, the probability distribution in each of the states and the initial state distribution [18]. 3. Performance of Classification Machines The unknown signal is processed and the extracted H and F statistic values are assigned to known signal categories by judiciously matching the component parameters. The performance of statistical classifier using simulation studies and the estimated H-statistic as well as F-statistic approximations, have been tabulated in Table 1. This approach and the featured statistical indicators possess increased robustness essential for the efficient capability of a classifier machine. Table 1: Estimated values of H and F statistics for signal sources. Signal sources Estimated H statistic Estimated F statistic Bagre 2420 5706 Outboard 1951 2791 Damsel 2115 3616 Sculpin 1172 933 Atlantic croaker 1987 3023 Spiny 2450 6076 BlueGrunt 2097 3570 Dolphin 2146 3455 01m 1172 940 Barjack 2021 3050 Bow1 2168 3939 Boat 1494 1451 Chord 2160 3783 3Blade 1837 2372 Torpedo 2563 9757 Rockhind 2075 3394 Snap1 2117 3632 Scad 1990 2893 Finwhale 2134 3875 Seal1 2051 3187 Garib 1896 2635 Grunt 1955 3259 Ocean Wave 2054 3558 Minke 2130 3476 Hump 2156 3838 Searobin 1844 2394 American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 5 Analytical studies have been carried out for validating the classification potential of the system, by selecting a suitable simulation platform. The system has been tested for different signals, and results have been tabulated. The source specific features are being utilized in the training of a twenty state Hidden Markov Model. Using simulation studies, unambiguous classification has been achieved for various signal sources. The signal waveforms emanating from the target of interest have been sampled and recorded as a wave file and used as the input to the HMM classifier system. The unknown target signal to be identified is processed for the extraction of the desired features and they have been used in the recognition phase of the proposed stochastic classifier. The state transition probability distribution forms another important parameter determining the classification capability of the model. The system behavior under Gaussian ambient noise conditions has also been analyzed using simulation studies. The Gaussian ambient noise compensation performance of the classifier is studied and results are shown. Table 2 shows the classifier performance under different conditions of operation. For the trained HMM, the Performance Score is found to be 88% under ideal noise free environment, whereas with Gaussian ambient noise, the Performance Score of the classifier is seen decreasing but being compensated the introduction of filters. The performance score relates to the classification efficiency or the success rate of the system. Table 2: Performance scores of DST based classifier machine. Conditions of Operations Performance Scores Without Ambient Noise 88% W it h G au ss ia n A m b ie n t n o is e (S N R d B ) 20 dB With Butterworth filter compensation 85% With Chebyshev type 1 filter compensation 84% 14 dB With Butterworth filter compensation 82% With Chebyshev type 1 filter compensation 82% 10 dB With Butterworth filter compensation 78% With Chebyshev type 1 filter compensation 76% Under nonlinear conditions of second degree 85% For increased Gaussian ambient noise levels, with SNR of 14 dB, the tenth order Butterworth filter-based compensation gives a Performance Score of 82% while the same for the fifth order Chebyshev type 1 filter- based system also gives 82%. For further increased Gaussian ambient noise levels, with SNR of 10 dB, the Butterworth filter-based compensation gives a Performance Score than the Chebyshev type 1 filter-based system. The classifier performance under nonlinear channel conditions, with a nonlinearity of second degree, has been studied, with Performance Score shown in Table 2. The feature set based classifier machine has enhanced the state of the art by its improved robustness in non-ideal and nonlinear underwater environments. American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 6 4. Conclusions The robust feature set for classification machines consists of the H-statistic as well as F-statistic approximations for different signal sources. These have been effectively utilized for the classification process as demonstrated by the simulation results. The system proposed for DST feature set in this paper makes use of a twenty state HMM for the detection and classification of various signals. The system performance under Gaussian ambient noise conditions and typical nonlinear conditions have also been analyzed in the studies. A tenth order Butterworth lowpass filter and a fifth order Chebyshev type 1 filter-based schemes are used for providing the required compensation for Gaussian noise. In the presence of a Gaussian ambient noise and also in nonlinear conditions, the proposed system gives robust performance, thus enhancing the success rate of the classification machine. Acknowledgements The authors gratefully acknowledge the Department of Electronics, Govt. Model Engineering College, Kochi, India, for providing the necessary facilities for carrying out this work. References [1] Binesh, T, Supriya M H and Saseendran Pillai, P R, “A Non-Parametric Estimation based Underwater Target Classifier”, Signal processing: An International Journal, vol 5, no.4, pp.156-164, Oct. 2011. [2] Prakash, C. and Gangashetty, S.V., "Fourier-Bessel cepstral coefficients for robust speech recognition," in Proc. International Conference on Signal Processing and Communications (SPCOM), 2012, vol., no., pp. 22-25. [3] Milner, B. and Darch, J., "Robust Acoustic Speech Feature Prediction from Noisy Mel-Frequency Cepstral Coefficients,”, IEEE Transactions on Audio, Speech, and Language Processing, vol.19, no.2, pp.338,347, Feb. 2011. [4] Kumar, K., Chanwoo Kim and Stern, R.M., "Delta-spectral cepstral coefficients for robust speech recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011, vol., no., pp.4784,4787, 22-27. [5] Siew Wen Chin., Kah Phooi Seng., Ang, Li-Minn., and King Hann Lim, "Improved voice activity detection for speech recognition system," in Proc. International Computer Symposium (ICS), 2010, vol., no., pp.518,523, 16-18. [6] Valero, X. and Alias, F., "Gammatone Cepstral Coefficients: Biologically Inspired Features for Non- Speech Audio Classification," IEEE Transactions on Multimedia, vol.14, no.6, pp.1684,1689, Dec. 2012. American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 7 [7] Zhang He., and Wan Lei; Sun Yushan, "A New Approach to Underwater Target Recognition," in Proc. 2nd International Congress on Image and Signal Processing, 2009. CISP '09, vol., no., pp.1,5, 17-19. [8] Cartmill, J., Azimi-Sadjadi, M.R., and Wachowski, N., "Buried Underwater Object Classification Using a Collaborative Multi-Aspect Classifier," in Proc. International Joint Conference on Neural Networks, IJCNN 2007, vol., no., pp.1807,1812, 12-17. [9] Bender, A., Williams, S.B., and Pizarro, O., "Classification with probabilistic targets," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012, vol., no., pp.1780,1786, 7-12. [10] Xueyao Li, Hua Xie and Bailing Cheng; , "Noisy Speech Enhancement Based on Discrete Sine Transform," in Proc. Computer and Computational Sciences, IMSCCS 2006, vol.1., no., pp.199-202, 20-24. [11] Binesh,T., Supriya,M.H., and P.R. Saseendran Pillai., “Discrete Sine Transform Based HMM Underwater Signal Classifier,” in Proc. International Symposium on Ocean Electronics, SYMPOL, 2011, vol., no. pp.152-155, 16-18. [12] Liying Ma, and Khorasani, K., "Constructive feedforward neural networks using Hermite polynomial activation functions," IEEE Transactions on Neural Networks, vol.16, no.4, pp.821-833, July 2005. [13] Bershad, N.J., Celka, P., McLaughlin, S., "Analysis of stochastic gradient identification of Wiener- Hammerstein systems for nonlinearities with Hermite polynomial expansions," IEEE Transactions on Signal Processing, vol.49, no.5, pp.1060,1072, May 2001. [14] Pawlak Miroslaw, “Nonparametric Identification of a Particular Nonlinear Time Series System”, in Proc. 29th IEEE Conference on Decision and Control, 1990, vol., no., pp. 216-217. [15] Rabiner, L.R., "A tutorial on hidden Markov models and selected applications in speech recognition," Proceedings of the IEEE, vol.77, no.2, pp.257-286, Feb 1989. [16] Sun, Y. and Tong, L., "A lower ordered HMM approach to blind sequence estimation," in Proc. Military Communications Conference, 1998. MILCOM 1998. IEEE, vol.3, no., pp.847-851, 18-21. [17] Donghui Li, Azimi-Sadjadi M.R. and Robinson, M, "Comparison of different classification algorithms for underwater target discrimination,", IEEE Transactions on Neural Networks, vol.15, no.1, pp.189- 194, Jan. 2004. [18] Baum, L.E., Petrie, T., Soules, G., and Weiss, N., “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains,” Annals of Mathematical Stat., vol 41, no.1, pp. 164-171., 1970. American Academic Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) - Volume 93, No 1, pp 1-8 8 [19] Greblicki, W., "Nonparametric identification of Wiener systems by orthogonal series," IEEE Transactions on Automatic Control, , vol.39, no.10, pp.2077,2086, Oct 1994. [20] Selesnick, I.W. and Burrus, C.S., "Generalized digital Butterworth filter design," in Proc. Acoustics, Speech, and Signal Processing ICASSP 1996, vol.3, no., pp.1367-1370, 7-10. [21] Huo Guo-ping; Miao Ling-juan; Ding Hui, "A new method for efficient design of Butterworth filter based on symbolic calculus," in Proc. International Conference on Computer Application and System Modeling (ICCASM), 2010, vol.8, no., pp., 22-24.