Equally - weighted compositions of Gaussian-noised – data - trained t wo - layer perceptrons in boost ing ensembles for high - accurate discontinuous... Проблеми трибології (Problems of Tribology) 2015, № 2 53 Romanuke V.V. Khmelnit skiy Nat ional University, Khmelnit skiy, Ukraine, E-m ail: romanukevadimv@gmail.com EQUALLY - WEIGHTED COMPOSITIONS OF GAUSSIAN - NOISED - DATA - TRAINED TWO - LAYER PERCEPTRONS IN BOOSTING ENSEMBLES FOR HIGH - ACCURATE DISCONTINUOUS TRACKING OF WEAR STATES REGARDING STATISTICAL DATA INACCURACIES AND SHIFTS UDC 539.375.6+539.538+519.237.8 Equally - weighted comp ositions of Gaussian - no ised-data - trained two - lay er p ercep trons are studied in order to track metal wear states more accurately at the high est level of statistical data in accur acies and shifts (noise). The noise r an ge is modeled throu gh four magn itudes char acterizin g ultimate jitters and shifts in wear influ encin g factors. Accuracy and v ari- ance gains of equally - weighted comp ositions seem to be increasin g when no ise intensities become lower. Wh en boosting ensembles are comp osed from ordinary classifiers, h igh-accurate trackin g fails. Only comp osing ensemb les from a lot of the best op timized p erceptrons, the accuracy imp roves by 1,5 % for the averaged track in g error rate and by 7,7 % for the tracking error rate at noise maximum. Her e, the boosting ap p ears to have its limit. But ensembles of equally -weighted co mp ositions of p erceptrons p erform even better than ensembles of p ercep trons weighted after training. And for ensurin g high- accurate dis- continuous tracking of wear states, we just need p erceptrons trained by quite different backp rop agation methods. Ke y wor ds: wea r state tracking, statistical data, data inaccuracies, data shifts, accuracy, two-layer perceptron, boosting, boosting ensemble, trac king error rate. Ge neration of statistical data inaccur acies and shifts in me asuring me tal we ar states While metal billets are processed, metal processing tools are worn within a range of metal wear states (MWS) being specific for the type of metal and peculiarit ies of processing. The range is ordered by wear accu- mu lation. While meta l details are run, the range is wider due to that intensity of wear is much less er. MWS are measured (scored) via wear influencing factors (WIF), including direct (speed, pressure, tem- perature, etc.) and indirect (time duration) ones. Measuring MWS is needful for controlling wear process and prevention of underuse and overuse of both tools and details. The measure ment is a ground for trac king MWS along the whole life cyc le. The t racke r is usually stated after arranging the huge statistical data (HSD). Ho wever, these data are inaccurate as wear is very stochastic process. Additionally, MWS measurements may have instru- mental and systematic b ias errors wh ich shift WIF being matched to the corresponding MWS. That is why statis- tical data inaccuracies and shifts (SDIS) in measuring MWS hinder in effective application of continuous differ- ential-equations-based-models for t racking MWS and forecasting them. And using stochastic differentia l equa- tions doesn’t help much as expectance evaluation is not so reliable and volatility is generated very high. Thus discontinuous models are preferable, though they require HSD to set reliable statistical correspondence [1] in the form of finite statistical data set (FSDS) inc luding each of  \ 1N  wear states. FSDS is   1, L L jj j F w   X by  \ 1, 1L N  and the wear state  1,jw N labeled as 1 j j i Q x X     X for Q WIF. The range of MWS is presumed to be wholly sampled into those WIF, and so      1 1, 1, L j j w N N   by the s -th state sj w labeled as the pure representative (PR) sj X . SDIS a re generated as a result of the wear is influenced with innume rable mu ltitude of factors, what lets treat those inaccu- racies as norma l noise. Due to that, the classifie r can be two-layer perceptron with nonlinear transfer functions (2LPNLTF) [1]. Th is is a universal classifier, performing great ly on Gaussian - noised data (GND). Nonetheless, possible shifts in FSDS and shifts in input data make single 2LPNLT F classifier noneffective at higher intens i- ties of the shift noise. But compositions of GND-tra ined 2LPNLTF may improve accuracy of tracking MWS [2]. Therefore, boosting ensembles of GND-tra ined 2LPNLTF are to be tried e xhaustively. Gain in accurac y of tr acking MWS at higher inte nsities of SDIS using boosting ense mbles Intensities of SDIS become higher when either the metal life cycle has run near its half or MWS are measured with poor-calibrated instrument. The accuracy of tracking MWS is tracking error rate (TER) being the percentage ratio of the classifier’s incorrect responses to the classifier’s total inputs , although the percentage of the classifier’s correct responses among its total inputs is imp lied by T ER. Equally - weighted compositions of Gaussian-noised – data - trained t wo - layer perceptrons in boost ing ensembles for high - accurate discontinuous... Проблеми трибології (Problems of Tribology) 2015, № 2 54 Based on a boosting technique stated in [2], ensemble of three 2LPNLTF solved a problem of trac king 24 MWS with 16 WIF in that the mean ensemble T ER was 6,82 % at the highest level of SDIS in every state. In term of T ER, the averaged gain (relative decre ment of T ER or something else) of the boosting exceeded 50 %, and variance of wear states’ T ER beca me more than 50 % lower with the ensemble. For the same problem, the classifier was optimized in [3] for improv ing accuracy of the tracking. Hav- ing increased number of 2LPNLTF within the ensemb le up to 30, the averaged gain with the optimized ensemble became about 56 % in respect of the best ensemble of three 2LPNLT F. Similarly, variance of TER over 24 MWS beca me about 53 % lower. However, nearly the same results were reg istered when the ensemble was composed without training, but just setting the weight of every 2LPNLTF to one thirtieth. The e xp lanation is that all those 2LPNLT F we re roughly simila rly-t rained GND classifiers, without focusing on specific SDIS. Hence, equally-we ighted compositions of  \ 1, 30B  GND-tra ined 2LPNLTF a re believed to track MWS in that problem more accurately. The goal of c omposing a boosting ensemble of GND-traine d 2 LPNLTF for high-acc urate tr acking Taken the spoken problem of tracking 24 MWS with 16 WIF fro m [2, 3], there should be composed a boosting ensemble of equally-we ighted GND-t rained 2LPNLT F for high-accurate tracking at the highest level of SDIS. The result is expected to be better than in [3]. In any case, evaluation of the ratio between TER for single 2LPNLT F and TER for the ensemble is going to be plotted. This implies both TER on average and TER at the highest level of SDIS. Similar plots of the boosting gain will be done for the variance of wear states’ TER. Accurac y and variance g ains of e qually-weighte d c ompositions of GND-traine d 2 LP NLTF In general statement, the boosting ensemble as B 2LPNLT F equally-we ighted composition output is * 1, arg max s N ss m    by   1 1 B s sm mB    (1) on the forecasted MWS  * 1, arg max s s N s m    (2) by the  -th 2LPNLTF [3], giving the value  sm  in its s -th output neuron. A 2LPNLTF itself is trained with training sets   : sjis isQ N iy y x Y and     10 01 1 , 1, , , 0, , 0, 1 , 0, , 0, : h h hHR h hr h h H h H H                          Y Y Y Ξ Θ Ξ ΘY         10 0, 1, , , 0, , 0, 1 , 0, , 0, 1is is is is sQ N Q Nh H h H H                          Ξ Θ Ξ ΘN N (3) for PR and SDIS correspondingly, where    1 0; 1s Qj i i x   and  0R   , and  0, 1N is the infinite set of standard norma l variate’s values. Term h Ξ in (3) is responsible for modeling jitter inaccuracies and omis- sions in statistical data or measurements. And term h  Θ in (3) models WIF shifts in every state. Coefficient 0 characterizes ultimate jitters and  is ratio between the suspected jitters and WIF shifts [1]. These ones were put to 0 0.25  and 1.5  whilst the problem of trac king 24 MWS was studied in [2, 3]. Denote by    the averaged TER of the  -th 2LPNLT F along with its TER  ;H   at SDIS ma ximu m. For the ensemble of B 2LPNLT F, B is the averaged TER and  B H  is TER at SDIS ma xi- mu m. Henceforwa rd, the accuracy gains are the averaged TER gain and the gain of T ER at SDIS ma ximu m      1TE R 1,minB Bg B         and       1TER 1,; min ;H B H HBg B           (4) respectively. For seeing the boosting gain for the variance of wear states’ TER, denote by  ,v N the aver- aged variance of N wear states’ TER of the  -th 2LPNLT F along with its variance  ; ,Hv N  at SDIS ma ximu m. For the ensemble of B 2LPNLTF,  Bv N is the averaged variance of N wear states’ TER and  ;B Hv N is variance at SDIS ma ximu m. Henceforwa rd, the variance gains are the averaged variance gain Equally - weighted compositions of Gaussian-noised – data - trained t wo - layer perceptrons in boost ing ensembles for high - accurate discontinuous... Проблеми трибології (Problems of Tribology) 2015, № 2 55 and the gain of variance at SDIS ma ximu m       1var 1,, min ,B Bg B N v N v N        and       1var 1,; , ; min ; ,H B H HBg B N v N v N         (5) respectively. The accuracy gains (4) for the proble m of trac king 24 MWS with 16 WIF a re shown in Figure 1 and Figure 2, where 1R  , 10H  , and SDIS a re modeled by  0 0.15, 0.2, 0.25, 0.3  and  1, 1.25, 1.5, 2 . (6) The variance gains (5) for the spoken problem are shown in Figure 3 and Figure 4, plotted by (6). The incre ment of SDIS intensity according to (6) is designated with dots, circ les, squares, and diamonds, respectively. 10 20 30 40 50 60 70 80 90 0 1 2 3 4 5 6 7 10 20 30 40 50 60 70 80 90 1 2 3 4 5 6 Fi g. 1 – The ave rage d TER gain Fi g. 2 – The gain of TER at SDIS maximum 10 20 30 40 50 60 70 80 90 0 1 2 3 4 5 6 7 8 9 10 20 30 40 50 60 70 80 90 1 2 3 4 5 6 7 8 9 10 11 Fi g. 3 – The gain of ave rage d varian ce of 24 wear states’ TER Fi g. 4 – The gain of varian ce at SDIS maximum Figures 1 - 4 show clearly that those gains unexpectedly are about those ones for 30 GND-trained 2LPNLT F weighted within the ensemble after training [3]. It means that tracking MWS more accurately fails when any  \ 1, 30B  GND-trained 2LPNLTF a re taken either into equally-we ighted composition or into ensemble to be trained. Notwithstanding the fail, high-accurate discontinuous tracking (HADCT) is available if ensemble is composed of GND-trained 2LPNLTF of higher accuracy (HA2LPNLTF). Sure ly, such HA2LPNLTF occur rarer than an ordinary GND-tra ined 2LPNLTF. A mong 17818 ord inary GND-tra ined 2LPNLTF with 70 neurons in hidden layer by R  1 and H  18, there happened to be found 60 HA2LPNLTF performing at    „ 1,17 for the problem by 0 0.25  0,25 and   1,5, and the equally-weighted composition of these HA2LPNLT F re inforced by resume-tra ining performs at 60  0,62,  60 20   4,43,  60 24 0.14v  0,14,  60 20 ; 24 5.4v   5,4. Thus, the cor- responding gains remain roughly as those ones in Figures 1 - 4, although HADCT is realizable. Discussion of the gains and findi ngs Having processed HSD for trying HADCT, the boosting appears to have its limit. Une xpectedly, but HADCT here is rea lizab le only by using a lot of the best HA2LPNLT F. The ga ins (4) and (5) see m to be increas- ing when intensities of SDIS beco me lo wer. But they have their own limits, nearly corresponding to the circled polylines in Figures 1 - 4. Co mparing the gains to those ones in [3], the accuracy improvement e xists anyway. It is about 1,5 % for the averaged TER and 7,7 % for T ER at SDIS ma ximu m. And ensembles of equally-weighted Equally - weighted compositions of Gaussian-noised – data - trained t wo - layer perceptrons in boost ing ensembles for high - accurate discontinuous... Проблеми трибології (Problems of Tribology) 2015, № 2 56 compositions of GND-trained 2LPNLT F perform even better than ensembles of GND-tra ined 2LPNLTF weighted after tra ining. For ensuring HADCT, we just need 2LPNLTF trained by quite different methods of backpropagation - “traingda”, “traingd x”, “tra ingdm” , “tra ingd”, “trainlm”, “tra inscg”, etc. References 1. Ro manuke V.V. Wear state discontinuous tracking model as two-layer perceptron with nonlinear transfer functions being trained on an extended general totality regarding statistical data inaccuracies and shifts / V. V. Ro manuke // Proble ms of tribology. – 2014. – N. 3. – P. 50-53. 2. Ro manuke V.V. Accuracy improve ment in wear state discontinuous tracking model regarding statis- tical data inaccuracies and shifts with boosting mini-ensemble of two-layer perceptrons / V.V. Ro manuke // Proble ms of tribology. – 2014. – N. 4. – P. 55-58. 3. Ro manuke V.V. Optimizing parameters of the two-layer perceptrons’ boosting ensemble training for accuracy improve ment in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts / V.V. Ro manuke // Proble ms of tribo logy. – 2015. – N. 1. – P. 65-68. UDC 539.375.6+539.538+519.237.8 Посту пила в р едакцію 15.04.2015 Романюк В.В. Рівнозва жені склади двошаро вих персептронів, навчених на даних з гаусовими шума- ми, у комітетах бустингу для високоточного дискретного відслідковування станів зносу з урахуванням похи- бок і зсувів у статистичних даних. Вивчаються р івнозважені склади двошар ових пер септр онів, навчених на даних з гау совими шу мами, для того, щоб відслідкову вати стани зносу металу більш точно за найвищого р івня похибок і зсу вів у статистичних да- них (шу му ). Діапазон цього шу му моделюється за чотир ма ампліту дами, що хар актер изу ють гр аничні флу кту аційні похибки та зсу ви у фактор ах впливу на знос. Вигр аші у точно сті та диспер сії цих р івнозважених складів здаються зр остаючими пр и зменшенні інтенсивностей шу му . Коли комітети бу стингу складаються зі звичайних класифікато- р ів, високоточне відслідкову вання не вдається. Лише пр и складанні комітетів зі значного ч исла найкр ащих оптимі- зованих пер септр онів точність покр ащу ється на 1,5 % для у сер едненого р івня помилок відсл ідкову вання і на 7,7 % для р івня помилок відслідкову вання за максимального шу му . Здається, бу стинг ту т має свою межу . Однак комітети р івнозважених скл адів пер септр онів фу нкціону ють навіть кр аще, ніж комітети пер септр онів, що зважу ються після навчання. І для забезпеч ення високоточного дискр етного відсл ідкову вання станів зносу нам необхідні пр осто пер се- птр они, навчені за доволі р ізними мето дами звор отного пошир ення. Ключові слова: відслідкову вання стану зносу , статистичні дані, по хибки у даних, зсу ви у даних, точність, двошаро- вий пер септр он, бу стинг, комітет бу стингу , р івень помилок відслідкову вання.