8_Romanuk.doc Accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts with boosting ... Проблеми трибології (Problems of Tribology) 2014, № 4 55 Romanuke V.V. Khmelnitskiy National University, Khmelnitskiy, Ukraine, E-mail: romanukevadimv@gmail.com ACCURACY IMPROVEMENT IN WEAR STATE DISCONTINUOUS TRACKING MODEL REGARDING STATISTICAL DATA INACCURACIES AND SHIFTS WITH BOOSTING MINI-ENSEMBLE OF TWO- LAYER PERCEPTRONS UDC 539.375.6+539.538+519.237.8 There is presented a method of improving accuracy in tracking metal tool wear states discontinuously, when the states’ finite set has been statistically tied to the set of representative wear influencing factors. Range of wear states is pre- sumed to be wholly sampled into those factors. The tracker is a static model based on boosting mini-ensemble of three two- layer perceptrons with nonlinear transfer functions. It regards statistical data inaccuracies and shifts. For making the ensem- ble, the AdaBoost technique is used. A distinction of the presented method of boosting from the AdaBoost is in the rule for finding the decreasing coefficient in order to re-distribute weights over training samples. Another one is that the ensemble is aggregated at once. The averaged gain of the boosting mini-ensemble in tracking 24 wear states with 16 influencing factors exceeds 50 %. The wear state tracking model is going to be perfected on optimizing two parameters of the training set and the naive rule for finding the decreasing coefficient before re-distributing training samples’ weights. Key words: wear state, statistical data, data jitter inaccuracies, data omissions, data shifts, tracking model, accuracy, two-layer perceptron, boosting, boosting ensemble, tracking error rate. Benefits of tracking wear states discontinuously In industrial metal processing, rationalized usage of billets and tools is desired. This is partially realized with tracking metal wear states letting prevent underuse and overuse. Discontinuousness of the tracking benefits because its reliability and accuracy are far beyond higher than for continuous approach needing corrections of coefficients in differential and difference equations as time goes by. Besides, decision making on the usage is always of a discrete set of wear states, whose tracking accuracy is never perfect. Approaches to wear state tracking accuracy improvement Improvement of the wear state tracking accuracy is predetermined with the way of tracking wear states. While tracking discontinuously, a finite set of wear states is controlled by a finite difference method or a method of statistical correspondence [1]. The finite difference method accuracy is increased only with more accurate sampling. And the accuracy rank of statistical approximation is dependent on the initial statistical data. With sta- tistically universal approximators based on two-layer perceptron with nonlinear transfer functions (2LPNLTF), it is possible to make the rank higher using specific training samples which could regard statistical data inaccura- cies and shifts [2]. Based on boosting technique [3, 4], ensembles of weak learners may increase the accuracy rank additionally. However, when the number of wear influencing factors (WIF) is of the order of tens [5], such ensembles requiring a few 2LPNLTF haven’t been tried on their performance. In particular, AdaBoost technique (ABT) uses just learners of small configuration [6, 7], although 2LPNLTF might be tried as well. The article goal Under supposition that there is a few tens of WIF corresponding to each known wear state within a fi- nite statistical data set (FSDS), we are to increase the wear state 2LPNLTF-tracking accuracy with mini- ensemble of 2LPNLTF. The ensemble performance is going to be boosted on the basis of ABT. Before boosting, the 2LPNLTF classifier along with its training routine is formalized. The real gain of the boosting will be dis- cussed to focus, probably, on wear state tracking problems having different numbers of WIF, wear states, and engaging various numbers of 2LPNLTF in ensembles. Tracking wear states with boosting mini-ensemble of 2LPNLTF The single object input of 2LPNLTF is [ ]1 Q i Q x X × = ∈ ⊂X ° with Q ∈ • WIF and the output of 2LPNLTF is the number [2] SHL 1 1 * 1, 1, 1 1 arg max 1 exp 1 exp arg max S Q ks i ik k s s s N s N k i s u x a h b v − − = = = =               ∈ + − ⋅ + − + + =                     ∑ ∑ (1) mailto:romanukevadimv@gmail.com Accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts with boosting ... Проблеми трибології (Problems of Tribology) 2014, № 4 56 of the current wear state for the number { }\ 1N ∈ • of total states, where SHLS is number of neurons in the sin- gle hidden layer of 2LPNLTF, and ( )SHL 1S Q N N⋅ + + + coefficients [ ] [ ] [ ] [ ]{ } SHL SHL SHL1 1 , , ,ik ksQ S S N k S Nsa u h b× × × × (2) are to be determined in the training process. FSDS is { } 1 , L jj j w = X by { }\ 1, 1L N∈ −• and 1 j j i Q x X ×  = ∈ X corresponding to the wear state { }1,jw N∈ . All possible wear states are represented in FSDS: { } { } { } 1 1, 1, L j j w N N = =I . Henceforward, the s -th state sj w is reflected with the pure representative sj X . The input of 2LPNLTF is fed successively with the training set [ ]{ }: sjis isQ N iy y x×= =Y (3) and { } { }{ 10 01 1 , 1, , , 0, , 0, 1 , 0, , 0, : h h hHR h hr h h H h H H= = −+ σ ⋅ + µ ⋅ σ ⋅ σ = σ ∀ = ∈ σ > = ξ ξ ∈ µ > = θ θ = ζ ∈=Y Y Y Ξ Θ Ξ ΘY •% % [ ] ( ) [ ] ( )}0 0, 1, , , 0, , 0, 1 , 0, , 0, 1is is is is sQ N Q N× ×+ σ ⋅ + µ ⋅ σ ⋅ σ = σ ∀ = ∈ σ > = ξ ξ ∈ µ > = θ θ = ζ ∈Ξ Θ Ξ Θ• N N (4) by { }0R ∈ • U and the infinite set ( )0, 1N of standard normal variate’s values. For making sure that the pure representatives (3) have not been disassociated from those N wear states, the input of 2LPNLTF is re-fed with the set (3). Having got 16Q = , 24N = , SHL 45S = , [ ]{ } 16 1 0; 1sji i x = ∈ in a problem of 24 wear states tracking, 2LPNLTF (1) was identified by three methods of determining coefficients (2) — “traingda” ( 1α = ), “traingdx” ( 2α = ), “trainscg” ( 3α = ). FSDS for ABT is formed via the set (4) by 1R = , 20H = , 0 0.25σ = , 1.5µ = , whereupon this set tP is re-generated for 100T = times. Thus FSDS for ABT is { } 100 1t t = P including ( ) ( )1 20 24 100 50400R H N T+ ⋅ ⋅ = + ⋅ ⋅ = training samples. At the q -th iteration of boosting, these samples have the weights in ( ) ( ) 1 50400 q d qτ ×=   D by 01,q q= at some final iteration number 0q . Initially, ( ) 11 50400d −τ = 1, 50400∀ τ = . Matrix [ ]3 50400aατ ×=A is of correct responses of classifi- ers, where 1aατ = is the correct classification of τ -th sample by the α -th 2LPNLTF, otherwise 0aατ = . The weighted errors are in the matrix ( ) ( ) 3 50400 q qα ×= η  E , where ( ) ( ) ( ) 50400 1 1q d q aα τ ατ τ= η = ⋅ −∑ , 1, 3α = . (5) Starting from 1q = , there are found each classifier’s weighted error (5), the best 2LPNLTF ( ) ( )* 1, 3 arg minq qα α= α ∈ η , (6) and the minimal weighted error ( ) ( )* 1, 3 minq qα α= η = η , (7) letting learn the coefficient ( ) ( )*1q qγ = − η (8) and calculate the new distribution ( )1q +D of weights ( ) 50400 1 1d q d dτ τ υ υ= + = ∑% % by ( ) ( ) ( )( )* ,exp 2 1qd d q q aτ τ α τ = ⋅ −γ ⋅ − % (9) over samples in { }1001t t =P . If ( ) 1 * 1q N −η < − then q q=% and 1q q= +% , and (5) — (9) are re-found. If ( ) 1* 1q N −η −Ö then 0q q= and there are calculated the following coefficients: Accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts with boosting ... Проблеми трибології (Problems of Tribology) 2014, № 4 57 ( ) ( ) ( ) 0 1 q p q q p = γ = γ γ∑% by 01,q q= for ( ) ( ) { } ( )0 *1, ,q q q q ∈ α=α β α = γ∑ % . (10) Denoting the α -th 2LPNLTF value sv as ( )sv α , the boosted classifier output is * 1, 24 arg max s ss v = ∈ % by ( ) ( ) ( ) ( ) ( ) ( )1 1 2 2 3 3s s s sv v v v= β + β + β% . (11) In the section below we’ll see the real gain of the boosting mini-ensemble of three 2LPNLTF. The gain is defined with the tracking error rate (TER) being the percentage of the classifier’s correct responses among the total inputs. Particularly, ratio between TER of a 2LPNLTF and TER of the ensemble in (11) is of interest. Results and discussion It has been exposed experimentally that TER of the ensemble in (11) is about 56 % lower than TER of the best single 2LPNLTF in tracking 24 wear states. At that the worst ensemble TER is no greater than 1.12 %, and 0.96 % is TER of the best-combined ensemble. At the highest level of jitter inaccuracies and omissions in statistical data or measurements and WIF shift in every state, the mean ensemble TER is 6.82 %, although the gain of the boosting is only 39 %. Another side of the boosting mini-ensemble effectiveness is that variance of wear states’ TER is more than 50 % lower. These results are evidence of the noticeable gain of the boosting based on just three 2LPNLTF. Nearly the same effects were locally registered on some other wear state tracking problems. Consequently, the boosting gain is expected also in solving problems having different numbers of WIF or (and) wear states. Very likely, engaging four and more 2LPNLTF in ensembles will force the gain, im- proving accuracy further. A distinction of the presented method of boosting from ABT is in the coefficient (8). Another one is that the ensemble is aggregated at once. It is fully realizable within MATLAB, and its code may be freely edited for adapting to specified problems of wear state tracking. And as the tracker is ensemble of 2LPNLTF then the classifier operation speed is high enough. While MATLAB-tracking on Intel Core i3-4150 CPU @ 3.50 GHz with 4 GB RAM within 64-bit Windows 7, the ensemble tracks over 2300 wear states per second. Importantly, that the improved averaged accuracy of the ensemble is maintained constant through the whole range of wear states. Conclusion The boosting mini-ensemble of three 2LPNLTF appears capable to track wear regarding statistical data inaccuracies and WIF shifts more accurately. By this model, some WIF at the input may be even omitted. Omis- sions are substituted with elements from ( )0, 1N . However, the accuracy might be improved more, because parameters 100T = and 20H = were assigned empirically. They could be optimized. Also the naive rule (8) is possibly nonoptimal for the decreasing coefficient ( )qγ in re-distributing weights (9). Hence, it is well- promising that the wear state tracking model is going to be perfected. References 1. Chungchoo C. On-line tool wear estimation in CNC turning operations using fuzzy neural network model / C. Chungchoo, D. Saini // International Journal of Machine Tools and Manufacture. – 2002. – Volume 42, Issue 1. – P. 29-40. 2. Romanuke V. V. Wear state discontinuous tracking model as two-layer perceptron with nonlinear transfer functions being trained on an extended general totality regarding statistical data inaccuracies and shifts / V. V. Romanuke // Problems of tribology. – 2014. – N. 3. – P. 50- 53. 3. Champion M. Sparse regression and support recovery with image-Boosting algorithms / M. Cham- pion, C. Cierco-Ayrolles, S. Gadat, M. Vignes // Journal of Statistical Planning and Inference. – 2014. – Volume 155. – P. 19-41. 4. Shen C. Fully corrective boosting with arbitrary loss and regularization / C. Shen, H. Li, A. van den Hengel // Neural Networks. – 2013. – Volume 48. – P. 44-58. 5. Hua J. A cobalt diffusion based model for predicting crater wear of carbide tools in machining tita- nium alloys / J. Hua, R. Shivpuri // Journal of Engineering Materials and Technology. – 2005. – Volume 127. – P. 136-144. 6. Nie Q. Probability estimation for multi-class classification using AdaBoost / Q. Nie, L. Jin, S. Fei // Pattern Recognition. – 2014. – Volume 47, Issue 12. – P. 3931-3940. 7. Sun Y. Unifying multi-class AdaBoost algorithms with binary base learners under the margin frame- work / Y. Sun, S. Todorovic, J. Li // Pattern Recognition Letters. – 2007. – Volume 28, Issue 5. – P. 631-643. Accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts with boosting ... Проблеми трибології (Problems of Tribology) 2014, № 4 58 Романюк В. В. Покращення точності у дискретній моделі відслідковування стану зносу з урахуванням похибок і зсувів у статистичних даних на основі міні-комітету бустингу двошарових персептронів. Представляється метод покращення точності дискретного відслідковування станів зносу металевого засобу, коли скінченна множина цих станів була статистично пов’язана з множиною репрезентативних факторів, що впли- вають на знос. Діапазон станів зносу вважається повністю розбитим за цими факторами. Відстежувачем є статична модель на основі міні-комітету бустингу трьох двошарових персептронів з нелінійними передавальними функціями. Вона враховує похибки і зсуви у статистичних даних. Для утворення згаданого комітету використовується техніка адаптивного бустингу. Одна з відмінностей методу бустингу, що представляється, від адаптивного бустингу полягає у правилі знаходження спадного коефіцієнта для того, щоб перерозподіляти ваги навчальних зразків. Ще одна полягає у тому, що комітет утворюється одразу. Усереднений виграш такого міні-комітету бустингу у відслідковуванні 24 станів зносу з 16 факторами впливу перевищує 50 %. Дана модель відслідковування стану зносу буде удосконалена завдяки оптимізації двох параметрів навчальної множини і наївного правила знаходження спад- ного коефіцієнта перед перерозподілом ваг навчальних зразків. Ключові слова: стан зносу, статистичні дані, флуктуаційні похибки у даних, пропуски у даних, зсуви у даних, модель відслідковування, точність, двошаровий персептрон, бустинг, комітет бустингу, рівень помилок відслідковування.