Title Science and Technology Indonesia e-ISSN:2580-4391 p-ISSN:2580-4405 Vol. 4, No. 3, July 2019 Research Paper Comparison of Two Priors in Bayesian Estimation for Parameter of Weibull Distribution Ferra Yanuar1*, Hazmira Yozza1, Ratna Vrima Rescha1 1Department of Mathematics, Faculty of Mathematics and Natural Science, Andalas University, Kampus Limau Manis, 25163, Padang-Indonesia *Corresponding author: ferrayanuar@sci.unand.ac.id Abstract This present study purposes to conduct Bayesian inference to estimate scale parameters, denoted by θ, with known location parameter or β, of Weibull distribution. There are two types of prior distributions used in this study, conjugate prior and non informative prior. As conjugate prior is inverse gamma, and as non-informative prior is Jeffreys’ prior. This research also aims to study several theoretical properties of posterior distribution based on prior used implement it to generated data and make comparison between both Bayes estimator as well. The method used to evaluate as the best estimator is based on the smallest Mean Square Error (MSE) value. This study proveds that Bayes estimator using conjugate prior produces better estimated parameter value estimate non-informative prior since it produces smaller MSE value, for condition θ > 1 based on analytic and simulation study. Meanwhile for θ < 1 both priors could not yield acceptable estimated parameter value. Keywords Bayesian inference, Weibull distribution, prior conjugate, non-informative prior, Mean Square Error Received: 20 June 2019, Accepted: 18 July 2019 https://doi.org/10.26554/sti.2019.4.3.82-87 1. INTRODUCTION Weibull models are used to describe various types of ob- served failures of components and phenomena. The Weibull distribution is widely used in life data analysis, reliability engineering and elsewhere due to its versatility and also its relative simplicity. Much of the attractiveness of the Weibull distribution is due to the wide variety of shapes which can as- sume by altering its parameters. The two-parameter Weibull distribution has one shape and one scale parameter. The random variable X follows Weibull distribution with the shape and scale parameters respectively denoted by β,θ >0 has probability density function of the form (Aslam et al., 2014) f(x|θ,β) = β θ x β−1 i exp [ − x β i θ ] ,x ≥ 0,θ,β > 0 In this study, we concern the estimation of the scale parameter of the Weibull distribution, with known shape parameter. Many studies has been done in estimating such parameter using the maximum likelihood estimator (MLE) and/or Bayes method, such as study conducted by Ieren and Oguntunde (2018), Aslam et al. (2014) or Abdulabaas et al. (2013). All studies obtained that Bayes method yielded better estimated value than MLE. In using Bayes method, most of researchers used the non- informative prior as the prior distribution, those are uniform prior or Jeffreys’ prior. Meanwhile many studies suggested to use the conjugate prior in order to obtain better estimated values (Thamrin et al., 2018), (Corrales and Cepeda-Cuervo, 2019). No work has been done in comparing between conju- gate prior and non-informative prior yet. Thus, this present study aims to explore the comparison between both priors distribution in Bayes estimation method analitically and using simulation study (Muharisa et al., 2018). The theorit- ical properties of both Bayes estimator are also studied. In Section 2 we presents the literature review related to Bayes methods including invers Gamma distribution as conjugate prior and Jeffreys’ method as non-informative prior. Results from theoritical and simulation study and also discussions are presented in Section 3. Meanwhile Conclusions from this study are provided in Section 4. 2. BASIC CONCEPTS IN BAYESIAN METHOD Suppose that X1,X2,. . . ,Xn are random sample from the distribution with pdf f(x;θ)where θ is the parameter of the distribution. Estimation of parameters θ will be based on random samples X1,X2,. . . ,Xn. The Bayes method is one parameter estimation method that can be used for https://doi.org/10.26554/sti.2019.4.3.82-87 Yanuar et. al. Science and Technology Indonesia, 4 (2019) 82-87 this purpose. The Bayes method combines the likelihood function and the prior distribution of the parameters to be estimated, so that the posterior distribution is obtained which will be the basis of the parameter estimation (Thamrin et al., 2018). The best estimator is the estimator which produces the smallest MSE value. The MSE of µ is calculated based on the following formulae MSE(µ) = V ar(µ)+[b(µ)]2 (Corrales and Cepeda-Cuervo, 2019). 2.1 Likelihood Function in Weibull Distribution If X1,X2, ..., are random samples from Weibull distribution, written as Wei(θ,β) with unknown β, so that its likelihood function is: f(x|θ) = n∏ i=1 f(xi; θ) = n∏ i=1 β θ x β−1 i exp [ − (xβi θ )] = (β θ )n( n∏ i=1 x β−1 i ) exp [ − (∑n i=1 x β i θ )] (1) 2.2 Invers Gamma Distribution as Conjugate Prior If a random variable Y has Invers Gamma distribution with scale parameter ω > 0 and shape parameter τ > 0 written as Y ∼ IG(τ,ω), are respectively (Aminzadeh and Deng, 2018) f(y; τ; ω) = ωτ Γ(τ) y−(τ+1)e− ω y ,y > 0 and 0 elsewhere Expectation and variance of Y ∼ IG(τ,ω) are formulated as follow; E(Y ) = ω τ − 1 ,τ > 2 V ar(Y ) = ω2 (τ − 1)2(τ − 2) ,τ > 2 In this study, it is assumed that X ∼ WEI(θ,β), with knownβ, and parameter θ will be estimated using Bayes method. It is assumed that parameter θ is a random vari- able and it has Invers Gamma distribution with parameter τ = 1 and ω = 1, written θ ∼ IG(1, 1) as conjugate prior distribution. Probability density function forθ ∼ IG(1, 1) is: f(θ) = 11 Γ(1) θ−(1+1) exp ( − 1 θ ) = θ−2 exp ( − 1 θ ) ,θ > 0 (2) Then the posterior distribution for θ will be determined using Equations (1) and (2) and modifying both of them with the formula as follows: (Maswadah et al., 2013) f(θ|x) = f(θ|x)f(θ)∫ f(θ|x)f(θ)dθ = f(x,θ) f(x) = βn ∏n i=1 x β−1 i θ −(n+2)exp [ − 1 θ ( Σni x β−1 i + 1 )] βn ∏n i=1 x β−1 i ( Σni=1x β i + 1 )−(n+1) Γ(n + 1) = ( Σni=1x β i + 1 )(n+1) Γ(n + 1) θ −(n+2)exp [ −1 θ ( Σni x β−1 i +1 )] (3) Based on Eq. (3) it’s identified that θ|x using conjugate prior has Invers Gamma distribution with parameters n + 1 and ∑n i=1 x β i +1 denoted by θIG|x ∼ IG ( n+1, ∑n i=1 x β i +1 ) . 2.3 Jefferys’ Prior As Non-informatif Prior Jeffreys’ prior is the most widely used noninformative prior in Bayesian analysis. This method is also attractive because it is proper under mild conditions and requires no elicitation of hyperparameters whatsoever. Jeffreys’ rule is derived from likelihood function then take the prior distribution to be the determinant of the square root of the Fisher information matrix (Corrales and Cepeda-Cuervo, 2019), denoted by f(θ) ∝ √ I(θ. Fisher’s information for the parameter θ, is defined as: I(θ) = −nE [ d2 dθ2 ln f(x|θ) ] The natural logarithm of the likelihood function from data that has Weibull distribution is formulated as ln(f(x|θ)) = ln ((β θ )n n∏ i=1 x β−1 i exp [ − (∑n i=1 x β i θ )]) = n[ln β − ln θ] + (β − 1) n∑ i=1 ln x1 ∑n i=1 x β i θ Therefore Fisher’s information for the parameter θ that has Weibull distribution is given by I(θ) = −nE [ n θ2 − 2 ∑ x β i θ3 ] = (n θ )2 Thus, Jeffreys’ prior is defined as f(θ) ∝ √ ( n θ )2 ∝ n θ (4) © 2019 The Authors. Page 83 of 87 Yanuar et. al. Science and Technology Indonesia, 4 (2019) 82-87 Then posterior distribution for θ is constructed as fol- lowing (f(x|θ) = f(x|θ)f(θ)∫ f(x|θ)f(θ)dθ = f(x,θ) f(x) = nβn ∏n i=1 x β−1 i θ −(n+1) exp [ − (∑n i=1 x β i θ )] nβn ∏ n i=1 x β−1 i ( ∑ n i=1 x β i )n Γ(n) = ( ∑n i=1 x β i ) 2 Γ(n) θ−(n+1) exp [ − (∑n i=1 x β i θ )] (5) Eq. (5) informed us that posterior distribution for θ|x using Jeffrey’s prior has Invers Gamma distribution with pa- rameters n and ∑n i=i x β i denoted by θj|x ∼ IG(n, ∑n i=1 x β i ). 3. RESULTS AND DISCUSSION In this section we will present the results of several the- oretical properties of Bayes inference for scale parameter from Weibull distribution using prior conjugate and nonin- formative prior. Both Bayes estimators will be compared using the best estimator criteria, that is the smallest value of Mean square Error (MSE). 3.1 Analytical Study In this section, it will be explained theoretically the estima- tion process of scale parameters θ of the Weibull distribution with known shape parameter β using the Bayes method. In estimating parameter, it uses two types of prior distributions, namely prior conjugate, used inverse Gamma distribution and noninformative prior, that is Jeffrey’s method. 3.1.1 Bayes Inference with Invers Gamma Distribu- tion as Conjugate Prior It has been proved that θIG|x ∼ IG(n + 1, ∑n i=1 x β i + 1). Thus the posterior mean for θIGis given by θ̂IG = E(θIG) = ∑n i=1 x β i + 1 (n + 1) − 1 = ∑n i=1 x β i + 1 n ,n > 0 (6) This posterior mean is also the estimator for θIG of Weibull distribution with known shape parameter. The posterior variance for θIG is expressed as (S′IG) = V ar(θIG) = ( ∑n i=1 x β i + 1) 2 n2(n− 1) ,n > 1 (7) The (1 −α)100% Bayesian credible interval for θ is ap- proximately θ̂Bayes−Zα 2 (S′Bayes) ≤ θ ≤ θ̂Bayes+Zα2 (S ′ Bayes). Thus (1−α)100% Bayesian credible interval for θIG is given by ∑n i=1 x β i + 1 n −Zα 2 (√ ( ∑n i=1 x β i + 1) 2 n2(n− 1) ) ≤ θIG ≤ ∑n i=1 x β i + 1 n + Zα 2 (√ ( ∑n i=1 x β i + 1) 2 n2(n− 1) ) (8) Then, the formula for MSE for θIG is determined by V ar(θ̂IG) and bias(θ̂IG). Formula for V ar(θ̂IG) and bias(θ̂IG) respectively are V ar(θ̂IG) = 1 n2 n∑ i=1 V ar(x β i + 1) = 1 n2 (nθ2) = θ2 n bias(θ̂IG) = E(θ̂IG) −θ = nθ + 1 n −θ = 1 n Therefore formulae for MSE is given by MSE(θ̂IG) = V ar(θ̂IG) + [bias(θ̂IG)] 2 = θ2 n + [ 1 n ]2 = nθ2 + 1 n2 (9) 3.1.2 Bayes Inference with Jeffreys’ Prior As Non- informative Prior Based on Eq. 5 is known that θj|x ∼ IG(n, ∑n i=1 x β i ) Es- timator of scale parameter θ̂j is provided by the posterior mean that is given by θ̂j = E(θj) = ∑n i=1 x β i n− 1 ,n > 1 (10) And the posterior variance is expressed as (S′j)2 = V ar(θj) = (∑n i=1 x β i ) (n− 1)2(n− 2) ,n > 2 (11) The (1 − α)100% Bayesian credible interval for θj is given by ∑n i=1 x β i n− 1 −Zα 2 (√ ( ∑n i=1 x β i ) 2 (n− 1)2(n− 2) ) ≤ θJ ≤ ∑n i=1 x β i n− 1 + Zα 2 (√ ( ∑n i=1 x β i ) 2 (n− 1)2(n− 2) ) (12) While formulae for MSE is approximately V ar(θ̂J) = V ar (∑n i=1 x β i n−1 ) = 1 (n−1)2 ∑n i=1 V ar(X β i ) = 1 (n−1)2 (nθ 2) = © 2019 The Authors. Page 84 of 87 Yanuar et. al. Science and Technology Indonesia, 4 (2019) 82-87 Table 1. Bayes Inference for Scale Parameter, True value for θ=2 Criteria Sample Size Conjugate Prior Non-informative Prior (Invers Gamma) (Jeffreys’ Prior) Estimated Mean n = 25 1.68624 1.71483 n = 100 1.80989 1.81807 n = 150 1.82652 1.83207 Estimated Variance n = 25 0.11847 0.12785 n = 100 0.03308 0.03372 n = 150 0.02239 0.02267 MSE n = 25 0.11533 0.13273 n = 100 0.03285 0.03406 n = 150 0.02228 0.02282 Width of n = 25 1.34927 1.40166 Bayesian Credible Interval n = 100 0.71305 0.71991 n = 150 0.58656 0.59033 nθ2 (n−1)2 and bias(θ̂J) = E(θ̂J) − θ = nθ n−1 − θ = θ n−1 . Thus MSE is expressed as MSE(θ̂J) = var(θ̂J) + [bias(θ̂J)] 2 = nθ2 (n− 1)2 + [ θ n− 1 ]2 = (n + 1)θ2 (n− 1)2 (13) 3.1.3 Comparison of Both MSEs Based on two formulaes for MSE, (θ̂IG) and MSE(θ̂J) ob- tained above, it will be determined which one is better estimator. It has been obtained that MSE(θ̂J) = (n+1)θ2 (n−1)2 . In the case θ > 1, implies θ2 > 1, so MSE(θ̂J) = nθ2 + θ2 (n− 1)2 > nθ2 + 1 (n− 1)2 > nθ2 + θ2 n2 = MSE(θ̂IG) For condition θ > 1, it is proven that MSE(θ̂J) > MSE(θ̂IG) it means the value of the MSE with prior con- jugate (Inverse Gamma) is smaller than MSE with non- informative priors (Jeffreys’ prior). This means that the Bayes estimator with conjugate priors is better estimator than noninformative priors. 3.2 Simulation Study This research then conducted a simulation study to imple- ment the result of analytical study above. The simulation study also aims to examine how similar the estimates of the parameters of the models are, compared with the true values of the parameters. In this simulation study, n = 25, 100 and 150 data were generated from Weibull distribution, WEi(2, 1). Based on n = 25, it is calculated that ∑25 i=1 x β i = 41.1560. Here, it is assumed that β = 1, the value of ,θ will be estimated using the Bayes method. In this study, we consider two conditions for θ, these are θ > 1 and θ < 1. 3.2.1 Bayes Inference for Parameter θ > 1 with Con- jugate Prior It has been obtained that the posterior distribution for θIG is Invers Gamma distribution, written as IG(n + 1, ∑n i=1 x β i + 1). So that the posterior distribution for θIG based on gen- erated data is IG(26 , 42.15606). Posterior mean for θIG is E(θIG) = ∑n i=1 x β i +1 n = 1.686242, posterior variance for θIG is V ar(θIG) = ( ∑n i=1 x β i )2 n2(n−1) = 0.1184755. Furthermore, the MSE value based on generated data is MSE(θ̂IG) = nθ2+1 n2 = 0.1153365. By taking α = 5%, the 95% Bayesian credible interval for θIG is approximately (1.011605 ; 2.360879). 3.2.2 Bayes Inference for Parameter θ > 1 with Non Informative Prior It is obtained that posterior distribution for θJ is Invers Gamma distribution, θJ ∼ IG(n, ∑n i=1 x β i ). Based on gen- erated data, it is obtained θJ ∼ IG(25, 41.15606). The pos- terior mean is E(θJ) = ∑n i=1 x β i n−1 = 1.714836 and posterior variance is V ar(θ̂J) = ( ∑n i=1 x β i )2 (n−1)2(n−2) = 0.1278549. Meanwhile MSE value is MSE(θ̂J) = (n+1)θ2 (n−1)2 = 0.1327382. The 95% Bayesian credible interval for θJ is approximately (1.014003 ; 2.415669) The same process then implemented to n = 100 and 150 generated from Weibull distribution as well. Complete re- sults of calculation for all three groups of data are presented in Table 1. From the Table 1 above, it can be seen that the Bayes estimator with prior Invers Gamma produces the estimated value is closer to the true value for θ = 2 compared to © 2019 The Authors. Page 85 of 87 Yanuar et. al. Science and Technology Indonesia, 4 (2019) 82-87 Table 2. Simulation Study for Estimate Scale Parameter θ <1 for n = 100 Criteria True Value for θ Invers Gamma Prior Jeffreys’ Prior Estimated Mean 0.3 11.1743 11.27708 0.5 1.12287 1.12411 0.7 0.9266 0.92586 0.9 0.83726 0.83561 Estimated Variance 0.3 1.26126 1.29767 0.5 0.01273 0.01289 0.7 0.00867 0.00874 0.9 0.00708 0.00712 MSE 0.3 1.24875 1.31052 0.5 0.0127 0.01302 0.7 0.00868 0.00883 0.9 0.00711 0.00719 Width of Bayesian Confidence Interval 0.3 4.40239 4.46549 0.5 0.44238 0.44512 0.7 0.36505 0.36662 0.9 0.32985 0.33088 Jeffrey’s prior for all three groups data. In general it can be seen that both estimators are consistent estimator because more data, the estimated value increasingly approaching the true value. This present study also proved that the value of variance, the MSE value of Inverse Gamma prior results in smaller value than Jeffrey’s prior for all three selected sample sizes. Invers Gamma Prior also results narrower 95% Bayesian confidence interval for all three sample sizes compared to Jeffreys’ prior. The same estimation process is also carried out for several true values for θ > 1 and linear results are obtained. Thus it can be concluded that for θ > 1, the prior conjugate is better estimator than prior non-informative for case of parameter of Weibull distribution. 3.2.3 Bayes Inference for Parameter θ < 1 In the same ways as in the previous section, simulations are carried out for several values for θ < 1. To save the space, simulation study was done for n = 100 with the true values for θ are 0.3, 0.5, 0.7 and 0.9. Complete results of the study are provided in Table 2. Based on Table 2 above, it can be seen that the Bayes estimator with the two selected priors produces estimated mean that does not approach the corresponding true values. The estimated value is further away from the true value of θ which is set for the value of θ which is close to 0. The estimated value here are not acceptable. Therefore, this simulation study proved that the scale parameter of the Weibull distribution for θ < 1 cannot be used. Based on Table 2 above, it can be seen that the Bayes estimator with the two selected priors produces estimated mean that does not approach the corresponding true values. The estimated value is further away from the true value of θ which is set for the value of θ which is close to θ. The estimated value here are not acceptable. Therefore, this simulation study proved that the scale parameter of the Weibull distribution for θ < 1 cannot be used. 4. CONCLUSIONS This study aims to compare the results of estimating scale parameters of the Weibull distribution by using prior conju- gates and non-informative priors on the posterior distribu- tion. Comparison is done analytically and with simulation studies. Theoretically it is proven that for the value of θ > 1, the MSE value of the prior conjugate will yield a smaller estimation value than non-informative priors, while the value for θ < 1 does not produce a consistent predictive value. While the results of simulation studies proved that the value of θ < 1 will produce a value that is far from the true value so that the estimated value is not acceptable. Whereas for the value of θ > 1, it is concluded that the estimated value approaches the true value by increasing the sample size. Invers Gamma prior produced better estimate mean compared Jeffreys’ prior. This conjugate prior also resulted smaller estimated variance, smaller MSE and nar- rower Bayesian credible interval than Jeffreys’ prior. Thus it can be concluded that inverse gamma prior (prior conjugate) gives the better estimator than Jeffreys’ prior. REFERENCES Abdulabaas, F., Y. M. Al Mayali, and I. A. Neama (2013). A Comparison Between the Bayesian and the Classical © 2019 The Authors. Page 86 of 87 Yanuar et. al. Science and Technology Indonesia, 4 (2019) 82-87 Estimators of Weibull Distribution. Journal of Kufa for Mathematics and Computer, 1(8); 21–28 Aminzadeh, M. S. and M. Deng (2018). Bayesian predictive modeling for Inverse Gamma-Pareto composite distribu- tion. Communications in Statistics - Theory and Methods, 0; 1–17 Aslam, M., S. Mohsin, A. Kazmi, I. Ahmad, and S. H. Shah (2014). Bayesian Estimation for Parameters of the Weibull Distribution. Science International (Lahore), 26(5); 1915– 1920 Corrales, M. L. and E. Cepeda-Cuervo (2019). A Bayesian Approach to Mixed Gamma Regression Models. Revista Colombiana de Estad́ıstica, 42(1); 81–99 Ieren, T. G. and P. E. Oguntunde (2018). A Compari- son between Maximum Likelihood and Bayesian Esti- mation Methods for a Shape Parameter of the Weibull- Exponential Distribution. Asian Journal of Probablity and Statistics, 1(1); 1–12 Maswadah, M., A. M. Seham, and M. Ahsanullah (2013). Bayesian Inference on the Generalized Gamma Distribu- tion Based on Generalized Order Statistics. Journal of Statistical Theory and Applications, 12(4); 356–377 Muharisa, C., F. Yanuar, and D. Devianto (2018). Simula- tion Study of the Using of Bayesian Quantile Regression in Non- normal Error. Cauchy - Jurnal Matematika Murni dan Aplikasi, 5(November); 121–126 Thamrin, S. A., M. N. Zoraida, A. K. Jaya, and Ansariadi (2018). Performance of bayesian using conjugate prior estimator for Weibull right censored survival data. Asian Journal of Scientific Research, 11(3); 376–382 © 2019 The Authors. Page 87 of 87 INTRODUCTION BASIC CONCEPTS IN BAYESIAN METHOD Likelihood Function in Weibull Distribution Invers Gamma Distribution as Conjugate Prior Jefferys' Prior As Non-informatif Prior RESULTS AND DISCUSSION Analytical Study Bayes Inference with Invers Gamma Distribution as Conjugate Prior Bayes Inference with Jeffreys’ Prior As Noninformative Prior Comparison of Both MSEs Simulation Study Bayes Inference for Parameter >1 with Conjugate Prior Bayes Inference for Parameter >1 with Non Informative Prior Bayes Inference for Parameter <1 CONCLUSIONS