EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS
2025, Vol. 18, Issue 2, Article Number 6162
ISSN 1307-5543 – ejpam.com
Published by New York Business Global

A Simulation Study of Some Logistic, Poisson, and
Multiple Ridge Regression Estimators

Jerson S. Mohamad1, Angelyn S. Delica1, Maydalyn H. Esperat1,
Aubrey G. Labastilla1, Shiela May M. Ledesma1, Doeyien D. Misil1

1Department of Mathematics and Statistics, College of Science and Mathematics,
Western Mindanao State University, 7000, Zamboanga City, Philippines

Abstract. This paper is a Monte Carlo simulation study of some logistic, Poisson, and multiple
ridge regression estimators. This study proposes new ridge regression estimators using linear
combinations of known ridge parameters, developed through grid search and methods that leverage
mean squared error (MSE) values from prior simulations. The performance of each known and
proposed estimators are then compared using MSE criterion. Results show that the proposed
estimators performed better on many cases. Furthermore, each estimator was applied to secondary
data and was compared based on their respective estimated coefficients.

2020 Mathematics Subject Classifications: 62J07, 62J12, 65C05

Key Words and Phrases: Ridge regression, ridge parameter, logistic regression, Poisson regres-
sion, multiple regression, Monte Carlo simulation, stats R package

1. Introduction

Regression analysis is a powerful tool in statistics, widely used to model relationships
between variables. However, challenges such as multicollinearity and data-specific require-
ments can limit the effectiveness of traditional regression methods. In response to these
challenges, ridge regression techniques have been developed to enhance model stability and
accuracy [1, 2]. The general form of ridge regression applies an L2 penalty to the coeffi-
cient estimates, modifying the standard maximum likelihood (ML) estimation to control
overfitting. This paper focuses on logistic ridge regression, Poisson ridge regression, and
multiple ridge regression. Related studies are found in [3–8].

DOI: https://doi.org/10.29020/nybg.ejpam.v18i2.6162

Email addresses: mohamad.jerson@wmsu.edu.ph (J. Mohamad),
angelyn.sanchez@wmsu.edu.ph (A. Delica), esperat.maydalyn@wmsu.edu.ph (M. Esperat),
aubrey.guerrero@wmsu.edu.ph (A. Labastilla), micubo.shiela@wmsu.edu.ph (S. Ledesma) ,
doeyien.misil@wmsu.edu.ph (D. Misil)

https://www.ejpam.com 1 Copyright: © 2025 The Author(s). (CC BY-NC 4.0)



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 2 of 10

Logistic Ridge Regression is applied in binary classification problems. The likelihood
function is pi = 1/(1 + e−xT

i β) and the ridge-regularized log-likelihood function is

J(β) = −
n∑

i=1

[Yi log(pi) + (1− Yi) log(1− pi)] + k

p∑
j=1

β2
j ,

where pi represents the predicted probabilities for the outcome Y , n is the number of
observations, p is the number of independent variables, Y is the dependent n× 1 vector,
X is the independent n × p matrix, β is the p × 1 coefficient vector, and k is the ridge
parameter.

Poisson ridge regression is used to model count data. The likelihood function is

L(β) =
n∏

i=1

ex
T
i βYi

Yi!
,

and the ridge-regularized log-likelihood function is

J(β) = −
n∑

i=1

[
Yi x

T
i β − ex

T
i β
]
+ k

p∑
j=1

β2
j .

The coefficients are estimated by solving

β̂ridge = argmin
β

J(β).

Multiple Ridge Regression applies the ridge penalty to multiple linear regression, which
models the relationship between a continuous dependent variable and multiple predictors.
The ridge estimator modifies the Ordinary Least Squares (OLS) estimator:

β̂ridge = (XTX + kI)−1XTY

2. Methodology

This study investigates the performance of ridge-regularized estimators applied to lo-
gistic regression, Poisson regression, and multiple linear regression models in the presence
of multicollinearity. The objective is to compare several known ridge parameter estima-
tors and newly proposed ones based on their ability to minimize the mean squared error
(MSE) of coefficient estimates. A Monte Carlo simulation framework is employed for the
evaluation.

2.1. Theoretical Framework for Ridge Estimation

The ridge estimator for the regression coefficients β is

β̂ridge = (XTX + kI)−1XTY



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 3 of 10

where k > 0 is the ridge parameter, I is the p× p identity matrix, and X is the matrix of
predictors. This estimator is particularly useful in the presence of multicollinearity, where
the OLS estimator becomes unstable.

To analyze the effect of ridge regularization, we express the linear model using spectral
decomposition. Let the matrix XTX have eigen-decomposition:

XTX = DΛDT ,

where D is an orthogonal matrix of eigenvectors and Λ = diag(λ1, λ2, . . . , λp) contains the
eigenvalues of XTX. Defining Z = XD and α = DTβ, the model becomes:

Y = Zα+ ε.

The ridge estimator of α is:

α̂(k) = (ZTZ + kI)−1ZTY.

The mean squared error (MSE) of α̂(k) can be decomposed into variance and bias
components and is given by:

MSE(α̂(k)) = σ2
p∑

i=1

(
λi

λi + k

)2

+

p∑
i=1

(
kα2

i

λi + k

)2

,

where σ2 is the error variance, and αi is the ith element of α. In practice, since σ2 and α
are unknown, the estimated MSE used for comparison is:

M̂SE(α̂(k)) = σ̂2
p∑

i=1

(
λi

λi + k

)2

+

p∑
i=1

(
kα̂2

i

λi + k

)2

.

This framework is also extended to logistic and Poisson regression models via their
respective iterative methods of approximations. For these models, the MSE is computed
based on the penalized likelihood estimates of β̂.

2.2. Ridge Parameter Estimators

The following are the known ridge parameters compared in this simulation study:

HK = k1 = k̂HK1 =
σ2

a2max

, where σ2 =

∑n
i=1(yi − µ̂i)

2

n− p− 1
[1];

k2 = k̂HKM =
1

a2max

[2];

k3 = k̂GM =
σ2(∏p

i=1 a
2
i

)1/p [9];
k4 = k̂MED = Median(m2

i ), where mi =
σ2

a2i
[10];



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 4 of 10

k5 = k̂SS = max(si), where si =
tiσ

2

(n− p)σ2 + tia2i
[10];

k6 = k̂KM2 = max

(
1

mi

)
[11];

k7 = k̂KM4 =

(
p∏

i=1

1

mi

)1/p

[11];

k8 = k̂KM5 = median

(
1

mi

)
[11].

2.3. Proposed Estimators

For logistic and Poisson regression, new ridge estimators k9l and k9p are introduced as
weighted linear combinations of the above estimators. The weights are computed based
on the inverse of the mean MSE from preliminary simulations:

ci =
1

Mean MSE of ki∑
j

1
Mean MSE of kj

.

For multiple linear regression, the proposed estimator k9m is determined using a grid
search method. A sequence of k values from 0.01 to 10 is tested, and the value minimizing
the MSE is selected as the optimal k, denoted ksearch. Regression analysis is then performed
on ksearch values using the known ridge parameters and model dimensions (n and p) to
generate a predictive formula for k9m.

2.4. Simulation Procedure

The simulation design consists of the following steps:

1. Parameter Setup. The parameter ranges are defined as follows: the sample size n
varies from 10 to 250; the number of predictors p ranges from 2 to 12; the correlation
level ρ is between 0.70 and 0.99 inclusively; and the intercept β0 is set to either 0 or
1. These parameter values are commonly used in simulation studies [3–5].

2. Generate Multicollinear Predictors. The matrix X ∈ Rn×p is generated using
the method of McDonald and Galarneau [12]:

xij =
√

1− ρ2 · zij + ρ · zi0, i = 1, . . . , n, j = 1, . . . , p,

where zij ∼ N (0, 1) are standard normal random variables, inducing correlation
among predictors.

3. Generate Response Variable. Set the true coefficients β such that∑
β2
i = 1. Generate the response vector Y appropriate for each regression model

(logistic, Poisson, or linear) using X and β.



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 5 of 10

4. Estimate Coefficients and Compute MSE. For each estimator, the regression
method is applied then optimized through the optim function in R to compute β̂
and evaluate the MSE:

MSE =
1

p
(β − β̂)T (β − β̂).

5. Repetition and Averaging. Perform steps 2 to 4 for a total of 1,000 times.
For each estimator, compute the average MSE across the 1,000 replications. The
estimator with the lowest average MSE is considered the best estimator.

3. Results

In developing new ridge estimators, the following results were determined and are
proposed based on their performance in their respective regression models:

k9l = 0.451 · k6 + 0.269 · k7 + 0.2797 · k8 for logistic regression,

k9p = 0.290 · k2 + 0.169 · k6 + 0.283 · k7 + 0.253 · k8 for Poisson regression, and

k9m = k̂search = 0.329 · n+ 0.464 · p− 0.282 · k2,−0.176 · k6 for multiple regression.

The following subsections discusses the results in tables for each regression model, Tables 1,
3 and 5 shows the simulation results where the proposed estimator has the least estimated
MSE.

3.1. Logistic Ridge Regression Simulation Results

For logistic ridge regression, the widely used ML estimator, the known HK, k2 to
k8 estimators and the proposed k9l estimator are applied and compared in simulations.
Table 1 shows some of the simulation results where the MSE values of each estimator are
compared when β0 = 0 and ρ = 0.99 for p = 2, 3, 4, 8, 12 with n = 20p+ 10, 30p, 40p, 60p,
and 100p. It is observed in the simulation that the new estimator k9l has the least MSE
in some cases. Specifically, when β0 = 0, ρ = 0.99, p = 2, 3, 4, 8, 12 and when n =
60, 80, 160, 170, 180, 240, and 250.

Table 1: Estimated MSEs for the Logistic Ridge Regression Simulation with β0 = 0 and ρ = 0.99

Estimators
p n ML HK k2 k3 k4 k5 k6 k7 k8 k9l
2 60 0.658 0.569 0.370 0.533 0.416 0.517 0.217 0.219 0.214 0.213
2 80 0.452 0.403 0.277 0.386 0.316 0.380 0.161 0.166 0.161 0.159
3 180 6.817 5.097 2.355 2.601 0.812 1.747 0.076 0.078 0.073 0.070
4 160 13.74 10.81 5.144 4.518 1.244 3.088 0.099 0.096 0.087 0.086
8 170 42.43 35.62 17.41 10.89 3.43 9.19 0.129 0.129 0.113 0.103
8 240 27.98 23.67 11.98 7.215 2.87 7.63 0.088 0.157 0.127 0.082
12 250 53.22 46.255 23.671 13.07 5.048 14.199 0.105 0.180 0.140 0.090



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 6 of 10

To apply the logistic estimators to secondary data, we used the breast cancer data
obtained from Taha (2022, Kaggle dataset). We set the dependent variable as the type
of cancer diagnosis (0=Benign Cancer, 1 = Malignant Cancer) with n = 180 and p = 3.
Three explanatory variables are included: radius mean (R), perimeter mean (P) and area
mean (A), which represent specific average values of the cancer image features.

Table 2 shows the coefficient results of the bivariate correlation r to Diagnosis and the
different logistic regression estimator’s coefficients. It can be observed that only the esti-
mated coefficients of k6, k7, k8, and the new estimator k9l has the same sign as that of the
correlation coefficients of the independent variables to the dependent. On the other hand,
the ML and HK estimates do not match the signs to that of the correlation coefficients, a
clear indicator of the negative effect of multicollinearity present in the secondary data.

Table 2: Correlation Coefficients r and Estimated Logistic Ridge Coefficients of each Estimator

Estimators
Var r ML HK k2 k3 k4 k5 k6 k7 k8 k9l
R 0.71 -43.9 -43.8 -42.4 -40.7 -11.4 -28.1 3.02 2.99 2.94 3.02
P 0.73 38.03 37.9 36.7 35.1 9.7 24.6 0.004 0.024 0.048 0.007
A 0.71 11.32 11.3 11.07 10.8 5.2 8.8 0.003 0.021 0.043 0.007

3.2. Poisson Ridge Regression Simulation Results

Table 3: Estimated MSEs for the Poisson Ridge Regression Simulation with βo = 0 and p = 2

Estimators
p n ML HK k2 k3 k4 k5 k6 k7 k8 k9p

0.85 15 0.102 0.077 0.032 0.074 0.072 0.070 0.054 0.034 0.034 0.032
0.85 20 0.064 0.054 0.023 0.045 0.040 0.050 0.028 0.022 0.020 0.020
0.90 15 0.142 0.091 0.031 0.065 0.062 0.087 0.054 0.035 0.033 0.031
0.90 20 0.076 0.061 0.025 0.044 0.041 0.060 0.030 0.026 0.022 0.022

0.95 10 2.392 0.714 0.107 0.484 0.129 0.352 0.122 0.067 0.076 0.064
0.96 10 4.892 0.474 0.084 0.282 0.113 0.241 0.134 0.075 0.087 0.073
0.96 20 0.123 0.080 0.025 0.077 0.074 0.081 0.030 0.024 0.021 0.021

0.97 10 2.256 0.594 0.114 0.413 0.147 0.336 0.129 0.075 0.085 0.072
0.98 10 2.785 0.454 0.079 0.294 0.111 0.246 0.131 0.074 0.085 0.069
0.99 20 0.468 0.151 0.025 0.109 0.063 0.123 0.026 0.019 0.018 0.016

In Poisson ridge regression, the ML estimator, the known HK, k2 to k8 estimators
and the proposed k9p estimator are applied and compared in simulations. Table 3 shows
some of the simulation results where the MSE values of each estimator are compared when
β0 = 0, ρ = 0.85, 0.90, 0.95, 0.96, 0.97, 0.98, 0.99, p = 2, 3 with n = 10, 15, 20, 30, 50. It
has been observed that the new Poisson ridge estimator (k9p) has the least MSE in some
cases. Specifically, when β0 = 0, p = 2, 3 at the different levels of ρ and values of n.



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 7 of 10

Applying the Poisson Ridge estimators to the secondary data from the Botswana De-
mographic and Health Survey (Central Statistics Office, 1988). We set the number of
children as the dependent variable, and year born and age as the independent variables
. Table 4 shows the coefficient results of the bivariate correlation coefficients r to the
number of children and the different Poisson regression estimator’s coefficients. It is ob-
served that k3 to k8 and the proposed k9p estimated coefficients coincides with the signs
in the correlation coefficients of the independent variables to the dependent. On the other
hand, the ML and HK estimated coefficients does not match the sign of the correlation
coefficient an indication of the negative effect of multicollinearity in the secondary data.

Table 4: Correlation Coefficients r and Estimated Poisson Ridge Coefficients of Estimators

Estimators
Var r ML HK k2 k3 k4 k5 k6 k7 k8 k9p
Year -0.73 4.9 0.393 1.32 -0.53 -0.57 -0.57 -0.59 -0.55 -0.58 -0.57
Age 0.74 5.53 1.00 1.93 0.074 0.03 0.029 0.012 0.047 0.022 0.031

3.3. Multiple Ridge Regression Simulation Results

In the multiple ridge regression, Table 5 shows some results of the simulation where
the MSE values of each estimator. Overall, it has been observed that the proposed ridge
estimator k9m has the least MSE in some cases. Specifically, when ρ ≥ 0.72, p ≥ 3 and
n ≥ 10.

Table 5: Estimated MSEs for the Multiple Ridge Regression Simulation with p = 3

Estimators
p n ML HK k2 k3 k4 k5 k6 k7 k8 k9m

0.72 10 0.083 0.064 0.039 0.047 0.437 0.049 0.026 0.037 0.032 0.024
0.72 20 0.032 0.029 0.023 0.027 0.024 0.027 0.018 0.024 0.023 0.014
0.72 100 0.007 0.007 0.006 0.004 0.005 0.006 0.006 0.007 0.007 0.004

0.85 10 0.157 0.106 0.051 0.066 0.060 0.084 0.026 0.041 0.034 0.022
0.85 20 0.057 0.047 0.031 0.031 0.032 0.045 0.021 0.033 0.029 0.012
0.85 150 0.006 0.006 0.006 0.005 0.005 0.006 0.005 0.006 0.006 0.005

0.90 10 0.220 0.134 0.058 0.072 0.063 0.102 0.022 0.036 0.030 0.018
0.90 20 0.083 0.064 0.037 0.040 0.041 0.063 0.022 0.037 0.033 0.013
0.90 30 0.049 0.041 0.027 0.026 0.027 0.041 0.019 0.030 0.028 0.009

0.95 10 0.472 0.276 0.103 0.128 0.082 0.161 0.019 0.030 0.024 0.018
0.95 20 0.161 0.105 0.047 0.056 0.056 0.098 0.020 0.037 0.029 0.012
0.95 30 0.109 0.080 0.042 0.046 0.048 0.079 0.022 0.038 0.033 0.010

Applying the estimators to secondary data obtained from the World Bank (2023),
which provides economic and labor data for the Philippines from 2010–2021 as shown in
Table 6. We set the dependent variable Unemployment as percent of total labor force
regressed by the independent variables consumer price index or CPI (2010 base year),



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 8 of 10

wholesale price index or WPI (2010 base year), total labor force or Labor (L), total
population (P), and household (H) final consumption expenditure (annual percent growth)
data from year 2010 to 2021.

Table 6: Some Philippine Indicator Values from World Bank (2023)

Year CPI WPI Labor(L) Population(P) Household(H) Unemployment

2010 100.0000 100.0000 38081598 38081598 3.589800 3.61
2011 104.7184 108.6630 39490704 96337913 5.552134 3.59
2012 107.8882 109.9084 40075145 98032317 6.798970 3.50
2013 110.6746 111.7949 40789298 99700107 5.819003 3.50
2014 114.6565 115.7143 42179845 101325201 5.784455 3.60
2015 115.4295 117.6191 42622144 103031365 6.444572 3.07
2016 116.8766 118.6996 43756699 104875266 7.149608 2.70
2017 120.2113 120.9341 42974771 106738501 5.957367 2.55
2018 126.5938 123.2875 43800369 108568836 5.765216 2.34
2019 129.6220 125.3114 45091808 110380804 5.866915 2.24
2020 132.7241 128.3150 42419079 112190977 -7.956529 2.52
2021 137.9364 132.2527 44857443 113880328 4.211487 3.40

Table 7 shows the bivariate correlation coefficients r of each independent variable to
the Unemployment variable and the different multiple ridge estimators’ coefficients. It can
be noted that k6 to k8 and the proposed k9m coefficients match the signs of the correlation
coefficients. On the other hand, the OLS and HK estimated coefficients do not match all
the signs of the correlation coefficients an indication of the presence of multicollinearity in
the secondary data.

Table 7: Correlation Coefficients r and Estimated Ridge Regression Coefficients of Estimators

Estimators
Var r ML HK k2 k3 k4 k5 k6 k7 k8 k9m
CPI -0.63 2.93 2.69 1.62 0.94 0.95 0.35 -0.10 -0.0006 -0.08 -0.095
WPI -0.63 1.09 1.10 1.04 0.79 0.795 0.34 -0.01 -0.005 -0.08 -0.098
L -0.69 -0.88 -0.93 -1.11 -1.06 -1.06 -0.74 -0.17 -0.37 -0.24 -0.195
P -0.70 -3.79 -3.51 -2.18 -1.31 -1.32 -0.59 -0.15 -0.27 -0.19 -0.167
H 0.22 0.28 0.30 0.38 0.36 0.36 0.24 0.04 0.10 0.059 0.047

4. Conclusion

The method of ridge regression is useful in reducing the negative effects of multi-
collinearity by using a small bias in the form of a ridge parameter k. There have been
many proposed ridge parameter, the best ridge parameter has yet been determined and
no constant value of k is certain to give an estimator that is consistently better in terms
of MSE than the ML or OLS.



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 9 of 10

In this study, we used Monte Carlo simulation with 1,000 replications in the R software.
In all the simulation combinations considered, where the number of independent variables
p, level of correlation ρ, intercept β0, and sample sizes n are varied, the simulation results
showed that overall the known and proposed ridge parameter estimators has lower MSE
than that of the OLS, ML, and HK estimators. Furthermore, the proposed estimators
performed best on some cases. Finally, the methods of the known and proposed estimators
were applied to multicollinear data from secondary sources and found that the proposed
estimators’ coefficients match the signs of the corresponding correlation coefficient.

Acknowledgements

This research is funded by the Higher Innovations Fund of the Western Mindanao State
University, Zamboanga City, Philippines.

References

[1] A.E. Hoerl and R.W. Kennard. Ridge regression: biased estimation for non-
orthogonal problems. Technometrics, 12:55–67, 1970.

[2] A.E. Hoerl and R.W. Kennard. Ridge regression: application to non-orthogonal
problems. Technometrics, 12:69–82, 1970.

[3] K. Månsson and G. Shukur. On ridge parameters in logistic regression. Communica-
tions in Statistics—Theory and Methods, 40:3366–3381, 2011.

[4] K. Månsson and G. Shukur. A poisson ridge regression estimator. Economic Mod-
elling, 28(4):1475–1481, 2011.

[5] A. Göktaş, Ö. Akkuş, and A. Kuvat. A new robust ridge parameter estimator based
on search method for linear regression model. Journal of Applied Statistics, 48(13-
15):2457–2472, 2020.

[6] O. Akbilgic and H. Bozdogan. Predictive subset selection using regression trees and
rbf neural networks hybridized with the genetic algorithm. European Journal of Pure
and Applied Mathematics, 4(4):467–485, 2011.

[7] S. Mermi, Ö. Akkuş, and A. Göktaş. How well do ridge parameter estimators proposed
so far perform in terms of normality, outlier detection, and mse criteria? Communi-
cations in Statistics - Simulation and Computation, 53(1):1–67, 2024.

[8] S. Mermi, Ö. Akkuş, A. Göktaş, and N. Gündüz. A new robust ridge parameter esti-
mator having no outlier and ensuring normality for linear regression model. Journal
of Radiation Research and Applied Sciences, 17(1):100788, 2024.

[9] B.M.G. Kibria. Performance of some new ridge regression estimators. Communica-
tions in Statistics—Theory and Methods, 32:419–435, 2003.

[10] M.A. Alkhamisi, G. Khalaf, and G. Shukur. Some modifications for choosing ridge
parameter. Communications in Statistics—Theory and Methods, 35:1–16, 2006.

[11] G. Muniz and B.M.G. Kibria. On some ridge regression estimators: an empirical com-
parison. Communications in Statistics—Simulation and Computation, 38:621–630,
2009.



J. Mohamad, et. al. / Eur. J. Pure Appl. Math, 18 (2) (2025), 6162 10 of 10

[12] G.C. McDonald and D.I. Galarneau. A monte carlo evaluation of some ridge-type
estimators. Journal of the American Statistical Association, 70(350):407–416, 1975.