









































Pa
ge

 
1



Pa
ge

 
12

7

American Journal of  Applied 
Statistics and Economics (AJASE)

Maximizing Predictive Regression and Dimensionality Reduction Techniques: Evidence 
from Monte Carlo’s Simulation Study

Oluwafemi Clement Onifade1*, Samuel Olayemi Olanrewaju1, Emmanuel Segun Oguntade1

Volume 4 Issue 1, Year 2025
ISSN: 2992-927X (Online)

DOI: https://doi.org/10.54536/ajase.v4i1.5938
https://journals.e-palli.com/home/index.php/ajase

Article Information ABSTRACT

Received: August 16, 2025

Accepted: September 19, 2025

Published: October 18, 2025

This study proposes a novel two-step sparse learning framework that combines Sparse 
Principal Component Regression (SPCR) with regularization methods, Lasso, Elastic Net, 
Ridge, and Smoothly Clipped Absolute Deviation (SCAD), to improve prediction and 
interpretability in high-dimensional settings. Simulation experiments were conducted under 
varying sample sizes, dimensionality levels, sparsity conditions, and predictor correlations 
to evaluate the performance of  the hybrid estimators in comparison to traditional 
penalization approaches. Results show that SPCR-Lasso and SPCR-Enet consistently deliver 
superior accuracy and stability in high-dimensional, multicollinear contexts, with SPCR-
Enet performing particularly well in extreme dimensionality. SPCR-SCAD demonstrated 
advantages in sparse, low-correlation scenarios, while Ridge regression contributed modest 
improvements. These findings underscore that estimator performance is strongly data-
dependent and highlight the value of  SPCR hybridization for mitigating multicollinearity 
while enhancing interpretability. The study offers practical guidance for applied researchers 
in fields such as genomics, finance, and climate science, and contributes methodologically 
by demonstrating the robustness of  SPCR-based regularization in handling complex high-
dimensional data structures.

Keywords

Elastic Net, High-Dimensional 
Data, Lasso, Multicollinearity, 
Regularization, SCAD, SPCR 

1 Department of  Statistics, Faculty of  Science, University of  Abuja, Abuja, Nigeria
* Corresponding author’s e-mail: onifade.oluwafemi@yahoo.com

INTRODUCTION 
High-dimensional modelling has emerged as a critical 
and transformative area of  research with profound 
implications across diverse domains, including data 
science, machine learning, and statistics. The prominence 
of  high-dimensional data can be attributed to the 
prevalence of  large-scale datasets and complex systems in 
various applications. In high-dimensional modelling, the 
term “high dimensional” refers to situations where the 
number of  explanatory variables, denoted as p, exceeds the 
number of  observations, n (i.e., p > n). This phenomenon 
has gained attraction due to the rapid advancements in 
technology, which enable the collection of  a vast number 
of  variables to better understand complex phenomena of  
interest. The applicability of  high-dimensional modelling 
spans multiple fields, including computational chemistry, 
Chemometrics with spectral data, genomics, fMRI data 
analysis, large-scale healthcare analytics, text/image 
analysis, astronomy, and many others.
The versatility of  high-dimensional modelling techniques 
has also been demonstrated in the field of  drug discovery 
and development. For example, Priya et al. (2022) focus 
on the application of  machine-learning approaches in 
chemo-informatics for drug discovery. Machine learning 
techniques, specifically QSAR (Quantitative Structure-
Activity Relationship), have effectively modelled 
various physicochemical properties of  drugs, including 
toxicity, absorption, and drug-drug interactions. These 
approaches, being a subset of  artificial intelligence, 
show great potential in drug discovery by handling non-
linear datasets and big data with increasing complexity. 

However, the curse of  dimensionality, a well-known 
challenge in high-dimensional data, poses significant 
obstacles to accurate predictions and efficient parameter 
estimation. The exponential growth of  data volume with 
increasing variables leads to sparse data points, which 
can hinder the effectiveness of  traditional methods. 
Multicollinearity, a common issue in high-dimensional 
datasets, further complicates parameter estimation and 
can result in inflated confidence intervals.
To address these challenges, sophisticated techniques are 
required that can effectively handle the complexities posed 
by high-dimensional data. Dimensionality reduction and 
variable selection methods have emerged as attractive 
strategies to tackle high-dimensional studies. Over the 
last two decades, regularization approaches such as 
lasso, elastic net, ridge regression, and Smoothly Clipped 
Absolute Deviation (SCAD) have become the methods 
of  choice for analyzing high-dimensional data. These 
regularization methods have been extensively applied in 
various disciplines, including statistics (Nwosu et al. 2024), 
chemo-informatics (Song et al., 2024), epidemiology 
(Cleophas et al. 2024), and bioinformatics (Kitano et al., 
2024), and many others. 
In recent years, Sparse Principal Component Regression 
(SPCR), and Sparse Partial Least Squares (SPLS), has 
garnered attention as a potential solution to improve 
predictive model accuracy. By identifying a small subset of  
the original predictor variables that capture most of  the 
variance in the data, SPCR facilitates highly interpretable 
models with enhanced predictive accuracy. SPCR 
has demonstrated promising results in various fields, 



Pa
ge

 
12

8

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

including medical research, finance, and environmental 
sciences. SPCR has also proven successful in QSAR 
modelling by identifying the most relevant molecular 
descriptors that greatly influence biological activity or 
molecule properties. For example, Zhang et al. (2024) 
demonstrated the effectiveness of  SPCR in identifying 
the most important features for predicting the antitumor 
activity of  molecules, leading to more reliable QSAR 
models. While previous studies have predominantly 
focused on combining principal component regression 
(PCR) with regularization techniques in low-dimensional 
settings, where the number of  predictors is less than the 
observations, there is a clear need for sparse PCR. Sparse 
PCR can be more advantageous in situations with a large 
number of  predictor variables, as it identifies a smaller 
subset of  the original predictors that are most crucial in 
predicting the response variable.
In this thesis, we aim to address the challenges posed 
by high-dimensional data through the application of  
regularization techniques and Sparse Principal Component 
Regression (SPCR). By developing a novel two-step 
sparse learning approach that integrates Sparse Principal 
Component Regression (SPCR) with regularization 
techniques (Ridge regression, Lasso, Elastic Net, and 
Smoothly Clipped Absolute Deviation). This combined 
approach seeks to enhance predictive accuracy and 
interpretability in high-dimensional datasets, particularly 
in scenarios involving multicollinearity and sparsity. The 
specific objectives include to:

i. Develop efficient framework that combine SPCR 
with regularization methods.

ii. Assess the performance of  the combined approach 
using traditional modeling techniques like Lasso and Ridge 
through predictive accuracy measures, i.e. mean square error.

iii. Design a simulation study to demonstrate the 
robustness of  the proposed approach across multiple 
high-dimensional datasets, varying in sample size, 
multicollinearity, and sparsity levels.

LITERATURE REVIEW
Empirical work on high-dimensional prediction 
has converged on two broadly successful strategies. 
The first reduces dimensionality via latent factors or 
components (e.g., Principal Component Regression — 
PCR), which mitigates multicollinearity and variance 
inflation (Jolliffe, 2002; Hastie et al., 2009). The second 
directly penalizes regression coefficients to induce 
shrinkage and (sometimes) sparsity (Ridge, Lasso, Elastic 
Net, SCAD), which controls overfitting and performs 
variable selection when appropriate (Hoerl & Kennard, 
1970; Tibshirani, 1996; Zou & Hastie, 2005; Fan & Li, 
2001). Empirical comparisons show neither approach 
dominates across all data regimes: PCR is robust under 
extreme multicollinearity but produces components that 
are not tailored to prediction of  the response, while 
penalized regressions are powerful for sparse signals but 
can struggle when predictors are highly correlated (Hastie 
et al., 2009).

To bridge the gap between unsupervised dimension 
reduction and predictive goals, researchers developed 
Sparse Principal Component Analysis (SPCA) and Sparse 
Principal Component Regression (SPCR). SPCA (Zou et 
al., 2006; Witten et al., 2009) imposes sparsity on loadings 
so principal components involve only a subset of  
predictors, improving interpretability without discarding 
the variance-reduction benefit of  PCA. Empirical 
studies in genomics, chemometrics, and neuroimaging 
have found SPCA yields components that are easier to 
interpret and often more useful as inputs for supervised 
tasks than dense PCA components.
SPCR, either formulated as a one-stage joint optimization 
of  component extraction and regression loss or as 
a carefully tuned two-stage procedure, goes further 
by explicitly constructing components that optimize 
predictive performance (Kawano, 2018; Zou et al., 2006). 
Empirical comparisons show SPCR often outperforms 
classical PCR when the directions of  maximal predictor 
variance differ from the directions most predictive of  
the outcome (i.e., when supervised signal does not 
align with principal variance directions). Applications 
in biological data and other high-dimensional domains 
report improved prediction and sparser, more actionable 
component loadings (Zou et al., 2006; Kawano, 2018).
There is substantial empirical evidence that different 
regularizers perform differently depending on correlation 
structure and sparsity. Ridge excels when many predictors 
carry signal but are highly correlated; it reduces variance 
without producing sparse solutions, often improving out-
of-sample prediction in dense-signal, collinear settings 
(Hoerl & Kennard, 1970). Also, Lasso provides both 
shrinkage and variable selection and works well when the 
true model is sparse and predictors are not excessively 
collinear; empirical studies show it can fail to reliably select 
the “correct” group in the presence of  strong predictor 
correlation (Tibshirani, 1996). Elastic Net empirically 
combines strengths of  Ridge and Lasso, grouping 
correlated predictors while performing variable selection; 
simulation and applied work show Elastic Net often 
outperforms Lasso under grouped-correlated designs 
(Zou & Hastie, 2005). SCAD and other nonconvex 
penalties (Fan & Li, 2001) demonstrate favorable oracle 
properties in theory and often reduced bias empirically 
compared to Lasso, but they require careful tuning and are 
more sensitive to initialization and optimization choices. 
Empirical simulation studies repeatedly demonstrate there 
is no uniformly best penalty: performance depends on 
(i) sparsity level, (ii) inter-predictor correlation, (iii) signal 
strength, and (iv) sample size. This motivates this study 
that systematically maps performance across different 
scenarios rather than relying on single-case comparisons.

MATERIALS AND METHODS
Development of  Novel Two-Step Sparse Learning 
Techniques
To address the challenges of  high-dimensional data 
analysis, this study integrates Sparse Principal Component 



Pa
ge

 
12

9

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Regression (SPCR) with regularization techniques such as 
Ridge, Lasso, Elastic Net, and SCAD. The proposed two-
step sparse learning approach combines the strengths 
of  dimensionality reduction (SPCR) with the variable 
selection and regularization capabilities of  these methods, 
ensuring both interpretability and predictive accuracy.

Step 1: Dimensionality Reduction Using Sparse 
Principal Component Regression (SPCR) Workflow
Compute sparse principal components T=XW by solving: 
minimize‖Y-XWα‖2

2+λ1 ‖W‖1+λ2 ‖W‖F’
2

Where W is the matrix of  sparse component weights, α 
is the regression coefficient vector, and λ1 and λ2 control 
sparsity and shrinkage.

i. Select the top k components based on the proportion 
of  variance explained and their relevance to Y.

ii. Output the reduced dataset T, a sparse representation 
of  X.

Step 2: Regularized Regression on Reduced Components
After dimensionality reduction, apply regularized 
regression techniques (Ridge, Lasso, Elastic Net, and 
SCAD) to the reduced dataset T to build predictive models 
while managing overfitting and multicollinearity. This 
means that the resulting equations involve combining the 
dimensionality reduction framework with the respective 
penalty functions of  the chosen regularization technique.

Ridge Regression with SPCR
Objective Function:
minimize‖Y-Tβ‖2

2+λ‖β‖2
2                                                                                           

Where, 
T=XW: Sparse components derived using SPCR,
λ: Regularization parameter controlling the degree of  
shrinkage.
β: Regression coefficients
‖β‖2

2 :L2-norm penalty that shrinks all coefficients toward 
zero but does not enforce sparsity,

Lasso Regression with SPCR
Objective Function:
minimize‖Y-Tβ‖2

2+λ‖β‖1,                                                                                              
Where:
T=XW: Sparse components from SPCR,
‖β‖1: L1-norm penalty that enforces sparsity by shrinking 
some coefficients to exactly zero,
Lasso regularization enhances variable selection by 
retaining only the most relevant components or predictors.

Elastic Net with SPCR
Objective Function:
minimize 
‖Y-Tβ‖2

2+λ1‖β‖1+λ2‖β‖2
2                                                                                                                                                                    

Where,
T=XW: Sparse components from SPCR,
β: Regression coefficients
‖β‖1:  Enforces sparsity (Lasso component),
‖β‖2

2: Mitigates multicollinearity and provides stability 

(Ridge component),
λ1: Controls sparsity,
λ2: Controls shrinkage.
Elastic Net is particularly effective when predictors 
are highly correlated, as it selects groups of  correlated 
components.

SCAD with SPCR
Objective Function:
minimize‖Y-Tβ‖2

2+∑j=1
ppλ (|βj|),                                                                                  

where, T=XW: Sparse components from SPCR,
β: Regression coefficients
pλ (|βj|) is the SCAD penalty function, 
λ controls the penalty’s strength.

Advantages of  Combining Regularization with SPCR
i. SPCR reduces dimensionality while Ridge or 

Lasso enhances the predictive power by handling 
multicollinearity or enforcing sparsity.

ii. SCAD further refines the predictor selection process, 
reducing bias for large coefficients while retaining sparse 
predictors.

Simulation Study
The simulation study aims to evaluate and compare 
the performance of  sparse learning methods (Lasso, 
Ridge, Elastic Net, SCAD, and SPCR) under controlled 
and varied conditions. This study focuses on predictive 
accuracy, interpretability, and computational efficiency, 
providing insights into the strengths and weaknesses 
of  these methods in high-dimensional settings. The 
following sections detail the design, dataset characteristics, 
evaluation metrics, comparative testing procedures, and 
the approach for data analysis and interpretation

Design of  the Simulation Study
The simulation study replicates real-world challenges 
by systematically varying key parameters: sample size, 
predictor dimensionality, levels of  multicollinearity, noise, 
and sparsity. These variations ensure a comprehensive 
evaluation of  the methods’ performance across diverse 
conditions, reflecting practical scenarios in high-
dimensional data analysis.

Sample Sizes
Four small sample scenarios are considered: n=30, 
n=50, n=70 and n=100. The small sample sizes (n=30 
and n=50) represent the most challenging setting where 
predictors (p) far exceed observations (p>n), crucial for 
assessing the methods’ ability to avoid overfitting. While 
higher small sample sizes (i.e. n=70 and n=100) explore 
scalability and performance in more balanced or low-
dimensional settings.

Predictor Dimensionality
Predictor dimensionality (p) varies from low (p=20) 
to high (p=200). Low-dimensional scenarios allow 
methods to demonstrate baseline predictive capabilities 



Pa
ge

 
13

0

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

without dimensionality-related challenges. Moderate 
and high-dimensional settings introduce significant 
computational and statistical challenges, such as sparsity 
and multicollinearity.

Multicollinearity
Multicollinearity is varied at four levels:

Low (ρ~0.1): Predictors are weakly correlated, 
minimizing interference among variables.

Moderate (ρ~0.5): Predictors form correlated blocks, 
testing methods like Elastic Net and SPRC designed to 
handle such scenarios.

High (ρ~0.9): Many predictors are extremely 
interrelated, challenging methods like Lasso, which may 
arbitrarily select variables from correlated groups.

Sparsity
Predictor sparsity is varied to test variable selection 
capabilities:

• Sparse (10% non-zero coefficients): Only a small 
fraction of  predictors are relevant, providing a benchmark 
for variable selection.

• Dense (30% non-zero coefficients): Many predictors 
have small, non-zero effects, testing the methods’ capacity 
to identify subtle contributions.

Data Generation Process
The data is generated as: Y = Xβ+ϵ      
where:
X~N(0,Σ), with Σij=ρ for i≠j), controlling the level of  
multicollinearity
β is a sparse vector with randomly assigned non-zero 
coefficients drawn from N(0,1),
∈~N(0,σ2) is Gaussian noise adjusted to achieve desired R2.

Evaluation Metrics
Performance will be assessed using the following metrics 

Mean Squared Error (MSE)
MSE measures the average magnitude of  the errors in the 
predicted values. It is often preferred over MSE as it is in 
the same units as the response variable, making it easier 
to interpret. The formula for MSE is:
MSE=1/n ∑i=1

n(yi-y^i)
2 

The estimation model with lowest MSE would be 
considered the best. Afterwards, the R-squared of  the 
returned best model would be assessed.

R-squared (R2)
Indicates the proportion of  variance explained by the 
predictors, with higher values representing better model 
fit:
R2=1-(∑i=1

n(yi-y^i)
2 )/(1/n ∑i=1

n(yi-y¯i)
2 )                                                                                                        

RESULTS AND DISCUSSION
Simulated Data Presentation and Preliminary Assessment
This section presents and discusses the simulated 
response variable and explanatory variables at different 
scenarios of  high dimensionalities as shown in Table 1, 
Table 2, Table 3 and Table 4. The simulated response 
variable and explanatory variables at each scenarios were 
generated using Y = Xβ+ϵ;                                                                                                                 
where:
X~N(0,Σ), with Σij= ρ for i≠j), controlling the level of  
multicollinearity
β is a sparse vector with randomly assigned non-zero 
coefficients drawn from N(0,1),
∈~N(0,σ2)is Gaussian noise adjusted to achieve desired 
R2.

Table 1: Summary Statistics of  the Simulated Response Variables and Last 7-Explanatory Variables at Sample Size 30
No of  Predictors 50
Variables Y X44 X45 X46 X47 X48 X49 X50
Mean 1.324 0.0594 -0.010 0.0731 -0.074 -0.022 0.228 0.147
Median 1.966 0.115 -0.060 0.117 -0.104 0.142 0.472 0.090
Min. -7.318 -1.916 -2.043 -1.994 -2.250 -2.129 -2.08 -1.615
Max. 11.453 2.401 2.479 2.309 2.065 1.410 2.158 2.236
No of  Predictors 70
Variables Y X64 X65 X66 X67 X68 X69 X70
Mean 1.386 -0.098 -0.025 0.039 0.039 0.038 0.082 0.009
Median 2.610 -0.092 0.039 0.090 0.080 0.048 -0.124 -0.055
Min. -12.311 -1.823 -1.564 -1.552 -1.609 -1.823 -1.678 -2.005
Max. 14.267 1.833 1.956 2.013 1.941 1.833 2.217 1.948
No of  Predictors 200
Variables Y X194 X195 X196 X197 X198 X199 X200
Mean -0.497 0.036 0.047 -0.006 0.088 0.153 0.194 0.147
Median 0.344 -0.202 -0.095 -0.175 -0.152 0.063 -0.048 -0.115
Min. -15.905 -2.442 -2.609 -2.907 -2.132 -2.079 -1.888 -2.019
Max. 13.879 2.539 2.532 2.787 2.764 2.834 2.566 2.488

Source: Researchers’ Compilations from R-Output



Pa
ge

 
13

1

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Table 2: Summary Statistics of  the Simulated Response Variables and Last 7 Independent Variables at Sample Size 50
No of  Predictors 70
Variables Y X44 X45 X46 X47 X48 X49 X50
Mean -0.048 0.062 -0.101 -0.125 -0.099 0.067 -0.106 -0.014
Median 0.237 0.065 -0.064 -0.145 0.114 -0.217 0.057 -0.167
Min. -8.288 -2.070 -2.035 -2.378 -3.486 -1.716 -3.099 -1.494
Max. 7.115 2.594 2.245 2.372 2.079 2.811 1.618 1.872
No of  Predictors 100
Variables Y X94 X95 X96 X97 X98 X99 X100
Mean -1.232 0.163 -0.106 -0.013 0.159 0.234 0.206 0.054
Median -0.968 0.161 -0.023 -0.024 0.066 0.128 0.188 -0.086
Min. -20.891 -1.975 -3.081 -1.798 -3.041 -2.014 -2.255 -1.978
Max. 25.595 2.494 2.305 1.374 2.568 3.348 2.241 2.529
No of  Predictors 200
Variables Y X194 X195 X196 X197 X198 X199 X200
Mean -0.375 0.069 0.096 0.071 0.080 0.065 0.063 0.071
Median -0.808 0.147 0.235 0.376 0.313 0.349 0.349 0.368
Min. -10.372 -2.383 -2.492 -2.409 -2.333 -2.398 -2.475 -2.467
Max. 20.619 1.669 1.574 1.607 1.627 1.641 1.577 1.572

Source: Researchers’ Compilations from R-Output

Table 3: Summary Statistics of  the Simulated Response Variables and Last 7 Independent Variables at Sample Size 70
No of  Predictors 100
Variables Y X94 X95 X96 X97 X98 X99 X100
Mean -0.472 0.027 0.006 0.085 0.024 0.026 -0.075 0.034
Median -1.048 -0.001 0.117 0.107 0.008 -0.079 -0.031 -0.051
Min. -10.022 -2.215 -2.177 -2.139 -1.665 -2.279 -3406 -2.084
Max. 10.800 2.041 1.928 1.884 2.376 2.274 2.203 2.419
No of  Predictors 150
Variables Y X144 X145 X146 X147 X148 X149 X150
Mean 0.537 -0.114 -0.148 -0.157 -0.159 -0.178 -0.148 -0.147
Median 1.510 -0.245 -0.202 -0.195 -0.243 -0.225 -0.202 -0.215
Min. -15.961 -2.112 -1.982 -1.688 -1.980 -1.995 -1.960 -1.995
Max. 19.764 2.894 3.079 3.063 2.869 3.028 3.328 3.134
No of  Predictors 200
Variables Y X194 X195 X196 X197 X198 X199 X200
Mean -0.444 0.022 0.003 0.001 0.013 -0.043 -0.026 -0.0001
Median -0.695 -0.124 -0.110 -0.194 -0.109 -0.077 -0.077 -0.101
Min. -7.515 -1.702 -1.643 -1.661 -1.711 -1.659 -1.575 -1.492
Max. 7.247 2.599 2.542 2.495 2.553 2.492 2.144 2.257

Source: Researchers’ Compilations from R-Output



Pa
ge

 
13

2

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Explicitly, Table 1 presents the summary statistics of  
simulated response-variables and last 7-explanatory 
variables at different scenarios of  p>(n=30), as p was 
varied across 50, 70 and 200. The n=30 is our first small 
sample settings representing the most challenging setting 
where predictors (p) far exceed observations. According 
to the table at (p=50)>(n=30), the simulated response-
variable has mean of  1.32 ranges between -17.32 and 
11.45. Also, the table show that at (p=70)>(n=30) the 
simulated response-variable has mean of  1.39 ranges 
between -12.31 and 14.27. Likewise, Table 1 reveals that 
at (p=200)>(n=30) the simulated response-variable has 
mean of  -0.497 ranges between -15.91 and 13.88. 
Similarly, Table 2 presents the summary statistics of  
simulated response-variables and last 7-explanatory 
variables at different scenarios of  p>(n=50), as p 
was varied across 70, 100 and 200. The n=50 is our 
second small sample settings also representing the 
most challenging setting where predictors (p) far exceed 
observations. According to the table at (p=70)>(n=50), 
the simulated response-variable has mean of  -0.048 
ranges between -8.29 and 7.12. Also, the table show 
that at (p=100)>(n=50) the simulated response-variable 
has mean of  -1.232 ranges between -20.89 and 25.59. 
In addition, Table 2 reveals that at (p=200)>(n=50) the 
simulated response-variable has mean of  -0.375 ranges 
between -10.37 and 20.62.
Furthermore, Table 3 presents the summary statistics 
of  simulated response-variables and last 7-explanatory 
variables at different scenarios of  p>(n=70), as p was 
varied across 100, 150 and 200. The n=70 is our first 
higher small sample size considered to explore scalability 
and performance in more balanced or low-dimensional 

settings. According to the table at (p=100)>(n=70), the 
simulated response-variable has mean of  -0.472 ranges 
between -10.022 and 10.800. Also, the table show that 
at (p=150)>(n=70) the simulated response-variable has 
mean of  0.537 ranges between -15.961 and 19.764. Table 
3 further reveals that at (p=200)>(n=70) the simulated 
response-variable has mean of  -0.444 ranges between 
-7.515 and 7.247.
Moreover, Table 4 presents the summary statistics of  
simulated response-variables and last 7-explanatory 
variables at different scenarios of  p>(n=100), as p was 
varied across 120, 150 and 200. The n=100 is our second 
higher small sample size considered to explore scalability 
and performance in more balanced or low-dimensional 
settings. According to the table at (p=120)>(n=100), the 
simulated response-variable has mean of  -0.086 ranges 
between -24.882 and 23.759. Also, the table show that 
at (p=150)>(n=100) the simulated response-variable has 
mean of  0.164 ranges between -10.883 and 12.130. Table 
4 further reveals that at (p=200)>(n=100) the simulated 
response-variable has mean of  0.449 ranges between 
-11.441 and 12.423. 
Based on the foregoing it is quite evident that simulated 
dataset obviously exhibits high-dimensionality problem 
(i.e. p>n), thus necessitate advanced methods of  
regression estimation other than the OLS.

Performance Assessment of  Ridge, Lasso, Elastic 
Net, SCAD and the Novel Two-Step Sparse 
Learning Methods under High Dimensionality and 
Multicollinearity 
This section presents and discusses the performances 
of  the celebrated ridge, lasso, elastic net, SCAD and our 

Table 4: Summary Statistics of  the Simulated Response Variables and Last 7 Independent Variables at Sample 
No of  Predictors 120
Variables Y X114 X115 X116 X117 X118 X119 X120
Mean -0.086 0.132 0.085 -0.052 0.044 -0.126 -0.065 -0.008
Median 1.026 0.158 -0.005 -0.001 0.051 -0.006 0.016 0.089
Min. -24.882 -2.411 -2.646 -3.345 -2.594 -2.825 -3.218 -2.975
Max. 23.759 1.936 2.944 3.344 2.349 2.028 2.685 3.341
No of  Predictors 150
Variables Y X144 X145 X146 X147 X148 X149 X150
Mean 0.164 0.047 -0.042 -0.053 0.089 -0.086 -0.202 -0.045
Median 0.588 0.020 0.049 -0.178 0.109 -0.144 -0.263 -0.018
Min. -10.883 -2.446 -2.409 -2.554 -2.487 -2.239 -2.630 -2.769
Max. 12.130 3.384 2.371 2.677 2.796 2.784 2.064 2.635
No of  Predictors 200
Variables Y X194 X195 X196 X197 X198 X199 X200
Mean 0.449 -0.146 -0.047 -0.079 -0.077 -0.076 -0.070 0.070
Median 0.333 -0.125 0.064 -0.089 -0.036 -0.184 0.053 0.082
Min. -11.441 -2.284 -2.749 -3.593 -2.269 -1.879 -3.431 -2.265
Max. 12.423 2.781 3.419 3.087 1.808 2.134 3.174 3.102

Source: Researchers’ Compilations from R-Output



Pa
ge

 
13

3

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

four novel two-steps sparse regression models towards 
providing a robust regression model for the simulated 
response-variables under the high dimensionality 
scenarios (as presented in the previous section) and 
multicollinearity problems.
Table 5 presents the assessment results (i.e. MSEs) of  
each Ridge, Lasso, Elastic-Net, SCAD and the novel two-
step sparse learning regression models under problem 
of  high dimensionality and multicollinearity at small 
sample sizes (i.e. 30 and 50).  Explicitly, for sample 

size 30 at 10% sparsity Table 5 reveals lowest MSEs of  
0.001, 0.0167 and 0.00006 for Lasso estimator when 
p=50 (i.e. low high-dimensional) at low correlation 
(r=0.1), moderate correlation (r=0.5)   and when p=200 
at moderate correlation (r=0.5) levels respectively. The 
table further depicts lowest MSEs for the novel SPCR-
Lasso estimator when  p=50; r=0.9 (mse = 0.0143) i.e. 
low high-dimension with high correlation, p=70; r=0.1 
(mse=0.0071),p=70;r=0.5 (mse=0.0025) & p=70; r=0.9 
(mse=0.0671) i.e. moderate high-dimensional with any 

Table 5: Summary Statistics of  the Simulated Response Variables and Last 7 Independent Variables at Sample 
n Sparsity 10

p 50 70 200
r 0.1 0.5 0.9 0.1 0.5 0.9 0.1 0.5 0.9

30 Ridge 22.2261 14.7271 5.5648 4.7699 7.8126 4.7783 2.8892 8.78595 4.3978
Lasso 0.0010 0.0167 0.1172 0.0638 0.0211 0.1235 0.000019 0.000063 0.00516
Enet 0.0041 0.0269 0.1988 0.0334 0.0192 0.2428 0.000082 0.000372 0.01729
SCAD 2.5382 0.7175 1.2063 0.1656 0.3210 0.9581 0.1454 0.2581 1.34518
SPCR-Ridge 1.7273 1.5237 0.5399 0.6035 0.2873 0.5755 0.0539 0.4525 0.17657
SPCR-Lasso 0.3215 0.1458 0.0143 0.0071 0.0025 0.0671 0.0000004 0.007986 0.003849
SPCR-Enet 1.0575 0.2012 0.0351 0.0672 0.0342 0.0997 0.000696 0.07211 0.011083
SPCR-SCAD 0.2566 0.0348 0.0577 0.1156 0.0055 0.1057 0.0001347 0.04886 0.044825

Sparsity 30
Ridge 38.9309 34.5543 5.0801 51.7066 26.5211 11.1444 15.8539 12.43887 4.99299
Lasso 0.0078 0.7775 0.7791 0.0516 0.0462 0.0665 0.04935 0.000873 0.00397
Enet 0.0871 1.2908 0.4971 0.1924 0.0218 0.1709 0.17285 0.000262 0.01215
SCAD 3.9459 0.9614 1.2167 1.6763 0.9853 4.9219 0.21469 0.81469 3.17956
SPCR-Ridge 38.7535 28.7759 17.0134 34.2091 22.8729 175.7768 15.5972 33.1529 15.2773
SPCR-Lasso 0.9465 1.8579 0.3021 0.0269 0.0173 0.0511 0.04379 0.06934 0.17663

SPCR-Enet 2.1343 5.9747 0.9162 7.4662 0.0632 0.2739 0.11784 0.30761 0.43285
SPCR-SCAD 0.9149 1.6942 1.1840 1.4363 0.0753 0.4019 0.08979 0.07658 0.26952

Sparsity 10
p 50 70 200

50 Ridge 14.27529 13.14787 3.99444 7.59169 11.5585 2.51970 10.93393 17.68289 3.68972
Lasso 0.024283 0.01111 0.26847 0.001597 0.00047 0.12309 0.000056 0.00016 0.00761
Enet 0.035625 0.03055 0.32753 0.002292 0.00199 0.10761 0.005829 0.00089 0.03019
SCAD 0.137990 0.37474 0.71736 0.135452 0.28281 0.34494 1.51859 0.20688 0.81663
SPCR-Ridge 1.59547 1.50312 0.69733 0.961469 0.41246 0.3449 1.16773 0.14019 0.19149
SPCR-Lasso 0.071019 0.20712 0.07296 0.031501 0.00401 0.14790 0.0000021 0.02395 0.00096
SPCR-Enet 01.80131 0.450867 0.08955 0.065121 0.03633 0.20077 0.009640 0.06199 0.00076
SPCR-SCAD 0.002342 0.20008 0.27585 0.008672 0.00831 0.05008 0.001545 0.00298 0.00113
Sparsity 30
Ridge 28.72535 24.78267 10.92322 46.11221 39.1727 9.20752 25.78999 27.94949 12.2576
Lasso 0.001752 0.04827 0.45687 0.00135 0.00376 0.12701 0.03429 0.05793 0.04912
Enet 0.00692 0.12742 0.76926 0.00513 0.01369 0.28181 0.35494 0.20659 0.27805
SCAD 0.39620 1.59569 4.17981 2.22021 4.83648 1.49247 0.49354 0.36052 23.3093
SPCR-Ridge 1.790068 2.43327 1.00552 1.90385 0.99053 0.91456 19.86799 2.06924 0.70623
SPCR-Lasso 0.15863 0.31728 0.25098 0.04437 0.04119 0.01670 0.30157 0.09142 0.00785
SPCR-Enet 0.08939 0.54537 0.30180 0.26648 0.05336 0.01793 1.04003 0.23314 0.00523
SPCR-SCAD 0.05241 0.79069 0.23878 0.07853 1.32367 0.01216 0.42382 0.08032 0.00686

Source: Researchers’ Compilations from R-Outputs



Pa
ge

 
13

4

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

correlation levels, and  p=200; r=0.1 (mse=0.0000004) & 
p=200; r=0.9 (mse=0.003849) i.e. high-dimensional with 
low and high correlation levels. In the same vein, Figure 
1 presents performance ranks of  each estimator under 
varied levels of  high dimensionality and multicollinearity 
for sample size 30 with 10% sparse. The figure similarly, 

ranks Lasso estimator best when p=50 & r=0.1, p=50 & 
r=0.5, and p=200 & r=0.5 while SPCR-Lasso returned 
best rank estimator when p=50 & r=0.9, p=70 & r=0.1, 
p=70 & r=0.1, p=70 & r=0.5, p=70 & r=0.9, p=200 & 
r=0.1, and p=200 & r=0.9.

Figure 1: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 30 with 10% Sparse 

Figure 2: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 30 with 30% Sparse

Similarly, for sample size 30 at 30% sparsity Table 5 reveals 
lowest MSEs of  0.0078, 0.7775, 0.00087 and 0.00397 for 
Lasso estimator when p=50 (i.e. low high-dimensional) at 
low correlation (r=0.1), moderate correlation (r=0.5) and 
high correlation (r=0.9) levels respectively. While when 
p=200 at moderate correlation (r=0.5), the table returned 
Elastic Net (Enet) estimator with lowest MSE of  0.00026. 
The table also depicts lowest MSEs for the novel SPCR-
Lasso estimator when  p=50; r=0.9 (mse = 0.3021) i.e. 
low high-dimension with high correlation, p=70; r=0.1 
(mse=0.0269),p=70;r=0.5 (mse=0.0173) & p=70; r=0.9 
(mse=0.0511) i.e. moderate high-dimensional with any 

correlation levels, and  p=200; r=0.1 (mse=0.04379) 
i.e. high-dimensional with low correlation level. In the 
same vein, Figure 2 presents performance ranks of  each 
estimator under varied levels of  high dimensionality and 
multicollinearity for sample size 30 with 30% sparse. The 
figure similarly, ranks Lasso estimator best when p=50 & 
r=0.1, p=50 & r=0.5, and p=200 & r=0.9. It also ranks 
Elastic Net estimator best when  p=200 & r=0.5 while 
SPCR-Lasso returned best rank estimator when p=50 
& r=0.9, p=70 & r=0.1, p=70 & r=0.1, p=70 & r=0.5, 
p=70 & r=0.9, and p=200 & r=0.1.



Pa
ge

 
13

5

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Figure 4: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 50 with 30% Sparse

Figure 3: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 50 with 10% Sparse

Furthermore, considering small sample size of  50 at 
10% sparsity Table 5 reveals the Lasso estimator with 
least MSEs of  0.0111, 0.0016, 0.0005 and 0.00016 when 
p=70 & r=0.5, p=100 & r=0.1, p=70 & r=0.5, and 
p=200 & r=0.5 respectively. Meanwhile the table depicts 
the novel; SPCR-Lasso estimator with least MSEs when 
p=70 & r=0.9 (mse=0.07296) and  p=200 & r=0.1 
(mse=0.0000032), SPCR-SCAD estimator with least 
MSEs when p=70 & r=0.1 (mse=0.002342) and  p=100 
& r=0.9 (mse=0.05008), and SPCR-Enet estimator 
with lowest MSE when p=200 & r=0.9 (mse=0.00076). 
Similarly, Figure 3 presents the performance ranks of  
each estimator under varied levels of  high dimensionality 
and multicollinearity for sample size 50 with 10% sparse. 
According to the figure, the Lasso estimator was ranked 
best (i.e. 1st) on four occasions namely, p=70 & r=0.5, 
p=100 & r=0.1, p=70 & r=0.5, and p=200 & r=0.5. the 
novel SPCR-Lasso estimator was ranked best on two 
occasions namely, p=70 & r=0.9  and  p=200 & r=0.1 
. Also, the novel SPCR-SCAD was ranked best on two 

occasions namely p=70 & r=0.1 and  p=100 & r=0.9. 
As well as our novel SPCR-Enet was ranked best when 
p=200 & r=0.9.
Considering small sample size of  50 at 30% sparsity 
Table 5 and Figure 4 reveal Lasso estimator returned 
with least MSE and 1st ranking on six occasions 
namely p=70 & r=0.1 (mse=0.001752),  p=70 & r=0.5 
(mse=0.04827), p=100 & r=0.1 (mse=0.00135), p=100 
& r=0.5 (mse=0.00376),  p=200 & r=0.1 (mse=0.03429) 
and  p=200 & r=0.5 (mse=0.00376). Additionally, Table 
5 and Figure 4 depict our novel SPCR-SCAD estimator 
returned with least MSE and 1st ranking on two occasions 
namely p=70 & r=0.9 (mse=0.23878) and p=100 & 
r=0.9 (mse=0.01216). Also, according to Table 5 and 
Figure 4 our novel SPCR-Enet returned with least MSE 
and ranked 1st when =200 & r=0.9 (mse=0.00523). 
Moreover, Table 6 presents the assessment results (i.e. 
MSEs) of  each Ridge, Lasso, Elastic-Net, SCAD and the 
novel two-step sparse learning regression models under 
problem of  high dimensionality and multicollinearity at 



Pa
ge

 
13

6

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

higher small sample sizes (i.e. 70 and 100). According to 
Table 6 and Figure 5 when considering sample size 70 with 
10% sparsity, Lasso estimator returned with least MSE 
and ranked 1st on three occasions namely p=100 & r=0.1 
(mse=0.01239), p=150 & r=0.1 (mse=0.000253) and 
p=150 & r=0.5 (mse=0.00860). Also, our novel SPCR-
Enet returned with the least MSE and ranked best (1st) 
on two occasions namely, p=150 & r=0.9 (mse=0.01112) 

and p=200 & r=0.9 (mse=0.01801). Similarly, our novel 
SPCR-SCAD returned with the least MSE and ranked 
best (1st) on two occasions namely, p=200 & r=0.1 
(mse=0.0000029) and p=200 & r=0.5 (mse=0.00497). 
Table 6 and Figure 5 reveal our novel SPCR-Lasso with 
the least MSE and 1st ranking when p=100 & r=0.1 
(mse=0.04099). 

Table 6: MSE of  Ridge, Lasso, Elastic-Net, SCAD and the Novel Two-Step Sparse Learning Regression Models 
under Problem of  High Dimensionality and Multicollinearity at Higher Small Sample Size 
n Sparsity 10

p 100 150 200
r 0.1 0.5 0.9 0.1 0.5 0.9 0.1 0.5 0.9

70 Ridge 16.2612 12.23781 4.03865 11.26075 11.60848 3.39042 8.64446 9.76361 2.67325
Lasso 0.01239 0.06057 0.18933 0.000253 0.00860 0.08186 0.00108 0.00857 0.11078
Enet 0.04216 0.04236 0.30014 0.008587 0.02651 0.16821 0.00337 0.01337 0.10399
SCAD 0.91239 0.42240 1.54271 0.061469 0.09442 0.70653 0.05592 0.04387 0.37114
SPCR-Ridge 7.81776 2.56198 0.65102 20.5621 0.10744 0.48016 0.17769 0.77086 0.37926
SPCR-Lasso 0.25674 0.26353 0.04099 0.39296 0.05496 0.01196 0.01360 0.01134 0.05829
SPCR-Enet 0.52689 0.39883 0.05059 0.55724 0.12907 0.01112 0.02494 0.05916 0.01801
SPCR-SCAD 0.19465 0.60541 0.14201 0.14136 0.05721 0.01919 0.0000029 0.00497 0.23454

Sparsity 30
Ridge 38.13883 37.66447 12.62623 24.65487 36.8325 11.61202 43.27625 40.24778 14.48422
Lasso 0.00662 0.00338 0.37802 0.000527 0.00093 0.06559 0.00039 0.00097 0.03899
Enet 0.01231 0.01996 0.58596 0.01063 0.00397 0.15603 0.00169 0.01392 0.10182
SCAD 0.54454 1.58249 7.48471 0.18944 0.75494 0.74484 0.38007 1.28358 1.70183
SPCR-Ridge 1.05483 1.59691 1.40662 1.38173 2.20109 0.63355 20.95064 41.6646 0.58520
SPCR-Lasso 0.01037 0.11502 0.10606 0.11955 0.08147 0.00544 0.37343 0.42448 0.00979

SPCR-Enet 0.10300 0.23069 0.04797 0.17829 0.14697 0.01071 1.25418 0.87494 0.01865
SPCR-SCAD 0.01718 0.06238 0.12671 0.15924 0.14211 0.06971 0.52191 0.50069 0.02307

Sparsity 10
p 120 150 200

10
0

Ridge 7.76100 12.6273 2.51623 15.30857 12.1868 3.17887 12.09962 12.77974 2.54699
Lasso 0.03929 0.09684 0.50299 0.01276 0.00601 0.20818 0.00835 0.00496 0.08117
Enet 0.04664 0.10807 0.51229 0.03871 0.02988 0.32757 0.01998 0.01209 0.13614
SCAD 0.09555 0.26393 0.97131 0.28428 0.18236 0.61504 0.07291 0.19182 0.60169
SPCR-Ridge 0.60385 2.70444 0.80414 1.10457 0.45551 0.67572 3.83138 0.55069 0.42212
SPCR-Lasso 0.10859 0.32131 0.04616 0.04605 0.11171 0.10353 0.27173 0.03944 0.00243
SPCR-Enet 0.20375 0.86351 0.15666 0.05838 0.08879 0.24179 0.74755 0.04253 0.14797
SPCR-SCAD 0.11849 0.47327 0.37146 0.03035 0.23513 0.42612 0.29195 0.00853 0.00307
Sparsity 30
Ridge 44.73326 43.83492 8.42825 25.9745 36.23507 9.66694 40.51713 26.12172 10.96855
Lasso 0.00553 0.02191 0.25439 0.00293 0.00363 0.16858 0.00104 0.00111 0.12742
Enet 0.01770 0.05789 0.38731 0.00866 0.01271 0.30531 0.00383 0.00347 0.26067
SCAD 0.42485 1.01778 1.44601 0.76685 0.50686 2.20988 0.28645 0.66346 0.88463
SPCR-Ridge 4.84146 1.15390 0.93489 1.80877 3.69042 0.79279 9.05412 3.75005 0.97242
SPCR-Lasso 0.09466 0.10141 0.07998 0.09151 0.19978 0.05461 1.69562 0.75744 0.05299
SPCR-Enet 0.63696 0.06535 0.16816 0.18488 0.50735 0.06652 4.53791 0.79444 0.06279
SPCR-SCAD 0.50354 0.32955 0.15189 0.20898 0.28904 0.06717 0.00176 0.46458 0.06440

Source: Researchers’ Compilations from R-Outputs



Pa
ge

 
13

7

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Figure 6: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 70 with 30% Sparse

Figure 5: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 70 with 10% Sparse

In addition, considering sample size 70 with 30% 
sparsity, Table 6 and Figure 6 reveal Lasso estimator 
with lowest MSE and ranked 1st on six occasions 
namely p=100 & r=0.1 (MSE=0.00662), p=100 & r=0.5 
(MSE=0.00338),p=150 & r=0.1 (MSE=0.000527), 
p=150 & r=0.5 (MSE=0.00093), p=200 & r=0.1 

(MSE=0.00039), and  p=200 & r=0.5 (MSE=0.00097). 
The table and figure further depict our novel SPCR-
Lasso estimator with the lowest MSE and best ranking 
estimator when p=150 & r=0.9 (MSE=0.00544),  and 
p=200 & r=0.9 (MSE=0.00979)  as well as SPCR-Enet 
when p=100 & r=0.9 (MSE=0.04797).

Figure 7: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 100 with 10% Sparse



Pa
ge

 
13

8

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

Figure 8: Performance Ranks of  Each Estimator under Varied Levels of  High Dimensionality and Multicollinearity 
for Sample Size 100 with 30% Sparse

Furthermore, considering sample size 100 with 10% 
sparsity, Table 6 and Figure 7 reveal Lasso estimator 
with lowest MSE and ranked 1st on six occasions 
namely p=120 & r=0.1 (MSE=0.00662), p=120 & 
r=0.5 (MSE=0.09684),p=150 & r=0.1 (MSE=0.01276), 
p=150 & r=0.5 (MSE=0.00601), p=200 & r=0.1 
(MSE=0.00835), and  p=200 & r=0.5 (MSE=0.00496). 
The table and figure establish our novel SPCR-Lasso with 
the least MSE and best ranking estimator when p=120 & 
r=0.9 (MSE=0.07998),  p=150 & r=0.9 (MSE=0.10353) 
and p=200 & r=0.9 (MSE=0.00243). 
Similarly, considering sample size 100 with 30% 
sparsity, Table 6 and Figure 8 reveal Lasso estimator 
with lowest MSE and ranked 1st on six occasions 
namely p=120 & r=0.1 (MSE=0.00553), p=120 & 
r=0.5 (MSE=0.02191),p=150 & r=0.1 (MSE=0.00293), 
p=150 & r=0.5 (MSE=0.00363), p=200 & r=0.1 
(MSE=0.00104), and  p=200 & r=0.5 (MSE=0.00111). 
The table and figure establish our novel SPCR-Lasso with 
the least MSE and best ranking estimator when p=120 & 
r=0.9 (MSE=0.04616),  p=150 & r=0.9 (MSE=0.05461) 
and p=200 & r=0.9 (MSE=0.05299). 

Findings Summary, Discussion of  Findings, And 
Conclusion
Findings by Small Sample Sizes and Dimensionality
At extremely small sample sizes (n=30), SPCR-Lasso 
consistently outperformed all other estimators, especially 
when dimensionality was high (p=7 or p=200). This 
demonstrates the strength of  SPCR-Lasso in small-

sample, high-dimensional contexts, where traditional 
Lasso, Ridge, or Elastic Net tend to become unstable. 
At moderately small sample sizes (n=50), results showed 
variation across conditions: SPCR-SCAD excelled in 
contexts of  low sparsity and low correlation. SPCR-Lasso 
and SPCR-Enet provided superior performance under 
higher correlation and dimensionality. As sample sizes 
increased further (n≥70), SPCR-Lasso and SPCR-Enet 
emerged as the most consistent and robust estimators 
across both moderate and high correlations. Notably, 
SPCR-Enet showed particular strength in very high-
dimensional scenarios (p=200), reflecting its ability to 
balance shrinkage and group variable selection.

Findings by Multicollinearity and Sparsity
The findings also highlight clear interactions between 
predictor correlation and sparsity:

• Under low correlation (r=0.1), traditional Lasso 
sometimes matched or exceeded SPCR-based methods in 
low-dimensional settings, suggesting SPCR hybridization 
may not always be necessary in weakly collinear designs.

• Under moderate (r=0.5) or high correlation (r=0.9), 
SPCR-Lasso and SPCR-Enet decisively outperformed 
alternatives, confirming the necessity of  the SPCR step 
for mitigating multicollinearity.

• With respect to sparsity, SPCR-SCAD performed 
best in highly sparse, low-correlation conditions, while 
SPCR-Lasso and SPCR-Enet proved more adaptable 
across both sparse and dense regimes.

Table 7: Overview of  Best Estimators under Different Considered Small Sample Sizes, High-Dimensionality, 
Multicollinearity and Sparsity Levels 
n r 0.1 0.5 0.9 0.1 0.5 0.9 0.1 0.5 0.9

p 50 70 200
30 10% Lasso Lasso SPCR-

Lasso
SPCR-
Lasso

SPCR-
Lasso

SPCR-
Lasso

SPCR-
Lasso

Lasso SPCR-
Lasso

30% Lasso Lasso SPCR-
Lasso

SPCR-
Lasso

SPCR-
Lasso

SPCR-
Lasso

SPCR-
Lasso

Enet Lasso



Pa
ge

 
13

9

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

70 100 200
50 10% SPCR-

SCAD
Lasso SPCR-

Lasso
Lasso Lasso SPCR-

SCAD
SPCR-
Lasso

Lasso SPCR-
Enet

30% Lasso Lasso SPCR-
SCAD

Lasso Lasso SPCR-
SCAD

Lasso Lasso SPCR-
Enet

100 150 200
70 10% Lasso Enet SPCR-

Lasso
Lasso Lasso SPCR-

Enet
SPCR-
SCAD

SPCR-
SCAD

SPCR-
Enet

30% Lasso Lasso SPCR-
Enet

Lasso Lasso SPCR-
Lasso

Lasso Lasso SPCR-
Lasso

120 150 200
100 10% Lasso Lasso SPCR-

Lasso
Lasso Lasso SPCR-

Lasso
Lasso Lasso SPCR-

Lasso
30% Lasso Lasso SPCR-

Lasso
Lasso Lasso SPCR-

Lasso
Lasso Lasso SPCR-

Lasso
Source: Researchers’ Compilations

Discussion of  Findings
Theoretical and Methodological Insights
The results validate the rationale for hybridizing SPCR 
with regularization penalties. SPCR effectively reduces 
dimensionality while preserving predictive features, 
and the addition of  regularization stabilizes estimates 
in the presence of  multicollinearity. Together, this 
hybrid approach delivers stronger predictive accuracy 
and interpretability than either dimension reduction or 
regularization alone.
The study also demonstrates that penalty choice must be 
data-dependent. Specifically:

• SPCR-Lasso and SPCR-Enet are best suited for high-
dimensional, correlated designs.

• SPCR-SCAD retains value under extreme sparsity 
with low correlation.

• SPCR-Ridge, while stabilizing, offers limited benefits 
compared to its sparse counterparts.
These findings align with empirical evidence in high-
dimensional statistics but extend prior work by 
systematically comparing multiple regularizers within an 
SPCR framework across diverse simulation conditions.

Practical Implications for Applied Research
For applied researchers working in genomics, finance, 
climate science, and social sciences, the study’s findings 
provide clear practical guidance:

• Use SPCR-Lasso or SPCR-Enet when predictors are 
highly correlated or dimensionality is large.

• Employ SPCR-SCAD in cases of  extreme sparsity 
with weak predictor correlation.

• Expect interpretability benefits from SPCR, as sparse 
principal components link outcomes to identifiable subsets 
of  predictors rather than opaque linear combinations.
This guidance equips researchers with a decision-making 
framework to select the most effective hybrid estimator 
given the structural characteristics of  their data.

Policy and Applied Modeling Implications
The findings also have implications for applied modeling 

in policy-relevant domains. Policymakers and analysts 
working with high-dimensional, multicollinear data 
(e.g., in economic forecasting, climate modeling, or 
epidemiological surveillance) can adopt SPCR-based 
methods to achieve more reliable predictions. By 
improving both accuracy and interpretability, these 
methods enhance the credibility of  evidence-based policy 
decisions.

Implications for Future Research
The findings suggest several avenues for further inquiry:

i. Extending the hybrid SPCR framework to nonlinear 
models (e.g., kernel methods, deep learning).

ii. Applying SPCR-regularization pipelines to real-
world datasets in genomics, finance, and environmental 
science to validate simulation results.

iii. Investigating stability selection and uncertainty 
quantification after SPCR to improve robustness of  
variable selection in practice.

iv. Exploring time-series extensions of  SPCR 
hybridization for forecasting applications.

CONCLUSION
This section has discussed the findings of  the simulation 
study and their implications for statistical methodology, 
applied practice, and policy. The results confirm that hybrid 
SPCR estimators substantially outperform traditional 
penalization methods in small-sample, high-dimensional, 
and multicollinear conditions. Among these, SPCR-
Lasso and SPCR-Enet emerge as the most versatile and 
reliable, while SPCR-SCAD shows targeted advantages in 
sparse, low-correlation settings. Collectively, the findings 
highlight the significance and necessity of  hybrid SPCR 
approaches as a methodological advancement for high-
dimensional data analysis. 

REFERENCES 
Ali, H., Shahzad, M., Sarfraz, S., Sewell, K. B., Alqalyoobi, 

S., & Mohan, B. P. (2023). Application and impact of  
Lasso regression in gastroenterology: a systematic 



Pa
ge

 
14

0

https://journals.e-palli.com/home/index.php/ajase

Am. J. Appl. Stat. Econ. 4(1) 127-140, 2025

review. Indian Journal of  Gastroenterology, 42(6), 780-790.
Chatterjee, I., & Baumgärtner, L. (2024). Unveiling 

Functional Biomarkers in Schizophrenia: Insights 
from Region of  Interest Analysis Using Machine 
Learning. Journal of  Integrative Neuroscience, 23(9).

Chen, J., Yang, S., Wang, Z., & Mao, H. (2021). 
Efficient sparse representation for learning with 
high-dimensional data. IEEE Transactions on Neural 
Networks and Learning Systems, 34(8), 4208-4222.

Cleophas, T. J., & Zwinderman, A. H. (2024). Application 
of  Regularized Regressions to Identify Novel Predictors in 
Clinical Research. Springer Nature.

Fan, J., & Li, R. (2001). Variable selection via nonconcave 
penalized likelihood and its oracle properties. JASA.

Gupta, V., Chen, Y., & Wan, M. (2024). Predictability 
of  weakly turbulent systems from spatially sparse 
observations using data assimilation and machine 
learning. arXiv preprint arXiv:2407.10088.

Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased 
estimation for nonorthogonal problems. Technometrics.

Jolliffe, I. T. (2002). Principal component analysis. Springer.
Kawano, S. (2018). Sparse Principal Component Regression 

(one-stage SPCR literature).
Kitano, T., & Noma, H. (2024). Ridge, lasso, and elastic-

net estimations of  the modified Poisson and least-squares 
regressions for binary outcome data. arXiv preprint 
arXiv:2408.13474.

Manzhos, S., & Ihara, M. (2022). Advanced machine 
learning methods for learning from sparse data in 
high-dimensional spaces: A perspective on uses 
in the upstream of  development of  novel energy 

technologies. Physchem, 2(2), 72-95.
Meinshausen, N. (2007). Relaxed Lasso and stability selection 

literature.
Nwosu, A., Aimufua, G. I. O., Ajayi, B. A., & Olalere, 

M. (2024). The Impact of  Regularization on Linear 
Regression Based Model. Journal of  Artificial Intelligence 
and Computer Science, 1(1).

Priya, A. K., Gnanasekaran, L., Rajendran, S., Qin, J., 
& Vasseghian, Y. (2022). Occurrences and removal 
of  pharmaceutical and personal care products from 
aquatic systems using advanced treatment-A review. 
Environmental Research, 204, 112298.

Song, J., Xu, L., & Wang, X. (2024, July). A Regularization 
Method for Enhancing the Robustness of  Regression 
Networks. In 2024 43rd Chinese Control Conference 
(CCC) (pp. 8524-8529). IEEE.

Tibshirani, R. (1996). Regression shrinkage and selection via the 
Lasso. JRSS-B.

Witten, D. M., Tibshirani, R., & Hastie, T. (2009). A 
penalized matrix decomposition, with applications to sparse 
principal components and canonical correlation analysis. 
Biostatistics.

Zhang, X., Sun, Q., & Kong, D. (2024). Supervised 
Principal Component Regression for Functional 
Responses with High Dimensional Predictors. Journal 
of  Computational and Graphical Statistics, 33(1), 242-249.

Zou, H., & Hastie, T. (2005). Regularization and variable 
selection via the Elastic Net. JRSS-B.

Zou, H., Hastie, T., & Tibshirani, R. (2006). Sparse 
principal component analysis. Journal of  Computational 
and Graphical Statistics.


