307 © 2025 The Author(s). Published by College of Education for Pure Science (Ibn Al-Haitham), University of Baghdad. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License Autoregressive Models Estimation Selection with Known Marginal Distribution Israa Amer Flayyih 1,2* and Basad Al-sarray 3 1 Department of Mathematics, College of Education for Pure Sciences (Ibn AL-Haitham), University of Baghdad, Baghdad, Iraq. 2 Department of Mathematics, College of Basic Education, University of Diyala, Diyala, Iraq. 3 Department of Computer, College of Science, University of Baghdad, Baghdad, Iraq. *Corresponding Author. Received: 10 February 2024 Accepted: 22 May 2024 Published: 20 October 2025 doi.org/10.30526/38.4.3927 Abstract A time series is a sequence of observations recorded at regular intervals. Time series analysis has applications in diverse fields such as finance, stock prices, economics, environmental science, and social network data analysis. The recorded series is used to represent a measurable quantity or attribute, such as temperature readings, economic indicators, or other variables, depending on the context of the analysis. The idea of time series analysis is to identify patterns, trends, or underlying structures within the data, as well as to make predictions or forecasts about future values based on previous observations. Autoregressive (AR) models are widely used in modeling and forecasting data from time series. This work focuses on AR model parameter estimation, emphasizing the significance of the likelihood function by defining the marginal distribution of the AR process, which is getting by representing the AR process with random shocks and assuming the error terms in a time series have a normal distribution with a zero mean and variance . Some of the simulated experiments are designed to fit the model for different model orders and sample size to find model parameter estimation by likelihood function with marginal distribution. The results of Mean Squares Errors (MSE) and Mean Percentage Errors (MPE) indicate the significance and robust estimation of the AR –models parameters estimators that are computed theoretically. Keywords: Autoregressive Time series Models, Marginal distribution of time series, Maximum likelihood Estimation. 1. Introduction A time series is a collection of data points produced successively over time. If the set is continuous, the time series is considered continuous and discrete if the set it belongs to is also discrete (1). In the early century, time series analysis heavily relied on an autoregressive model, as researched by Yule in 1927 (1). The autoregressive model utilized regression analysis to estimate the values of the series at a particular time by applying a linear function of past values (2). Time series analysis employs this model to illustrate the relationship among time series data; refer to (2) for further details. https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com https://orcid.org/0009-0006-0718-2902 mailto:israa.aamer2103@ihcoedu.uobaghdad.edu.iq https://orcid.org/0000-0003-2419-0710 mailto:basad.a.alsarray@gmail.com IHJPAS. 2025,38(4) 308 Generalized autoregressive (GAS) models are a type of observational time series model that is proposed in (3). This novel method offers a unified and consistent framework for adding time-varying parameters to a wide range of nonlinear models. Using multivariate point processes with time-varying parameters and novel model requirements, this method can result in a new formulation of observational models. We present detailed studies of the models along with experimental and simulation evidence. In (4), an optimal power spectrum estimator using the mixed ARMA (1,1) model for time series data that adheres to a normal distribution was conducted. An extensively used model for analyzing time series data is the autoregressive (AR) model. The innovation noise in AR was traditionally described as a Gaussian in (5). But since many time series applications, like financial time series data, for example, are not Gaussian, the AR model with more broadly applicable substantial improvements is recommended. For the first time, an effective framework based on stochastic approximation expectation maximization (SAEM) combined with a Markov chain Monte Carlo (MCMC) approach is proposed to handle the problem of missing values in incomplete time series. Authors in (6) constructed an autoregressive model incorporating an exogenous variable. They use two methods: the first method employed was the threshold approach, utilizing two proposed approaches to identify the optimal cut-off point for future forecasting and predictability in the time series; their goal was achieved through the threshold point indicator; and the second method involved using seasonal B-J models, which relied on the principles of the two approaches above to determine the most suitable seasonal model. The enhancement of the estimate of a third-order autoregressive model by employing the LDR and WLSE estimation techniques was studied by researchers in (7). By creating a time series for the AR(3) model with normally and non-normally distributed error terms, the researcher in (7) found that enhancing the estimation of the autoregressive model using the LDR and WLSE methods is contingent upon the sample size for all considered error distributions. In (8), the best and most efficient artificial neural network models for solving linear and nonlinear time series behavior were considered. The researchers concluded that the ideal neural networks are the backpropagation network (BP) and the recurrent neural network (RNN) for solving time series, whether linear, quasi-linear, or nonlinear. The results showed an improvement over the modern methods of time series forecasting. Furthermore, the author in (9) combined ARMA models with EGARCH models to create a hybrid model: ARMA –EGARCH . This hybrid model was used to analyze time series data on average temperatures. He determined that the optimal model is ARMA –EGARCH ; see (9). In (10), researchers presented the fundamental genetic algorithm (CGA) to estimate the log-likelihood parameter function of a first-order moving average model. After comparing it based on mean squared error (MSE) with the intraday technique, they determined that CGA could yield more dependable outcomes. In (11), researchers presented a new ARIMA model and applied it to a monthly chemical sales dataset in the United States. After comparing it with other models, they found that the updated ARIMA model is more accurate. In (12) they conducted the first study with a small sample size that focused on predicting errors based on the concept of the Gaussian noise process. In (13), researchers studied monthly rainfall data from the Baghdad Meteorological Station to study the temporal behavior of the data series for many ARIMA models that have been tested. They verified the adequacy of the best one. They concluded that the seasonal ARIMA model for orders SARIMA(2,1,3)x(0,1,1) is the best. IHJPAS. 2025,38(4) 309 The idea of this work is finding theoretically the estimation of the autoregressive model by likelihood function by defining the marginal distribution of the AR process when the error terms of the series are normal distribution with zero mean and constant variance. The rest of the paper is section 2 gives the theatrical properties of the AR model; section 3 presents the details of the maximum likelihood function for the AR model, at least the simulation and results given in section 4. 2. Autoregressive Time Series Model The autoregressive time series model of order p may be written as (14-19): (1) Where p is called the order of the AR model. of the linear combination are the model parameters and . By using the backward shift operator, B, which is ( . Then the in Equation 1 can be written as the following: Which is simplified as below: . Hence, (2) where . By dividing both sides of Equation 2 on ,we get: , where . The parameters of an AR (p) process must satisfy certain conditions for the process to be stationary. 3. Likelihood Function of Model As in (19-23), we suppose that the order stationary autoregressive model is considered as follows: . That is (3) The formula for the exact likelihood function is: | | (4) where ∑ ∑ ∑ . Also, | | [ ] (5) Where are the theoretical covariance of the process and | | | | In our work, we have three cases for the likelihood functions. Therefore, for Equation 4, we have three cases as follows: Maximum Likelihood for AR (1) The general likelihood function is | | . where = = = , and | | | | As a result, the exact likelihood function is | | ∑ (6) The log likelihood of Equation 6 is : IHJPAS. 2025,38(4) 310 ∑ (7) Differentiate both sides of Equation 7 with respect to : ∑ . Therefore, ∑ ∑ Set ∑ ∑ (8) Since , where and . Then, Equation 8 become ∑ ∑ (9) substitute ∑ , in Equation 9 ∑ ∑ ∑ ( )∑ ∑ ∑ ∑ .Hence, ∑ ∑ ̂ ∑ ∑ . Maximum Likelihood for AR(2) For , the exact likelihood function is | | (10) Then, Equation 5 be ∑ . Where , and { | |} . Hence, ∑ (11) Then , - | | | | (12) Substitute Equations 11 and 12 in Equation 10 | | ∑ } (13) The log likelihood of Equation 13 is ∑ (14) Differentiate both sides of Equation 14 with respect to : ∑ ∑ ∑ .Set ∑ ∑ ∑ . Therefore, ∑ ∑ ∑ (15) Differentiate both sides of Equation 14 with respect to : ∑ ∑ ∑ ∑ ∑ ∑ Set . ∑ ∑ ∑ (16) Substitute Equation 15 in Equation 16. IHJPAS. 2025,38(4) 311 ∑ ∑ ∑ ∑ ∑ ∑ After simplification , ∑ ∑ ∑ ∑ ∑ (17) Hence, ̂ ∑ ∑ ∑ ∑ ∑ . Substitute Equation 17 in Equation 15. Then: ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ . After simplification, ∑ ∑ ∑ ∑ ∑ . Hence. ̂ ∑ ∑ ∑ ∑ ∑ Maximum Likelihood for AR (3) For the exact likelihood function will be: | | , - (18) The Equation 5 will be ∑ . So we have , , and { | |} . Hence, ∑ (19) Then | | | | (20) By substitute Equations 19 and 20 in Equation 18, the exact likelihood function will be | | , ∑ - (21) The log likelihood of Equation 21 is = ∑ (22) Differentiate both sides of Equation 22 with respect to : ∑ ∑ ∑ ∑ . We make , so we have ∑ ∑ ∑ ∑ . After simplification, ∑ ∑ ∑ ∑ (23) Differentiate both sides of Equation 22 with respect to : ∑ ∑ ∑ ∑ . We make , so we get ∑ ∑ ∑ ∑ . After simplification, ∑ ∑ ∑ ∑ (24) Differentiate both sides of Equation 22 with respect to . Then IHJPAS. 2025,38(4) 312 ∑ . Also, ∑ ∑ ∑ ∑ . Set . Then ∑ ∑ ∑ ∑ . ∑ ∑ ∑ ∑ (25) Substitute Equation 23 in Equation 24 to get that ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ . Thus, ∑ ∑ ∑ ∑ ∑ ∑ (26) Hence,, ̂ ∑ ∑ ∑ ∑ ∑ ∑ . Substitute Equation 26 in Equation 24 then ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ That is ∑ ∑ ∑ ∑ ∑ (27) Hence, ̂ ∑ ∑ ∑ ∑ ∑ . Substitute Equations 26 and 27 in Equation 23 to consider the following: ̂ ∑ ∑ ∑ ∑ ∑ ∑ - ∑ ∑ ∑ ∑ ∑ ∑ 3.1. Marginal distribution of AR time series in case known error distribution In (23-26), the general form for the random shock form is (28) where is a random error, , and is a back shift operator that is ( ) Then . Using back shift operator, give us Then ∑ , where . Since (29) Substituting Equation 28 in Equation 29, we get ( of ( Hence, IHJPAS. 2025,38(4) 313 ∑ (30) of ( Then, , ∑ where (31) of ( + Then, ∑ ( ) where and (32) Assuming that represents random errors that are uniformly distributed. It received a mean of zero and a variance of that is It finds the marginal distribution for { by using the characteristic function if the data follow the autoregressive model . The model can be written in terms of random errors according to Equations 30, 31, 32, and ( ). For we have the following cases: of AR(1) [ ] * ∑ + [ ]. So, ∏ When { the character function for errors will be ∏ , where ∏ = (33) Then, ( ) of AR(2) Since ∑ Then [ ] * ∑ + * (( ) )+= * ( ) +. * ( ( )) +. ∏ ( ( )) (34) When { for errors is IHJPAS. 2025,38(4) 314 Hence, ∏ ( ( )) [ ] (35) Then, ( ( ) ) of AR(3) Since ∑ ( ) .Then [ ] * ∑ ( ) + Then , * (( ) )+ Hence , ∏ ( ) (36) where { for errors is Hence , ∏ . ( ) / [ ] (37) Then, ( ( ) ). 3.2 Maximum likelihood function with marginal distribution The Normal distribution function is √ where By using marginal distribution for when and , where is normal distribution by mean zero and variance (27-30). The maximum likelihood function when and √ where =∏ ( ) ∑ (38) The log likelihood of Equation 38 is: . / ∑ ( ) ∑ (39) Differentiate both sides of Equation 39 with respect to . ∑ ∑ (40) Make Equation 40 equals to zero. ∑ ∑ ∑ IHJPAS. 2025,38(4) 315 Then, ∑ ∑ √ ∑ ∑ ̂ √ ∑ ∑ The maximum likelihood function when and ( ) √ where =∏ ( ( ) ) ∑ ( ) (41) The log likelihood of Equation 41 is . ( ) / ∑ ( ) ( ) ( ( ) ) ∑ . ( ) / ( ) (( ) ) ∑ . ( ) / (42) Differentiate both sides of Equation 42 with respect to . ( ) ∑ By simplification, ( )∑ (43) Make Equation 43 equals to zero. ( )∑ = 0 ( )∑ ( )∑ (44) Differentiate both sides of Equation 42 with respect to . ∑ ∑ (45) Make Equation 45 equals to zero, ∑ ∑ ∑ Hence , ∑ ∑ (46) Substitute Equation 44 in Equation 46; ( ( )∑ ) ( ∑ ∑ ) IHJPAS. 2025,38(4) 316 ∑ ∑ After simplification, ∑ ∑ ̂ ∑ ∑ (47) Substitute Equation 47 in Equation 46 . ∑ ∑ / ∑ . ∑ ∑ /∑ ∑ ∑ ∑ ∑ Hence , ̂ (∑ ) ∑ (∑ ) ( ∑ ) The maximum likelihood function when and ( ) √ where =∏ ( ( ) ) ∑ ( ) (48) The log likelihood of Equation 41 is . ( ) / ∑ ( ) ( ) ( ( ) ) ∑ . ( ) / ( ) (( ) ) ∑ . ( ) / (49) Differentiate both sides of Equation 49 with respect to . ( ) (( ) ( ))∑ ( ) By simplification, ( ) ( ) ∑ ∑ ( ) ( ) (50) Make Equation 50 equals to zero , ( ) ∑ ∑ ( ) = 0 ∑ ∑ ∑ ∑ ( ∑ ) ( ∑ ) ( ∑ ) ∑ (51) Differentiate both sides of Equation 49 with respect to . Then IHJPAS. 2025,38(4) 317 ( ) ∑ ( ) ∑ ( ) (52) Make Equation 52 equals to zero , ∑ ∑ Hence , ∑ (53) Differentiate both sides of Equation (49) with respect to . Then ( ) ∑ ( ) ∑ ( ) (54) Make Equation 54 equals to zero , ∑ ∑ ∑ (55) Substitute Equation 55 in Equation 54; ∑ ∑ After simplification, ∑ ∑ (56) Substitute Equation 56 in Equation 55; ∑ ∑ ∑ ∑ ∑ ∑ (57) Substitute Equation 57 in Equation 55; ∑ ∑ ∑ ∑ By simplification, ∑ ∑ ∑ ∑ Hence, ∑ ∑ (58) Substitute Equation 58 in Equation 56; ∑ ∑ ∑ ∑ ∑ ∑ IHJPAS. 2025,38(4) 318 (∑ ) ∑ ∑ (59) Therefore, ̂ (∑ ) ∑ ∑ Substitute Equation 59 in Equation 58; ( (∑ ) ∑ ∑ ) ∑ ∑ ∑ (60) Hence , ̂ ∑ Substitute Equation 60 in Equation 57; . ∑ / ∑ . ∑ / ∑ . ∑ / By simplification, ∑ ∑ ∑ Hence, ̂ ∑ ∑ ∑ 4. Simulation and Results. This study presents AR model estimation via maximum likelihood methods by defining marginal distribution of the given time series, for this some of simulated experiments designed and . Different initial values for the parameters were assumed for each experiment. The results of these experiments are given in Tables 1-9 for some statistical criteria, mean square error (MSE), and the absolute mean percentage error (MPE). The first experiment was designed to calculate MLE and MAR for time series values for , with ( and ), sample size ( and ), and number of replicate ( and ), the results of this experiment are given by Tables 1, 2, and 3. Regarding the MSE and MPE criteria, we can see from the given tables , the values of MSE and MPE decrease with increasing sample size n for each value of for MLE and MAR. The values of MPE of both methods become closer as the sample size increases for different values of . The MSE and MPE values decrease with decreasing replication for each value of . The values of MPE indicate MAR is better than MLE where most of the different model parameters and sample size Moreover, Figures 1, 2, and 3 represent the estimated model for the time series when , respectively, the red graph refers to the generated simulated model, the blue graph refers to the model estimated via exact likelihood, and the green graph refers to the estimated model by using the likelihood function via the marginal distribution of the time series. In Figures 1, 2, and 3, we notice that all-time series are stable at different values of the sample size and each number of replication . IHJPAS. 2025,38(4) 319 Table 1. Model for MLE and MAR with . The results of Table 1; as shown in Figure 1. ,r =100 ,r =100 ,r =100 ,r=100 ,r =100 ,r =100 ,r=100 ,r=100 ,r =100 ,r =100 ,r =100 ,r =100 Figure 1. ) Model for MLE and MAR with r =100. For number of replication , the results are given in Table 2 and Figure 2. n MLE MAR MSE MPE MSE MPE 30 1.5065e-32 46.9825 9.4843e-30 26.2206 1.1237e-32 50.6085 9.6416e-31 30.6140 1.9921e-33 48.6619 3.1081e-30 30.4533 4.9416e-31 40.1620 4.1241e-30 31.8221 50 1.9921e-33 22.2352 1.6181e-30 11.9847 0 21.6314 3.9045e-31 15.3154 1.1205e-31 18.3909 1.0085e-30 14.0371 2.1041e-32 16.3906 8.3717e-31 13.4952 100 1.2450e-34 7.3882 1.3466e-30 4.3022 5.4907e-32 7.5252 7.9683e-33 4.6729 2.8014e-32 7.7938 8.4165e-32 5.0796 7.1715e-32 5.7299 1.9921e-31 4.8566 IHJPAS. 2025,38(4) 320 Table 2. Model for MLE and MAR with . n MLE MAR MSE MPE MSE MPE 30 1.9921e-33 249.0715 6.3367e-28 156.3922 2.5817e-30 240.5369 1.3884e-28 142.6665 3.3487e-30 249.4303 5.0201e-28 159.3458 8.6808e-31 189.4300 1.9144e-30 154.6298 50 7.1715e-32 105.8937 2.9528e-28 62.9162 8.4165e-30 109.3878 6.6348e-29 62.0760 1.1506e-29 104.1844 6.2410e-29 64.2770 2.9407e-29 79.9037 5.6593e-28 59.7555 100 2.4359e-30 mse_mar = 2.6394e-28 mpe_mle = 39.9870 39.9870 2.6394e-28 21.2546 1.9328e-29 38.2339 2.4543e-28 21.6598 1.9574e-29 34.3012 4.8790e-29 22.1899 6.7627e-29 28.4785 1.0327e-29 20.5642 The plot of the results in Table 2, as show in Figure 2. ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 ,r =500 Figure 2. Model for MLE and MAR with For the number of replication, see the following Table 3 and Figure 3: IHJPAS. 2025,38(4) 321 Table 3. Model for MLE and MAR with r =1000. n MLE MAR MSE MPE MSE MPE 30 2.8766e-30 478.8193 2.3028e-28 290.8784 3.1873e-30 494.8485 2.9650e-29 291.8135 3.3537e-29 457.6450 2.3573e-28 290.2070 7.6527e-29 386.0229 2.2490e-28 317.8166 50 8.9955e-31 233.6525 5.7334e-27 124.4451 7.5749e-31 226.1102 1.3022e-27 124.1611 2.2024e-30 210.4893 2.6904e-28 139.5043 8.5978e-29 171.0632 5.4177e-28 127.1026 100 1.3466e-30 79.4712 1.3466e-30 42.9843 1.5779e-29 76.0834 9.3883e-28 42.8568 1.9623e-29 69.6642 2.3150e-27 43.1488 1.1461e-27 59.9664 3.1794e-28 43.9696 Draw the results in Table 3, as shown in Figure 3. ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 ,r =1000 Figure 3. Model for MLE and MAR with . The second experiment was designed to calculate MLE and MAR for time series values for with ( and ), and ( and ), sample size ( and ), and number of replicate ( and ). The results of this experiment are given by Tables 4, 5, and 6. Regarding the MSE and MPE criteria, we can see from the given tables that the values of MSE and MPE decrease with increasing IHJPAS. 2025,38(4) 322 sample size n for each value of for MLE and MAR. The values of MPE of both methods become closer as the sample size increases for different values of . The MSE and MPE values decrease with decreasing replication for each value of . The values of MPE indicate MAR is better than MLE where most of the different model parameters and sample size . Moreover, Figures 4, 5, and 6 represent the estimated model for the time series when , respectively, the red graph refers to the generated simulated model, the blue graph refers to the model estimated via exact likelihood, and the green graph refers to the estimated model by using the likelihood function via the marginal distribution of the time series. In Figures 4, 5, and 6, we notice that all-time series are stable at different values of the sample size and each number of replication . The following table, as well as the figure that is related to it, represents the information for the second experiment in this paper. For the number of replication , the results are given in Table 4 and Figure 4. Table 4. Model for MLE and MAR with . n MLE MAR MSE MPE MSE MPE 30 7.4211e-40 6.6789e-41 0.0030 0.0016 1.2450e-34 1.9921e-33 1.4253 4.8494 4.2745e-39 4.7495e-40 0.0111 0.0096 0 0 1.4116 27.5439 0 2.9684e-41 0.0024 0.0017 0 4.9802e-34 1.4233 6.4939 1.8553e-42 2.8988e-42 4.4416e-04 1.9749e-04 3.1126e-35 3.1126e-35 1.4428 1.3447 50 2.6716e-40 7.4211e-42 0.0023 0.0013 1.2450e-34 1.2450e-34 0.8658 1.4350 1.8998e-37 1.2159e-37 0.0250 0.0228 7.0034e-35 2.0399e-30 0.8381 107.9930 1.1874e-40 4.7495e-40 0.0017 0.0015 7.0034e-35 3.1126e-35 0.8637 0.4945 2.9684e-41 1.8553e-42 6.1100e-04 2.4482e-04 7.0034e-35 3.1126e-33 0.8769 3.6684 100 1.8998e-39 2.9684e-41 0.0015 8.5164e-04 7.0034e-35 7.9683e-3 0.4533 7.7977 3.0397e-38 7.5992e-39 0.0115 0.0103 3.1126e-35 7.9683e-31 0.4305 30.1493 4.7495e-40 7.4211e-42 8.6626e-04 6.1362e-04 0 1.7929e-32 0.4541 7.8653 0 2.6090e-43 3.2966e-04 2.3317e-05 3.1126e-35 3.1873e-32 0.4669 7.6017 The plot of the results in Table 4, as shown in Figure 4. IHJPAS. 2025,38(4) 323 Figure 4. for MLE and MAR Model with . For the number of replication ; the results are given in Table 5 and Figure 5. Table 5. ) Model for MLE and MAR with n MLE MAR MSE MPE MSE MPE 30 9.6177e-39 8.5787e-39 0.0042 0.0034 4.9802e-34 4.9802e-34 1.4365 5.1715 1.5388e-37 4.7495e-38 mse_ml = 4.7495e-38 0.0254 0.0221 4.4822e-33 1.4056e-29 1.4125 99.4448 7.5992e-39 5.8181e-39 0.0038 0.0032 1.3151e-33 4.0339e-32 1.4287 14.2587 2.4536e-40 4.1743e-42 5.1827e-04 1.2615e-04 1.5252e-33 1.7508e-33 1.4517 0.9196 50 2.6716e-40 2.4536e-40 8.8243e-04 3.2758e-04 3.7663e-33 6.1007e-33 0.8814 3.4285 4.8635e-37 2.0945e-37 0.0083 0.0072 2.8091e-33 5.0997e-31 0.8582 14.0768 1.0716e-38 2.4044e-39 0.0020 0.0014 1.1205e-33 6.1007e-33 0.8675 1.3755 7.4211e-42 4.8991e-42 8.6381e-04 4.9186e-05 1.7508e-33 1.2450e-32 0.8805 3.3063 100 2.6716e-38 5.0166e-39 0.0014 7.5191e-04 1.9454e-34 1.4522e-30 0.4573 7.8931 7.5992e-39 0 0.0099 0.0085 7.7815e-36 1.5618e-30 0.4349 24.5102 2.9684e-41 1.0686e-39 9.4607e-04 4.3304e-04 0 5.7571e-31 0.4552 7.6444 1.8553e-40 1.0436e-42 1.8933e-04 4.1176e-05 7.7815e-34 2.8686e-31 0.4687 7.7983 IHJPAS. 2025,38(4) 324 The plot of the results in Table 5, as shown in Figure 5. Figure 5. Model with . For the number of replication the results are given in Table 6 and Figure 6. Table 6. Model for MLE and MAR with n MLE MAR MSE MPE MSE MPE 30 7.4211e-38 1.0686e-39 0.0046 0.0025 4.4822e-33 7.0034e-31 1.4337 9.1800 4.0200e-36 8.3781e-37 0.0275 0.0250 7.0034e-33 2.4989e-29 1.4137 102.4699 7.5992e-39 9.0908e-39 0.0028 0.0013 4.4822e-33 2.6345e-31 1.4317 11.8544 3.7569e-41 1.2627e-40 4.9649e-04 2.3499e-04 9.5324e-33 1.0653e-32 1.4550 1.2305 50 8.6559e-38 6.8392e-38 0.0040 0.0029 4.4822e-33 5.2323e-32 0.8680 1.6382 8.3781e-37 2.7357e-37 0.0196 0.0176 1.9921e-33 6.1706e-29 0.8484 72.0161 2.1640e-38 2.0846e-38 0.0021 0.0013 1.7508e-33 8.9955e-33 0.8681 1.5923 1.6697e-39 8.9795e-40 7.1442e-04 3.6851e-04 2.8091e-33 1.2749e-31 0.8864 3.8168 100 1.8998e-37 2.0067e-38 0.0015 0.0011 1.1237e-32 1.4522e-30 0.4588 7.9779 2.7357e-37 1.0050e-36 0.0110 0.0098 7.9683e-33 3.2638e-29 0.4351 25.8946 2.6716e-40 1.0716e-38 8.8139e-04 6.4366e-04 3.4317e-33 3.5140e-30 0.4591 7.9920 8.1817e-40 2.4536e-40 2.1775e-04 6.3309e-05 3.8130e-34 0 0.4734 7.5106 IHJPAS. 2025,38(4) 325 The following Figure helps us show the results in Table 6. Figure 6. Model for MLE and MAR with . The third experiment is designed to calculate MLE and MAR for time series values for , with ( and ), ( and ), and ( and ), sample size ( and ), and the number replicate ( and ). The results of this experiment are given by Tables 7, 8, and 9. Regarding the MSE and MPE criteria, we can see from the given tables, the values of MSE and MPE decrease with increasing sample size n for each value of for MLE and MAR. The values of MPE of both methods become closer as the sample size increases for different values of . The MSE and MPE values decrease with decreasing replication for each value of . The values of MPE indicate MAR is better than MLE where most of the different model parameters and sample size Moreover, Figures 7, 8, and 9 represent the estimated model for the time series when , respectively, the red graph refers to the generated simulated model, the blue graph refers to the model estimated via exact likelihood, and the green graph refers to the estimated model by using the likelihood function via the marginal distribution of the time series. In Figures 7, 8, and 9, we notice that all-time series are stable at different values of the sample size and each number of replication . IHJPAS. 2025,38(4) 326 Table 7. Model for MLE and MAR with . n MLE MAR MSE MPE MSE MPE 30 1.7508e-35 3.1873e-32 4.9802e-34 0.6308 56.2287 6.0444 1.7929e-32 0 2.0399e-30 17.1093 6.3666 575.9633 0 3.1873e-32 4.9802e-34 0.0214 38.5691 3.0948 4.9802e-34 1.2450e-34 4.5897e-30 16.2997 5.1487 440.9363 4.8635e-37 2.8686e-31 1.9921e-33 0.2512 103.1897 9.6427 7.9683e-33 0 0 16.6317 5.6625 496.3091 7.7815e-36 3.1873e-32 4.9802e-34 0.3762 35.2640 3.2239 4.9802e-34 1.9921e-33 4.5897e-30 16.6013 5.6163 491.2326 50 3.1126e-35 0 1.9921e-33 0.5560 26.8586 3.5300 1.9921e-33 4.4822e-33 1.8359e-29 10.6481 4.5445 256.8860 0 5.0997e-31 1.2450e-34 0.0089 61.2589 4.8212 7.9683e-33 4.9802e-34 5.0997e-31 9.6300 3.0853 154.9406 1.9454e-36 1.1474e-30 4.4822e-33 0.1122 70.7262 6.7510 4.4822e-33 1.2450e-34 4.5897e-30 9.8909 3.4879 181.0627 4.3771e-36 3.1873e-32 3.1126e-35 0.2402 20.5466 2.0373 1.2450e-32 4.9802e-34 5.0997e-31 9.9961 3.6442 191.5910 100 3.1126e-35 3.1873e-32 1.9921e-33 0.3113 12.1245 1.8428 1.9921e-33 4.9802e-34 2.0399e-30 5.5194 2.6377 76.1240 3.0397e-38 3.1873e-32 0 0.0094 19.0202 1.9054 4.4822e-33 4.4822e-33 0 5.0358 2.0113 51.8760 1.2159e-37 5.0997e-31 4.4822e-33 0.0500 35.1164 3.5198 0 1.2450e-34 1.2749e-31 4.9543 1.8936 47.7893 1.2159e-35 1.2749e-31 4.9802e-34 0.1258 13.6534 1.5854 4.4822e-33 4.9802e-34 1.1474e-30 5.1427 2.1598 57.2322 The following Figure shows the information in the table above. , , , , , , IHJPAS. 2025,38(4) 327 , , , , , , Figure 7. Model for MLE and MAR with . For the number of replication , the results are given in Table 8 and Figure 8. Table 8. Model for MLE and MAR with . n MLE MAR MSE MPE MSE MPE 30 7.7815e-36 1.1474e-30 5.4907e-32 1.1129 41.6021 6.2432 2.4104e-31 4.4946e-32 2.4683e-28 18.8798 8.5553 852.8920 1.1874e-38 5.0997e-31 1.2450e-34 0.0151 71.6615 5.4075 1.6136e-31 4.9802e-34 9.9954e-29 16.2684 4.9303 417.3141 7.7815e-36 3.1873e-30 1.1205e-31 0.1940 115.0466 10.0319 1.9921e-31 1.0085e-32 1.3055e-28 16.5396 5.3600 462.5379 7.0034e-35 1.2749e-31 7.0034e-33 0.3779 35.6797 3.0645 0 6.1007e-33 9.9954e-29 16.5729 5.4118 468.0983 50 2.8014e-34 1.5618e-30 1.7929e-32 0.3748 33.2431 3.5554 3.1873e-32 2.8014e-32 9.9954e-29 10.1978 3.8494 205.2482 0 8.1595e-30 4.0339e-32 0.0131 78.4409 6.7044 2.1963e-31 2.4403e-32 1.2749e-29 9.8868 3.3887 174.1100 2.8014e-34 0 1.2450e-32 0.2287 46.7047 4.6974 4.4822e-31 3.1873e-32 5.0997e-29 10.0690 3.6620 192.3506 8.2192e-35 1.2749e-31 1.1237e-32 0.2531 19.8713 1.8715 1.7929e-32 1.7929e-32 1.3055e-28 10.0014 3.5618 185.5815 100 5.6222e-34 1.3466e-30 1.0085e-32 0.2117 14.8984 1.7558 2.4403e-32 4.9802e-32 2.4989e-29 5.1762 2.1664 57.2946 1.8998e-39 1.2749e-31 4.9802e-34 0.0047 27.1229 2.3497 1.9921e-33 1.2450e-32 1.0327e-29 4.9021 1.7712 43.5548 3.9394e-35 8.1595e-30 1.2749e-31 0.0650 32.7943 3.2813 1.4393e-31 1.2450e-32 6.2471e-30 4.9854 1.8957 47.7256 7.7815e-36 9.7612e-32 8.9955e-33 0.1105 9.6529 0.9837 1.7978e-31 6.1007e-33 1.8359e-29 5.0245 1.9529 49.6866 IHJPAS. 2025,38(4) 328 The values in Table 8 via the Figure are shown as follows: , , , , , , , , , , , , Figure 8. Model for MLE and MAR with For the number of replication , the results are given in Table 9 and Figure 9: Table 9. ) Model for MLE and MAR with . n MLE MAR MSE MPE MSE MPE 30 7.7815e-34 8.1595e-30 6.0260e-32 0.6687 55.1572 5.8641 5.4234e-31 1.7045e-31 1.2749e-29 17.2401 6.3891 577.0794 3.0397e-38 1.4738e-28 1.6136e-31 0.0045 240.8549 21.6782 1.1957e-30 6.5863e-32 7.3436e-29 16.9682 5.9919 531.7196 2.3831e-35 6.7444e-29 6.4543e-31 0.1511 116.0243 10.5361 0 3.1873e-32 1.6523e-28 mpe_m = 0.1511 16.7239 5.6242 490.9766 3.5455e-34 3.8567e-30 4.9802e-32 0.4434 30.9070 3.0517 1.5065e-30 1.1205e-31 9.9954e-29 17.0314 6.0854 542.2655 50 4.8635e-35 2.8766e-30 1.4393e-31 0.5620 27.3096 3.6123 1.4522e-30 1.4393e-31 7.7567e-28 10.7926 4.6473 263.9899 2.0689e-36 1.2749e-29 1.1957e-30 0.0092 84.1834 6.7660 3.1126e-31 0 1.7454e-28 9.7663 3.1902 161.2271 4.3771e-36 1.1474e-28 7.5749e-31 0.1259 64.8504 6.3882 8.3717e-31 4.9802e-32 1.4738e-28 10.0636 3.6429 190.9959 IHJPAS. 2025,38(4) 329 n MLE MAR MSE MPE MSE MPE 2.1448e-34 2.0399e-30 3.1126e-35 0.2248 21.1864 2.0627 1.6136e-31 0 2.4989e-29 10.0493 3.6217 189.5649 100 5.4646e-33 8.1595e-30 7.7815e-32 0.3224 12.4381 1.8716 7.1914e-31 9.7612e-32 1.2252e-28 5.5823 2.6766 77.4493 2.8896e-36 6.4543e-29 3.5982e-32 0.0064 30.6033 2.4344 3.3666e-31 1.7929e-32 1.1474e-30 4.8274 1.6496 39.6038 9.5324e-35 5.3579e-29 1.0471e-31 0.0737 26.3076 2.4699 4.9802e-32 6.0260e-32 5.0997e-29 4.9277 1.8042 44.6310 4.9802e-34 2.3028e-30 3.1873e-32 0.1460 7.9169 0.9263 3.6306e-31 5.4907e-32 2.4683e-28 5.2147 2.2136 59.0183 The results in the Table above are shown in the figures below. , , , , , , , , , , , , Figure 9. Model for MLE and MAR with . 5. Conclusion Recently, time series analysis is necessary for many fields and applications with huge and complex data. This paper presents the problem of autoregressive model parameter estimation with maximum likelihood function and model selection. The model parameter estimators are derived theoretically by defining the marginal distribution of the AR time series, where the distribution is got by random shocks representing the AR process with the assumption of a IHJPAS. 2025,38(4) 330 normal distribution of the white noise errors term. Here, AR model order is adopted for and . Exact maximum likelihood function and maximum likelihood with time series marginal distribution are derived. To show the ability of the theoretical computing of the AR parameters, some of the experiments are implemented with simulation for the AR parameter estimation and model selection. The experiments are designed for different values of model order (p) , sample size (n), and number of replications. The results show the efficiency of the model parameter estimation by computing mean squares errors, and mean percentage errors. For different settings of and sample size , the results show the stationarity of the predicted model with estimated parameters for different values of , sample size , and replications. The values of MPE and MSE show the ability of MAR for model parameter estimation, where at most (MPE and MSE)-based MAR are better than MLE. Acknowledgment The authors express gratitude to the reviewers for their insightful feedback and recommendations to enhance the paper. Conflict of Interest The authors assert that they do not have any conflicts of interest. Funding There is no financial assistance available for preparing the publication. References 1. Davis RA, Brockwell PJ. Introduction to time series and forecasting. Springer publication; 2016. 2. Wei W.W. Time series analysis: univariate and multivariate methods. USA, Pearson Addison Wesley, Segunda edicion .2006; Cap, 10: 212-235. 3. Creal D, Koopman SJ, Lucas, A. Generalized autoregressive score models with applications. J Appl Econ .2013; 28(5):777-795. http://dx.doi.org/10.1002/jae.1279. 4. Al-Nasser AM, Tariq S. Robust Estimations for power spectrum in ARMA (1, 1) model simulation study. J Econ Admin Sci . 2017; 23(98). https://doi.org/10.33095/jeas.v23i98.287. 5. Liu J, Kumar S , Palomar D.P. Parameter estimation of heavy-tailed AR model with missing data via stochastic EM. IEEE Trans Sign Proc .2019; 67(8):2159-2172. https://doi.org/10.1109/TSP.2019.2899816. 6. Juma AA, AL-Mohana, FAM. A Modified Approach by Using Prediction to Build a Best Threshold in ARX Model with Practical Application. Baghdad Sci J. 2019; 16(4 Supplement). http://dx.doi.org/10.21123/bsj.2019.16.4(Suppl.).1049 7. Naser J.A. Estimate AR(3) by Using Levinson-Durbin Recurrence & Weighted Least Squares Error Methods. Ibn AL-Haitham J Pure Appl Sci. 2017; 26(3):357- 378. https://jih.uobaghdad.edu.iq/index.php/j/article/view/447. 8. Ashour MA. H. Optimized Artificial Neural network models to time series. Baghdad Sci J. 2022; 19(4): 0899-0899. http://dx.doi.org/10.21123/bsj.2022.19.4.0899. 9. Mudhir AA. Mixing ARMA Models with EGARCH Models and Using it in Modeling and Analyzing the Time Series of Temperature. Iraqi J Sci . 2021; 2307-2326. http://dx.doi.org/10.24996/ijs.2021.62.7.19 10. Hussain B.A, Al-Dabbagh R. A. D. A canonical genetic algorithm for likelihood estimator of first order moving average model parameter.Neural Network World. 2007 ;17(4): 271. http://dx.doi.org/10.1002/jae.1279 https://doi.org/10.33095/jeas.v23i98.287 https://doi.org/10.1109/TSP.2019.2899816 http://dx.doi.org/10.21123/bsj.2019.16.4(Suppl.).1049 https://jih.uobaghdad.edu.iq/index.php/j/article/view/447 http://dx.doi.org/10.21123/bsj.2022.19.4.0899 http://dx.doi.org/10.24996/ijs.2021.62.7.19 IHJPAS. 2025,38(4) 331 11. Salah OM, Mahdi GJM, Al-Latif IAA. A modified ARIMA model for forecasting chemical sales in the USA. In J Physics: Conference Series. 2021; 1879(3): 032008. IOP Publishing. http://dx.doi.org/10.1088/1742-6596/1879/3/032008 12. Chrétiena S, Wei T, Al-sarray B. A. H. Joint estimation and model order selection for one dimensional ARMA models via convex optimization: a nuclear norm penalization approach.arXiv preprint.2015;arXiv:1508.01681.https://doi.org/10.48550/arXiv.1508.01681. 13. Ali SM. Time series analysis of Baghdad rainfall using ARIMA method. Iraqi J Sci . 2013;54(4): 1136-1142. 14. Montgomery DC, Jennings CL, Kulahci, M. Introduction to time series analysis and forecasting, John Wiley & Sons; 2015. 15. Palma W. Time series analysis, John Wiley & Sons; 2016. 16. Mills TC. Applied time series analysis: A practical guide to modeling and forecasting, Academic press; 2019. 17. Kirchgässner G, Wolters J,Hassler, U. Introduction to modern time series analysis. Springer Science & Business Media; 2012. 18. Anderson T.W. The statistical analysis of time series. John Wiley & Sons ; 2011. 19. Guidolin M, Pedio M. Essentials of time series for financial applications. Academic Press; 2018. 20. Brockwell PJ, Davis RA. Time series: theory and methods. Springer science & business media; 1991. 21. Box G. Time series analysis, forecasting and control. In A Very British Affair: Six Britons and the Development of Time Series Analysis During the 20th Century. London: Palgrave Macmillan UK; 2013; pp. 161-215. 22. Hamilton J. D. Time series analysis, Princeton university press; 2020. 23. Millar RB. Maximum likelihood estimation and inference: with examples in R, SAS and ADMB, John Wiley & Sons; 2011. 24. Al-sarray B .Comparison among Forecasting Methods of Markov and Mixed Models by using Simulation. master thesis, College of Science, University of Baghdad ,2001. 25. Bauwens L, Rombouts JV. On marginal likelihood computation in change-point models. Computational Stat Data Anal. 2012;56(11), 3415-3429. http://dx.doi.org/10.1016/j.csda.2010.06.025. 26. Cudeck R, Harring J. R, du Toit S. H. Marginal maximum likelihood estimation of a latent variable model with interaction. J Educational Behavioral Stat. 2009; 34(1), 131-144. https://doi.org/10.3102/1076998607313593. 27. Balakrishna N. Non-Gaussian autoregressive-type time Series. Singapore. Springer; 2021. 28. Ullrich T. On the autoregressive time series model using real and complex analysis. Forecasting. 2021; 3(4), 716-728. https://doi.org/10.3390/forecast3040044. 29. Wilson GT. Time Series Analysis: Forecasting and Control. By George EP Box, Gwilym M. Jenkins, Gregory C. Reinsel and Greta M. Ljung. Published by John Wiley and Sons Inc., Hoboken, New Jersey 2015. 30. Forbes C, Evans M, Hastings, N, Peacock B. Statistical distributions. John Wiley & Sons 2011. http://dx.doi.org/10.1088/1742-6596/1879/3/032008 https://doi.org/10.48550/arXiv.1508.01681 http://dx.doi.org/10.1016/j.csda.2010.06.025 https://doi.org/10.3102/1076998607313593 https://doi.org/10.3390/forecast3040044