Eser, M.T. (2021). Comparison of bayesian and frequentist factor analysis methods: Buss and perry aggression questionnaire example. International Online Journal of Education and Teaching (IOJET), 8(4). 2871-2887. Received : 14.06.2021 Revised version received : 09.09.2021 Accepted : 12.09.2021 COMPARISON OF BAYESIAN AND FREQUENTIST FACTOR ANALYSIS METHODS: BUSS AND PERRY AGGRESSION QUESTIONNAIRE EXAMPLE (Research article) Mehmet Taha Eser https://orcid.org/0000-0001-7031-1953 Aydın Adnan Menderes University, Turkey m.taha.eser@adu.edu.tr Biodata: Mehmet Taha Eser has been Assistant Professor in Aydin Adnan Menderes University Faculty of Education since 2020. His research interests are data analysis, item response theory and statistical modeling. Copyright © 2014 by International Online Journal of Education and Teaching (IOJET). ISSN: 2148-225X. Material published and so copyrighted may not be published elsewhere without written permission of IOJET. https://orcid.org/0000-0001-7031-1953 mailto:m.taha.eser@adu.edu.tr http://orcid.org/xxxx Eser 2872 COMPARISON OF BAYESIAN AND FREQUENTIST FACTOR ANALYSIS METHODS: BUSS AND PERRY AGGRESSION QUESTIONNAIRE EXAMPLE Mehmet Taha Eser m.taha.eser@adu.edu.tr Abstract This study aims to compare the results of the factor analysis performed with Frequentist and Bayesian approaches. The number of sub-dimensions of the measurement tool obtained from different methods, the variation of the items in the sub-dimensions, and the fit statistics' differentiation were examined. 778 students constitute the study sample. Buss and Perry Aggression Questionnaire-short form was used in the study. The results obtained from Bayesian factor analysis and maximum likelihood, minimum residuals, principal axis factoring, ordinary least squares, weighted least squares, generalized least squares factor extraction methods, which are frequentist factor analysis methods, were compared. As a result of the study, the number of factors obtained from different factor extraction methods was the same, and the measurement tool had a four-dimensional structure. The analyses were performed in R,and JASP the factor loading cut-off was set as .40. As a result of study, Bayesian approach also yields valid results in terms of factor analysis, similar to the frequentist approach. Bayesian factor analysis method allowed to keep the measurement tool's structure, and there is no need to remove items from the measurement tool. So the researchers were recommended to include the Bayesian factor analysis in their studies. Keywords: Bayes, factor analysis, measurement model. 1. Introduction Two different theoretical approaches, the Classical (or Frequentist, Berkeley statistics) approach, and the Bayesian approach, were dominant in statistics' development process. Frequentist and Bayesian approaches have been seen as the other's alternative for addressing and processing many issues and concepts. Being an alternative approach having its discipline, many statistical concepts are interpreted differently by the Bayesian approach (Ekici, 2005; Mc Charty, 2007; Ekici, 2009; Etz & Vandekerckhove, 2016). Thomas Bayes, an English mathematician, laid the foundations of the Bayesian approach in 1763. In the paper "An Essay Towards Solving a Problem in the Doctrine of Chances," the approach was published by his friend Richard Price a certain time after Thomas Bayes' death. This article also forms the basis of the Bayesian approach used today (Link & Barker, 2006; Savchuk & Tsokos, 2011). Bayes' theorem is important in mathematical statistics. This theorem aims to produce results by using universal truths and observations in modeling. The most important feature that distinguishes this approach from other classical statistical methods is using observations and subjective opinions in predicting uncertain knowledge (Box & Tiao, 1992; Congdon, 2003; Ekici, 2009; Çevik, 2009; Link & Barker, 2006). The Bayesian approach has gained popularity mailto:m.taha.eser@adu.edu.tr International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2873 first in natural sciences and health sciences. In recent years, it became popular, especially in the social sciences, upon the science's open and ethical realization. The reasons for choosing the Bayesian approach are reliability, accuracy (especially in case of noisy data and small samples), the possibility of including prior knowledge in the analysis, and the intuitive and simple interpretation of the results (Berger, 1985; Carlin & Louis, 2000; Erkan, 2019). Frequentist and Bayesian approaches have theoretically differed from each other as they developed separately over time, and their distinctive features became evident. The frequentist approach is parallel to the deductive method, whereas the Bayesian approach shows parallelity to the inductive method. Usually, the frequentist approach seems closer to the descriptive interpretation of the causality principle, while the Bayesian approach is closer to the probabilistic interpretation of the causality principle (Ekici, 2009; Kruschke, 2010). In classical approaches, analyzes are performed under the assumption of normal distribution. According to this assumption, the distribution of data approaches the normal distribution in large samples. However, researchers may fail to meet this assumption. It may not be possible to achieve a large sample size, especially in studies conducted in medicine and psychology, or meet the assumption of multivariate normality in studies with incomplete observations such as in behavior and social sciences. In this case, the parameter and standard error estimates will tend to give biased results (Ekici, 2009; Ozechowski, 2014; Erkan, 2019). However, in the analyses based on the Bayesian approach, posterior distributions are obtained using raw data and prior knowledge. In this way, the Bayesian approach provides the most appropriate solution in cases where the analyzes based on classical approaches are insufficient. The frequentist approach generally focuses on testing the null hypothesis and often increases science's replication crisis due to misuse of the p-value. In the frequentist approach, the null hypothesis only survives until the next test, which is an important limitation of the frequentist approach, causing researchers to find themselves on the "wrong side of the null hypothesis" (Bernardo & Smith, 1994; Gelman et al., 2004; Lee, 2004). There is general acceptance that the Bayesian approach is a way to handle these and other problems (Szucs & Ioannidis, 2016; Benjamin et al., 2018; Etz & Vandekerckhove, 2016). Researchers can have good prior knowledge from the analysis of similar or historical data or other sources for many biomedical and behavioral sciences problems. This knowledge about unknown parameters obtained from previous studies is called prior. The most important feature that distinguishes Bayesian inference from frequentist inference is that prior knowledge is used in the Bayesian approach. Bayes theorem is mainly based on combining prior probability distributions to generate posterior probabilities. A priori distribution of a parameter is a probability distribution that contains imprecise information about the parameter before analyzing the data. According to the Bayesian approach, without making any observations for parameter estimation, one can have an idea about the value of theta by using the previous information. From this point of view, it can be thought that there is a pre-knowledge distribution consisting of the values that theta can take (Meyer & Millar, 1999; Millar, 2002; Lee, 2012). In Bayesian statistics, the prior probability corresponds to the event's probability before new data are collected. Bayes theorem, which forms the basis of the Bayesian approach, is shown in Equation 1. 𝑝(𝐴\𝐵) = 𝑝(𝐵\𝐴) 𝑝(𝐴) 𝑝(𝐵) Eser 2874 In this equation obtained from the conditional probability definition, p(A\B) shows the probability of event A occurring, given that event B occurred. The expression p(B\A) in the numerator part of this equation shows the probability of occurrence of event B, given that event A occurred, and p (A) and p (B) show the probability of occurrence of events A and B, respectively (Winkler, 2003). Bayes theorem allows the interpretation of parameters using prior knowledge after collecting prior knowledge. Previous studies, experiments, and expert opinions are accepted as prior knowledge. In the Bayesian approach, the posterior knowledge shows the current state of the knowledge about all unknown parameters, and it is obtained using prior information and the likelihood function (Equation I). Figure 1 shows the basic logic of Bayesian statistics. Posterior probability= Likelihood x prior probability / Proof of the prior probability (Equation I) Figure 1. Calculation of bayesian posterior probability. source: Adams et al. (2013) The expression p (A\B) given in Equation 1 shows the posterior probability, the expression p (A) shows the prior probability, the expression p(B\A) the likelihood function, and the expression p(B) shows the proof of the prior probability. This is considered the best rational assessment of a posterior probability based on available knowledge before an experiment is conducted. The posterior probability is the revised probability of an event that occurs after new knowledge is taken into account. The posterior probability is calculated by updating the prior probability using the Bayes theorem. Accordingly, the posterior probability statistically means "the probability of event A to occur given that event B occurred." In the Bayesian approach, the prior distribution must be determined to obtain the posterior distribution. Determining prior distribution for model parameters and permitting them to explain their probability directly emerges as a distinctive feature compared to the classical approach. However, to achieve this, the researcher needs to select the prior distribution of model parameters before analysis. It can be said that this stage is perhaps the most challenging part for the researcher (Ekici, 2009; Ozechowski, 2014; Erkan, 2019). The data sets govern our prior beliefs about parameters and thus give rise to current beliefs. But how is the prior selection carried out? Prior choice is related to how much information we have before collecting data and how accurate we believe this information is. Prior distributions are divided into two as informative and non-informative according to the amount of information they have. According to the amount of information, the preliminary distribution can directly affect the final distribution, as well as through the likelihood function. Whether this percentage distribution contains information is the most important selection criterion in determining the prior distribution (Lee, 2004; Walker, Gustafson & Frimer, 2007; Yuan & MacKinnon, 2009). International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2875 1.1. Exploratory Factor Analysis and Bayesian Approach Thurstone (1934) suggested some criteria for predicting and describing factor models with perfectly simple structures, where each measurement is associated with at most one latent factor, and he developed various analytical methods. For him, the models with simple structures are transparent and easily interpreted. He developed the "oblique" rotation method for factor analysis, suggesting that interrelated factors are a more reasonable representation of reality (Thurstone, 1947). Exploratory factor analysis is accepted as a classical method developed for discovering the perfect simple structure that Thurnstone (1934) tried to reach (Gorsuch, 1983, 2003). Almost all of the classical Exploratory Factor Analyzes' steps are at the researcher's arbitrariness to a certain extent (Conti et al., 2014). Classical factor analysis consists of four steps: choosing the model's sub-dimensions, assigning the items to factors, estimating factor loadings, and removing the items assigned on more than one factor. Also, various methods are available to select the dimensional latent structure, remove items, and rotate factors (Jenrich, 2001; Gorsuch, 2003; Costello & Osborne, 2005). The researcher aims to perform Bayesian factor analysis to carry out an analysis that better reflects the theories and prior beliefs. The analysis is carried out using Markov Chain Monte Carlo (MCMC) methods, which are the models that simulate observations derived from complex distributions and allow working with high dimensional models. The Monte Carlo method is an experimental method that aims to reach the result by experimenting with random numbers. Monte Carlo refers to the partial random simulation process. It is also known as the Metropolis algorithm and aims to go to predictive solutions in problems where it is difficult to reach exact solutions. The Markov chain is; Considering the previous value, a new value from the a priori distribution represents the sampling process. These iterative process values constitute the markov chain value of the sample drawn from the a priori distribution (Box & Tiao, 1992; Congdon, 2003; Congdon, 2010; Karadağ, 2011) The model's parameters and latent variables are simulated from the posterior distribution, which has a systematically equivalent equation. Since full posterior distributions can be deduced for unknown parameters and latent variables in MCMC, the large sample assumption is not needed (Ozechowski, 2014). Therefore, the Bayes estimation performs well in small sample sizes. Besides, unlike the Maximum Likelihood (ML) estimation, the multivariate normal distribution assumption is not sought in Bayes estimation in analyzes. In terms of computation, Bayes estimation allows estimating more complex models than the Maximum Likelihood estimation (Asparouhov & Muthen, 2010; Schmitt, 2011). The most frequently used methods in MCMC are MCMC sampling and Gibbs sampling, which are based on the Metropolis-Hastings algorithm. The Metropolis-Hastings algorithm was invented by American physicist and computer scientist Nicholas C. Metropolis. The algorithm is simple and practical and is used to obtain random samples from an arbitrary complex target distribution of any size, known as the normalization constant. "Gibbs sampling", a special case of the Metropolis-Hastings algorithm, requires decomposing the combined posterior distribution into full conditional distributions for each parameter in the model and their sample. According to some researchers, this algorithm is more useful since the auxiliary distribution is not required for the application of the Metropolis algorithm. The classical factor analysis's rotation stage is not included in the functioning of the Bayesian factor analysis. The gap created by the absence of rotation is overcome with strong priors and MCMC methods. Factor loadings of the items formed by Bayesian factor analysis Eser 2876 are affected by prior distributions; therefore, they differ from factor loadings created by classical factor analysis. In Bayesian factor analysis, all model parameters (loadings, intercepts, variances, correlations) have prior distributions (Mair, 2018; Piatek, 2019). Bayesian factor analysis does not only load the item scores obtained from the measurement processes to any dimension but also uses all obtained information. Unlike classical factor analysis, in Bayesian factor analysis, the number of factors is not specified at the first stage. The number of factors is estimated with other parameters. The definition criteria of classical factor analysis are included in the Bayesian factor analysis, and posterior distributions, which constitute stable and interpretable models, are formed during the inclusion process. Hinkin (1995) underlined that approximately 71% of the literature studies where exploratory factor analysis was used were publications involving dimension reduction and scale development. Thus, the Bayesian method's introduction as an alternative to frequentist methods is important in contributing to the field. With this study, it is possible to use the posterior distribution obtained in previous studies as a prior in a new study in case of new observations. Besides, MCMC methods allow performing the probability theory analyses in small samples without meeting normality assumption, which assigns an important role to Bayesian factor analysis. Bayesian factor analysis methods that eliminate the hypothetical limitations of frequentist factor analysis can be used as an alternative to classical exploratory factor analysis, which is frequently used in scale development studies, especially in educational and social sciences. This study aims to compare the factor analysis results obtained from frequentist and Bayesian approaches. For this reason, the following subjects were discussed in the study: the number of sub-dimensions in the measurement tool obtained by different methods, the variation of the items in the sub- dimensions, and the differentiation of the fit statistics. The study also aims to compare some statistical results of frequentist and Bayesian factor analysis; to show the advantages of the Bayesian method and contribute to the idea that Bayesian factor analysis can be used as an alternative to classical factor analysis or as a validation method. The research's problem statement was specified as "Do the factor analysis results obtained from frequentist and Bayesian factor analysis differ?". 2. Method This study compares classical exploratory factor analysis results based on the frequentist and Bayesian method. Since it is aimed to reveal similar and different aspects of the analysis results obtained from two different methods, the study is under the comparative studies model, one of the quantitative research methods (Mills, Bunt, & Brujin, 2006). 2.1. Study Sample The study sample consists of 778 students between the ages of 15-18 from five different high schools. Approximately 35% of the participants were male, and 65% were female students. The sample selection was based on volunteerism, and no personal information was collected from the participants. The sample of the study includes students from the 9th, 10th, 11th, and 12th grades. The stratified purposeful sampling method, one of the purposeful sampling methods, was used to discover and describe the characteristics of certain subgroups of the sample used in the study (Fraenkel, Wallen, & Hyun, 2012). 2.2. Data Collection Instruments International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2877 In the study, Buss and Perry Aggression Questionnaire-short form was used as the data collection tool. Regarding the literature, there are many measuring tools developed to determine the level of aggression (Buss & Perry, 1992; Palmstierna & Wistedt, 1987; Raine et al., 2006). Among these measurement tools, Buss and Perry Aggression Questionnaire is the most used one. The scale consists of 29 items and 4 sub-dimensions: physical aggression, verbal aggression, anger, and hostility (Buss & Perry, 1992). Bryant and Smith developed the short form of the scale to create a much simpler data collection tool with three items in each sub-dimension. The short form has been adapted to different cultures, including the Netherlands, Argentina, and China (Reyna et al., 2011); the psychometric properties of the Turkish version such as gender, age, measurement equivalence, and changing item function were examined by Kuzucu and Sarıot-Ertürk (2020). The results showed that the Aggression Scale's Short Form is a measurement tool that consists of four factors, has an acceptable level of reliability for Turkish culture, does not show a measurement difference according to gender, and can measure aggression for adolescents and adults. 2.3. Data Analysis Today, bayesian analyzes can be performed using popular software such as WinBUGS, R, JASP, SAS, MPLUS and SPSS. SAS and SPSS are commercial softwares and other softwares are free. Among these softwares, WinBUGS, R and JASP stand out for being free. Open source and free R and JASP softwares were used in the research . Within the scope of the R program, the "BayesFM" package was used, in which the number of latent factors and the assignment of explanatory variables (manifest variables) to the factors were determined during the MCMC sampling process (Piatek, 2019). In the original publication where Buss and Perry Aggression Questionnaire-short form was developed, four factors of the scale were reported to be interrelated. They used the principal axis factoring method as the factor extraction method for exploratory factor analysis and the ProMax method as the rotation method. In Social Sciences, the relevant structure's factors are generally expected to be related to each other (Costello & Osborne, 2005). However, the selection of the correct rotation method depends on the researcher's objective. It is necessary to check whether the used rotation method accurately reflects pre-existing assumptions about the data and whether the rotated data matches the model expected from the theory (Allen, 2017). Different factor extraction methods were used to compare related sub-dimensions. In the analysis, factor loading cut-off was set as 0.40 considering that low values of factor loading cut-off cause some items to be assigned to more than one factor and cross-loading occurs (Costello & Osborne, 2005); with a low factor loading cut-off value, it is more likely that subjectivity will be more involved in the decision-making process regarding the items that should be kept in the measuring tool; and a higher cut-off value allows the items to represent better the factors to which they are attached (Comrey & Lee, 1992; Tabachnick & Fidell, 2007). Important assumptions in factor analysis methods are: (i) the quality of the data set, (ii) the suitability of the variables in the data set to the analysis, (iii) meeting the multivariate normality assumptions of the data, and (iv) the adequate sample size (Floyd & Widaman, 1995). In classical exploratory factor analysis, KMO is used to test the sample size's adequacy; Bartlett's test of sphericity is performed to decide whether the data matrix is the unit matrix and whether the correlation between the variables is sufficient. Accordingly, Kaiser-Mayer-Olkin (KMO) value should be greater than 0.80 (Kaiser, 1970), and the result of the Bartlett test is expected to be statistically significant (Hair, Anderson, Tattham, & Black, 1995; Tabachnick & Fidell, 2007). The study's KMO value was found to be .81, and the Bartlett test was found to be statistically significant (χ2 = 2558, df = 66, p <.05). Regarding the studies in the literature on classical exploratory factor analysis, the recommended sample size should be at least 300, and Eser 2878 the number of observations for each variable (item) should be between 5 and 10 (Comrey & Lee, 1992; Tabachnick & Fidell, 2007). The Mardia test was used to test for multivariate normality, and it was concluded that both kurtosis and skewness met multivariate normality. Since the data set used in the study is based on the data collected from 778 individuals, it can be said that the sample size is sufficient. Considering the KMO and Bartlett test results and the sample size, it can be said that important assumptions of the classical exploratory factor analysis are met. 3. Results This part of the research includes the findings and comments about the results of the performed analyzes. First, scree plots showing the number of factors obtained from real and simulated data using different factor extraction methods are shown in Figure 2. Figure 2. Scree plots of factor extraction methods Figure 2 shows that the number of factors obtained from different factor extraction methods is the same, and the measurement tool has four factors. Then, the models established to estimate the factor structures by the observed variables were compared. Table 1 shows the items to be removed, explained variance, RMSEA, TLI, and the pseudo-chi-square value for the classical exploratory factor analysis performed by using different factor extraction methods, where promax rotation method, and the factor loading cut-off value is specified as 0.4. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2879 Table 1. The Results of Classical Exploratory Factor Analysis Factoring Method Rotation Method Cut- Off Removed Item Explained variance (%) RMSEA TLI χ²/df P- value for χ²/df Model 1 Maximum likelihood Promax 0.4 None 50 0.0210 0.99 2 31.5/2 4 0.139 Model 2 Minimum residuals Promax 0.4 None 50 0.0220 0.99 1 32.4/2 4 0.116 Model 3 Principal axis factoring Promax 0.4 None 50 0.0220 0.99 1 32.4/2 4 0.115 Model 4 Ordinary least squares Promax 0.4 None 50 0.0220 0.99 1 32.4/2 4 0.116 Model 5 Weighted least squares Promax 0.4 None 50 0.0220 0.99 1 32.3/2 4 0.117 Model 6 Generalized least squares Promax 0.4 None 50 0.0220 0.99 1 32.3/2 4 0.117 Regarding Table 1, the explained variance, Root Mean Square Error (RMSEA), Tucker- Lewis Index (TLI), and pseudo-chi-square values computed by using the different factor extraction methods and rotation methods are the same. When Table 1, which includes the results of the models established within the scope of 6 different extraction methods, is examined, it is striking that the explained variances (50%), RMSEA values, Tucker Lewis indexes (TLI) and pseudo chi-square values are very close to each other. When Table 1 is re- examined, it is striking that there is no requirement for item deletion for the models established within the scope of different factor extraction methods. At the same time, when the goodness of fit indices in Table 1 are examined, it is striking that the RMSEA, TLI and χ²/df values indicate a good fit. In Figure 3, there are factor loading tables related to the results of frequentist exploratory factor analysis performed within the scope of different factor extraction methods. Eser 2880 Figure 3. Factor loading tables for different factor extraction methods When the factor loadings tables, which are the output of frequentist exploratory factor analysis performed within the scope of different factor extraction methods in Figure 3, are examined, it is striking that the factor loading values of the relevant item in each table are very close to each other. At the same time, it has been calculated that the variance values explained by the relevant factor in each table are very close to each other. When Figure 3 was examined again, it was calculated that the variances explained for the six different models were very close to each other. It is striking that item 4 has the lowest factor loading and item 11 has the highest factor load. After classical exploratory factor analysis, Bayesian exploratory factor analysis was performed. The BayesFM package used in the Bayesian exploratory factor analysis employs the MCMC sampling method, based on the Metropolis-Hastings algorithm; therefore, the Bayesian exploratory factor analysis carried out in the research is based on the Metropolis- Hastings algorithm. In this study, the bayesian model specification was made assuming non- informative and uncorrelated priors. Specifically, it was assumed a multivariate normal distribution centered at zero with diagonal variance and covariance matrix of 0.001 precision for factor loadings. The MCMC method was performd to 50000 iterations with a burn-in sample of 5000 and thinning interval equal to 100. It is noticeable that all 12 variables positively affect the factor score. In this study, The Heidelberger and Welch’s convergence diagnostic test was used to test the null hypothesis that the values-sampled by MCMC are from a stationary distribution, a requisite for good inference model. This test uses the "Cramer-von Mises Statistic". In this case, the test shows that for most of the variables, the null hypothesis is not rejected. The factor loading matrix of the Bayesian exploratory factor analysis is shown in Figure 4. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2881 Figure 4. Factor loading matrix The factor loading matrix of the Bayesian exploratory factor analysis in Figure 4 shows that item 11 is the item with the highest factor loading of 0.85, and item 4 is the one with the lowest factor loading. As a result of the calculation performed using factor loadings (Kim & Mueller, 1978), the explained variance was found to be approximately 50%. The measurement tool was found to have four factors for both frequentist and Bayesian factor analysis in the study. The difference between the methods arises from the number of items to be removed from the scale. It is striking that the variance values explained for the models developed within the scope of frequentist exploratory factor analysis and for the model developed within the scope of bayesian exploratory factor analysis are almost the same. At the same time, when Figure 3 and Figure 4 were evaluated together, it was concluded that the item with the least factor loading was item 4 and the item with the highest factor loading was item 11. 4. Discussion, Conclusion and Recommendations It is thought that exploratory factor analysis is related to item reduction through dimension reduction approach. The Buss and Perry Aggression Questionnaire-short form used in the study consists of 12 items, and the fact that the structure formed as a result of Bayesian exploratory factor analysis consists of 12 items shows that the Bayesian approach also yields valid results in terms of factor analysis, similar to the frequentist approach. Bayesian factor analysis method allowed to keep the measurement tool's structure, and there is no need to remove items from the measurement tool. In other words, within the scope of the research, the factor loading limit value of 0.4 in the original article was adhered to, and as a result, the structure formed within the scope of both frequency exploratory factor analysis and bayesian exploratory factor analysis did not require any item deletion and the explained variance values were very close to each other. structures were determined. In the exploratory factor analysis, the variables assigned to a factor are kept within the structure, whereas the variables assigned to more than one factor or not assigned to any factor are removed from the structure (Hinkin, 1998; Tabachnick & Fidell, 2007). The most popular factor loading cut-off, called "good" in the literature, is 0.40 (Hinkin, 1998). Several researchers suggested 0.30 (Costello & Osborne, 2005), 0.32 (Tabachnick & Fidell, 2001), and 0.45 (Tabachnick & Fidell, 2007) as well. Hair, Black, Babin & Anderson (2009) stated that factor loadings between 0.3 and 0.4 are acceptable and suggested that if it is desired to select practically significant items, the items having factor loading 0.5 or higher than 0.5 should be kept. In the light of this information, factor loading cut off value can be determined as 0.5 for exploratory factor analyzes in terms of both the selection of items of practical importance and Eser 2882 the minimum exposure to cross-loading problem within the scope of scale development and adaptation studies. Although the cut-off point for the factor load value was determined as 0.4 with the Bayesian factor analysis method carried out within the scope of the research, it was determined that the structure created within the scope of the original article would be preserved, in other words, there would be no need for item deletion when we took this value as 0.5. However, in case the factor loading cut off value is 0.4, it has been determined that item deletion should be performed for all frequentist exploratory factor analysis results performed within the scope of different factor extraction methods. Regarding the steps to be followed in exploratory factor analysis, it can be said that these procedures are complex. Even researchers with years of experience remain uncertain about some nuances and details regarding the structure discovered by the exploratory factor analysis (Osborne, 2015). Considering this situation, evidence-gathering processes for the structures formed in classical exploratory factor analysis, which researchers in the literature often prefer, can be enhanced by using Bayesian exploratory factor analysis. Different methods can be introduced to the researchers who started their postgraduate education to help them decide the method they should use for the relevant structure in factor analysis. What is important at this stage is to provide the strongest set of arguments for each step of the structure-building process (Brown, 2009c). Including a preliminary criterion or criteria in the report that can be proven to be convincing and useful will increase the results' validity. Factor analysis based on different rotation methods, testing whether the same structure is observed, can increase the study's validity and strengthen the results. In Bayesian exploratory factor analysis, one should be very careful about writing scripts (codes). The slightest misspelling made while defining the model and transferring the data to the program will cause the model not to work. Another point to be considered in the Bayesian exploratory factor analysis is setting the convergence value and burn-in period. The lack of a definite rule on this issue is a disadvantage for Bayesian models. The Bayesian approach provides researchers with advantages in terms of time and cost, especially by working with small samples and overcoming multiple calculations in complex models. But the absence of a definite rule for convergence value and burn-in period may require the researcher to do more trials to reach appropriate statistical values. When the literature is examined, it is observed that bayesian methods have gradually started to be used instead of estimation methods, which have been used for years, especially within the scope of latent variable modeling. For the frequentist structural equation model, point estimation, asymptotic confidence limits and test statistics are generally used using the highest likelihood (ML) method. In Bayesian approach, latent variables are estimated by MCMC with the help of full posterior distribution for parameters. Bayesian method uses MCMC algorithms to obtain posterior distributions. MCMC uses a set of conditional distributions to pull parameter values with random walk simulation. Thanks to these conditional distributions, it is tried to obtain a distribution that converges to the composite posterior distribution. The most important advantage of MCMC is that it does not need asymptotic normality in samples and gives reliable results in small samples. The flexible nature of Bayesian methods, allowing researchers to include their own experiences into posterior distributions, achieving lower errors thanks to MCMC methods, and working well in small samples make it a method that is increasingly used in many areas today. Considering the advantages of Bayesian methods, especially MCMC, it is recommended to use Bayesian exploratory factor analysis as a validation method for the results of basic analysis or frequentist factor analysis within the scope of scale development and scale adaptation studies. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2883 Although user-friendly statistics programs are available for Bayesian methods, applying these methods requires mastery in statistics and a very good understanding of the theoretical background. Within the scope of the study, analyzes were carried out using the answers obtained from 778 people who answered the Buss and Perry Aggression Questionnaire-short form. A similar study can be carried out comparatively using simulative data generated under different conditions. Eser 2884 References Allen, M. (2017). The Sage Encyclopedia Of Communication Research Methods. (Vols. 1-4). Thousand Oaks, CA: SAGE Publications. doi: 10.4135/9781483381411 Asparouhov, T. , & Muthén, B. (2010). Bayesian analysis of latent variable models using mplus. Technical Report. (Report No. 4). Retrieved from: http://www.statmodel.com/download/bayesadvantages18.pdf Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E. J., Berk, R., … others. (2018). Redefine statistical significance. Nature Human Behaviour, 2(1), 6. doi: 10.1038/s41562-017-0189-z Berger, J.O. (1985) Statistical Decision Theory and Bayesian Analysis. Springer-Verlag: New York. doi: 10.1007/978-1-4757-4286-2 Bernardo J. M., Smith., A.F.M. (1994). Bayesian Theory. Newyork: Wiley Box, G.E.P., & Tiao, G.C. (1992). Bayesian Inference In Statistical Analysis. New York: John Wiley & sons. Brown, J. D. (2009). Statistic Corner. Questions And Answers About Language Testing Statistics: Choosing The Right Number Of Components Or Factors In PCA And EFA. Shiken: JALT Testing & Evaluation SIG Newsletter, 13(3), 19-25. Bryant, F. B., & Smith, B. D. (2001). Refining the architecture of aggression: A measurement model of the buss-perry aggression questionnaire. Journal of Research in Personality, 35(2), 138-167. doi: 10.1006/jrpe.2000.2302 Buss, A. H., & Perry, M. (1992). The aggression questionnaire. Journal of Personality and Social Psychology, 63(3), 452. doi: 10.1037/0022-3514.63.3.452 Carlin, B.P., & Louis T.A. (2000). Bayes And Empirical Bayes Methods For Data Analysis. London: Chapman & Hall. Comrey, L.A. & Lee, H.B. (1992). A First Course In Factor Analysis (2nd ed.). NJ: Lawrence erlbaum associates. Congdon, P. (2003). Modelling spatially varying impacts of socioeconomic predictors on mortality outcomes. J Geograph Syst 5, 161–184. doi: 10.1007/s10109-003-0099-7 Congdon, P., (2010). Applied Bayesian hierarchical methods, Chapman&Hall/CRC, Taylor&FrancisGroup, Boca Raton. Costello, A. B., & Osborne, J. W. (2005). Best Practices In Exploratory Factor Analysis: Four Recommendations For Getting The Most From Your Analysis. Practical Assessment, Research, & Evaluation, 10, 1-9. Çevik, M. (2009). Doğrusal olmayan Bayesçi regresyon ve yüksek frekanslı ses sistemle- rinde bir uygulama. (Yüksek lisans tezi, Yıldız Teknik Üniversitesi, İstanbul). Erişim adresi: http://dspace.yildiz.edu.tr/xmlui/bitstream/handle/1/4206/0042381.pdf?sequence=1&is Allowed=y Ekici, O. (2005). Bayesçi regresyon ve Win- BUGS ile bir uygulama. (Yüksek lisans tezi, Istanbul Üniversitesi, Istanbul). Erişim Adresi: Http://Nek.Istanbul.Edu.Tr:4444/Ekos/Tez/40722.Pdf Ekici, O., (2009). İstatistikte Bayesçi ve klasik yaklaşımın farklılıkları. Balıkesir Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 12(21), 89-101. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2885 Erkan, G. (2019). Klasik ve Bayesci Yapısal Eşitlik Modellerinde Parametre Tahminlerinin Karşılaştırılması: Sıralı Kategorik Verilerle Bir Uygulama. (Yüksek lisans tezi. Hacettepe Üniversitesi, Ankara). Erişim adresi: http://www.openaccess.hacettepe.edu.tr:8080/xmlui/bitstream/handle/11655/6516/1023 7080.pdf?sequence=1&isAllowed=y Etz, A., & Vandekerckhove, J. (2016). A bayesian perspective on the reproducibility project: Psychology. PloS One, 11(2): e0149794. doi: 10.1371/journal.pone.0149794 Floyd, F. J. & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286-299. doi: 10.1037/1040-3590.7.3.286 Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How To Design And Evaluate Research In Education, (Eight Edition). New York: McGraw-Hill. Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B., (2004) Bayesian Data Analysis. London: Chapman & Hall. Gorsuch, R. (1983). Factor Analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Gorsuch, R. L. (2003). Factor analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook Of Psychology. Research Methods In Psychology, 2, 143–164. John Wiley & Sons Inc. G. Conti, S. Frühwirth-Schnatter & J.J. Heckman, R. Piatek (2014). Bayesian exploratory factor analysis. Journal of Econometrics,183(1), 31-57. doi: 10.1016/j. jeconom.2014.06.008. Hair Jr., J. F., Black, W. C., Babin, B. J. & Anderson, R. E. (2009). Multivariate Data Analysis ,(7th ed.). Upper Saddle River, NJ: Pearson Prentice-Hall. Hinkin, T. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21(5), 967-988. doi: 10.1177/014920639502100509 Hinkin, T. R. (1998). A Brief Tutorial On The Development Of Measures For Use In Survey Questionnaires. Organizational Research Methods, 1(1), 104-121. Kaiser, H. (1970). A Second Generation Little Jiffy, Psychometrika, 35(4), 401-415. doi: 10.1007/BF02291817 Karadağ, Ö. (2011). Bayesçi Hiyerarşik Modeller, (Hacettepe Üniversitesi, Fen Bilimleri Enstitüsü, Yüksek Lisans Tezi, Ankara) Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14(7), 293–300. doi: 10.1016/j.tics.2010.05.001 Kuzucu, Y., ve Sarıot-Ertürk, Ö. (2020). Psychometric properties of turkish version of aggression questionnaire short form in adolescents and adults. Journal of Measurement and Evaluation in Education and Psychology, 11(3), 243-265. doi: 10.21031/epod.683176 Lee, P. M. (2004) Bayesian Statistics: An Introduction. Newyork: Wiley. Link, W.A. & Barker, R.J. (2006). Model weights and the foundations of multimodel inference. Ecology, 87, 2626–2635. doi: 10.1890/0012-9658 Mair, P. (2018). Modern Psychometrics With R. Springer-Verlag, New York: Springer. McCarthy M. A. (2007). Bayesian Methods For Ecology. Cambridge University Press, Cambridge. Eser 2886 Mills, M., Bunt, G. G. & Brujin, J. (2006). Comparative research persistent problems and promising solutions. InternationalSociology,21(5), 619-631. doi :10.1177/0268580906067833 Osborne, Jason W. (2015) What is rotating in exploratory factor analysis. Practical Assessment, Research, and Evaluation, 20,2. doi: 10.7275/hb2g-m060 Ozechowski T. J. (2014). Empirical Bayes MCMC estimation for modeling treatment processes, mechanisms of change, and clinical outcomes in small samples. Journal Of Consulting And Clinical Psychology, 82(5), 854–867. doi: 10.1037/a0035889 Palmstierna, T., & Wistedt, B. (1987). Absence of acquired tolerance to neuroleptics in schizophrenic patients. Am J Psychiatry 144, 1084–1085. Piatek, R. (2019). BayesFM: Bayesian Inference for Factor Modeling. (version 0.1.3) [Computer software]. https://CRAN.R-project.org/package=BayesFM Raine, A., Dodge, K., Loeber, R., Gatzke-Kopp, L., Lynam, D., Reynolds, C., StouthamerLoeber, M. & Liu, J. (2006). The Reactive–Proactive Aggression Questionnaire: Differential Correlates Of Reactive And Proactive Aggression In Adolescent Boys. Aggressive Behavior, 32, 159-171. Reyna, C., Sanchez, A., Ivacevich, M. G. L., & Brussino, S. (2011). The buss-perry aggression questionnaire: Construct validity and gender invariance among argentinean adolescents. International Journal of Psychological Research, 4(2), 30-37. Doi: 10.21500/20112084.775 Savchuk, V.P., & Tsokos, C.P. (2011). Bayesian theory and methods and applications, Atlantis Press, Paris. doi: 10.2991/978-94 Schmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304-321. doi: 10.1177/0734282911406653 Szucs, D., & Ioannidis, J. P. (2016). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. Plos Biology, 15(3), e2000797. doi: 10.1371/journal.pbio.2000797 Tabachnick, B.G., & Fidell, L.S. (2001) Using Multivariate Statistics. (4th ed.), Boston: Allyn and Bacon. Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics (5th ed.). Boston: Allyn & Bacon/Pearson Education. Thurstone, L. L. (1934). The vectors of mind. Psychological Review, 41(1), 1–32. doi: 10.1037/h0075959 Thurstone, L. L. (1947). Multiple-Factor Analysis; A Development And Expansion Of The Vectors Of Mind. University of Chicago Press. Yuan, Y., & MacKinnon, D. P. (2009). Bayesian mediation analysis. Psychological Methods, 14, 301– 322. doi:10.1037/a0016972 Walker, L. J., Gustafson, P., & Frimer, J. A. (2007). The application of Bayesian analysis to issues in developmental research. International Journal of Behavioral Development, 31, 366– 373. doi:10.1177/0165025407077763 Winkler G. (2003). The Bayesian Paradigm. In: Image Analysis, Random Fields And Markov Chain Monte Carlo Methods. Applications of Mathematics (Stochastic Modelling and International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2871-2887. 2887 Applied Probability), 27. Springer, Berlin: Heidelberg. doi: 10.1007/978-3-642-55760- 6_2