Uzun, A.M. (2019). Psychometric properties of the perceived ICT literacy scale among Turkish university students. International Online Journal of Education and Teaching (IOJET), 6(3). 644-656 http://iojet.org/index.php/IOJET/article/view/561 Received: 31.10.2018 Received in revised form: 10.06.2019 Accepted: 30.06.2019 PSYCHOMETRIC PROPERTIES OF THE PERCEIVED ICT LITERACY SCALE AMONG TURKISH UNIVERSITY STUDENTS Research Article Ahmet Murat Uzun Afyon Kocatepe University auzun@aku.edu.tr Ahmet Murat Uzun received his PhD degree in Computer Education and Instructional Technology from Middle East Technical University, in Turkey. His research interests include instructional design, cognitive load theory, multimedia learning and ICT ethics. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. http://iojet.org/index.php/IOJET/article/view/561 mailto:auzun@aku.edu.tr Uzun 644 PSYCHOMETRIC PROPERTIES OF THE PERCEIVED ICT LITERACY SCALE AMONG TURKISH UNIVERSITY STUDENTS1 Ahmet Murat Uzun auzun@aku.edu.tr Abstract This study aims to assess the psychometric properties of the 17-item 3-factors perceived ICT literacy scale in a sample of Turkish undergraduate students. Previous research on measuring ICT literacy mostly focused on the technical competence dimension of the ICT use. The current scale not only considers the technical literacy but it also evaluates the information literacy. The scale consists of three factors, each of which refers to one dimension of the Information and Communication Technology (ICT). Data were collected from 284 undergraduate students educated at different departments. Confirmatory Factor Analysis (CFA) using maximum likelihood estimation procedure was applied to test the psychometric properties of the scale. Results of the CFA indicated that the scale showed acceptable levels of validity. Considering the reliability values, all the subscales and the scale as a whole showed adequate reliability values. The limitations of the scale were discussed and some implications were suggested. Keywords: ICT literacy, information literacy, validity, reliability 1. Introduction Today, Information and Communication Technology (ICT) has surrounded our life. In order to survive, one should become more literate in the 21st century, because abilities of researching and communicating information by means of technology are indispensable requirements. Besides, advancements in the ICT have been the triggering force for most of the developed and developing countries’ economic growth (Tadesse, Gillies, & Campbell, 2018) and many popular occupations necessitate ICT-oriented skills (Ellis, 2001, as cited in Katz, 2005). European Commission (2018) has regarded ICT literacy as one of the key competencies for lifelong learning. Considering the importance of the issue, countries have started various initiatives to benefit from ICT in schools (Gök & Yıldırım, 2015). Policy makers and researchers offered different frameworks and competencies to emphasize the importance of ICT literacy (Zhang & Zhu, 2016). Using ICT to find and process information is a significant obstacle that today’s scholarly world, labor force, and society encounter (Katz, 2005). This may be because of having insufficient ICT skills. Although the significance of ICT literacy is well known, it has been stated that there is a serious gap between the levels of ICT skills and knowledge that students possess and the levels required by today’s 21st century. Besides, there is a false belief that ICT literacy refers to performing technical skills such as using technologies and the internet effectively, leading to underestimation of the information literacy. This has prompted researchers and policy makers to propose a more holistic ICT literacy approach, which considers not only technical dimensions of ICT use such as computer and internet literacy, but also higher order skills such as information literacy (Katz, 2005; Lau & Yuen, 2014). Furthermore, measuring ICT literacy is as important as defining and conceptualizing it. The 1 A part of this study was presented at the II. International Congress on Science and Education, 28-30 September, 2018, Afyonkarahisar, Turkey. mailto:auzun@aku.edu.tr International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 645 literature on ICT literacy did not provide sufficient measures to evaluate students’ ICT literacy. Besides, in most of the scales, sub dimensions of the ICT literacy were not well discriminated, supposing that ICT literacy is unidimensional. Given this rationale, the current study aims to evaluate psychometric properties of the perceived ICT literacy scale (Lau & Yuen, 2014) among Turkish university students. 2. Defining and Measuring ICT Literacy In order to define ICT literacy, first, one should define the term “literacy”. However, there is not a consensus on the definition of the literacy in the literature (Bawden, 2001). Literacy has been defined as the ability to read and write by Turkish Language Association. However, as referred by Dinçer (2017), today, the concept has a broader meaning. “Literacy can be defined as having the skills one needs to make the connection to the information necessary to survive in society” (Olsen & Coons, 1989, as cited in Bawden, 2001, p. 222). Considering information literacy, American Library Association suggested the following definition: “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information” (ALA, 1989, as cited in Katz, 2005). Considering ICT literacy, several concepts were used interchangeably in the literature. To name a few, these include digital competence, ICT competence, internet literacy, media literacy, digital literacy, computer literacy, and ICT literacy (Dinçer, 2017; Zhang & Zhu, 2016). Young generation of today devotes considerable time to use their laptops, tablets, smartphones, PCs, and etc. However, this doesn’t necessarily mean that they are literate on using ICT (Siddiq, 2016). Therefore, previous conceptualizations of ICT literacy have been criticized by the researchers in that they were used to refer only to the technical competence of ICT use neglecting higher order skills such as problem solving and critical thinking. However, effective and efficient use of ICT necessitate not only technical competence but also information literacy skills. That is, ICT usage ability should be combined with information literacy skills that include researching and communicating information (Lau & Yuen, 2014; Rodríguez-de-Dios, Igartua, & González-Vázquez, 2016), because having expert in technical operation of ICT does not mean using ICT effectively (Katz, 2005). The concept of technology should not be delimitated to the merely usage of hardware and software but it should be considered along with social, ethical and intellection issues related to it (Rodríguez-de-Dios et al., 2016). In line with this rationale, in an international panel, Educational Testing Service (ETS) defined ICT literacy as “using digital technology, communications tools, and/or networks to access, manage, integrate, evaluate, and create information in order to function in a knowledge society” (ICT Literacy Panel, 2007, p. 2). It was also defined by Claro et al. (2012) as the capacity to solve problems of information, communication and knowledge in digital environments. Various frameworks were suggested by the researchers, institutions and policy makers to determine the indicators of ICT literacy. According to Educational Testing Service (ETS), ICT proficiency includes the following abilities: access, manage, integrate, evaluate, and create information (ICT Literacy Panel, 2007). According to Australian Council for Educational Research (ACER), components of ICT literacy were determined as accessing, managing, and evaluating information, developing new understandings, communicating with others and using ICT appropriately (Meiers, Knight, & White, 2009). According to Organization for Economic Co-Operation and Development (OECD), ICT literacy includes abilities of accessing, managing, integrating, evaluating, and creating information (Dinçer, 2017). According to European Commission (2018), digital competence framework includes “information and data literacy, including management of content, communication and collaboration, and participation in society, digital content creation, including ethical principles, safety and problem solving” (p. 51). Uzun 646 Although conceptualizations on ICT literacy have been widely discussed by the literature, little research was devoted to the development of ICT literacy scale (Lau & Yuen, 2014; Tadesse et al., 2018). Besides, most of the measures were limited in that they mostly regarded ICT literacy as capabilities of using computer hardware and software. However, as the definition of ICT is broader in today’s conditions, former scales are regarded to be limited to measure all aspects of the ICT literacy (Rodríguez-de-Dios et al., 2016). Different scales were used to measure ICT literacy of the various groups of students. For example, studying with pre-service science teachers, Üstündağ, Güneş and Bahçivan (2017) adapted digital literacy scale (Ng, 2012) into Turkish culture. It was found that the 10 items loaded into one factor explaining 40% of the total variance. Rodríguez-de-Dios et al. (2016) conducted a study on the development and validation of a digital literacy scale for teenagers. Accounting for 44.3% of the total variance, exploratory factor analysis showed the existence of six factors: technological skill, personal security skill, critical skill, devices security skill, informational skill, and communication skill. Goldhammer, Naumann and Keßel (2013) developed a basic computer skills (BCS) scale based on the data of German PISA 2009 field trial. BCS is regarded as the basic computer use scale of the broader concept of the ICT literacy. Results of the Confirmatory Factor Analysis (CFA) indicated that the BCS scale included one dimension labeled as BCS speed and ability. Richter, Naumann and Horz (2010) used Practical Computer Knowledge (PRACOWI) to measure ability of solving computer related problems, which is a subscale of computer literacy inventory. The subscale was found to have a good internal reliability value and was one-dimensional (cited in Greiff, Kretzschmar, Müller, Spinath, & Martin, 2014). Lau and Yuen (2014) developed a three-factor 17 items perceived ICT literacy scale for the secondary school students. The scale consisted of three factors including all dimensions of Information and Communication Technologies: Information (Information), the Internet (Communication) and Computer (Technology) literacy. Results showed that the three subscales explained 65.83% of total variance and it had a good reliability value (Cronbach’s alpha = .92). Nasser AL-Nuaimi, Bouazza, Abu-Hilal, and Al-Aufi (2017) developed an information-ethics questionnaire for undergraduate students. One of the sub dimensions of the scale was perceived ICT literacy self-efficacy. The researchers barrowed items from the perceived ICT literacy scale developed by Lau and Yuen (2014) and from information literacy survey developed by Pinto (2010). Different from Lau and Yuen (2014), Nasser AL-Nuaimi et al. (2017) tested the psychometric properties of the developed scale with undergraduate students rather than secondary school students. Exploratory factor analysis showed that the scale explained 58.93 % of the total variance. In addition, the results of the CFA indicated satisfactory results of model fit indices. To sum up, although the conceptualization and definition of ICT literacy have been widely discussed by the literature, scales on measuring ICT literacy including all dimensions of ICT remain insufficient (Lau & Yuen, 2014). In addition, most of the studies on developing ICT literacy scale and measuring it consider ICT literacy as a homogenous unidimensional construct, in which cognitive and technical proficiency are combined (Tadesse et al., 2018). On the other hand, few studies addressed the issue of taking ICT literacy as a construct that involves sub-correlated dimensions such as information, internet and computer literacy (Lau & Yuen, 2014; Nasser AL-Nuaimi et al., 2017). Developing an ICT literacy scale is important in that such a measure could be used to determine students’ prerequisite knowledge regarding the media literacy skills, when implementing media literacy education (Arke & Primack 2009; Buckingham 2009). Besides, as referred by Zhang and Zhu (2016), to put into action of ICT literacy education effectively, one should determine these skills as proxies of the prerequisites. Considering the lack of International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 647 research in Turkish culture about the issue, it could be argued that there is a need for such a scale to use it in Turkish culture. To this end, the purpose of this study is to assess the psychometric properties of the perceived ICT literacy scale in a sample of Turkish undergraduate students. Accordingly, the following research question was posed: Does the Turkish version of the perceived ICT literacy scale have satisfactory levels of validity and reliability. 3. Method 3.1. Item Selection and Scale Translation The current study barrowed items from a three-factor 17-item perceived ICT literacy scale (3F-PICTLS) developed by Lau and Yuen (2014). The scale was originally developed for the secondary school students. However, during the item generation, the researchers applied for various frameworks (e.g., Educational Testing Service, 2003), which were originally suggested for the context of higher education. In addition, Nasser AL-Nuaimi et al. (2017) tested the adapted version of the scale (ICT self-efficacy subscale) in a sample of undergraduate students and showed that it had appropriate levels of validity and reliability values. Considering the lack of such a scale for Turkish undergraduate population, the target group for the current study was also undergraduate students. Therefore, referring to Nasser AL-Nuaimi et al. (2017), some of the items barrowed from Lau and Yuen (2014) were modified in a way that suits the levels of undergraduate students. The 3F-PICTLS scale consists of three factors, each of which refer to one dimension of the ICT). These factors are information literacy, internet literacy, and computer literacy, which denote to Information (I), Communication (C) and Technology (T) respectively. Information Literacy factor includes seven items that are about recognizing the needed information and abilities such as locating, evaluating, and using the needed information (ICT literacy Panel, 2007). The following, one item of Information Literacy is given as example: “I am able to interpret and represent information, such as using ICT to synthesize, summarize, compare and contrast information from multiple sources”. Internet Literacy factor included five items, which are about technical competencies needed when using internet. A sample item related to the Internet Literacy factor could be given as follows: “I am able to use email to communicate.” Finally, Computer Literacy factor is related to the offline computer usage abilities. The following, a sample item of the Computer Literacy scale was given: “I am able to plot a graph and chart using spreadsheet software”. All items are scored on a 5-point Likert type scale. A one - point (1) refers to strongly disagree and a five - point (5) refers to strongly agree. In the process of scale translation, the following procedures were followed: Items were translated by four scholars separately, two of whom are expert in educational sciences and two are in educational technology. All translators have the PhD degree in their field and they speak English well. Then, the four translated versions of the scale were carefully reviewed and compared with each other by the researcher of the current study. Experts’ opinions regarding the items were also considered in this phase and the necessary modifications were applied to the items based on the suggestions. Finally, back translation was conducted by an expert working in the English language teaching department. This form of the scale was compared with the original form and it was observed that the two scales were similar. Finally, the Turkish form of the scale was sent to a Turkish language expert to put it in final form based on the appropriateness of the items in terms of the grammar, wording, clarity, and spelling. Uzun 648 3.2. Participants and Data Collection A non-random convenient sampling procedure was used to collect data from the participants. Convenient sampling is used by the researchers, when participants of the study were determined based on their availability and accessibility (Creswell, 2012). The participants were undergraduate students studying different majors at different public universities in Turkey. In order to conduct factor analysis, different criteria for a minimum of required sample size was suggested by different researchers. According to Guilford (1954), the sample size should be more than 200. According to Hair, Black, Tatham and Anderson (2010), at least five participants for each parameter are needed but 10 participants are preferred. Data were collected through both online and offline methods during summer school of 2017- 2018 academic year. Online data were collected via Google form. Offline data were collected face to face by using a paper-based format. In the current study, initially, there were 295 respondents. Of all participants, 55% (N =164) of them are female and 45 % (N = 131) are male. Their mean age is 22 (SD = 3.56). As the data had missing values and univariate or multivariate outliers, 11 of all cases (5 males, 6 females) were excluded from the analysis yielding a total number of 284 respondents. This sample size was deemed sufficient according to the criteria suggested by the researchers. 3.3. Data Analysis IBM AMOS version 24.0 was used to analyze the data. Confirmatory Factor Analysis (CFA) using maximum likelihood estimation procedure was applied to test the psychometric properties of the scale and the strength of the factor solution suggested by the original study. Also known as the measurement model, CFA is used to test the relationships between the observed variables and the latent structures considered to be measured by these observed variables (Kline, 2015; Weston & Gore, 2006). Before conducting the analysis, outlier and missing case analysis were conducted and problematic cases were removed. Outliers were detected by univariate boxplots and Mahalanobis distance. After that, assumptions of CFA such as univariate normality and multivariate normality were tested. According to Kline (2015), distributions with skewness values greater than the absolute value of 3 and kurtosis values greater than the absolute value of 10 should not be considered as normal. In the current study, it was found that all the items distributed normally based on this suggestion. To test multivariate normality, Mardia’s coefficient was evaluated. According to Raykov and Marcoulides (2008), the suggested value for p (p + 2) = 17(19) = 323, where the p is the number of the items (as cited in Lau & Yuen, 2014). In the current study, the coefficient was calculated as 139.881, which was below the suggested value. Therefore, multivariate normality assumption was met. 4. Results 4.1. Validity Results To test the validity, various model fit indices could be used. However, there is not a consensus in the literature regarding which indices should be reported (İlhan & Çetin, 2014). Kline (2015) indicates that “a minimum set of fit statistics that should be reported whenever it is possible to do so” is “model chi-square with its degrees of freedom and p value, Root Mean Square Error of Approximation (RMSEA), Bentler Comparative Fit Index (CFI) and Standardized Root Mean Square Residual (SRMR)” (p. 269). In the current study, χ2/df (Chi- Square/Degree of Freedom), Comparative Fit Index (CFI), Standardized Root Mean Square Residual (SRMR), Root Mean Square Error of Approximation (RMSEA) and NNFI (TLI) were reported. International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 649 Comparative Fit Index (CFI) supposes that latent constructs are uncorrelated. This null model is compared with the sample covariance matrix (Hooper, Coughlan, & Mullen, 2008). Values close to one indicates a good fit for the model. Standardized Root Mean Square Residual (SRMR) is the standardized form of Root Mean Square Residual (RMR), which refers to mean of absolute covariance residual. Values close to zero indicates a good fit (Kline, 2015). Like SRMR, Root Mean Square Error of Approximation (RMSEA) is a badness of fit index. Values of RMSEA should be close to the zero, as values close to the zero shows minimum error between observed and reproduced matrices (Meydan & Şeşen, 2011). Normed-fit index (NFI) evaluates the strength of the model by comparing χ2 value of the current model to the χ2 value of the null model. Values of NFI range between zero and one, with values close to one indicate a good model fit. The disadvantage of this index is that for small sample sizes, (N < 200), it underestimates the model fit. However, the introduction of Non-Normed Fit Index (NNFI), which is also known as Tucker-Lewis Index (TLI), solved this problem (Hooper et al., 2008). The suggested model fit indices for perfect and acceptable model fit were summarized by İlhan and Çetin (2014) based on the various references. It was given in the Table 1. Table 1. Suggested values for perfect and acceptable model fit Model fit indices Perfect fit Acceptable fit χ 2/sd 0 ≤ χ 2/sd ≤ 2 2 ≤ χ 2/sd ≤ 3 CFI .95 ≤ CFI ≤ 1.00 .90 ≤ CFI ≤ .95 SRMR .00 ≤ SRMR ≤ .05 .05 ≤ SRMR ≤ .10 RMSEA .00 ≤ RMSEA ≤ .05 .05 ≤ RMSEA ≤ .08 NNFI (TLI) .95 ≤ NNFI (TLI) ≤ 1.00 .90 ≤ NNFI (TLI) ≤ .95 In the current study, the following model fit values were found. χ2/sd = 2.42, CFI = .94, SRMR = .053, RMSEA = .071, NNFI (TLI) = .92. All the values could be interpreted as having acceptable values (see Table 1). In other words, the model fits data well. Factorial structure of the tested theoretical model is provided in the Figure 1. Unstandardized and standardized regression weights and the squared multiple correlations (R2) values are provided in Table 2. All standardized regression weights (i.e., factor loadings) are significant at the .001 alpha level. The squared multiple correlations (R2) refer to the amount of variance of the observed variables (items) explained by the latent variable (factor). Correlations between factors are found as .82 between Information Literacy (INFL) and Internet Literacy (INTL), .79 between Computer Literacy (COMPL) and INTL and .70 between COMPL and INFL. Uzun 650 Figure 1. Factorial structure of the Turkish version of 3F-PICTLS. Note. INFL = Information Literacy, INTL = Internet Literacy, COMPL = Computer Literacy. International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 651 Table 2. Regression weights and the squared multiple correlations Factors Items Unstandardized estimates Standardized estimates R2 INFL INFL1 1 .618 .382 INFL2 1.131 .775 .601 INFL3 1.326 .849 .721 INFL4 1.331 .810 .656 INFL5 1.377 .792 .627 INFL6 1.179 .663 .440 INFL7 1.153 .679 .461 INTL INTL1 1 .678 .460 INTL2 .735 .688 .473 INTL3 .694 .693 .480 INTL4 .327 .612 .375 INTL5 .687 .719 .517 COMPL COMPL1 1 .767 .588 COMPL2 1.108 .731 .534 COMPL3 1.155 .810 .656 COMPL4 1.120 .761 .579 COMPL5 1.102 .623 .388 Note. INFL = Information Literacy, INTL = Internet Literacy, COMPL = Computer Literacy. 4.2. Reliability Results In order test whether the scale and its sub factors have sufficient reliability values, Cronbach’s Alpha values, which refers to internal consistency were calculated. According to Nunnally (1978), values above .70 are regarded to be a sufficient reliability value. According to the results, INFL has a reliability value of .90. Deleting none of the items from the scale increased the reliability value. INTL has a reliability value of .78 and deleting items did not increase the reliability value of the scale; therefore, items of the INTL were decided to be retained. Finally, the reliability value of COMPL was found to be .86 and none of the items seemed to be problematic based on their item deletion values. In addition to sub dimensions, the Cronbach’s alpha reliability value of the ICT literacy as a whole was assessed, which was found to be as .92. From these results, it could be argued that the scale has sufficient Cronbach’s alpha reliability values. 5. Discussion and Conclusion The purpose of this study was to evaluate psychometric properties of the perceived ICT literacy scale (3F-PICTLS) in a sample of the Turkish undergraduate students (N = 284). To this end, the scale, originally developed by Lau and Yuen (2014), was adapted into Turkish language and then validity and reliability issues were addressed. To test validity, CFA was applied to the data to confirm the factor structure proposed by the original study. As indices of Uzun 652 the model fit, values of χ2/sd, CFI, SRMR, RMSEA, NNFI (TLI) were used. Results of the CFA indicated that the scale showed adequate levels of validity indices values indicating that the measurement model shows a good fit. Factor loadings were all significant, meaning that all the standardized regression weights for the latent variables in the prediction of the observed variables were significant. Therefore, the theoretical measurement model suggested by the original study was confirmed by the data collected from Turkish undergraduate students. Considering the reliability, all the subscales and the scale as a whole showed adequate levels of Cronbach’s alpha values. In addition, deleting none of the items increased levels of Cronbach’s alpha values. In sum, the 3F-PICTLS scale was proven to have adequate levels of validity and reliability values in the sample of Turkish undergraduate students. This is in good agreement with the original scale developed by Lau and Yuen (2014). Results are also consistent with Nasser AL- Nuaimi et al. (2017)’s study that found acceptable validity and reliability results. Besides, as referred by other studies in the literature, ICT measurement tools should evaluate not only technical skills but also information literacy skills including problem solving and critical thinking, which are needed to function in the societies of the 21st century (Katz, 2005; Lau & Yuen, 2014; Rodríguez-de-Dios et al., 2016; Siddiq, 2016). Using internet and computer effectively necessitate information literacy skills, as having knowledge about technical skills does not imply being literate in ICT. For example, as referred by Lau & Yuen (2014), the item of “I am able to search for information on the internet using the advanced search options made available by search engines such as Google, Yahoo etc.” is significantly related to the item of “I can gather and retrieve information in digital environments” (Lau & Yuen, 2014; Nasser AL-Nuaimi et al., 2017), showing the relationship between cognitive and technical dimensions of the ICT literacy. Therefore, the effect of students’ information literacy skills on the effective and efficient use of ICT resources should be addressed in further measures dealing with the ICT literacy. Considering the lack of research in measuring ICT literacy from this perspective, it could be argued that the results of the current study would be an important contribution in fulfilling such a need in the current research base in a multifaceted manner. Another issue was the practicability of the scale. As referred by Üstündağ et al. (2017), in addition to validity and reliability, practicability of a scale should also be a concern for the researchers, as the scales with too many items may not be feasible for students to when responding to the items (Fraenkel, Wallen, & Hyun, 2012). As 3F-PICTLS consists of 17 items, one may argue that the scale seems feasible to collect data. The Turkish version of the 3F-PICTLS scale can be used in measuring Turkish undergraduate students’ ICT literacy and the results would be beneficial to guide or design in their ICT education. Turkish researchers and instructors would take beneficiary in using this Turkish translated scale. In technology integration or ICT literacy education studies, the scale could be beneficial to understand students’ entry levels of ICT usage skills as proxies for determining their prerequisites (Arke and Primack 2009; Buckingham 2009; Zhang & Zhu, 2016). Furthermore, the scale may be useful for practitioners, who are working in the field of value education to determine the relationship between ICT literacy and ethical use of ICT. The study undoubtedly has some limitations. First, the 3F-PICTLS scale did not include items addressing ethical issues of accessing and using information with ICT, although various international associations such as Association for Educational Communications Technology (AECT) (Molenda & Januszewski, 2008) and European Commission (2018) included ethics dimension in their conceptualizations of the ICT use. Second, the current 3F-PICTLS scale did not include items about the secure use of the ICT, which could also be addressed by the future studies (Rodríguez-de-Dios et al., 2016). Last, the scale did not include items regarding the International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 653 complex technical skills such as troubleshooting techniques, when faced with a problem during ICT use. Future studies may address this issue. Uzun 654 References Arke, E. T., & Primack, B. A. (2009). Quantifying media literacy: Development, reliability, and validity of a new measure. Educational Media International, 46(1), 53–65. Buckingham, D. (2009). The future of media literacy in the digital age: Some challenges for policy and practice. Medienimpulse, 2, 69–82. Bawden, D. (2001). Information and digital literacies: A review of concepts. Journal of Documentation, 57(2), 218–259. Claro, M., Preiss, D. D., San Martín, E., Jara, I., Hinostroza, J. E., Valenzuela, S., … Nussbaum, M. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers and Education, 59(3), 1042–1053. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Upper Saddle River, NJ: Pearson Education. Dinçer, S. (2017). Ortaokul öğrencilerinin bilgisayar okuryazarlık düzeylerinin belirlenmesi ve ölçme – Değerlendirme araçlarının yapısı [Assessing computer literacy of secondary school students and structure of assessment - Evaluation instruments]. Elementary Education Online, 16(3), 1329–1342. Educational Testing Service. (2003). Succeeding in the 21st century: What higher education must do to address the gap in information and communication technology. Princeton, NJ: National Higher Education ICT Initiative. European Commission (2018). Council recommendation on key competences for lifelong learning. Retrieved June 30, 2019, from http://data.consilium.europa.eu/doc/document/ST-5464-2018-ADD-2/EN/pdf Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education. New York, USA: McGraw-Hill Education. Goldhammer, F., Naumann, J., & Keßel, Y. (2013). Assessing individual differences in basic computer skills. European Journal of Psychological Assessment, 29(4), 263-275. Gök, A., & Yıldırım, Z. (2015). Investigation of FATİH project within the scope of teachers, school administrators and YEGITEK administrators’ opinions: A multiple case study. Mersin University Journal of the Faculty of Education, 11(2), 6–8. Greiff, S., Kretzschmar, A., Müller, J. C., Spinath, B., & Martin, R. (2014). The computer- based assessment of complex problem solving and how it is influenced by students’ information and communication technology literacy. Journal of Educational Psychology, 106(3), 666-680. Guilford, J. P. (1954). Psychometric methods (2th ed.). New York: McGraw-Hill. Hair, J. F., Black, W.C., Tatham, R. L., & Anderson, R. E. (2010). Multivariate data analysis. Upper Saddle River, NJ: Prentice Hall. Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60. ICT Literacy Panel. (2007). Digital transformation: A framework for ICT literacy. Princeton, NJ: Educational Testing Service. Retrieved September 8, 2018, from http://data.consilium.europa.eu/doc/document/ST-5464-2018-ADD-2/EN/pdf International Online Journal of Education and Teaching (IOJET) 2019, 6(3), 644-656 655 http://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/32_digitaltransformation .pdf İlhan, M., & Çetin, B. (2014). Comparing the analysis results of the structural equation models (SEM) conducted using LISREL and AMOS. Journal of Measurement and Evaluation in Education and Psychology, 5(2), 26–42. Katz, I. (2005). Beyond technical competence: Literacy in information and communication technology. Educational Technology, 45(6), 44-47. Kline, R. B. (2015). Principles and practice of structural equation modeling. Guilford publications. Lau, W. W. F., & Yuen, A. H. K. (2014). Developing and validating of a perceived ICT literacy scale for junior secondary school students: Pedagogical and educational contributions. Computers and Education, 78, 1–9. Meiers, M., Knight, P., & White, G. (2009). The Digest edition 2009/1: The use of ICTs in schools in the digital age: what does the research say? Retrieved September 10, 2018, from https://research.acer.edu.au/digest/6/ Meydan, H. C. & Şeşen, H. (2011). Yapısal eşitlik modellemesi AMOS uygulamaları[Structural equation modeling AMOS applications]. Ankara: Detay Yayıncılık. Molenda, M., & Januszewski, A. (2008). Educational technology: A definition with commentary. New York: Routledge. Nasser AL-Nuaimi, M., Bouazza, A., Abu-Hilal, M. M., & Al-Aufi, A. (2017). The psychometric properties of an information-ethics questionnaire. Performance Measurement and Metrics, 18(3), 166–179. Ng, W. (2012). Can we teach digital natives digital literacy? Computers and Education, 59(3), 1065–1078. Nunnally, J. C. (1978). Psychometric Theory. New York: McGraw-Hill. Pinto, M. (2010). Design of the IL-HUMASS survey on information literacy in higher education: A self-assessment approach. Journal of Information Science, 36(1), 86-103. Rodríguez-de-Dios, I., Igartua, J.-J., & González-Vázquez, A. (2016). Development and validation of a digital literacy scale for teenagers. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality - TEEM ’16, 1067–1072. Siddiq, F. (2016). Assessment of ICT Literacy. A comprehensive inquiry of the educational readiness for the digital era (Doctoral dissertation). Retrieved July 1, 2019, from https://www.duo.uio.no/bitstream/handle/10852/53359/PhD-Siddiq- DUO.pdf?sequence=1&isAllowed=y. Tadesse, T., Gillies, R. M., & Campbell, C. (2018). Assessing the dimensionality and educational impacts of integrated ICT literacy in the higher education context. Australasian Journal of Educational Technology, 34(1), 88–101. Üstündağ, M. T., Güneş, E., & Bahçivan, E. (2017). Turkish adaptation of digital literacy scale and investigating pre- service science teachers ’ digital literacy. Journal of Education and Future, (12), 19–29. http://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/32_digitaltransformation.pdf http://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/32_digitaltransformation.pdf https://research.acer.edu.au/digest/6/ https://www.duo.uio.no/bitstream/handle/10852/53359/PhD-Siddiq-DUO.pdf?sequence=1&isAllowed=y https://www.duo.uio.no/bitstream/handle/10852/53359/PhD-Siddiq-DUO.pdf?sequence=1&isAllowed=y Uzun 656 Weston, R., & Gore Jr, P. A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34(5), 719-751. Zhang, H., & Zhu, C. (2016). A study of digital media literacy of the 5th and 6th grade primary students in Beijing. Asia-Pacific Education Researcher, 25(4), 579–592.