Microsoft Word - [8]242-2789-4-CE2_new2.doc Australasian Journal of Educational Technology, 2014, 30(1). ascilite 106 Students' self-regulated learning, online information evaluative standards and online academic searching strategies Sheng-Chau Tseng, Jyh-Chong Liang, and Chin-Chung Tsai National Taiwan University of Science and Technology, Taiwan Online information searching strategies (OISS) used by students can be viewed as a key indicator in online learning environments. Therefore, developments in their OISS may also involve variables such as self-regulated learning (SRL) and online information evaluative standards (OIES). Three instruments, an OISS, a SRL and an OIES were used to investigate the roles of SRL (including basic and advanced SRL) and OIES (including naive and advanced standards) in the OISS of 307 high school students in Taiwan. The results of a structural equation model confirmed that both students' SRL and OIES have positive predictions on their OISS. In addition, students' advanced OIES mediated the relationships between basic SRL and OISS. However, when the students took advantage of the advanced SRL in employing OISS, the significance of the OIES was not shown. This study contributes to the understanding of students' online academic information searching strategies. Introduction Since the advent of web technology, there has been an explosion of teaching and learning via the Internet. Extending information and communication technology (ICT) skills has become one of the educational goals generally set by many secondary schools to better meet the needs of the digital age (Kuhlemeier & Hemker, 2007). More and more high school teachers are encouraged to use Internet resources and integrate Internet activities into their curricula. High school students are expected to learn computer skills for doing their schoolwork or completing assignments (Kuhlemeier & Hemker, 2007). It comes naturally for many teachers to assume that secondary students are acquainted with Internet skills, such as surfing the web or locating online information. However, not every high school student has developed sufficient Internet skills or critical ability needed for schools. In fact, a survey from 2003 to 2008 by Kimber and Wyatt-Smith (2010) have pointed out that high school students' capabilities in curricular-related online searching tasks are still insufficient, despite the prevalence of the new technology. As secondary students are frequent users of the Internet by now, an understanding of their thoughts about online resources is very important. While completing learning tasks, students are increasingly involved in searching for information on the Internet (Tsai, 2009a). There has been a growing amount of studies concerning information seeking or searching practice for learning (Hofer, 2004; Tsai, 2009b; Tsai, Hsu, & Tsai, 2011; Zhou & Xu, 2007). It is generally held that different searching strategies used by learners lead to different searching results, and thus different performance (Wu & Tsai, 2007). As searching and processing information is a complex cognitive process which requires learners to verify, evaluate, organize and integrate information from a variety of sources (Mason & Boldrin, 2008; Walraven, Brand-gruwel, & Boshuizen, 2008), it has become essential for researchers to study variables related to learners' online searching behaviours in web-based learning environments. Self-regulated learning (SRL) and online information evaluative standards (OIES) are found to be influential in online learning in related literature (Winters, Greene, & Costich, 2008; Wu & Tsai, 2005). Therefore, this study attempts to explore the roles of SRL and OIES in high school student online information searching strategies (OISS). Related Literature Self-regulated learning (SRL) SRL refers to students' ability to control themselves while trying to achieve specific goals during the learning process (Zimmerman, 2000), especially how to plan, monitor and evaluate their own learning (Pintrich, 2004). Cheng, Liang, and Tsai (2013) therefore proposed to categorize SRL into basic (i.e., the Australasian Journal of Educational Technology, 2014, 30(1). 107 planning phase) and advanced levels (i.e., the monitoring, controlling and reflection phases). Many studies have shown that SRL skills are beneficial to individuals' learning. Most of them also have highlighted the relationship between self-regulation and academic performances (Duncan et al., 2007; McClelland, Morrison, & Holmes, 2000; Winters et al., 2008). It is proposed that SRL is the key factor that separates successful learners from less successful learners (Willoughby, Anderson, Wood, Mueller, & Ross, 2009; Zimmerman, 1989). That is, students who self-regulate their learning well are likely to perform better than those who do not (Schunk & Zimmerman, 2006). Hypermedia learning environments are prevailing and have many advantages for fostering learning. Nowadays when students are looking for information, they often go directly to the Internet. However, in web-based environments, students are shown to have difficulties when learning (Winters et al., 2008) and thus may need more personal skills to integrate the online information which is characteristic of multiple representations in non-linear formats (Zimmermen, 2008). Recently, with the advent of ICT integration in the classroom context, SRL has particularly attracted researchers' attention (e.g., Lee & Tsai, 2011; Tsai, 2009b). A review of various studies has indicated that SRL is associated with academic success in computer based learning environments (Winters et al., 2008). For example, the studies by Azevedo and colleagues (Azevedo, 2005; Azevedo, Guthrie, & Seibert, 2004;) have shown that students with more metacognitive SRL tended to be more successful in hypermedia learning tasks. It therefore has been concluded that SRL is an essential element for successful online learning (Tsai, 2009a) as it helps students organize and integrate information into viable mental models when learning online (Azevedo & Cromley, 2004). To sum up, SRL activities have been directed at learning performance in both traditional learning settings and web-based learning settings in past literature (Boekaerts, Pintrich & Zeidner, 2000; Hsu & Shiue, 2005; Vermunt & Vermetten, 2004). Based on the literature, the current study will focus on SRL in online academic searching activities at the high school level. Online information evaluative standards (OIES) OIES in this study refers to students' evaluative standards when verifying online information. To obtain information, students often use web search engines, among other methods. Through such online searching activities, they repeatedly make judgments about how useful information is to their particular needs and form judgments about the relevance of the information to the learning task based on their criteria. That is, they need to evaluate the usefulness as well as the accuracy of the online materials obtained before analysing and selecting them for use (Wu & Tsai, 2005). According to Wu and Tsai's (2005) perspective, students' evaluative criteria for web materials involves advanced standards and naïve standards. As the Internet has become the primary vehicle for various information-sharing tasks (Dimmick, Chen, & Li, 2004; Gray, Klein, Noyce, Sesselberg, & Cantrill, 2005), there have been many studies concerning the uncertain nature of online information and users' ability to evaluate it (Flanagin & Metzger, 2007; Metzger, Flanagin, & Zwarun, 2003). For example, when investigating college students' use of web information, Metzger et al. (2003) found that students relied heavily on the web for both general and academic information. They also found that college students relied more on online information and verified less than adults. Rieh and Hilligoss (2008) revealed that college students often compromised information credibility for speed and convenience. Wu and Tsai (2005) further found a causal relationship between university students' information evaluative standards and their searching strategies. Simply stated, how learners evaluate web-based information and how their searching strategies are related to these evaluative standards have become essential issues for educators. As searching for information on the web has become frequent part of learning activities taking place in high school classrooms, the present study was therefore conducted to address this issue specifically in relation to high school students. Online information searching strategies (OISS) OISS in this study refers to learners' purposive behaviours in finding relevant or useful academic information in an online environment. As the amount of information available online is vast and the possible sources of information for students are much wider than they were before, making successful use of web information requires skills that enable students to find target information quickly. Tsai and Tsai Australasian Journal of Educational Technology, 2014, 30(1). 108 (2003), for example, have defined these skills under three domain factors, namely behavioural, procedural and metacognitive. In an era of information explosion, it is easy to find information on the Internet, but it could be very difficult to find relevant and accurate information in a reasonable time (Harrison, 2009). Most teachers believe that many of their students are competent in using social networking pages or online games, but they have concerns about students' Internet skills regarding searching for and using curriculum-based information (Ladbrook & Probert, 2011). Searching for suitable and reliable information online in an effective manner is a vital skill for students to learn. Therefore, researchers and educators are trying to ascertain what strategies or variables related to these strategies are involved when students are engaged in online searching. Several educational studies have investigated factors influencing students' online searching behaviours. For example, Tsai and Tsai (2003) found that high Internet self-efficacy students had better OISS and learned better than those with low Internet self-efficacy in a web-based learning task. Tabatabai and Shore (2005) revealed that the use of clear evaluation criteria for web sites was correlated with successful searching strategies. Tu, Shih, and Tsai (2008) pointed out that students' epistemological beliefs played an important role in students' OISS. Some studies even reported that students' OISS was improved when they completed related online training activities (Hwang & Kuo, 2011; Tekinarslan, 2008). Factors related to students' OISS are still under extensive investigation. However, little research has been conducted on senior high school students. In addition, factors such as SRL and OIES, which play important roles in online learning environments (Winters et al., 2008; Wu & Tsai, 2005), were also ignored. Thus, in this study we focus on the influence of SRL and OIES on high school students' OISS to bridge the research gap. Purposes of the study Information technology offers not only new opportunities but also challenges to online learners. Different instruments have therefore been developed with different objectives for addressing issues related to online learning. For example, Cheng et al. (2013) have developed an SRL instrument to assess students' SRL in online academic searching. Wu and Tsai (2005) developed an OIES instrument for exploring students' online information standards. Tsai (2009a) developed an OISS instrument to gauge students' online searching strategies. However, some of them examined the validity and reliability of the instruments only by exploratory factor analysis (e.g., Cheng et al., 2013; Tsai, 2009a). Therefore, one of the major purposes of this study was to further confirm the validity and reliability of these instruments. Also, these instruments were modified to explore secondary students' ideas about these constructs in the context of online academic searching, as searching online for academic purposes is quite common for them in the contemporary age. In addition, to date, research has examined various issues with regard to OISS (Tsai & Tsai, 2003; Wu & Tsai, 2005). However, most of this research has been conducted at the tertiary level and little has been done on high school students. Moreover, it is imperative to consider the roles of SRL and OIES in online academic information searching (Lee & Tsai, 2011; Lin & Tsai, 2008). Therefore, another major purpose of the current study was to examine the possible structural relationships among learners' SRL, OIES and OISS on the Internet, with a particular focus on secondary education. Based on the review of the literature, the hypothesized model for SRL, OIES and OISS in this study is as follows: First, the present study adopts Cheng et al.'s (2013) approach to categorize the SRL into two levels (basic level SRL and advanced level SRL). We hypothesize that in an academic online searching task, students' SRL has played a positive role in their OISS (Azevedo, 2005; Azevedo & Cromley, 2004; Tsai, 2009a, 2009b). Second, the current study employs Wu and Tsai's (2005) framework that students' OIES can be separated into four dimensions (multiple source, authority, content and technical). We hypothesize that students' advanced OIES (multiple source and content dimensions) has positive effects on their OISS; and naïve OIES (authority and technical dimensions) are negatively related to their OISS (Wu & Tsai, 2005). Third, we hypothesize that students' SRL positively guides their advanced OIES and Australasian Journal of Educational Technology, 2014, 30(1). 109 plays a negative role in their naïve OIES (Lee & Tsai, 2011). Our research model is illustrated in Figure 1. Figure 1. The hypothesized model for the SRL, OIES and OISS scales. Note. Solid lines indicate positive effects, and dashed lines indicate negative effects. The present study aims to address the above hypotheses by using structural equation modelling (SEM) techniques, including confirmatory factor analysis (CFA) and path analysis. Over the past three decades, SEM techniques have emerged as a powerful multivariate data analysis tool in social science research communities (Curran, 2003). For example, CFA, unlike exploratory factor analysis (EFA), allows researchers to simultaneously estimate the measurement of constructs (Aaker & Bagozzi, 1979). With regard to path analysis, it enables researchers to examine all structural effects or relevant paths in fewer steps, including mediators that may occur in a sequence of the data sets (McClell & Judd, 1993). Hence, SEM-based techniques have become a good choice when researchers are seeking appropriate tools to estimate complex models and avoid measurement error (Iacobuuci, Saldanha, & Deng, 2007). Research questions On the basis of the aforementioned literature review and purposes of the study, the research questions of this study are as follows: 1. Through CFA, are the SRL, OIES and OISS sufficiently valid and reliable for assessing high school students' online SRL, online information standards and OISS in the context of online academic searching? 2. By SEM analysis, do high school students' online SRL and online information standards have an effect on their searching strategies with respect to online academic searching? Does high school student OIES mediate the relationship between their SRL and OISS in online learning tasks? Method Sample The respondents of this study were 307 high school students in Taiwan, of which 163 were males and 144 females. The range of participants' ages was 16 to 18 years. All of the students reported that they had experience of online academic information searching in the past. In fact, over half (59.3 percent) of them reported that they sometimes use the Internet for academic information searching and a quarter (27.7 percent) of them said that they usually do so. Australasian Journal of Educational Technology, 2014, 30(1). 110 Survey instruments In this study, three questionnaires, namely SRL, OIES and OISS presented with a 7-point Likert scale (from 1, not at all true to 7, very true) were administered to obtain high school students' SRL, OIES and OISS when they were looking for information online. Additional items were added at the beginning of the three questionnaires to investigate whether they had experience of online information searching and their frequency of Internet use. The printed questionnaires were distributed and collected in classes to ensure the highest possible response rate. School approvals for their students to take part in the research were obtained before the investigation. Of 363 questionnaires delivered, 307 were completed and returned, with an overall response rate of 84.57%. SRL questionnaire This study used an SRL questionnaire developed by Cheng et al. (2013) to explore the students' perceptions of SRL when they were involved in searching online academic information. This study also followed Cheng et al.'s (2013) approach to categorization. That is, the planning phase (the initial phase of SRL proposed by Pintrich, 2004) was considered as basic SRL while the monitoring, controlling and reflection phases (the following second to fourth phases of SRL suggested by Pintrich, 2004) were considered advanced SRL. These four phases illustrate a time sequence loop that individuals go through in a given task, starting from the planning phase followed by the monitoring, controlling and reflection phases. Therefore, the SRL questionnaire in this study consists of two factors with 10 items, namely basic SRL and advanced SRL. The items used in the SRL questionnaire in this study were the same as those in Cheng et al.'s (2013). The descriptions of the two factors in the SRL questionnaire are presented below: 1. The Basic SRL factor is to assess the extent to which the students have an awareness of their insufficient academic knowledge and activate themselves to plan or set goals for relevant learning tasks in online environments. The sample item in this factor is "When I am searching for online academic information, I can set the searching goals based on topics or things I am interested in." 2. The Advanced SRL factor is to gauge the extent to which the students take action to monitor their online learning processes for course-related purposes, use applicable strategies to gather relevant online resources, and evaluate their learning outcomes. The sample item in this factor is "When I am searching for online academic information, I can integrate the learning materials I retrieved from the Internet into practical use." OIES questionnaire The OIES questionnaire adapted from Wu and Tsai (2005, 2007) was utilized to investigate the students' OIES in the situations of judging online academic information. The scale in Wu and Tsai (2005, 2007) included four factors with 16 items for the evaluative standards for web materials. Under these four factors, Multiple sources and Authority were the two factors used for gauging students' standards for accuracy of online academic information whereas Content and Technical issues were the two factors used for assessing the usefulness of online academic information. In addition, "multiple" and "content" commonly expressed by experts were defined as advanced standards while "authority" and "technical issues" were naive standards (Cheng et al., 2013). The same 16 items in the OIES scale (Wu & Tsai, 2005, 2007) were used in this study. The questions of all factors were stated with regard to the precondition "If I get some dubious information while searching online academic information…" Following are the sample questions of the four factors: 1. Multiple sources: I will try to find more websites to validate whether the information is correct. 2. Authority: I will believe in its accuracy if the information is posted in well-known websites. 3. Content: If its content fits my searching goal, I will consider the information as useful to me. 4. Technical issues: If it does not take much time to be retrieved, the information is useful to me. Australasian Journal of Educational Technology, 2014, 30(1). 111 OISS questionnaire This study used the OISS questionnaire to assess student information searching strategies in a web-based learning environment. The OISS developed by Tsai (2009a) is a 25-item questionnaire consisting of seven strategy aspects under three domain factors, namely behavioural, procedural and metacognitive. It was originally developed for students above secondary school levels (Tsai & Tsai, 2003) and later developed into a high school version as it was also suitable to use this instrument for the assessment of younger students’ online searching strategies (Tsai, 2009a). Following are the descriptions of the three factors of the OISS questionnaire: 1. Behavioural domain: This domain includes the strategies of "control" and "disorientation." It describes skills required for basic Internet manipulation and navigation. The sample item in this domain factor is "When I am searching online academic information, I know how to utilize advanced-search functions provided by search engines." 2. Procedural domain: This domain consists of the strategies of "trial and error" and "problem solving." It concerns with student content-general searching approaches on the Internet. The sample item in this domain factor is "When I am searching online academic information, I would try some possible entrance websites when I cannot find enough information." 3. Metacognitive domain: This domain contains the strategies of "purposeful thinking," "selecting main ideas," and "evaluating information." It indicates skills involved in higher-order and content-related cognitive activities on the Internet. The sample item in this domain factor is "When I am searching online academic information, I usually make sure the goals before starting my online searching." Data analysis To achieve the main purposes of this study, SEM techniques were used for data analysis. CFA was performed to test more precisely the hypothesized structure of the set of factors in each scale. Path analysis was performed to explore the relationships of these scales shown in Figure 1. Fit indices, such as GFI (Goodness of Fit Index≥0.90), CFI (Comparative Fit Index≥0.90), RMSEA (Root Mean-Square Error of Approximation≤ 0.08), RMR (Root Mean Square Residual≤ 0.10), AGFI (Adjusted Goodness- of-Fit Index≥0.80), NFI (Normed Fit Index≥0.90), NNFI (Non-Normed Fit Index≥0.90, RFI (Relative Fit Index≥0.90), IFI (Incremental Fit Index ≥0.90) were used as cut off criteria for good fit indices (Browne & Cudeck, 1989; Hoyle & Panter, 1995). ECVI (Expected Cross Validation Index), proposed as a means to assess the likelihood that the model cross-validates across similar-sized samples from the same population, was used as an indicator for the evaluation of a model's overall fit (Browne & Cudeck, 1989). The smallest value for ECVI would indicate the model with the greatest potential for replication. Results Instrument validation and its reliability To examine the measurement model of the previous survey instruments (i.e., SRL, OIES and OISS), CFA was performed using LISREL 8.54. The final set of items in each factor must show construct reliabilities and Cronbach's alphas that exceed recommended standards (i.e., factor loadings ≥ 0.5, t-value > 1.96 (p < 0.05), Composite Reliability≥ 0.6) for reliability. The results are presented in Tables 1, 2 and 3. Table 1 provides the validation result for the SRL scale. As shown in Table 1, the confirmatory factor analysis shows that the two factor model (i.e., basic SRL and advanced SRL) with seven items has a good fit with the latent constructs (RMR = 0.04, GFI = 0.97, AGFI = 0.94, NFI = 0.98, RFI = 0.97, IFI = 0.99, NNFI = 0.98, CFI = 0.99, RMSEA = 0.06, 90% confidence intervals of the RMSEA = 0.03–0.09, ECVI = 0.20 ). Convergent validity examines the relationship among items in the same construct. Bagozzi (1980) has recommended that for convergent validity to occur, all factors’ loadings must be large (i.e., factor loading of .50 or greater) and statistically significant. Three items were removed from the original scale because they had a low factor loading (i.e., factor loading less than 0.5). One reason could have been that the items were unclear and the wording was problematic for high school students. Each factor of the final Australasian Journal of Educational Technology, 2014, 30(1). 112 model of the SRL has four and three items respectively. The Composite Reliability value of the Basic SRL is 0.83. The Composite Reliability value of the Advanced SRL is 0.81, indicating good construct reliability and adequate convergent validity. The two factors in the final model are consistent with Cheng et al.'s (2013) study. Table 1 Validation for the Self-Regulation factors (N=307) Factor Item Mean SD Factor loading t-value Composite Reliability Basic SRL BSR1 5.56 0.82 0.62 11.17* 0.83 BSR2 0.80 15.81* BSR3 0.83 16.55* BSR4 0.71 13.49* Advanced SRL ASR5 4.99 0.92 0.75 14.03* 0.81 ASR6 0.84 16.36* ASR7 0.71 13.24* Note. Overall α = 0.86; Degrees of Freedom=13; * p < 0.05 Similarly, Table 2 provides the validation result for the OIES scale. In Table 2, the CFA indicates that the four factor model (i.e., multiple sources, authority, content and technical issues) with fifteen items has a good fit with the latent constructs (RMR = 0.07, GFI = 0.95, AGFI = 0.88, NFI = 0.92, RFI = 0.91, IFI = 0.95, NNFI = 0.94, CFI = 0.95, RMSEA = 0.07, 90% confidence intervals of the RMSEA = 0.05–0.08, ECVI = 0.93). Only one item was removed from the original scale for the low factor loading (Bagozzi, 1980). In the final model, one factor of the OIES has three items, and the other three factors all have four items. The Composite Reliability value of the Multiple sources is 0.65. The Composite Reliability values of Authority, Content and Technical issues are 0.81, 0.89 and 0.78 respectively. The values indicate good construct reliability and adequate convergent validity. These factors are consistent with those found in previous studies (Wu & Tsai, 2005, 2007), with overall α of 0.81. Hence, these results indicate that the OIES questionnaire is an adequate instrument to reflect high school students' online information evaluative standards when accessing academic online information. Table 2 Validation for the OIES factors (N=307) Factor Item Mean SD Factor loading t-value Composite Reliability Multiple Sources (MS) MS1 5.01 0.86 0.57 8.67* 0.65 MS2 0.65 9.73* MS3 0.63 9.41* Authority (AU) AU4 4.85 1.01 0.71 13.11* 0.81 AU5 0.74 13.84* AU6 0.80 15.18* AU7 0.64 11.37* Content (CO) CO8 5.63 0.81 0.80 16.48* 0.89 CO9 0.89 19.28* CO10 0.86 18.22* CO11 0.73 14.41* Technical Issues (TEC) TEC12 4.78 1.04 0.50 8.51* 0.78 TEC13 0.79 14.78* TEC14 0.85 16.03* TEC15 0.57 9.98* Note. Overall α = 0.81; Degrees of Freedom=84; * p < 0.05 Australasian Journal of Educational Technology, 2014, 30(1). 113 Table 3 provides the validation result for the OISS scale. According to Table 3, the CFA shows that the three factor model (including behavioural, procedural and metacognitive) with fifteen items has a good fit with the latent constructs (RMR = 0.06, GFI = 0.91, AGFI = 0.87, NFI = 0.95, RFI = 0.94, IFI = 0.97, NNFI = 0.96, CFI = 0.97, RMSEA = 0.08, 90% confidence intervals of the RMSEA = 0.06–0.08, ECVI = 0.99). Each of the three factors of the final model of the OISS has four, five and six items respectively. The Composite Reliability value of the Behavioural is 0.83. The Composite Reliability values of Procedural and Metacognitive are 0.81 and 0.84 respectively. The values indicate good construct reliability and adequate convergent validity. The results of the CFA are exactly in accordance with those factors identified by Tsai (2009a). The overall α of the three factors is 0.89, suggesting that they are sufficiently reliable to assess high school students' perceptions of OISS in the context of online academic searching. Table 3 Validation for the OISS factors (N=307) Factor Item Mean SD Factor loading t-value Composite Reliability Behavioural Beh1 5.60 1.04 0.75 14.38* 0.83 Beh2 0.70 13.08* Beh3 0.71 13.44* Beh4 0.79 15.47* Procedural PRO5 5.48 0.77 0.72 13.48* 0.81 PRO6 0.82 16.07* PRO7 0.57 10.02* PRO8 0.63 11.40* PRO9 0.65 12.00* Metacognitive MET10 5.52 0.81 0.54 9.63* 0.84 MET11 0.71 13.51* MET12 0.76 14.72* MET13 0.72 13.67* MET14 0.69 13.01* MET15 0.67 12.54* Note. Overall α = 0.89; Degrees of Freedom=87; * p < 0.05 Structural model: the relations among the scales SEM path analyses were conducted to explore the structural relations among the observed variables using the LISREL program. The two levels of students' SRL were used as the predictor variables while factors of students' OIES and OISS were used as the outcome variables for the analyses. The path model of this study is depicted in Figure 2, with the significance of the t-value as indicated by the asterisks for the model. The goodness-of-fit of the model was evaluated by the following criteria: the goodness-of-fit index (GFI) ≥ 0.90, the comparative fit index (CFI) ≥ 0.90, normal fit index (NFI) ≥ 0.90, incremental fit index (IFI) ≥ 0.90 and the root mean square residual (RMSR) ≤ 0.10 (Hoyle & Panter, 1995). The values that were obtained for the goodness-of-fit indices were as follows: GFI = 0.92, CFI = 0.92, NFI = 0.92, IFI = 0.92 and RMSR = 0.07, indicating that the model is plausible. According to Figure 2, SRL and OIES are significant predictors in explaining high school students' OISS. The paths from basic self-regulation to behavioural (β = 0.27) and procedural searching strategies (β = 0.28) and from advanced self-regulation to metacognitive search strategies (β = 0.26) are statistically significant. In addition, the paths from multiple sources to behavioural, procedural and metacognitive searching strategies are significant (β = 0.15, 0.15, 0.12 respectively) and those from content to behavioural, procedural and metacognitive searching strategies are also significant, but are larger in size Australasian Journal of Educational Technology, 2014, 30(1). 114 (β = 0.13, 0.22, 0.25 respectively). Finally, the paths from basic SRL to OISS through multiple sources (β = 0.32) and content (β = 0.43) are significant. In conclusion, it may be worth making three points about Figure 2 in terms of the path from SRL to OISS, the path from OIES to OISS, and the path from SRL to OISS through OIES. Firstly, basic self-regulation plays a positive role in behavioural and procedural searching strategies while advanced self-regulation only has positive effect on metacognitive searching strategies. Secondly, advanced OIES (i.e., Multiple sources and Content) has positive predictions on all OISS while naïve OIES (i.e., Authority and Technical issues) does not play any significant role in OISS. Thirdly, the relationship between basic SRL and OISS is mediated only through advanced OIES.                                 Figure 2. The final model for the SRL, OIES and OISS scales with significant structural relationships. Note. Significant t-value: * p<0.05, ** p<0.01, *** p<0.001 Discussion and conclusions This study aimed to investigate connections among high school students' perceptions of SRL, OIES and OISS when they were engaged in online academic information searching. To attain this goal, three instruments were used to collect data for the study. These instruments, by employing confirmatory factor analyses, showed adequate validity and reliability for assessing the students' perceptions of SRL, OIES and OISS. For researchers, these instruments may serve as useful tools to understand and delineate high school learner characteristics of searching on the Internet. Based on the results, it could be inferred that students' basic SRL guided their cognitive domain OISS (i.e., behavioural and procedural searching strategies) while their advanced SRL guided their metacognitive OISS. SRL in this study seemed to play a crucial role in students' OISS. Azevedo's (2005) study indicated that students using more metacognitive SRL tended to perform more successfully in online science learning tasks. Similarly, Tsai (2009a) referred to metacognitive domain strategies as the most critical variables in successful online inquiry-based learning. According to Azevedo and Cromley (2004), learners with SRL training could effectively regulate their learning with hypermedia and gain a deeper conceptual understanding of the learning task. As a lack of skill in using OISS is likely to put a student in a competitively disadvantaged position, educators therefore should consider implementing online tools to help students gain better SRL ability. For example, during the hypermedia learning tasks, different levels of electronic SRL prompts might be added as instructional supports when students are involving in OISS. Australasian Journal of Educational Technology, 2014, 30(1). 115 In addition to SRL, the role of OIES was also examined. It was found that advanced OIES had a positive effect on student OISS while naïve OIES did not. According to Wu and Tsai (2005), the advanced evaluative standards normally held by expert learners lead to their utilization of more sophisticated information searching strategies in web-based learning environments while the naïve evaluative standards usually held by novice learners did not. Similarly, in this study, high school students' advanced OIES did have significant effects on the OISS when they were engaged in online academic searching tasks. Kimber and Wyatt-Smith (2010) mentioned that today's educators should be charged with preparing young people for critical technology use more than basic technology proficiency. That is, digital worlds require higher order thinking and problem solving ability. They also pointed out that evaluative practices, including online information assessment and sources selection, are crucial online knowledge-seeking behaviours. With so many new information items generated via the Internet, some of which are misleading or erroneous, it is vital that educators give students explicit instructions in how to adequately evaluate this information (Cheong & Cheung, 2008). Therefore, future research is required to develop instructional methods to enhance students' advanced OIES inside and outside online learning environments. It is also worth noting that students' OIES, specifically the advanced aspect, mediated the relationships between basic SRL and OISS. One the other hand, when the students took advantage of the advanced SRL in employing metacognitive OISS, the significance of the OIES was not shown. One possible explanation for this is that in advanced SRL, students maintain attention for issues that concern them and for answers to the questions they have (Bates, 2002). Therefore, they may be directed to involve in metacognitive searching strategies without the mediation of OIES. One may argue that the RMSEA values in the current study are relatively large. Up until the early nineties, an RMSEA of below 0.08 was considered as a good fit (MacCallum, Browne, & Sugawara, 1996). However, more recently, a cut-off value close to 0.05 seems to be the general consensus among researchers in this area. Therefore, future study could helpfully focus on how to enhance the validity and reliability of the questionnaires used in this study. The results of this study provide evidence for the relationship among high school student SRL, OIES and OISS. Accordingly, educators should consider helping students develop better SRL abilities as well as advanced OIES so they are more prepared for online academic information searching tasks. However, future researchers still need to establish the generality of these findings by comparing different academic or age groups. In addition, in-depth interviews should be used to gain a fuller understanding of students' online academic searching processes. Moreover, the present study only examined whether learners' SRL and OIES influenced their OISS. Further studies are required to address whether their OISS also, in turn, influences their SRL or OIES. Acknowledgements Funding for this research work is supported by the National Science Council, Taiwan, under grant numbers 99-2511-S-011-005-MY3 and NSC-99-2511-S-011-008-MY3. References Aaker, D. A., & Bagozzi, R. P. (1979). Unobservable variables in structural equation models with an application in industrial selling. Journal of Marketing Research, 16(2), 147–158. Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209. Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students' learning with hypermedia? Journal of Educational Psychology, 96(3), 523-535. Australasian Journal of Educational Technology, 2014, 30(1). 116 Azevedo, R., Guthrie, J. T., & Seibert, D. (2004). The role of self-regulated learning in fostering students' conceptual understanding of complex systems with hypermedia. Journal of Educational Computing Research, 30, 87–111. Bagozzi, R. P. (1980). Causal modeling in marketing. New York: Wiley. Bates, M. J. (2002). Toward an integrated model of information seeking and searching. New Review of Information Behaviour Research, 3, 1-15. Boekaerts, M., Pintrich, P. R., & Zeidner, M. (2000). Handbook of self-regulation. San Diego, CA: Academic Press. Browne, M. W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24, 445−455. Cheng, K.-H., Liang, J.-C., & Tsai, C.-C. (2013). University students' online academic help seeking: The role of self-regulation and information commitments. The Internet and Higher Education, 16, 70-77. Cheong, C. M., & Cheung, W. S. (2008). Online discussion and critical thinking skills: A case study in a Singapore secondary school. Australasian Journal of Educational Technology, 24(5), 556-573. Retrieved from http://www.ascilite.org.au/ajet/ajet24/cheong.pdf Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38(4), 529–569. Dimmick, J., Chen, Y., & Li, Z. (2004). Competition between the Internet and traditional news media: The gratification-opportunities niche dimension. Journal of Media Economics, 17(1), 19-33. Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., … Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. Flanagin, A. J., & Metzger, M. J. (2007). The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media & Society, 9(2), 319-342. Gray, N. J., Klein, J. D., Noyce, P. R., Sesselberg, T. S., & Cantrill, J. A. (2005). Health information- seeking behaviour in adolescence: the place of the Internet. Social Science and Medicine, 60(7), 1467- 1478. Harrison, N. (2009). Analysis of online searching strategies for engineering students. Engineering Education, 4(1), 68-72. Hofer, B. K. (2004). Epistemological understandings as a metacognitive process: Thinking aloud during online searching. Educational Psychologist, 39(1), 43-56. Hoyle, R. H., & Panter, A. T. (1995). Writing about structural equation models. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 158-176). Thousand Oaks, CA: Sage. Hsu, Y. C., & Shiue, Y. M. (2005). The effect of self-directed learning readiness on achievement comparing face-to-face and two-way distance learning instruction. International Journal of Instructional Media, 32(2), 143-155. Australasian Journal of Educational Technology, 2014, 30(1). 117 Hwang, G.-J., & Kuo, F.-R. (2011). An information-summarising instruction strategy for improving the web-based problem solving abilities of students. Australasian Journal of Educational Technology, 27(2), 290-305. Retrieved from http://www.ascilite.org.au/ajet/ajet27/hwang.pdf Iacobuuci, D., Saldanha, N., & Deng, X. (2007). A meditation on mediation: Evidence that structural equation models perform better than regressions. Journal of Consumer Psychology, 17(2), 139–153. Kimber, K., & Wyatt-Smith, C. (2010). Secondary students' online use and creation of knowledge: Refocusing priorities for quality assessment and learning. Australasian Journal of Educational Technology, 26(5), 607-625. Retrieved from http://www.ascilite.org.au/ajet/ajet26/kimber.pdf Kuhlemeier, H., & Hemker, B. (2007). The impact of computer use at home on students' Internet skills. Computers & Education, 49(2), 460-480. Ladbrook, J., & Probert, E. (2011). Information skills and critical literacy: Where are our digikids at with online searching and are their teachers helping? Australasian Journal of Educational Technology, 27(1), 105-121. Retrieved from http://www.ascilite.org.au/ajet/ajet27/ladbrook.pdf Lee, S. W.-Y., & Tsai, C.-C. (2011). Students' perception of collaboration, self-regulated learning, and information seeking in the context of Internet-based learning and traditional learning. Computers in Human Behavior, 27(2), 905-914. Lin, C.-C., & Tsai, C.-C. (2008). Exploring the structural relationships between high school students' scientific epistemological views and their utilization of information commitments toward online science information. International Journal of Science Education, 30(15), 2001-2022. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130-149. Mason, L., & Boldrin, A. (2008). Epistemic metacognition in the context of information searching on the Web. In M. S. Khine (Ed.), Knowing, knowledge and beliefs: Epistemological studies across diverse cultures (pp. 377-404). Perth, Western Australia: Springer. McClell, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 14(2), 376–390. McClelland, M. M., Morrison, F. J., & Holmes, D. L. (2000). Children at risk for early academic problems: The role of learning-related social skills. Early Childhood Research Quarterly, 15(3), 307- 329. Metzger, M. J., Flanagin, A. J., & Zwarun, L. (2003). College student Web use, perceptions of information credibility, and verification behavior. Computers & Education, 41(3), 271-290. Retrieved from http://www.sciencedirect.com/science/article/pii/S0360131503000496 Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385-407. Rieh, S. Y., & Hilligoss, B. (2008). College students' credibility judgments in the information-seeking process. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth, and credibility (pp. 49-71). Cambridge, MA: The MIT Press. Schunk, D. H., & Zimmerman, B. J. (2006). Competence and control beliefs: Distinguishing the means and the ends. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 349-367). Mahwah, N.J.: Lawrence Erlbaum Associates. Australasian Journal of Educational Technology, 2014, 30(1). 118 Tabatabai, D., & Shore, B. (2005). How experts and novices search the web. Library & Information Science Research, 27, 222–248. Tekinarslan, E. (2008). Blogs: A qualitative investigation into an instructor and undergraduate students' experiences. Australasian Journal of Educational Technology, 24(4), 402-412. Retrieved from http://www.ascilite.org.au/ajet/ajet24/tekinarslan.pdf Tsai, M.-J. (2009a). Online information searching strategy inventory (OISSI): A quick version and a complete version. Computers & Education, 53(2), 473-483. Tsai, M.-J. (2009b). The model of strategic e-learning: Understanding and evaluating student e-learning from metacognitive perspectives. Educational Technology & Society, 12(1), 34-48. Tsai, M.-J., Hsu, C.-Y., & Tsai, C.-C. (2011). Investigation of high school students' online science information searching performance: The role of implicit and explicit strategies. Journal of Science Education and Technology, 21(2), 246-254. Tsai, M.-J., & Tsai, C.-C. (2003). Information searching strategies in web based science learning: the role of Internet self-efficacy. Innovation in Education & Teaching International, 40(1), 43-50. Tu, Y. W., Shih, M., & Tsai, C.-C. (2008). Eighth graders' web searching strategies and outcomes: The role of task types, web experience and epistemological beliefs. Computers and Education, 51(3), 1142-1153. Vermunt, J. D., & Vermetten, V. J. (2004). Patterns in student learning: Relationships between learning strategies, conceptions of learning, and learning orientations. Educational Psychology Review, 16(4), 359-384. Walraven, A., Brand-gruwel, S., & Boshuizen, H. P. A. (2008). Information-problem solving: A review of problems students encounter and instructional solutions. Computers in Human Behavior, 24(3), 623-648. Willoughby, T., Anderson, S. A., Wood, E., Mueller, J., & Ross, C. (2009). Fast searching for information on the Internet to use in a learning context: The impact of domain knowledge. Computers & Education, 52(3), 640–648. Winters, F. I., Greene, J. A., & Costich, C. M. (2008). Self-regulation of learning within computer-based learning environments: A critical analysis. Educational Psychology Review, 20(4), 429-444. Wu, Y. T., & Tsai, C.-C. (2005). Information commitments: Evaluative standards and information searching strategies in web-based learning environments. Journal of Computer Assisted Learning, 21(5), 374-385. Wu, Y. T., & Tsai, C.-C. (2007). Developing an information commitment survey for assessing students' web information searching strategies and evaluative standards for web materials. Educational Technology and Society, 10(2), 120-132. Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329-339. Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13-39). Burlington, MA: Elsevier Academic Press. Australasian Journal of Educational Technology, 2014, 30(1). 119 Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166-183. Zhou, G., & Xu, J. (2007). Adoption of educational technology ten years after setting strategic goals: A Canadian university case. Australasian Journal of Educational Technology, 23(4), 508-528. Retrieved from http://www.ascilite.org.au/ajet/ajet23/zhou.html Corresponding author: Jyh-Chong Liang, aljc@mail.ntust.edu.tw Australasian Journal of Educational Technology © 2014. Please cite as: Tseng, S.-C., Liang, J.-C., & Tsai, C.-C. (2014). Students' self-regulated learning, online information evaluative standards and online academic searching strategies. Australasian Journal of Educational Technology, 30(1), 106-121. Australasian Journal of Educational Technology, 2014, 30(1). 120 Appendix The following factors and items represent the SRL, OIES and OISS questionnaires that were used in this study. Items of the Self-Regulation Factor Item Question Basic SRL BSR1 When searching online, I set goals for searching for the materials I am interested in. BSR2 When searching online, I know my lack in knowledge and skills and try to get online information to make up for that. BSR3 When searching online, I collect information so that I can learn more about the learning topic. BSR4 When searching online, I can find new learning materials for myself. Advanced SRL ASR5 I can integrate the learning materials I get on the Internet. ASR6 I can search online for the information I need at my own learning pace. ASR7 I can evaluate the effectiveness of the searching results to fit my needs in the following learning and studying. Items of the OIES Factor Item Question Multiple Sources (MS) MS1 When I view some information unknown on the Internet, I will discuss with teachers or peers, and then to judge whether the information is correct. MS2 When I view some information unknown on the Internet, I will explore relevant content from books (or print materials), and then to evaluate whether the information is correct. MS3 When I view some information unknown on the Internet, I will try to find more websites to validate whether the information is correct. Authority (AU) AU4 When I view some information unknown on the Internet, I will believe in its accuracy if the information is posted in famous websites. AU5 When I view some information unknown on the Internet, I will believe in its correctness if the information appears in government websites. AU6 When I view some information unknown on the Internet, I will believe in its accuracy if the information is posted in professional (official) websites. AU7 When I view some information unknown on the Internet, I will believe in its correctness if the information appears in some websites recommended by experts. Content (CO) CO8 When I view or navigate information on the Internet, if its content fits my searching goal, I will think the information is useful to me. CO9 When I view or navigate information on the Internet, if it can provide more related links, the information for me is useful. CO10 When I view or navigate information on the Internet, if it can help me search relevant information further, I will think the information is useful to me. CO11 When I view or navigate information on the Internet, if it is highly related to my intended searching content, the information for me is useful. Australasian Journal of Educational Technology, 2014, 30(1). 121 Factor Item Question Technical Issues (TEC) TEC12 When I view or navigate information on the Internet, if there is no need to search for further information, the information is useful to me. TEC13 When I view or navigate information on the Internet, if it does not take much time to be retrieved, the information is useful to me. TEC14 When I view or navigate information on the Internet, if it does not require password or registration, I will think the information is useful to me. TEC15 When I view or navigate information on the Internet, if it is shown in more beautiful websites, I will believe in its usefulness. Items of the OISS Factor Item Question Behavioural BEH1 I know how to use a web browser, like IE or Netscape. BEH2 I look through the titles or hyperlinks in order to catch the main ideas in a webpage. BEH3 I know how to utilize advanced-search functions provided by search engines. BEH4 I know how to login a specific website with its URL. Procedural PRO5 I try some possible entrance websites when I cannot find enough information. PRO6 I try other databases when I cannot get any information in one database. PRO7 I try some other search engines when my search is not successful. PRO8 I think of some resolutions when I am frustrated with searching problems. PRO9 I do my best to resolve any problem occurred during a searching. Metacognitive MET10 I select main ideas provided in each webpage as possible as I can. MET11 I think of how to present and organize the data that I have searched from the web. MET12 I keep on evaluating the relationships among the information searched from the web. MET13 I compare information that has been collected from different websites. MET14 I decide if the information provided in a website is worth for reference. MET15 I think of how to utilize the searched information.