Microsoft Word - [9]197-2631-1-CE-1.doc Australasian Journal of Educational Technology, 2013, 29(3). ascilite 416 Effects of online procedural scaffolds and the timing of scaffolding provision on elementary Taiwanese students' question-generation in a science class Fu-Yun Yu National Cheng Kung University, Taiwan Han-Chang Tsai Kaohsiung Municipal Fuh-Hsing Primary School, Taiwan Hui-Ling Wu National Cheng Kung University, Taiwan This study examined the effects of online procedural scaffolds (in the form of generic question-stems with context-specific examples) and the timing of scaffolding provision (immediate versus delayed) on supporting the online student question-generation learning process in a science class. A total of 78 fifth-grade Taiwanese students participated in eight online question-generation sessions. An online learning system equipped with a customizable scaffolding design in terms of content and timing of access was used. The results of ANOVAs on the students' week-by-week question-generation performance showed the immediate positive effects of immediate procedural scaffolding. However, the delayed procedural scaffolds group did not statistically distinguish themselves from the no- scaffolds group in any of the eight question-generation performances, nor did the delayed approach engender productive failure, as postulated by some researchers. The significance of this study is discussed, along with suggestions for related instructional implementations, online systems and future research. Introduction Foundations and current state of student question-generation Based on current educational theories, educational activities should be designed to maximize learners' opportunities to construct meaningful personal knowledge and develop higher-order thinking abilities, such as metacognition, which has been shown as the most important factor contributing to learning (American Psychological Association, 1997; as cited in Mok, 2005). Congruent with this line of thought, practice of student question-generation has attracted the attention of both researchers and practitioners for several decades. A number of theoretical perspectives have been developed to explain how student question-generation can be of value to learning and cognitive development. In particular, constructivists posit that learning is most likely to occur in contexts where individuals are allowed to reflect and build their knowledge based on learning experiences to which they have been exposed (Bodner, Klobuchar, & Geelan, 2001; von Glasersfeld, 1987). Constructivism emphasizes that learners construct their own interpretations of the world of information around them in terms of their own conceptual structures (Steffe, 1991). The goal of instruction, from the perspective of constructivists, is thus to create situations that enhance individual interpretations and reflections, rather than mirroring the representations or fixed structures of the external, objective world (Steffe & Kieren, 1994; von Glasersfeld, 1987). As such, a constructivist approach supports having students create their own questions, in contrast to the situation that commonly occurs in traditional classrooms, where they answer questions that teachers regard as important based on the content of the study material (Yu, 2011). Another concept that is widely cited by researchers examining student question-generation is metacognition, which emphasizes the role of executive processes in overseeing and regulating a person's cognitive processes, such as planning, monitoring, predicting, evaluating and revising (Brown, 1987; Flavell, 1979; Livingston, 2003). When learners need to generate questions based on material they have studied, this triggers many metacognitive processes, thus aiding learning. Put another way, when asked to create questions, students need to reflect on whether there are any parts of the material that seem important, but which they do not comprehend, as well as how the core concepts can be understood, and then rephrased and used in test items. Based on metacognition theory, students who generate questions Australasian Journal of Educational Technology, 2013, 29(3). 417 are more likely to be aware of the state of their own knowledge and competence, and become more intellectually active and engaged in the learning process (Yu, 2005). With the support of sound theoretical foundations, a number of studies that investigate the effects of student question-generation have been published since the 1960s. Studies demonstrating the cognitive, affective and social effects of question-generation include its aiding comprehension of the learned content (Barlow & Cates, 2006; Brown & Walter, 2005; Drake & Barlow, 2008; Gillespie, 1990), and the promotion of cognitive and metacognitive strategies (Andre & Anderson, 1978-79; Bondy, 1984; Rosenshine, Meister, & Chapman, 1996; Yu & Hung, 2006; Yu & Liu, 2008), active learning behaviours (Liu & Yu, 2004), diverse and flexible thinking (Andre & Anderson, 1978-79; Brown & Walter, 2005; English, 1997), intra-group communication (Yu & Liu, 2005), self-confidence (Whitin, 2004) and motivation (Chin, Brown, & Bruce, 2002). Recently, based on the features of networked technology (e.g., less time-bound, more place and device- independence, large storage space, high processing speeds, multi-media capability, anonymity, and so on), a dozen or so online learning systems that focus on student question-generation have been developed – such as QPPA by Yu, Liu, and Chan (2002, 2005), POP-B and POP-C by Akira, Tsukasa, and Akira (2004), QAIS by Barak and Rafaeli (2004), MCIDA by Fellenz (2004), Concerto II by Hirai and Hazeyama (2007), ExamNet by Wilson (2004), PeerWise by Denny, Hamer, Luxton-Reilly, and Purchase (2008), QuARKS by Yu (2009), CodeWrite by Denny, Luxton-Reilly, Tempero, and Hendrickx (2011), QPIS by Lan and Lin (2011) and StudySieve by Luxton-Reilly (2012). While most existing systems allow students to generate questions of different types and media formats, and include an element of peer- assessment, the need to incorporate online scaffolding to support student question-generation activities has rarely been acknowledged. However, such support is often needed, as research indicates that a considerable proportion of students do not experience question-generation during their formal schooling (Moses, Bjork, & Goldenberg, 1993; Vreman-de Olde & de Jong, 2006), and view such activities as challenging (Yu, 2009). It is thus important to develop pedagogically sound online scaffolds to help students become more adept and comfortable at generating questions, and to build an empirical basis for the effects of this support. Effects, functions and types of scaffolding Scaffolding, a term coined by Wood, Bruner, and Ross (1976), is a form of support to help learners reach pedagogical goals which they may not be able to achieve if unaided. In particular, it attempts to bridge the gap between the learners' current abilities (actual development) and the intended goals (potential development). Researchers have adopted various scaffolds to support different activities, including scientific thinking (Squire & Jan, 2007), explanations (Sandoval & Reiser, 2004), reasoning (Seethaler & Linn, 2004), problem solving (Ge & Land, 2004; Saye & Brush, 2002) and online discussions (Choi, Land, & Turgeon, 2005). Although studies have found that the characteristics of different kinds of scaffolding influence the effects on learning (e.g., studies with no fading had higher effect sizes than studies with fixed fading), generally speaking, scaffolding positively impacts student learning (Belland, Walker, Olsen, & Leary, 2012). Based on a meta-analysis of 180 research studies, Swanson (1999) concluded that scaffolding is one of the nine most effective instructional interventions More recently, the effects of scaffolding on student cognitive outcomes were demonstrated by a meta-analysis of science, technology, engineering, and mathematics education at the K-12, college, graduate, and adult levels, and the results showed that scaffolding positively impacts student learning, producing an average effect size of .53 (Belland et al., 2012). The various functions and types of scaffolding have been analysed from a number of different perspectives. For instance, Jackson, Krajcik, and Soloway (1998) presented three scaffold designs that serve different purposes in software systems: supportive, reflective and intrinsic. Supportive scaffolding focuses on offering advice and support alongside the task, and serves such purposes as guiding, coaching, and modelling. For example, a built-in 'Show me an example' button provides learners with context- sensitive help if they need it. Reflective scaffolding helps learners to reflect upon the ways they personally go about solving or conceptualizing tasks by using forms and prompts to elicit statements from learners for planning, explaining, testing and evaluating. Finally, intrinsic scaffolding aims to reduce the complexity of a task and help learners think about concepts through the use of visual support, such as maps and models. Australasian Journal of Educational Technology, 2013, 29(3). 418 Hannafin, Land, and Oliver (1999) identified four kinds of scaffolding, conceptual, metacognitive, procedural, and strategic, that can be used to foster student learning in open-ended learning environments. Conceptual scaffolding helps learners to understand the complex problem space currently under consideration, or to clarify misconceptions by providing structural maps, content trees or explicit hints. Metacognitive scaffolding helps learners manage their individual thinking processes by reminding them to reflect upon their goals, or proposing self-regulatory strategies and related monitoring processes. Procedural scaffolding helps learners utilize the available tools and resources, usually through the use of a 'balloon' or 'pop-up' help window. Finally, strategic scaffolding provides learners with alternative approaches or techniques for learning, or suggestions for initial questions. Ge and Land (2004) identified three kinds of question prompts, procedural, elaborative, and reflective, that can be used as scaffolds to support the solving of ill-structured problems. Procedural prompts guide learners to complete specific tasks, and may help them to identify and analyse the important features of these. Elaboration prompts help learners to articulate their thoughts, construct explanations, make justifications, and carry out reasoning with the use of prompting questions like: "Why is it important?" and "How does .... affect ...?" Reflective prompts help learners to reflect, and encourage them to self- monitor during the process by embedding questions, such as "What is our plan?" and "Have our goals changed?" Focus and aims of the study Although there are currently various scaffolding types, each of which may have the potential to support student question-generation, the effects of one specific type of scaffolding—procedural scaffolding or supportive scaffolding, to use Jackson, Krajcik, and Soloway's (1998) term, serve as the focus of this investigation. As noted above, most students lack experience of question-generation, and perceive the task as difficult (Moses, Bjork & Goldenberg, 1993; Vreman-de Olde & de Jong, 2004; Yu, 2009; Yu & Liu, 2009). Since the aim of procedural scaffolding is to support the completion of specific tasks through guiding, coaching and modelling (Ge & Land, 2004; Jackson, Krajcik, & Soloway, 1998), its effects with regard to supporting online student question-generation performance is examined in this work. Additionally, although most related studies have found evidence of the positive effects of scaffolding with regard to stimulating and enhancing thinking about goals (Azevedo, Winters, & Moos, 2004), planning (Hmelo, Holton, & Kolodner, 2000), monitoring (Lajoie, Lavigne, Guerrera, & Munsie, 2001), problem understanding (Zydney, 2010), problem solving (Kaffman, Ge, Xie, & Chen, 2008) and reflecting in the form of metacognitive prompts (Azevedo et al., 2004; Berthold, Nuckles, & Renkl, 2007; Hmelo et al., 2000; Lajoie et al., 2001) and procedural prompts (Kaffman et al., 2008; Wu & Pedersen, 2011; Zydney, 2010), one factor which may influence the results is the timing of scaffolding provision. Supporters of scaffolding being provided at the start of a learning task emphasize its ability to avoid floundering and frustration (Rummel & Kramer, 2010), clarify procedural or conceptual misunderstandings or incomplete knowledge (Sharma & Hannafin, 2007), and decrease student working memory load while dealing with novel information (Kirschner, Sweller, & Clark, 2006). For instance, Sharma and Hannafin (2007) suggested the provision of metacognitive scaffolds right from the start of a task can help students cope with difficulties and clarify misunderstandings when they are engaged in learner-centred environments in which they are required to act on their metacognitive skills. On the other side of the argument are researchers who support the delayed provision of scaffolds, in order to encourage the students to learn by doing, engage recall from the long-term memory, and facilitate knowledge checks and activation (Koedigner & Aleven, 2007; Kapur, 2008), all of which will enhance student performance over the long run. Recently, Kapur (2011) demonstrated that while delaying the provision of instructional scaffolds may lead to some difficulties and initial failures on the students' part, it may enhance subsequent learning better than support given right from the start, a phenomena identified as "productive failure" (Kapur, 2008). Specifically, in an experimental study Kapur (2011) compared the learning outcome of seventh-grade students in a math class who were (a) provided with different levels of scaffolding in the form of teacher facilitation (including teacher-led discussion, corrective feedback, prompts for elaboration and explanations, and so on) throughout a problem solving process, versus (b) delayed scaffolds. The results showed that students who were provided with delayed support turned out to have better performance in generating a greater number of problem representations and methods for solving complex problems than their counterparts in the continuous support group. Pathak, Kim, Jacobson, and Zhang (2009) also demonstrated the value of delayed scaffolds. They compared the results from six pairs of students who worked on a science inquiry task either with or without delayed scaffolds. The results showed that students who used a delayed scaffolding approach were "more deeply engaged in the inquiry process and performed better on model-based explanations" (p. 199) than those had consistent scaffolding Australasian Journal of Educational Technology, 2013, 29(3). 419 throughout the activities. However, there are few studies in the literature that examine the effects of the timing of scaffolding provision, and those that do exist have conflicting findings. Therefore, another focus of this study is to examine whether and how delaying the provision of online procedural scaffolds influences student question-generation performance. Finally, in order to gain a more comprehensive understanding of the effects of online procedural scaffolds, student performance on question-generation is examined and analysed throughout the learning process, rather than simply at the end of it. In summary, the aim of this study is to examine the comparative effects of no-scaffolds, immediate-scaffolds and delayed-scaffolds with regard to supporting the online student question-generation learning process. Methods Participants and learning context Four fifth grade classes (N=78) from one elementary school in Taiwan participated in the study. An online student question-generation activity was introduced to support student science learning. Because science classes at this grade level are all taught by the same science teacher, the instructional content covered each week was identical. Two units were covered during the study: Plant World (lessons: the structure and function of plants, reproduction of plants, and plants vs. animals) and The Sun (lessons: sunrise and sunset, astrology in four seasons, and the origin of heat). Additionally, all students at the participating school started taking computer classes when they were in the third grade, and thus possessed the basic skills needed to use a computer. Finally, the students who participated in this study had no prior experience of generating questions related to instructional content. The online learning system An online learning system with an emphasis on student question-generation, called QuARKS, was adopted for the study. Like all other similar online systems, QuARKS enables multimedia files to be included as parts of the question, texts of different fonts, size and styles can be used (see Figures 1 and 3), and questions can easily be saved, retrieved, revised and deleted by users. However, QuARKS's procedural scaffolding to support online student question-generation activities (see Figure 2), the focus of this investigation, is, to the best of the authors' knowledge, not yet available with other learning systems. Figure 1. Screenshot of short-answer question-generation (for the no-scaffolds group, without access to online procedural scaffolding). Procedural scaffolding in the form of a set of generic question stems was adopted due to their proven effects on enhancing content comprehension and learning (Demetriadia, Papadopoulos, Stamelos, & Fischer, 2008; King, 1991; Rosenshine, Meister, & Chapman, 1996). With QuARKS's customizability, Click any icon/button in this section to activate various formatting and editing functions and to include a multimedia file as part of the question and answer. Australasian Journal of Educational Technology, 2013, 29(3). 420 different sets of procedural scaffolds can be incorporated, updated and made available in a timely and user-friendly fashion by individual instructors. A set of question stems were created for use in this study with reference to generic question stems developed by King (1994) and Tung (2005). These stems mainly focus on comprehension and integration prompts. Comprehension questions ask for the description or definition of a concept or process, for example, "Please use your own words to describe…," "What was the main idea…?" Integration questions deepen or provoke thinking by linking and integrating ideas, for instance, "What conclusion can you draw about…?," "How is…related to ….that we studied before?" (King, 1994). Three science teachers from the participating school were invited to assess the appropriateness and usefulness of the question stems for the subject matter (science) and participants (fifth graders), and based on this, fourteen question-stems were included and made accessible to both immediate- and delayed- scaffolds group at different points in time (see the middle column of Figure 2). Additionally, to overcome the insufficiency of context-general scaffolds for supporting learning (Tabak, Smith, Sandoval, & Reiser, 1996), examples of the use of each question stem based on the science content from the previous semester were produced, and given to the students assigned to both scaffolds groups as context-specific hints (see the far right column of Figure 2). For a detailed description of QuARKS, including the framework guiding the development of the system and associated scaffolding designs, please refer to Yu (2009). The various functions that the different treatment groups used within the system are described below. Figure 2. Procedural scaffolding for student question-generation with context-specific examples (accessible to the immediate- and delayed-scaffolds groups). Research design and experimental treatments To examine the comparative effects of no-scaffolds, immediate-scaffolds and delayed-scaffolds on student performance in question-generation, three treatment conditions were established. For all treatment groups, the students were directed to construct short-answer questions, consisting of a question and answer written in the corresponding spaces in the QuARKS interface (i.e., Figure 1 for the no-scaffolds group and Figure 3 for the immediate- and delayed-scaffolds group). During the question-generation activities, the students in all the groups were encouraged to refer to the study materials and supplementary handouts, including the question-generation criteria given by instructors. Click here for this specific question stem to be imported to the “question” field Procedural scaffolding in the form of generic question stems Context-specific examples for generic question stems Australasian Journal of Educational Technology, 2013, 29(3). 421 In Treatment A (no-scaffolds), no procedural scaffolding was provided while the students completed the online student question-generation tasks throughout the study. To prompt students regarding some types of higher-order questions that could be generated around the study material, students in both Treatments B and C had access to online procedural scaffolding in the form of a set of generic question stems with context-appropriate examples (see Figure 2) when engaged in question-generation. In Treatment B (immediate-scaffolds), procedural scaffolding was accessible at the start of the activity. Students constructed short-answer questions by first clicking on the 'Guide' button placed above the question field (Figure 3) to be directed to a pop-up window containing the question stems list plus examples for reference (Figure 2). After reading through the list and deciding on an appropriate question-stem to meet their current needs, students then click on the 'IMPORT' button (the far left column of Figure 2) beside any question stem to have the targeted stem automatically imported to the 'question' field for question completion (Figure 3). In Treatment C (delayed-scaffolds), online procedural scaffolding was not accessible until the third question-generation session. That is, only after the group had two weeks' experience generating questions was the 'Guide' button activated so that the students could access the set of online question stems and examples for reference. Finally, to account for both major components of scaffolds – support building and support withdrawal/fading (Fretz, Wu, Zhang, Davis, Krajcik, & Soloway, 2002; Oh, 2005) – access to online procedural scaffolding for both Treatments B and C was denied at the beginning of the final week (the eight question-generation session). Figure 3. Screenshot of short-answer question-generation with access to online procedural scaffolding via the 'Guide' button (for the immediate- and delayed-scaffolds groups). Experimental procedure Informed consent was secured from all the participants prior to the study. To minimize scheduling and administrative problems, intact classes were used and randomly assigned to one of three treatment conditions, with the exception of one class that was randomly assigned to three different groups. Although unstreamed classes at the primary educational level are mandated by the Ministry of Education, Taiwan, to ensure the equivalence of the classes randomly assigned to different conditions, the students' academic performance in the focal subject (i.e., science) during the previous semester was collected and analysed. The ANOVA results indicated that there were no significant differences among the three treatment groups prior to the study (F = 0.08, p = 0.92). On a weekly basis, students in all treatment Click here to be directed to a set of generic question stems with examples (below) (see Figure 2 for a larger image) After clicking on the “IMPORT” button placed aside one of the generic s question stem (Figure 2), the chosen stem is automatically imported here for further question completion Click any icon/button in this section to activate various formatting and editing functions and to include a multimedia file as part of the question and answer. Australasian Journal of Educational Technology, 2013, 29(3). 422 groups headed to a computer lab during their regularly scheduled 40-minute morning study sessions to engage in online student question-generation activities. Students worked individually for eight weeks to generate short-answer questions in accordance with the current instructional topics. A training period was arranged prior to the study to familiarize students with the learning system, and enable them to practice question-generation. Following Rosenshine et al., (1996), elements supporting student question-generation were explained to all treatment groups, including: (a) the criteria that the teacher would use to assess the performance of student question-generation (see next section for details), (b) models of appropriate questions, and (c) specific rules discriminating a question from a non-question (i.e., unsolvable questions, or questions without all the essential conditions/information), and a good question from a poor one (e.g., a question focusing only on small details, or one that cannot be answered with the information provided in the material). Additional training with regard to accessing online procedural scaffolds and applying each of the procedural prompts was arranged for both scaffolded groups at appropriate points in time, i.e., the first question-generation session for the immediate-scaffolds group, and the third question-generation session for the delayed-scaffolds group. For each question-generation session, students created questions on the QuARKS system based on the content of the previous class. Feedback on question-generation was provided each week by the participating science teacher to highlight the strengths and weaknesses of three purposefully selected pieces of student work. Furthermore, the teacher also addressed any misconceptions the students had about the activity and learning content, based on the student-generated questions. A flow diagram summarizing the major experimental procedures of this study is given in Figure 4. Figure 4. Experimental procedures of this study. Measures To assess student performance in question-generation, a set of criteria was developed based on the Torrance creativity index (1974) and King's classification of questions of different cognitive levels (1992). Each question the students generated during the question-generation activities was assessed using these criteria. Specifically, each question was analysed and scored along four dimensions (fluency, elaboration, originality and cognitive level), with these then summed to form a composite score. Definitions and examples of each of the four dimensions of question quality were given to the students during the training session, and could be referred to during the question-generation activities (see Table 1). To ensure the reliability of the scoring procedure, the analysis was carried out separately by two teachers at the participating school (one home-room teacher and another science teacher). Any inconsistencies between the raters were then discussed and resolved. Australasian Journal of Educational Technology, 2013, 29(3). 423 Table 1 Four dimensions of question quality with associated definition and examples Dimensions Definition Examples Fluency (0-3) The correctness of word use and punctuation (1); The clarity of meaning (1); correctness of the answer (1) Example: Q1: What does 'instar' mean during the hatching and development of insects? A2: After an insect hatches, its "1st instar" begins. Each time an insect molts, it adds an instar (1st instar -> 2nd instar -> 3rd instar…) (Non-example: "What is the meaning of 'intar'?" which has a spelling error and is not phrased concisely to exclude other possible but incorrect answers) Elaboration (0-2) The interconnectedness between the currently covered topic/unit and prior topics/units (1); Self-derived examples (not cited in textbook) (1) Example1 (Linking across units): Q1: We learned about raising animals in the previous unit. Just now we learned about raising insects. Both need a clean environment and sufficient food. What other conditions are essential when it comes to raising insects and animals? A2: The place is well- ventilated, and there is enough fresh food and hiding or resting areas. Example2 (Self-derived example): Q1: How are locusts and butterflies similar? Identify at least three features. A2: They are both insects. Both have wings, six legs, a pair of antennae, and their bodies are divided into three parts: head, chest, and abdomen. (Self-derived example: Locusts are not contained in the textbook as examples of insects) Originality (0-2) The uniqueness of a specific question as compared to those of peers in terms of content (1) and presentation style (1) Example (uniqueness): Q1: When creating an environment for raising butterflies, why is it best to place larvae-topped leaves inside a glass bottle with water and then in a cage? A2: This is because, with water, leaves can continually grow for an extended period of time. This arrangement also helps air circulation in the cage. Altogether, this helps to recreate conditions similar to an outdoor natural environment. (Only one pupil in the class asked questions about the experiment.) Cognitive level (0-3) Level 1 Fact: the use of language taken from the learned materials (1); Level 2 Comprehension: students use their own words to define or describe learned content (2); Level 3 Integration: a link has been built across topics/units and explanations have been provided to support this connection (3) Examples: Level 1 Fact Q1: The body of an insect is composed of which three parts? A1: Head, chest, and abdomen. (Facts taken from the textbook) Level 2 Comprehension Q1: The design of thermoses is based on the principle of connecting pipes. What other device found in daily life, not mentioned in the textbook, is also designed based on this principle? A2: Pipeline beneath the sink. (Requesting an example not taught in class, demonstrating student comprehension of the topic) Level 3 Integration Q1: What roles do insects play for plants and animals? A2: Insects help plants to propagate by spreading pollen and seeds. As for animals, in addition to being eaten by or eating other animals, insects also play the role of scavenger, dismembering and eating the dead animals and plants. (Answers provided demonstrate the ability to link across topics and link lectures with lab work.) Note. 1Question; 2Answer to the generated question Australasian Journal of Educational Technology, 2013, 29(3). 424 Results Week-by-week student performance in the question-generation activities is given in Table 2 and graphed in Figure 5. As shown in Figure 5, the immediate-scaffolds group performed consistently better than the no-scaffolds group throughout the whole activity. Furthermore, the immediate-scaffolds group performed better than the delayed-scaffolds throughout the question-generation period with the exception of one session (the seventh), where both groups performed at the same level. Finally, although the performance of the delayed-scaffolds group was slightly below or equal to that of the no-scaffolds group for the first two sessions (when online scaffolds were not accessible to the group), students in the delayed-scaffolds group performed better after the introduction of the scaffolds in the third session, with the exception of the last session (where a significant fall was observed when online procedural scaffolding was withdrawn). Table 2 Descriptive and F-test statistics of student question-generation performance No-scaffolds (n=27) Immediate- scaffolds (n=26) Delayed- scaffolds (n=25) F p η2 QGa 1st, M (SD) 4.93 (3.40) 8.46 (5.56) 3.88 (4.42) 7.23* .001 .162 2nd, M (SD) 3.70 (2.33) 8.15 (4.95) 3.64 (4.28) 10.94* .000 .226 3rd , M (SD) 4.11 (4.01) 8.54 (4.49) 5.12 (3.54) 8.66* .000 .188 4th , M (SD) 4.67 (3.59) 9.04 (6.58) 6.08 (4.87) 4.95* .010 .117 5th , M (SD) 5.44 (2.98) 7.11 (5.31) 6.00 (7.78) .559 .552 .016 6th , M (SD) 6.33 (4.88) 10.69 (9.02) 8.60 (10.61) 1.764 .178 .045 7th , M (SD) 7.78 (4.84) 10.38 (7.74) 10.40 (7.56) 1.301 .278 .034 8th, M (SD) 9.07 (6.28) 11.81 (7.23) 8.60 (6.26) 1.78 .178 .045 Total, M (SD) 46.11 (18.61) 74.19 (35.49) 52.32 (35.91) 5.99 .004 .138 Note. *The mean difference is significant at the .05 level. a QG: Question-Generation 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Time Q ue st rio n- ge ne ra tio n pe rf or m an ce No Scaffolds Immediate Scaffolds Delayed Scaffolds Figure 5. Week-by-week performance of student question-generation among the three groups. Australasian Journal of Educational Technology, 2013, 29(3). 425 Table 3 Pairwise comparisons among three groups on student question-generation performance Comparison Mean differences (I-J) SE p a 95% Confidence Interval (I) (J) Lower Upper 1st QG No-scaffolds Immediate- scaffolds -3.53561 * 1.24547 .017 -6.5856 -.4856 Delayed- scaffolds 1.04593 1.25810 1.000 -2.0350 4.1268 Immediate- scaffolds Delayed- scaffolds 4.58154 * 1.26967 .002 1.4723 7.6908 2nd QG No-scaffolds Immediate- scaffolds -4.45014 * 1.09577 .000 -7.1335 -1.7668 Delayed- scaffolds .06370 1.10687 1.000 -2.6469 2.7743 Immediate scaffolds Delayed scaffolds 4.51385 * 1.11706 .000 1.7783 7.2494 3rd QG No-scaffolds Immediate scaffolds -4.42735 * 1.10991 .000 -7.1454 -1.7093 Delayed- scaffolds -1.00889 1.12116 1.000 -3.7545 1.7367 Immediate- scaffolds Delayed- scaffolds 3.41846 * 1.13148 .010 .6476 6.1893 4th QG No-scaffolds Immediate- scaffolds -4.37179 * 1.41422 .008 -7.8350 -.9086 Delayed- scaffolds -1.41333 1.42855 .977 -4.9117 2.0850 Immediate- scaffolds Delayed scaffolds 2.95846 1.44170 .131 -.5721 6.4890 5th QG No-scaffolds Immediate- scaffolds -1.67094 1.55087 .854 -5.4688 2.1269 Delayed- scaffolds -.55556 1.56659 1.000 -4.3919 3.2808 Immediate- scaffolds Delayed- scaffolds 1.11538 1.58101 1.000 -2.7563 4.9871 6th QG No-scaffolds Immediate- scaffolds -4.35897 2.32188 .193 -10.0449 1.3270 Delayed- scaffolds -2.26667 2.34542 1.000 -8.0103 3.4769 Immediate- scaffolds Delayed- scaffolds 2.09231 2.36701 1.000 -3.7042 7.8888 7th QG No-scaffolds Immediate- scaffolds -2.60684 1.87081 .503 -7.1882 1.9745 Delayed- scaffolds -2.62222 1.88978 .508 -7.2500 2.0056 Immediate- scaffolds Delayed- scaffolds -.01538 1.90717 1.000 -4.6858 4.6550 8th QG No-scaffolds Immediate- scaffolds -2.73362 1.81539 .409 -7.1793 1.7120 Delayed- scaffolds .47407 1.83380 1.000 -4.0166 4.9648 Immediate- scaffolds Delayed- scaffolds 3.20769 1.85067 .261 -1.3243 7.7397 Total QG No-scaffolds Immediate- scaffolds -28.08120 * 8.48063 .004 -48.8491 -7.3133 Delayed- scaffolds -6.20889 8.56660 1.000 -27.1873 14.7695 Immediate- scaffolds Delayed- scaffolds 21.87231 * 8.64544 .041 .7008 43.0438 Note. *Mean differences significant at .05 level. a Adjusted for multiple comparisons using the Bonferroni. Australasian Journal of Educational Technology, 2013, 29(3). 426 The results of the data analysis using the analysis of variance (ANOVA) indicated that the three groups had statistically significant differences in the first four sessions, as well as in their overall performance, F (2, 75) = 7.23, 10.94, 8.66, 4.95 and 5.99; p < .05, η2 =.162, .226, .188, .117 and .138 (see Table 2). Using Cohen's (1988) criteria (for η2: small, 0.01 to 0.05; medium, 0.06 to 0.13; large > 0.13), all significance levels reached the large effect size except one, which was at the medium effect. Additionally, no statistically significant differences were found among the three groups after the fifth question-generation session. Follow-up post hoc comparisons, using the Bonferroni adjustment to control for type I errors, further indicated that the immediate-scaffolds group outperformed the no-scaffolds group in the first four question-generation sessions, and outperformed the delayed-scaffolds group in the first three (see Table 3). Furthermore, the immediate-scaffolds group performed significantly better than both no-scaffolds and delayed-scaffolds groups in overall question-generation performance, as seen by the total scores. Finally, no statistically significant differences were found between the no-scaffolds and delayed-scaffolds groups in all the comparisons. Discussion and Conclusions This study explored the effects of online procedural scaffolds and the timing of scaffolding provision on student question-generation performance. The results of the data analysis revealed several important findings. First, students in the immediate-scaffolds group had significantly better question-generation performance than those in the no-scaffolds group in the first-half of the learning (i.e., from the first up to the fourth session). This highlights the immediate positive effects of online procedural scaffolds being introduced at the start of the activity when the students had no prior experience of question-generation. Second, the delayed-scaffolds group did not outperform the immediate-scaffolds one in any of the eight question-generation sessions, and no productive failure was seen, in contrast to the findings of Kapur (2011) and Pathak et al. (2009). Kapur's (2011) experimental study found that delaying the provision of scaffolds in various forms of teacher-based scaffolds (e.g., teacher-led consolidation discussion, corrective feedback, elaborative prompts) enhanced student problem-solving performance to a greater extent than when support was given continuously right at the start of the process. Similarly, Pathak et al. (2009) reported that students who used a delayed scaffolding approach were more deeply engaged in the inquiry process and performed better on model-based explanations than those who had scaffolding throughout the activities. However, in this study, students in the immediate online procedural scaffolds group were found to have significantly better question-generation performance than those in the delayed- scaffolds group at the initial stage of the activity (i.e., the first three sessions). In other words, the lack of assistance experienced by the delayed-scaffolds group at the start of the learning process did not enhance their question-generation performance. In contrast, explicit online guidance and modelling using generic question stems with context-specific examples right from the beginning was found to provide the students with effective assistance in making and maintaining specific kinds of connections among pieces, topics and units of studied content necessary for question-generation, as claimed by King and Rosenshine (1993). As a result, the immediate inclusion of online procedural scaffolds guided students to complete the assigned task more productively at the early stage of the activity (as seen in the results for the first three sessions) and as a whole (as seen in the overall performance). Third, the delayed-scaffolds group did not statistically distinguish themselves from the no-scaffolds group in any of the eight question-generation sessions. Furthermore, online procedural scaffolding did not produce significant positive effects on the question-generation performance of the delayed-scaffolds group, even after its inclusion in the third question-generation session. It seems that for this group of students (fifth graders with no prior experience of question-generation), once they gained some experience in generating questions, procedural scaffolding's supportive effects, while still appearing useful as compared to the no-scaffolds group (see Figure 4), may not have been so strong as to reach a statistically significant level. In sum, the results confirmed the effectiveness of online procedural scaffolding and the significance of the timing of scaffolding provision for student question-generation. More specifically, immediate procedural scaffolds, rather than delayed procedural prompts, were found to be better at supporting online student question-generation. Australasian Journal of Educational Technology, 2013, 29(3). 427 Significance and implications of the study The exploration of the effects of online procedural scaffolds and the timing of scaffolding provision on student question-generation performance carried out in this study has some significant empirical, instructional, methodological, online system development and theoretical implications. First of all, the empirical superiority of immediate online procedural scaffolds in the form of generic question stems, as compared to delayed procedural scaffolds and no procedural scaffolds, for student question-generation performance was substantiated for the first time in this study. One important implication can be drawn from this for instructional implementation, and this is that instructors who are interested in applying question-generation, but have concerns about the initial performance of their students who lack prior experience of doing this, should include procedural scaffolds from the very beginning of the process. As for the methodological significance of the study, the superiority of the immediate effects of immediate procedural scaffolds could not have been observed without the present design, i.e., plotting week-by-week student performance throughout the whole process, rather than the more frequently adopted and easily dealt with one-time end-of-session data analysis. It is thus suggested that researchers interested in understanding the effects of procedural scaffolds in particular, as well as scaffolds in general, should collect and analyse both process and outcome data to achieve a more comprehensive understanding of the observed phenomenon. Furthermore, while a dozen or so online learning systems have been developed to support student question-generation activities over the past decade, the importance of providing online scaffolds in these has rarely been recognized, and thus they have not often been incorporated. In light of the current findings, developers of online learning systems that have student question-generation as a focus are advised to embed online procedural scaffolds from the start for better learning support, as are users of other learning strategies when the students' current performance level or related experience are very limited. Last but not least, there is a basic theoretical disagreement in the literature as to how the timing of scaffolding provision may affect learning. On one side of the argument are researchers, such as Kirschner, Sweller and Clark (2006) and Rummel and Kramer (2010), who are in favour of immediate scaffolds, and Sharma and Hannafin (2007), who support the positive effects of metacognitive and reflective scaffolds on learning. On the other side are those like Kapur (2011) and Pathak et al. (2009), who have stressed the value of delaying the provision of various forms of instructor support, including corrective feedback and elaborative prompts. The results of the current study provide additional empirical evidence as to the optimal timing of scaffolds under specific contexts. Specifically, immediate online procedural scaffolds were found to be more effective than delayed ones with regard to supporting Taiwanese elementary student question-generation performance in a science class, where the students lacked previous experience of this task. Limitations of the study and suggestions for future studies This study has a number of limitations that suggest avenues for future research. First, the study involved 78 fifth-grade Taiwanese students and examined the effects of online procedural scaffolding on student question-generation performance in a science class. Moreover, due to the small sample size of each participating class (less than 20 students), four classes were involved and both random assignment of classes and students to different treatment groups were employed. The small sample size, the localized and specific nature of the study and different random assignment methods employed to different classes, may limit the contributions of the study; therefore, its external generalizability to a larger sample size in other contexts should not be taken for granted. Second, this study examined the effects on student question-generation performance of one specific type of scaffolding, procedural scaffolding in the form of generic questions stem with context-specific examples. The applicability of the findings to other forms of procedural scaffolding (e.g., signal words, main ideas, story grammar, question types, the answer is, and so on) and other scaffolding types (e.g., reflective and intrinsic in Jackson, Krajcik and Soloway's, 1998, terms, conceptual, metacognitive and strategic in Hannafin, Land and Oliver's, 1999, terms, or elaborative and reflective in Ge and Land's, 2004, terms) for the support of different learning tasks may thus be constrained, and should be exercised with great caution. Australasian Journal of Educational Technology, 2013, 29(3). 428 Some possible directions for future studies are as follows. First, replications of this study using larger and different groups of students to support different learning tasks are highly recommended to increase the generalizability of the results. Second, while this study found immediate positive effects of immediate online procedural scaffolding on student question-generation performance, the effects with regard to promoting subsequent learning outcomes (e.g., academic performance in and attitudes toward the applied subject matter) need to be examined further. Finally, in view of the fact that scaffolds can take different forms and provide different kinds of support in a learning environment (Lin, Hmelo, Kinzer, and Secules, 1999), and each type of scaffolding may have the potential to support student question-generation in different contexts, the effects of these are another topic that should be examined in future work. Acknowledgements This paper was funded by research grants from the National Science Council, Taiwan (NSC 99-2511-S- 006 -015 -MY3). References Akira N., Tsukasa H., & Akira T. (2004). A support environment for learning by describing problem map in mathematical word problems. In E. McKay, B. Collis, & R. Mizoguchi (Eds.), Acquiring and constructing knowledge through human-computer interaction: Creating new visions for the future of learning venue (pp.213-225). Melbourne, Australia: RMIT University. Retrieved from http://doi.ieeecomputersociety.org/10.1109/CIE.2002.1185881 Andre, M., & Anderson, T. (1978-79). The development and evaluation of a self-questioning study technique. Reading Research Quarterly, 14(4), 605-622. Retrieved from http://2020engineer.iss.utep.edu/world/Research%20Literature/self- questioning%20study%20techniques.pdf Azevedo, R., Winters, F. I., & Moos, D. C. (2004). Can students collaboratively use hypermedia to learn science? The dynamics of self- and other- regulatory processes in an ecology classroom. Journal of Educational Computing Research, 31(3), 215-245. Retrieved from http://baywood.metapress.com/openurl.asp?genre=article&id=doi:10.2190/HFT6-8EB1-TN99-MJVQ Barak, M., & Rafaeli, S. (2004). On-line question-posing and peer-assessment as means for web-based knowledge sharing in learning. International Journal of Human-Computer Studies, 61(1), 84-103. Retrieved from http://dx.doi.org/10.1016/j.ijhcs.2003.12.005 Barlow, A., & Cates, J. M. (2006). The impact of problem posing on elementary teachers' beliefs about mathematics and mathematics teaching. School Science and Mathematics, 106, 64-73. doi:10.1111/j.1949-8594.2006.tb18136.x Belland, B. R., Walker, A., Olsen, W., & Leary, H. (2012, April). Impact of scaffolding characteristics and study quality on learner outcomes in STEM education: A meta-analysis. Paper presented at the 2012 AERA Annual Convention, Vancouver, British Columbia, Canada. Berthold, K., Nuckles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17, 564-577. Retrieved from http://dx.doi.org/10.1016/j.learninstruc.2007.09.007 Bodner, G. M., Klobuchar, M., & Geelan, D. R. (2001). The many forms of constructivism. Journal of Chemical Education, 78(8), 1107. Retrieved from http://pubs.acs.org/doi/abs/10.1021/ed078p1107.4 Bondy, E. (1984). Thinking about thinking. Childhood Education, 6(4), 234-238. Retrieved from http://sanfrancisco.kudzu.com/uploadImages/15000000/15962836/attachment9p9awl.pdf Brown, A. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 65-116). Hillsdale, NJ: Lawrence Erlbaum. Brown, S. I., & Walter, M. I. (2005). The art of problem posing (3rd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Australasian Journal of Educational Technology, 2013, 29(3). 429 Chin, C., Brown, D. E., & Bruce, B. C. (2002). Student-generated questions: A meaningful aspect of learning in science. International Journal of Science Education, 24(5), 521-549. Retrieved from http://dx.doi.org/10.1080/09500690110095249 Choi, I., Land, S. M., & Turgeon, A. J. (2005). Scaffolding peer-questioning strategies to facilitate metacognition during online small group discussion. Instructional Science, 33, 483-511. Retrieved from http://www.springerlink.com/content/727927jq4828u29v/ Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Demetriadia, S., Papadopoulos, P., Stamelos, I., & Fischer, F. (2008). The effect of scaffolding students' context-generating cognitive activity in technology-enhanced case-based learning. Computers & Education, 51, 939-954. Retrieved from http://www.sciencedirect.com/science/article/pii/S0360131507001169 Denny, P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008). PeerWise: students sharing their multiple choice questions. In M. E. Caspersen (Ed.), Proceeding of the Fourth international Workshop on Computing Education Research (pp 51-58). New York, NY: Association for Computing Machinery. Retrieved from http://doi.acm.org/10.1145/1404520.1404526 Denny, P., Luxton-Reilly, A., Tempero, E., & Hendrickx, J. (2011, March). Codewrite: Supporting student-driven practice of java. Proceedings of the 42nd ACM technical symposium on Computer science education, SIGCSE '11 (pp. 471–476). New York, NY. Retrieved from http://doi.acm.org/10.1145/1953163.1953299 Drake, J. M., & Barlow, A. T. (2007). Assessing students' levels of understanding multiplication through problem writing. Teaching Children Mathematics, 14(5), 272- 277. Retrieved from http://www.nctm.org/profdev/content.aspx?id=23586 English, L. D. (1997). Promoting a problem-posing classroom. Teaching Children Mathematics, 4(3), 172-179. Flavell, J. (1979). Metacognition and cognitive monitoring: A new area of cognitive-development inquiry. American Psychologist, 34, 906-911. doi:10.1037/0003-066X.34.10.906 Fellenz, M.R. (2004). Using assessment to support higher level learning: the multiple choice item development assignment. Assessment & Education in Higher Education, 29(6), 703-719. Retrieved from http://dx.doi.org/10.1080/0260293042000227245 Fretz, E., Wu, H.-K., Zhang, B., Davis, E., Krajcik, J., & Soloway, E. (2002). An investigation of software scaffolds supporting modeling practices. Research in Science Education, 32, 567–589. Retrieved from http://www.springerlink.com/content/m085684224442310/ Ge, X., & Land, S. (2004). A conceptual framework for scaffolding ill-structured problem-solving processes using question prompts and peer interactions. Educational Research Technology and Development, 52(2), 1042-1629. Retrieved from http://www.springerlink.com/content/a1m94820663g6g18/ Gillespie, C. (1990). Questions about student-generated questions. Journal of Reading, 34(4), 250-257. Retrieved from http://www.jstor.org/stable/40014543 Hannafin, M. J., Land, S., & Oliver, K. (1999). Open Learning Environments: Foundations, methods, and models. In C. Reigeluth (Ed.), Instructional-design theories and models (Vol. 2 pp. 115-140). Mahwah, NJ: Erlbaum. Australasian Journal of Educational Technology, 2013, 29(3). 430 Hirai, Y., & Hazeyama, A. (2007). Concerto II: a learning support system based on question-posing. In J. M. Spector, D. G. Sampson, T. Okamoto, Kinshuk, S. A. Cerri, M. Ueno, & A. Kashihara (Eds.), Proceedings of the 7th IEEE International Conference on Advanced Learning Technologies (pp. 338– 339). Washington, DC: IEEE Computer Society Press. Retrieved from http://doi.ieeecomputersociety.org/10.1109/ICALT.2007.101 Hmelo, C., Holton, D., & Kolodner, J. (2000). Designing to learn about complex systems. The Journal of the Learning Sciences, 9(3), 247-298. Retrieved from http://dx.doi.org/10.1207/S15327809JLS0903_2 Jackson, S. L., Krajcik, J. S., & Soloway, E. (1998). The design of guided learner-adaptable scaffolding in interactive learning environments. In C.-M. Karat, A. Lund, J. Coutaz, & J. Karat (Eds.), Proceedings of ACM CHI '98 Human Factors in Computer Systems. New York, NY: Association for Computing Machinery. Retrieved from http://dx.doi.org/10.1145/274644.274672 Kauffman, D. F., Ge, X., Xie, K., & Chen, C.-H. (2008). Prompting in web-based environments: Supporting self-monitoring and problem solving skills in college students. Journal of Educational Computing Research, 38(2), 115-137. Retrieved from http://baywood.metapress.com/link.asp?id=b175688145g31332 Kapur, M. (2008). Productive failure. Cognition and Instruction, 26, 379-424. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/07370000802212669 Kapur, M. (2011). A further study of productive failure in mathematical problem solving: Unpacking the design components. Instructional Science, 39(4). 561-579. Retrieved from http://www.springerlink.com/content/76x5626155k8t7k6/ King, A. (1991). Improving lecture comprehension: Effects of a metacognitive strategy. Applied Cognitive Psychology, 5, 331-346. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/acp.2350050404/abstract King, A. (1992). Facilitating elaborative learning through guided student-generated questioning. Educational Psychologist, 27(1), 111-126. Retrieved from http://dx.doi.org/10.1207/s15326985ep2701_8 King, A. (1994). Guiding knowledge construction in the classroom: Effects of teaching children how to question and how to explain. American Educational Research Journal, 31(2), 338-368. Retrieved from http://aer.sagepub.com/content/31/2/338.abstract King, A. & Rosenshine, B. (1993). Effects of guided cooperative questioning on children's knowledge construction. Journal of Experimental Education, 61(2), 127-148. Retrieved from http://www.jstor.org/stable/20152365 Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry- based teaching. Educational Psychologist, 41(2), 75–86. Retrieved from http://dx.doi.org/10.1207/s15326985ep4102_1 Koedinger, K., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239-264. Retrieved from http://www.springerlink.com/content/u8k26g2j24p37168/ Lajoie, S. P., Lavigne, N. C., Guerrera, C., & Munsie, S. D. (2001). Constructing knowledge in the context of BioWorld. Instructional Science, 29(2), 155-186. Retrieved from http://link.springer.com/article/10.1023%2FA%3A1003996000775?LI=true Lan, Y. F., & Lin, P. C. (2011). Evaluation and improvement of students question posing ability in a web- based learning environment. Australasian Journal of Educational Technology, 27(4), 581–599. Retrieved from http://www.ascilite.org.au/ajet/ajet27/lan.html Australasian Journal of Educational Technology, 2013, 29(3). 431 Lin, X., Hmelo, C., Kinzer, C. K., & Secules, T. J. (1999). Designing technology to support reflection. Educational Technology Research and Development, 47(3), 43-62. Retrieved from http://link.springer.com/article/10.1007%2FBF02299633 Liu, Y. H., & Yu, F. Y. (2004). Active learning through student generated questions in physics experimentation classrooms. Proceedings of the International Conference on Engineering Education, October 16-21, University of Florida, Gainesville, FL. Retrieved from http://www.ineer.org/Events/ICEE2004/Proceedings/Papers/170_ICEE2004acitve_learning_via_quest ion-posing_%281%29.pdf Livingston, J. A. (2003) Metacognition: Overview (ERIC Document Reproduction Service No. ED474273). Retrieved from http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED474273 Luxton-Reilly, A. (2012). The design and evaluation of StudySieve, a tool that supports student- generated free-response questions, answers and evaluations. (Unpublished doctoral dissertation). The University of Auckland, Auckland, New Zealand. Mok, M. M. C. (2005, December). A conceptual framework for the design of web-based self-directed learning curriculum. Keynote presented at the International Conference on Education and Information Technology, National Ocean University, Keelung, Taiwan. Moses, B. M., Bjork, E., & Goldenberg, E. P. (1993). Beyond problem solving: problem posing. In S. I. Brown & M. I. Walter (Eds.), Problem posing: reflections and applications (pp. 178-188). NJ, Hillsdale: Lawrence Erlbaum Associates. Oh, P. (2005). Discursive roles of the teacher during class sessions for students presenting their science investigations. International Journal of Science Education, 27(15, 16), 1825–1851. Retrieved from http://dx.doi.org/10.1080/09500690500239714 Pathak, S., Kim, B., Jacobson, M. J., & Zhang, B. (2009). Failures and successes in collaborative inquiry: Learning the physics of electricity with agent-based models. In C. O'Malley, D. Suthers, P. Reimann, & A. Dimitracopoulou (Eds.), Computer-supported collaborative learning practices: CSCL2009 conference proceedings (pp. 199-203). Rhodes, Greece: International Society of the Learning Sciences. Retrieved from http://dl.acm.org/citation.cfm?id=1600083 Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181-221. Retrieved from http://rer.sagepub.com/content/66/2/181.abstract Rummel, N., & Kramer, N. (2010). Computer-supported instructional communication: A multidisciplinary account of relevant factors. Educational Psychology Review, 22, 1-7. Retrieved from http://www.springerlink.com/content/k6v311xr52755717/ Sandoval, W. A., & Reiser, B.J. (2004). Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88(3), 345-372. Saye, J. W., & Brush, T. (2002). Scaffolding critical reasoning about history and social issues in multimedia-supported learning environments. Educational Technology Research and Development, 50(3), 77-96. Retrieved from http://www.springerlink.com/content/m138727lg24472jw/ Seethaler, S., & Linn, M. (2004). Genetically modified food in perspective: An inquiry-based curriculum to help middle school students make sense of tradeoffs. International Journal of Science Education, 26(14), 1765-1785. Retrieved from http://dx.doi.org/10.1080/09500690410001673784 Sharma, P., & Hannafin, M. (2007). Scaffolding in technology-enhanced learning environments. Interactive Learning Environment, 15(1), 27-46. Retrieved from http://dx.doi.org/10.1080/10494820600996972 Australasian Journal of Educational Technology, 2013, 29(3). 432 Squire, K. D., & Jan, M. (2007). Mad city mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), 5-29. Retrieved from http://dx.doi.org/10.1007/s10956-006-9037-z Steffe, L. P. (1991). The constructivist teaching experiment: Illustrations and implications. In E. von Glasersfeld (Ed.), Radical constructivism in mathematics education (pp. 177-194). Dordrecht: Kluwer Academic Publishers. Retrieved from http://rd.springer.com/chapter/10.1007/0-306-47201-5_9 Steffe, L. P., & Kieren, T. (1994). Radical constructivism and mathematics education. Journal for Research in Mathematics Education, 25(6), 711-733. Retrieved from http://www.jstor.org/stable/749582 Swanson, H. L. (1999) Instructional components that predict treatment outcomes for students with learning disabilities: support for a combined strategy and direct instruction model. Learning Disabilities Research & Practice, 14(3), 129-140. Retrieved from http://dx.doi.org/10.1207/sldrp1403_1 Tabak, I., Smith, B. K., Sandoval, W. A., & Reiser, B. J. (1996). Combining general and domain-specific strategic support for biological inquiry. In C. Frasson, G. Gauthier, & A. Lesgold (Eds.), Intelligent Tutoring Systems (pp.288-297). Berlin, Germany: Springer-Verlag Retrieved from http://www.springerlink.com/content/71777131h432784m/ Torrance, E. P. (1974). Torrance tests of creative thinking. Bensenville, IL: Scholastic Testing Service. Tung, K. J. (2005). The design and evaluation of online student question-generation learning system. (Unpublished Masters thesis). National Central University, Taiwan. von Glasersfeld, E. (1987). The construction of knowledge: Contributions to conceptual semantics. Seaside, CA: Intersystems Publications. Vreman-de Olde, C., & de Jong, T. (2006). Scaffolding learners in designing investigation assignments for a computer simulation. Journal of Computer Assisted Learning, 22(1), 63-73. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2729.2006.00160.x/abstract Whitin, P. (2004). Promoting problem-posing explorations. Teaching Children Mathematics, 11, 180- 186. Retrieved from http://www.nctm.org/publications/article.aspx?id=21560 Wilson, E. V. (2004). ExamNet asynchronous learning network: augmenting face-to-face courses with student-developed exam questions. Computers & Education, 42(1), 87-107. Retrieved from http://dx.doi.org/10.1016/S0360-1315(03)00066-6 Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology & Psychiatry & Allied Disciplines, 17, 89-100. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.1976.tb00381.x/abstract Wu, H. L., & Pedersen, S. (2011). Integrating computer- and teacher-based scaffolds in science inquiry. Computers & Education, 57(4), 2352-2363. Retrieved from http://dx.doi.org/10.1016/j.compedu.2011.05.011 Yu, F. Y. (2005). Promoting metacognitive strategy development through online question-generation instructional approach. In C. K. Looi, D. Jonassen, & M. Ikeda, (Eds.), Towards sustainable and scalable educational innovations informed by the learning sciences (pp. 564-571). Amsterdam, the Netherlands: IOS Press. Retrieved from http://dl.acm.org/citation.cfm?id=1563407 Yu, F. Y. (2009). Scaffolding student-generated questions: Design and development of a customizable online learning system. Computers in Human Behavior, 25(5), 1129-1138. Retrieved from http://dx.doi.org/10.1016/j.chb.2009.05.002 Australasian Journal of Educational Technology, 2013, 29(3). 433 Yu, F. Y. (2011). Multiple peer assessment modes to augment online student question-generation processes. Computers & Education, 56(2), 484-494. Retrieved from http://dx.doi.org/10.1016/j.compedu.2010.08.025 Yu, F. Y., & Hung, C. C. (2006, October). An empirical analysis of online multiple-choice question- generation learning activity for the enhancement of students' cognitive strategy development while learning science. In T. Simos & G. Maroulis (Eds.), Lecture Series on Computer and Computational Sciences: Recent Progress in Computational Sciences and Engineering (pp. 585-588). Chania, Crete, Greece. Yu, F. Y., & Liu, Y. H. (2005). Potential values of incorporating multiple-choice question-construction for physics experimentation instruction. International Journal of Science Education, 27(11), 1319- 1335. Retrieved from http://dx.doi.org/10.1080/09500690500102854 Yu, F. Y., & Liu, Y. H. (2008). The comparative effects of student question-posing and question- answering strategies on promoting college students' academic achievement, cognitive and metacognitive strategies use. Journal of Education and Psychology, 31(3), 25-52. Yu, F. Y., & Liu, Y. H. (2009). Creating a psychologically safe online space for a student-generated questions learning activity via different identity revelation modes. British Journal of Educational Technology, 40(6), 1109-1123. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1467- 8535.2008.00905.x/abstract Yu, F. Y., Liu, Y. H., & Chan, T. W. (2002, December). The efficacy of a web-based domain independent question-posing and peer assessment learning system. Proceedings of International Conference on Computers in Education (pp. 641-642). Auckland, New Zealand. Retrieved from http://dl.acm.org/citation.cfm?id=838932 Yu, F. Y., Liu, Y. H., & Chan, T. W. (2005). A web-based learning system for question-posing and peer assessment. Innovations in Education and Teaching International, 42(4), 337-348. Retrieved from http://dx.doi.org/10.1080/14703290500062557 Zydney, J. M. (2010). The effect of multiple scaffolding tools on students' understanding, consideration of different perspectives, and misconceptions of a complex problem. Computers & Education, 54, 360-370. Retrieved from http://dx.doi.org/10.1016/j.compedu.2009.08.017 Corresponding author: Fu-Yun Yu, fuyun.ncku@gmail.com Australasian Journal of Educational Technology © 2013. Please cite as: Yu, F. Y., Tsai, H. C., & Wu, H. L. (2013). Effects of online procedural scaffolds and the timing of scaffolding provision on elementary Taiwanese students' question-generation in a science class. Australasian Journal of Educational Technology, 29(3), 416-433.