chou-cm.pdf Australasian Journal of Educational Technology 2012, 28(4), 719-739 Influence of teachers’ perceived e-portfolio acceptance on teacher evaluation effectiveness in Taiwan Chun-Mei Chou National Yunlin University of Science and Technology This study examines technological and vocational school teachers’ perceived e- portfolio acceptance, computer self-efficacy, and evaluation effectiveness in Taiwan. Teachers’ perceived e-portfolios acceptance includes four factors, namely, staff commitment, performance expectancy, performance expectancy, and technology training. Computer self-efficacy includes four factors, namely, playfulness, ease of use, effectiveness, and usefulness. Evaluation effectiveness includes four factors, namely, classroom management, instructional design, dedicated attitude, professional development. Participants (N=318) responded to a 5-point Likert-type scale for each factor. Analysis was conducted using the structural equation modeling (SEM), and a good model fit was found for both the measurement and structural models. Findings demonstrate that technological and vocational schools teachers’ e-portfolios acceptance significantly and directly influences evaluation effectiveness, and e- portfolio acceptance influences evaluation effectiveness by computer self-efficacy. The teachers’ e-portfolio acceptance and computer self-efficacy fit the influence model and empirical data of evaluation effectiveness. Introduction Taiwanese vocational and technical schools seek to meet the needs of outstanding education and teacher evaluations and promote teacher evaluation systems. Since the 2006 academic year, vocational schools have been subsidised by the Ministry of Education to engage in a trial run of the “Teacher Professional Development Evaluation Implementation Plan” (Ministry of Education, 2008). Teacher evaluation methods are divided into teacher self-evaluation and intramural evaluation, the purposes of which are to supervise teacher professional development while providing suitable assistance. The school provides affirmation and feedback for teacher professional performance based on the evaluation results, and also can understand the overall teacher development needs and provide teachers with opportunities for training while on the job (Harvey & Williams, 2010; Sung, Chang, Yu & Chang, 2009; Yao, Aldrich, Foster & Pecina, 2009). Beginning in 2009, the Ministry of Education promulgated modifications, deleting the trial and implementing a plan for expanded implementation during the next four years (Ministry of Education, 2010). It was primarily voluntary, but schools were encouraged to participate. The schools that applied for the trials had to submit evaluation reports, but these reports only included statistical data, and it was unclear whether there were positive effects on teacher professional development and evaluation effects. One future trend in education is to cope with changes in the external environment through making appropriate adjustments, with the assistance of teacher professional 720 Australasian Journal of Educational Technology, 2012, 28(4) development evaluations, to better develop education quality and the professional know-how of teachers (Tsai, Tsai & Hwang, 2011; Yao, Aldrich, Foster & Pecina, 2009). The purpose of teacher professional development evaluations is to ensure instructional quality, and establish standards and performance levels for teacher professional development (Meyer, Abrami, Wade, Aslan & Deault, 2010). Ifinedo (2006) suggested that teaching portfolios should meet the three requirements of performance evaluation, the promotion of teacher professional development, and giving teachers better chances to obtain employment. Through the systematic collection of data on teacher instruction, student counseling, administration and other services, teaching portfolios can be used to verify teacher professional development. Teacher e-portfolios can also be used to evaluate the results of student learning, can be used to evaluate teacher professional development and instructional performance, and can also be used as criteria for teacher license renewal (Joyes, Gray & Hartnell-Young, 2010; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011; Swan, 2009). Some studies have suggested that teachers’ perceived e-portfolio acceptance will affect teacher evaluation effectiveness (Garrett, 2011; Nicholson, 2004; Seldin, Miller, Seldin, McKeachie, 2010). From the social cognitive perspective, teacher acceptance of e- portfolios is affected by factors such as social pressure and the school environment, which in turn affect teacher motivation, attitude, and the adoption of e-portfolios (Bandura, 2007; 2008; Saade & Kira, 2009; Shepherd & Bolliger, 2011). A number of schools have established e-portfolio and web-portfolio symposium websites in the hopes of effectively promoting teacher professional evaluation systems to help enhance school effectiveness (Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011; Wong & Li, 2011). Teacher perceptions of the usage of e-portfolios not only improve actual teaching performance, but also increase the information and feedback from student learning accomplishments. In addition, the perceived usage of teacher evaluation systems can be used to remove unsuitable teachers (Chang & Tseng, 2009). In terms of the theory of planned behaviour, it is possible to explore teachers’ personal voluntary behavior, external influences, and the reasons for obstacles when they adopt new technology or accept innovative management mechanisms. The theory of planned behaviour stresses teachers’ perceived behavioural control. When observing behavioural intentions, attitude, subjective norms, and perceived behavioural control will all affect personal behavioural intentions and in turn affect behaviour (Chai & Lim, 2011; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011). Teachers’ perceived e-portfolio acceptance will in turn affect their perceived behavioural control. This reflects their past experiences and expected obstacles, which are adjusted through internal and external personal behavioural controls and then shown by their teacher evaluation effectiveness. Most studies relating to teacher e-portfolios have focused on student learning accomplishments (Chang, 2008; Chang & Tseng, 2009; Chau & Cheng, 2010), teacher training (Chuang, 2008), instructional accomplishments (Avraamidou & Zembal-Saul, 2002), the acceptance of information (Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011), technological leadership (Chang, Chin & Hsu, 2008), and instructional reflections (Sung, Chang, Yu & Chang, 2009; Strudler & Wetzel, 2005), but there have been few studies that have explored the extent to which working teachers accept professional evaluations and the testing of their evaluation effects. In addition, it is difficult to understand the professional development of teachers, and further evaluation of its effects is even more difficult. Evaluation is a formal procedure, but in professional development evaluation there are only documentary records without actual evaluation; the evaluation content is too superficial and tends to be Chou 721 focused on surface effects. Thus, the influence of teachers’ perceived e-portfolio acceptance on teacher evaluation effectiveness is an important issue in the exploration of teacher evaluation effectiveness. Schools promote teaching portfolio measures in order to achieve the needs of teacher professional evaluations, allowing teachers to have effective management and production. Additionally, teachers’ computer self-effectiveness abilities have become increasingly important (Nicholson, 2004; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011). The internal factors that affect teachers’ computer self-effectiveness include personal differences, information techniques and abilities, personal will, attitude cultivation, and memory. Among these, some can be improved through educational training and experience (Guriting, Chunwen & Ndu, 2007; Vajargah & Jahani, 2010; Koc & Bakir, 2010; Saade & Kira, 2009). The external factors that affect teachers’ computer self-effectiveness include time, opportunities, and assistance from others. Changes in time, context, or strategic transformations cause new behavioural intentions (Anderson & Maninger, 2007; Guriting, Chunwen & Ndu, 2007; Hoffer & Alexander, 1992; Koc & Bakir, 2010). The technology acceptance model aims to explain the decisive factors of users’ acceptance of information technology, and analyses the influences of external factors on users’ beliefs, attitudes, and intentions concerning technology use (Saade & Kira, 2009; Teo, 2009a, 2009b). Analysis of the factors of teachers’ evaluation effectiveness, through e-portfolio acceptance and computer self-effectiveness, can help to probe into their perceived usefulness and ease of use. These two factors will influence users’ attitudes, behavioural intentions, and usage behaviours toward technology. Through the above, the perceptions of teachers when they participate in e-portfolios, the influence of teacher perceptions of e-portfolios on teacher evaluation effectiveness, and whether teachers’ computer self-effectiveness is beneficial for teachers, can all be shown. Analysis of the factors that influence teacher evaluation effectiveness could help schools to better understand vocational and technical school teacher needs in e- portfolio and professional development evaluations, in order to engage in improvements of digital instruction and instructional guidance and cope with the urgent needs of vocational school teacher professional development and the sustainable management of schools. Literature review E-portfolio acceptance Electronic portfolios (e-portfolios) refer to the use of computer technology to record and store learning portfolio files. These may include multimedia forms such as text, pictures, images, and sounds, for the display of richer and more diverse file contents (Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011). When e-portfolio learning file contents are shown, managed and produced through the Internet, they are referred to as web-based portfolios. In this article, teacher e-portfolios refers to teachers displaying their teaching portfolios on the World Wide Web, where they can create and update personal learning portfolio files through their browsers. In the context for this research, effective e-portfolios should contain: (1) a description of teaching responsibilities; (2) reflections on the teaching objectives; (3) a collection of instructional data; (4) student, peer and administrative evaluations and the subsequent 722 Australasian Journal of Educational Technology, 2012, 28(4) reflection; and (5) descriptions of instructional activity improvements. The data collected for teacher e-portfolios may come from the process of instruction, including instructional design, class management, research development and advancement, and from the accumulated results of a diligent spirit, as well as from evidence of teachers’ personal reflections. Teacher instructional portfolios may be divided into seven types: showcase portfolios, progress portfolios, process/product portfolios, reflective portfolios, rhetorical portfolios, teaching portfolios, and accreditation portfolios. Among these, accreditation portfolios may take the form of university or department evaluations; the angle and needs of the evaluation encompass the perspectives of universities or departments (Lee, 2009; Shepherd & Bolliger, 2011; Shroff, Deneen & Ng, 2011; von Konsky & Oliver, 2012). Teachers’ e-portfolio acceptance indicates that according to instructional content and strategies, they have abilities in applying computer media, the Internet, and information technology as instructional tools (Garrett, 2011; Hsu, Ju, Yen, & Chang, 2007; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011). According to the Technology Acceptance Model (TAM), the determining factors for teacher acceptance of information technology include the effect of external factors on internal beliefs, attitudes, and intentions, which affect the conditions of technological use. Scholars believe that the factors affecting teacher acceptance of teacher evaluation policies can be explored from the three aspects of teachers, policies, and environmental variables. Among these, teacher factors can be divided into personal background factors and personal work factors, such as staff commitment and expected level of effort. Policy factors include performance expectancies, and school environment variables include school size or location, and technology training. Computer self-effectiveness Computer self-effectiveness or efficacy refers to the personally perceived ability to use computer technology effectively to express behaviour. It also refers to the determination and perception of an individual toward one’s own ability to use a computer to complete a certain task (Ahmad, Basha, Marzuki, Hisham & Sahari, 2010; Wilfong, 2006). Computer self-effectiveness is concerned with the perceived ability of an individual relating to computers, and it emphasises determination to finish a task with a computer. When users have more experience using information systems, they have higher computer self-effectiveness (Guriting, Chunwen, Ndu, 2007; Koc & Bakir, 2010; Saade & Kira, 2009). Davis (1989) referred to the Technology Acceptance Model (TAM) according to Fishbein and Ajzen's Theory of Reasoned Action (TRA). TRA theory is that human use of appropriate information is often quite rational and systematic (Fishbein & Ajzen, 1975; Ajzen, 1988). The technology acceptance model theory indicates that perceived usefulness of information technology (perceived usefulness), and ease of use (perceived ease of use) are the two main determinants of intent (usage) behavior. This includes the four variables of perceived usefulness, confirmation, satisfaction, and willingness for continued use. It believes that perceived usefulness and satisfaction will affect willingness for the continued use of information systems. Perceived usefulness and confirmation will also affect user satisfaction (Ifinedo, 2006; Larsen, Sørebø & Sørebø, 2009; Ryu, Kim & Lee, 2009). Therefore, teacher willingness to use e- portfolios will affect satisfaction and perceived usefulness, and will indirectly influence the confirmation factor of willingness of use. Chou 723 Some studies have pointed out that computer self-effectiveness has a decisive influence on teacher instructional behaviour. The sense of effectiveness produced by computer self-effectiveness is not only related to the future instructional context of computer use by teachers, but will also affect whether they are willing to understand computers and apply them to instructional activities. If there is low teacher computer self-effectiveness, they may resist using computers for e-portfolios. Teachers who use technology in schools and have a positive attitude and a sense of effectiveness toward computers can help students have positive attitudes toward computer technology, which will help the school promote professional development evaluations with e- portfolios. When technology assists teachers in recording student learning processes and in complex preparation work, it can help teachers in completing e-portfolio databases. Teachers can then have more time to analyse student needs and engage in individualised instruction. When teachers are more familiar with the characteristics of technology and resources, they can easily use technology to improve personal instruction professionalism and teacher professional evaluation effectiveness. This shows that the level of teacher computer effectiveness not only influences personal computer behavior and learning intentions, but can also help with e-portfolio acceptance and teacher evaluation effectiveness. Teacher evaluation effectiveness Teacher evaluation refers to the value determination of teacher performance and the decision-making process, which assists with teacher professional development, in order to understand instructional performance and promote school reform. The purpose of teacher evaluation is to enhance teacher instructional effectiveness, inspire teacher professional development, and remove unsuitable teachers. There are two characteristics of teacher evaluations. The first is a summary evaluation, inclined toward evaluation methods with a performance responsibility orientation. The second is a formative evaluation, inclined toward evaluation methods with a professional development orientation. Teacher evaluations can be assessed from the three aspects of teacher competence evaluations and teacher quality evaluations, teacher performance evaluations, and teacher effectiveness evaluations (Cai & Lin, 2006; Chai & Lim, 2011). Teacher evaluation indicators include: (1) a focus on the planning and implementation of instruction; (2) evaluation, including instructional evaluation, feedback, and improvement; (3) planning a learning environment that is safe and conducive to learning; (4) enhancing communication and connections with parents and communities; and (5) an emphasis on professional ethics and development. In Taiwan, the teacher professional development evaluation indicators are divided into course design and instruction, class management and counseling, research development and advancement, and a diligent spirit and attitude. The perspective of Task-Technology Fit (TTF) states that for information technology to effectively improve personal work performance, such technology must be accepted by the users, and it should accommodate the supported tasks. In other words, if the technological functions to be used by the teacher can make the task smoother, the user’s costs in carrying out the task will be lowered. Only when it can support the task to be completed will there be good work performance (Wong & Li, 2011). 724 Australasian Journal of Educational Technology, 2012, 28(4) Teachers’ personal evaluations do not consider the effect of the organisation on teacher evaluation and professional development, which can contribute to adverse teacher evaluation effects. The procedures and content of teaching portfolio evaluations should consider the content items and performance standards to be evaluated, the scope and size of the files, a concrete explanation of the requirements of the constructed files, and a definitive evaluation of the file content. Anderson & Maninger (2007) studied the effects of teacher professional development evaluations, exploring the five aspects of participant responses, participant learning, organisational support and change, new know-how from participant usage, and student learning results. In summary, teacher evaluation refers to the value determination of teacher performance and the decision-making process, which assists with teacher professional development, in order to understand instructional performance and promote school reform. Through professional development evaluation, teachers are inspired to engage in teaching reflection and the pursuit of accomplishments, giving them a willingness and motivation for development that can effectively improve teacher instructional quality and student learning accomplishments. Teacher professional development evaluations utilise diverse evaluation methods, collecting performance data on course design and instruction, classroom management and counseling, research development and advancement, and a diligent spirit and attitude. Through the establishment of teacher professional development evaluations mechanisms, teacher instruction professionalism and its effects can be measured with objective evaluation standards. Influence of teachers using e-portfolios to improve the evaluation of teacher effectiveness In e-portfolio professional performance, teacher self-selections and self-reflections can reflect teacher autonomy and individuality. In the instructional process, e-portfolios encourage teachers to collect and record data in a focused and organised way, regarding personal instructional know-how and performance (Ahmad, Basha, Marzuki, Hisham & Sahari, 2010). Chang and Tseng (2009) indicate a higher acceptance of e-portfolios, there can be better promotion of teacher practices through continuous reflection and thought, as well as enhanced personal professionalism, to guide the enhancement of instructional effectiveness and achieve the objectives of teacher instruction (Anderson & Maninger, 2007). Through the criteria of establishing e-portfolios and the digitisation process, there can be participation in teacher evaluations, and systematic, conscious, and responsible efforts can be used to promote teacher instructional performance (Ahmad, Basha, Marzuki, Hisham & Sahari, 2010). Chang (2008) and Churchill (2009) pointed out that e-portfolios as formative evaluations for teachers to improve their instruction can promote teacher instructional effectiveness and achieve teaching objectives. Instructional effectiveness is the evaluation of personal instructional abilities and the extent of influence for student learning in the course of teaching (Churchill, 2009; Davis, 1989). This type of ability evaluation of computer self-efficacy is a subjectively conscious perception or belief, which will influence personal behaviour (Ahmad, Basha, Marzuki, Hisham, & Sahari, 2010; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011; Tsai, Tsai & Hwang, 2011). Teachers’ perceived e-portfolios acceptance or beliefs in their own e-teaching ability evaluation will affect their instructional behaviour. Furthermore, teacher computer self-efficacy can also explain their instructional behaviour and teacher evaluation Chou 725 effectiveness (von Konsky & Oliver, 2012; Swan, 2009). Highly effective teachers use more difficult and challenging instructional techniques and execute innovative plans. In other words, there is a positive correlation between instructional effectiveness and instructional behaviour (Garrett, 2011; von Konsky & Oliver, 2012). The development of these records and the traces of their gradual deployment can be recorded with teaching portfolios; teachers can continue to elevate their sense of instructional effectiveness from reflecting on the files, and can promote the realisation of teaching objectives. Teacher performance responsibility and professional development conversion is the trend in school education development. In school education and in teacher professional development, teaching portfolios can play a proactive and intermediary role (Ahmad, Basha, Marzuki, Hisham & Sahari, 2010). Through the establishment of teaching portfolios, teachers can provide records of professional development and proof of their accomplishments. In this way, teacher professional development is guaranteed, student learning accomplishments are assured, and instruction can be affirmed by the public at large (Wong & Li, 2011). In sum, the establishment of teaching portfolios is being guided by teacher evaluation and diverse evaluation considerations in education, and they have gradually become a part of teacher professional development. Teachers should proactively consider and reflect upon the meaning of teaching portfolios for class instruction and school education. At the same time, the establishment of teaching portfolios can lead to personal professional development, and they can cope with future trends in school education applications. Purposes of this study The purposes of this study are to address the following issues in teacher acceptance of e-portfolios and effectiveness of teacher evaluation: 1. What are the relationships between technological and vocational school teachers' perceived e-portfolios acceptance, computer self-efficacy, and teacher evaluation effectiveness? 2. Can we identify a suitable model that relates these factors and helps to identify important implications for using e-portfolios to improve the evaluation of teacher effectiveness? Methodology Research design This study employed structural equation modelling (SEM) to analyse the relationships between technological and vocational school teachers’ perceived e-portfolios acceptance, computer self-efficacy, and teacher evaluation effectiveness. Data were collected through a survey questionnaire containg questions on demographics and multiple items for each construct in the study. Normal distribution testing of the related variables in the model of this study is shown in Table 1. Although all observation variables do not reach normal distribution (p<.05), multi-variance normal test is insignificant (p>.05), which demonstrates normal distribution. According to the 726 Australasian Journal of Educational Technology, 2012, 28(4) conditions of maximum likelihood (ML), within the most commonly used approach in SEM, one of the conditions should be a simple random sampling that meets multi- variance normal distribution. Samples of this study meet the conditions of ML. Some scholars suggest that it is influential only when kurtosis is above 25; therefore, when kurtosis is below 25, ML is still an applicable analysis (Bagozzi & Yi, 1988; Bentler & Bonett, 1980; Hair, 2010). Boomsma and Hoogland (2001) compared and probed into the completeness of different estimation methods, and concluded that in regard to models with observation variables above 6 or 8 with non-normal distribution, ML has better statistical traits. Thus, although overall variables of this study meet the multi- variance normal distribution of ML, a single variable does not meet normal distribution. Kurtosis of multi-variance distribution is not large (<25); therefore, this research estimates the model by ML. Table 1: Mean and standard deviations of variables and normal distribution test Variables Mean SD Skewness Kurtosis X2 p-value Staff commitment 4.12 .61 -.14 .11 3713.07 .000 Effort expectancy 4.01 .55 -.21 -.25 3924.11 .000 Performance expectancy 4.11 .52 -.37 .34 4823.59 .000 Technology training 3.76 .62 -.36 .19 3418.26 .000 Classroom management 4.14 .61 .23 .31 3361.76 .000 Instructional design 4.26 .56 -.27 -.54 5235.35 .000 Dedicated attitude 4.23 .61 -.36 -.61 4128.56 .000 Professional development 4.12 .47 -.32 -.24 3879.51 .000 Playfulness 4.24 .59 -.24 -.34 3648.22 .000 Ease of use 4.18 .53 .13 .28 4279.29 .000 Effectiveness 3.65 .65 -.12 .11 3721.54 .000 Usefulness 3.88 .59 -.19 -.13 4013.62 .000 Total 4.03 .58 -.27 -.12 230.21 1.000 Participants This study treated teachers of technological and technological and vocational schools as the population, and adopted random sampling and cluster sampling for a survey. A total of 318 valid samples were collected, and the analysis of their gender, seniority, current post, current level, teaching background, school attributes, school category, and number of students is shown in Table 2. Measures A 42-item survey questionnaire was developed to measure participants’ e-portfolio acceptance, computer self-efficacy, and teacher evaluation effectiveness. The scale for e-portfolios acceptance included five constructs, namely, staff commitment (3 items), effort expectancy (3 items), performance expectancy (3 items), and technology training (4 items). The scale for computer self-efficacy included playfulness (3 items), ease of use (3 items), effectiveness (3 items), and usefulness (3 items). The scale for evaluation effectiveness included classroom management (4 items), instructional design (4 items), dedicated attitude (5 items), and professional development (4 items). The items reflected in the questionnaire could allow participants to take reference from their personal experiences when responding. Each item was measured on a five-point Likert scale of 1=strongly disagree to 5=strongly agree. A total of 13 items were used to measure e-portfolios acceptance, 12 items were used to measure computer self-efficacy, and 21 items measure evaluation effectiveness. These items are listed in Table 1. Chou 727 Table 2: Distribution of participants’ background in formal scales (N=318) Basic information Group No of people % Male 184 58%Gender Female 134 42% University (or below) 38 12% Master 76 24% Educational background Doctor 204 64% 5 years (and below) 58 18.3% More than 5 years and less than 10 years 88 27.6% More than 10 years and less than 15 years 73 23.0% Seniority Over 15 years 99 31.2% Full time teachers and administration staff 137 43.2%Current post Full time teachers 181 55.8% Professor 29 9.2% Associate professor 124 39.1% Assistant professor 73 23.1% Current level Lecturer 94 29.6% Science, engineering, agriculture and design 115 36.3% Business, management, tourism and recreation 130 41.0% Livelihood, health and medical care 32 10.1% Teaching background Liberal arts, law, education and general knowledge 41 12.6% Public 169 53.3%School attributes Private 149 46.7% Uni. of technology (inc. Coll. of technology) 170 46.8%School category Vocational school 148 53.2% 5,000 (and below) 136 42.7% More than 5,001 and less than 10,000 115 36.1% Number of students Above 10,000 67 21.2% Research tool The research tool was the “Investigation of factors in technological and vocational school teachers’ teacher evaluation effectiveness”. The compilation of this scale was based on the concepts from the computer self-efficacy scale by Karsten & Roth (1998), e-portfolios acceptance by Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck (2011) and Saade & Kira (2009), and the teacher evaluation effectiveness scale by Churchill (2009) and Anderson and Maninger (2007). In this study, four experts evaluated the fitness of the questions in order to verify the expert fitness of the scale. Ten teachers from technological and technological and vocational schools were invited to answer the questionnaire in order to enhance face validity. Five technological and technological and vocational schools were selected for a pre-test, with 115 teachers as the subjects. A total of 108 valid samples were collected; with a valid return rate of 93.9%. The scale in this study was a self-reported inventory, based on a Likert 5-point scale, where the range of “agree” to “disagree” is denoted by 5 to 1, respectively. The factor names, number of items, validity, and reliability levels of each aspect in this scale are as shown in Table 3. Data analysis Regarding data processing of formal survey, the returned questionnaires were coded. Linear structural relations (LISREL) was used to validate the correlation and influences among creative teaching self-efficacy, e-portfolios acceptance, and creative teaching 728 Australasian Journal of Educational Technology, 2012, 28(4) effect by Statistical Package for Social Sciences, SPSS 10.0. The statistical test criterion in this study is α = 0.05. Table 3: Factors, number of items, validity and reliability of technological and vocational school teachers’ perceived e-portfolios acceptance and computer self-efficacy on scale for teachers’ evaluation effectiveness. Factor Compositionof scales No of items Factor loading Cronbach alpha Accumu- lated explained variance KMO Total reliability Cronbach alpha Staff commitment 3 27.61 .91 Effort expectancy 3 12.30 .84 Performance expectancy 3 14.47 .83 Scale of e-portfolio acceptance Technology training 4 11.21 .92 65.59 .901 .93 Playfulness 3 29.13 .89 Ease of use 3 21.54 .90 Effectiveness 3 10.49 .92 Computer self-efficacy Usefulness 3 7.78 .91 68.94 .874 .93 Classroom management 4 24.58 .92 Instructional design 4 23.14 .89 Dedicated attitude 5 10.04 .89 Teacher evaluation effectiveness Professional development 4 8.61 .87 66.37 .881 .92 Results Fit test for influence model of evaluation effectiveness materials This study validated the model by LISREL 8.52. The estimation method was determined after examining the samples, and model estimation was carried out by software. Parameters after software estimation are shown in Table 4. Before the model fit test, whether the estimation coefficient is over the defined scope was verified. Only when parameter coefficients estimated do not violate the estimation can the fit test be conducted. Table 4: Normalised coefficients of path analysis of influence model of technological and technological and vocational schools teachers’ evaluation effectiveness Parameter Standarddeviation t value Normalised coefficient Parameter Standard deviation t value λ1 λ2 λ3 λ4 λ5 λ6 λ7 λ8 λ9 λ10 λ11 λ12 0.65 0.66 0.73 0.71 0.59 0.68 0.69 0.63 0.48 0.63 0.70 0.52 19.39* 18.14* 19.25* 18.09* 11.46* 19.20* 17.89* 15.59* 12.38* 15.49* 18.23* 12.56* 0.25 0.26 0.32 0.26 0.48 0.23 0.36 0.25 0.29 0.27 0.29 0.28 ε1 ε2 ε3 ε4 ε5 ε6 ε7 ε8 ε9 ε10 ε11 ε12 0.34 0.29 0.44 0.33 0.22 0.31 0.39 0.28 0.37 0.29 0.38 0.29 16.20* 14.14* 12.58* 13.01* 13.6* 12.67* 14.10* 11.67* 14.45* 13.55* 14.03* 12.49* Note: Those without standard deviations are criterion indicators of *p<.05 Chou 729 According to the definitions of Hair, Anderson, Tatham and Black (1998), three items can be used to examine estimation violations: 1. negative error variable exists; 2. normalised coefficient is above or approximate to 1 (0.95 is the usual threshold), and 3. significant standard deviation. According to Table 4, parameter estimations reveal positive error variables, and there are no negative variables. Normalised coefficient is 0.25~0.95, and remaining within the scope. Therefore, parameters estimations are not violated, and a fit could be conducted. This study conducted model fit testing by general criteria, according to fit measures estimated by the statistical method, in order to find out the fitness between the research data and model. Measures of this study are based on absolute fit, incremental fit, and parsimonious fit, as classified by Hair, Anderson, Tatham & Black (1998). In addition, construct reliability and validity tests of variables were used to determine the internal structural fit. Analytical results are shown below. Overall fit Based on the above, overall fit can be measured by absolute fit, incremental fit, and parsimonious fit. After estimation by LISREL 8.52, according to Table 5, the chi-square of the model is 213.91, p<.05, which is significant, and shows that there are significant differences between the covariance matrix of the model and the empirical data. Chi- square testing can be easily influenced by the number of samples and normality of the data. Therefore, when evaluating overall model fit, this study includes other measures. This study first examines theoretical validation of the model regarding unsatisfying evaluation standards. After reviewing ML, the model is modified according to the rationality of the theory. Although model fit after modification is enhanced, estimates of usefulness and computer self-efficacy do not reach a level of significance, and re- estimation is required. Although modified chi-squared testing fails to reach a statistical significance level, the model fit is improved and mostly satisfies the standards. In addition, tests of overall fit are generally positive. Table 5: Results of overall model fit test of technological and vocational school teachers’ evaluation effectiveness Evaluation items and outcome (N=318) Evaluation standard Researchers Fit Absolute fit χ2 = 213.91 d.f. = 45 χ2/d.f. = 4.75 GFI = 0.91 AGFI = 0.83 SRMR = 0.09 RMSEA = .012 <5 >0.9 >0.8 <0.1 <0.08 Hair et al. (1998) Hair (2010) Hair (2010) Hu & Bentler (1999) Jarvenpaa et al. (2000) Acceptable Acceptable Acceptable Acceptable Acceptable Incremental fit NFI = 0.92 NNFI = 0.91 IFI = 0.92 CFI = 0.91 >0.9 >0.9 >0.9 >0.9 Bentler & Bonett (1980) Bentler & Bonett (1980) Bentler & Bonett (1980) Bagozzi &Yi (1988) Acceptable Acceptable Acceptable Acceptable Parsimonious fit PNFI = 0.68 PGFI = 0.57 >0.5 >0.5 Bentler & Bonett (1980) Bentler & Bonett (1980) Acceptable Acceptable According to the analytical results of the revised model in Table 5, in absolute fit measures, GFI = 0.91, which is slightly higher than the standard of 0.9. According to Gefen and Straub (2000), GFI should be above 0.90, thus, this model is acceptable. 730 Australasian Journal of Educational Technology, 2012, 28(4) AGFI = 0.83, which is higher than the standard of 0.8. Gefen and & Straub (2000) suggests that AGFI should be above 0.80, thus the model is acceptable. RMSEA = .012 and <0.08. According to Jarvenpaa, Tractinsky and Vitale, (2000), RMSEA should be lower than 0.08, thus, based on the measures above, the absolute fit of this model is good. As to incremental fit and parsimonious fit measures, according to Gefen and Straub (2000) and Hair et al. (1998), when NFI, IFI, RFI and CFI are above 0.9, PNFI and PGFI should be above 0.5 in order for the model to be accepted. According to tghe data tested, NFI = 0.92, which is above the standard 0.9. It shows that the model is accepted. IFI = 0.92, which is above 0.9 means that the model is accepted. RFI = 0.89, which is lower than the standard 0.9, it means that the model is almost acceptable. CFI = 0.91, thus, the model is relatively acceptable. PNFI = 0.68, which is above the standard 0.5; the model is relatively acceptable. PGFI = 0.57, which is above 0.5; the model is acceptable. According to the incremental and parsimonious fit measures above, the models of this study are acceptable. However, absolute fit is good, which suggests that the models are still acceptable, as the overall model meets empirical data. Structural fit Regarding structural model fit, Hair et al (1998) suggested the measurement significance test and the R2 of latent dependent variables of structural parameters. The R2 of dependent variables should be lower than the standard of 0.5, and correlation among the latent variables should be higher than 0.90. Regarding the structural fit test, according to the structural parameters of the influence model of teachers’ evaluation effectiveness materials in Table 4, e-portfolios acceptance and computer self-efficacy are significant (t = 3.13, p <. 05). In addition, as to the evaluation of R2 latent dependent variables, according to Table 40, e-portfolios acceptance, and computer self-efficacy are 0.65 and 0.73, respectively. The R2 of computer self-efficacy is lower than 0.50, which meets the evaluation standards; therefore, the structural model fit of this study is good. Correlation coefficients of the three latent variables are 0.72~0.91 (Table 6 and Figure 1). The correlation coefficient of teacher evaluation effectiveness on e-portfolios acceptance is higher than 0.90, while the remaining are lower than 0.90. Thus, the three latent variables may affect the structural model fit due to overly high correlations. Discussion This study aimed to analyse the correlations among technological and vocational school teachers’ perceived e-portfolios acceptances, computer self-efficacy, and evaluation effectiveness in Taiwan. Technological and vocational school teachers’ e-portfolios acceptance, performance expectancy, and technology training show significant influence on classroom management and instructional design, in teacher’s evaluation effectiveness. Effort expectancy allows teachers to recognise objectives for e-portfolios in schools. When schools encounter environmental changes and competition, teachers gradually understand and trust the measures of the schools’ innovative leadership. Through performance expectancy and rewards by schools, teachers can fulfill their creativity Chou 731 and further develop courses and e-teaching design. This will allow them to accomplish school objectives and instructional design in order to meet environmental changes (Chau & Cheng, 2010; Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011; Damanpour & Wischnevsky, 2006; von Konsky & Oliver, 2012). Table 6: Average variances extracted and the correlation coefficient of latent variables on the influence model of evaluation effectiveness materials Latent variables R2 E-portfolioacceptance Teacher evaluation effectiveness Computer self-efficacy E-portfolio acceptance 0.91 1 Teacher evaluation effectiveness 0.72 0.72 1 Computer self-efficacy 0.81 0.91 0.81 1 Figure 1: Path of technological and vocational school teachers’ evaluation effectiveness materials Technological and vocational school teachers’ e-portfolio acceptance, staff commitment, and technology training show significant influence on ease of use and effectiveness of computer self-efficacy. Instructional design and dedicated attitude in technological and vocational school teachers’ evaluation effectiveness significantly influence playfulness and ease of use of computer self-efficacy. Teachers’ perceived technology training and staff commitment will enhance teachers’ new teaching methods, evaluations, and implementation of e-portfolios. It is one of the key factors on teachers’ computer self-efficacy (Chai & Lim, 2011; Karsten & Roth, 1998; Joyes, Gray & Hartnell-Young, 2010; Mohrman, Cohen & Mohrman, 1995; Swan, 2009). .69 .68 .59 .52 .70 .64 .48 .65 .66 .73 .71 .91* .81 .72 Staff commitment Playfulness Performance expectancy Technology training Effort expectancy Ease of use Effectiveness Usefulness Classroom management Instructional design Dedicated attitude Teacher evaluation effectiveness E-portfolios acceptance Computer self-efficacy .37 .38 .29 .34 .33 .44 .29 .22.31.39 .29 Professional development .28 .63 732 Australasian Journal of Educational Technology, 2012, 28(4) Technological and vocational school teachers’ e-portfolio acceptance significantly and directly influences teacher evaluation effectiveness. E-portfolio acceptance significantly and indirectly influences evaluation effectiveness through computer self-efficacy. Technological and technological and vocational schools teachers’ e-portfolio acceptance and computer self-efficacy fit the model and empirical data of evaluation effectiveness. Therefore, schools should respect and support instructional technology training. Through recognition of e-portfolios and evaluation contents, teachers can constantly ponder on e-portfolio training and information technology. They will enhance instructional design and adopt a dedicated attitude. Teachers’ active promotion of web-based portfolio ideas in e-portfolios and practical use of e- instruction will encourage them in their teaching. They identify with performance expectancy through e-portfolio acceptance in order to develop teachers’ e-portfolio plans, and constantly adopt and implement e-portfolios. Thus, they will result in useful e-portfolios and a more dedicated attitude (Pynoo, Devolder, Tondeur, van Braak, Duyck & Duyck, 2011; Angle & Van de Ven, 2000; Hage & Aiken, 1970; Igbahia & Iivari, 1995; Susser & Ariga, 2006). Although the model of this study reveals goodness of fit, the fit effect remains unsatisfying, which suggest that there are some latent variables that have not been elaborated. Variables in models sometimes fail to reveal the ideal explanatory effect; thus, this study further probes into the model with a more complete overall fit. Limitations of the study This study focused on the influence of technological and vocational school teachers’ perceived e-portfolio acceptance on teacher evaluation effectiveness. Chou, Shen, Hsiao and Chen (2010a) found that technological and vocational school teachers who also undertake administrative jobs tend to identify more closely with e-portfolio acceptance. When teachers undertake administrative jobs, they use the school network system and are engaged in activities of e-processing. Would such activities cause them to identify highly with evaluation effectiveness? This is a limitation of this study. This study measures teachers’ perceptions during the first, middle and final stages of teacher evaluation effectiveness. Some teachers perceived e-portfolios as a means to evaluate teacher effectiveness. Teachers’ perceived computer and evaluations of e- portfolios are insufficient; meaning they may lack knowledge in two scales, namely, “computer self-efficacy” and “evaluation effectiveness”, and any effects between the two would be limited. Scoring according to scales is based on teachers’ self-perceived; therefore, this study cannot eliminate samples that were influenced by situations, attitudes, real respondents, emotions, or are seemingly unmatched with reality. Although the use of self-reports to collect data has its benefits, it may lead to a common method variance, namely, a situation that may inflate the true associations between variables. This study aimed to probe into the factors of technological and vocational school teachers’ evaluation effectiveness, which is significantly affected by school feedback regarding evaluation outcomes. Future studies can include measurements of this variable, and modifying the model in order to further probe into the cause-and-effect relations among the variables. Chou 733 Implications for practice The findings from this study demonstrate that “performance expectancy” and “technology training” of technological and vocational school teachers’ “e-portfolio acceptance” significantly influence “teacher evaluation effectiveness”. E-portfolio users’ use intentions depend on their attitudes toward computer self-efficacy. Computer resources and organisational support will indirectly affect users’ intentions to use e-portfolio systems (Igbaria, Guimaraes & Davis, 1995). Administrators of technological and vocational schools should promote teachers’ e-portfolio acceptance, thereby encouraging teachers to continually contemplate on performance expectancy approaches, and through technology training and performance expectancy, energise their active adoption of professional development for e-portfolios and continuous implementation. Second, according to results of this study, technological and vocational school teachers’ “e-portfolios acceptance” significantly influences “computer self-efficacy”. In the post- acceptance model (PAM), the process by which teachers perceive new playfulness or concepts through ease of use, effectiveness, and usefulness, usually requires a long period of use to gain acceptance by teachers. E-portfolios acceptance in schools will influence teachers’ use intentions of computer technology. After e-portfolios content and technology training are introduced within an organisation, if it is supported by teachers and supervisors, the users’ use attitude and intention would be indirectly influenced by the increased e-portfolios use opportunities, and subsequent evaluation experience (Sorensen, Mathiasen & Dalsgaard, 2009; Wong & Li, 2011). Third, e-portfolios are more suited to formative evaluations, such as instructional improvement and professional development. When they are applied to summary evaluations such as performance auditing and the hiring and evaluation of teachers, a number of factors must be considered: 1. teacher e-portfolios are open, and it is difficult to compare instructional quality among teachers; 2. teacher e-portfolios must be filed and evaluated in detail; 3. e-portfolios should be focused on the dimensions or criteria by the evaluator; and 4. some instructional qualities cannot be demonstrated through e-portfolios. Finally, the findings of this study indicate that “ease of use” and “effectiveness” of “computer self-efficacy” can enhance teachers’ evaluation effectiveness. Besides computer knowledge and techniques, computer self-efficacy also includes teacher attitudes towards computer technology, such as usefulness, teacher cognition, evaluation value, and motive (Anderson, Klassen & Johnson, 1981). When teachers have a greater knowledge of evaluation effectiveness and performance, they are more likely to have intentions to use the web-based portfolios. There are significant and positive correlations among computer knowledge and skills, and teachers’ e- evaluation and performance (Nicholson, 2004; Pynoo, Devolder, Tondeur, van Braak, Duyck, & Duyck, 2011; Seldin, Miller, Seldin & McKeachie, 2010). Measures to enhance technological and vocational school teachers’ evaluation effectiveness are as follows: 1. allow teachers to perceive the importance and growing trends in e-portfolio through teacher instructional design and classroom management; 2. assist teachers with e- portfolios in order to integrate the courses and resources of e-evaluation system; 3. instruct teachers to reorganise current teacher evaluation resources into a web-based portfolio system; and 4. plan e-portfolio training with encouragement in order to energise teachers' enthusiasm to adopt e-evaluation information in an e-portfolio system. 734 Australasian Journal of Educational Technology, 2012, 28(4) Acknowledgments This study was financially sponsored by the National Science Council, Taiwan, under Grant No. NSC98-2511-S-224-002-MY2. References Anderson, S. E. & Maninger, R. M. (2007). Preservice teachers' abilities, beliefs, and intentions regarding technology integration. Journal of Educational Computing Research, 37(2), 151-172. http://dx.doi.org/10.2190/H1M8-562W-18J1-634P Ahmad, T. B. T., Basha, K. M., Marzuki, A. M., Hisham, N. A. & Sahari, M. (2010). Faculty's acceptance of computer based technology: Cross-validation of an extended model. Australasian Journal of Educational Technology, 26(2), 268-279. http://www.ascilite.org.au/ajet/ajet26/ahmad.html Ajzen, I. (1988). Attitudes, personality and behavior. Milton Keynes (UK): Open University Press. http://books.google.com.au/books?id=dmJ9EGEy0ZYC&printsec=frontcover#v=onepage& q&f=false Bagozzi, R. P. & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marking Science, 16(1), 74-94. http://dx.doi.org/10.1007/BF02723327 Bandura, A. (2007). Much ado over a faulty conception of perceived self-efficacy grounded in faulty experimentation. Journal of Social and Clinical Psychology, 26(6), 641-658. http://dx.doi.org/10.1521/jscp.2007.26.6.641 [also at http://des.emory.edu/mfp/Bandura2007JSCP.pdf] Bandura, A. (2008). An agentic perspective on positive psychology. In S. J. Lopez (Ed.), Positive psychology: Exploring the best in people (Vol. 1, pp. 167-196). Westport, CT: Greenwood Publishing Company. http://des.emory.edu/mfp/Bandura2008PP.pdf Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606. http://dx.doi.org/doi:10.1037/0033-2909.88.3.588 Boomsma, A. & Hoogland, J. J. (2001). The robustness of LISREL modeling revisited. Psychometrika, 51, 313-325. http://www.ppsw.rug.nl/~boomsma/ssi.pdf Cai, Y. & Lin, C. (2006). Theory and practice on teacher performance evaluation. Frontiers of Education In China, 1(1), 29-39. http://dx.doi.org/10.1007/s11516-005-0004-x Chai, C. S. & Lim, C. P. (2011). The Internet and teacher education: Traversing between the digitized world and schools. The Internet and Higher Education, 14(1), 3-9. http://dx.doi.org/10.1016/j.iheduc.2010.04.003 [also at http://repository.ied.edu.hk/dspace/handle/2260.2/11128] Chang, C. C. (2008). Enhancing self-perceived effects using Web-based portfolio assessment. Computers in Human Behavior, 24(4), 1753-1771. http://dx.doi.org/10.1016/j.chb.2007.07.005 Chang, C. C. & Tseng, K. H. (2009). Use and performances of Web-based portfolio assessment. British Journal of Educational Technology, 40(2), 358-370. http://dx.doi.org/10.1111/j.1467- 8535.2008.00885.x Chau, J. & Cheng, C. (2010). Towards understanding the potential of e-portfolios for independent learning: A qualitative study. Australasian Journal of Educational Technology, 26(7), 932-950. http://www.ascilite.org.au/ajet/ajet26/chau.html Chou 735 Chou, C. M., Shen, C. H., Hsiao, H. C. & Chen, S. C. (2010a). The influence of innovative organizational management of technological and technological and vocational schools on innovative performance - using e-portfolios acceptance as the mediator variable. World Transactions on Engineering and Technology Education, 8(2), 237-242. http://www.wiete.com.au /journals/WTE&TE/Pages/Vol.8,%20No.2%20(2010)/19-12-CHOU.pdf Chuang, H. H. (2008). Perspectives and issues of the creation for weblog-based electronic portfolios in teacher education. British Journal of Educational Technology, 39(1), 170-174. http://dx.doi.org/ 10.1111/j.1467-8535.2007.00748.x Churchill, D. (2009). Educational applications of Web 2.0: Using blogs to support teaching and learning. British Journal of Educational Technology, 40(1), 179-183. http://dx.doi.org/10.1111/j.1467-8535.2008.00865.x Damanpour, F. & Wischnevsky, J. D. (2006). Research on innovation in organizations: Distinguishing innovation-generating from innovation-adopting organizations. Journal of Engineering and Technology Management, 23(4), 269-291. http://dx.doi.org/10.1016/j.jengtecman.2006.08.002 Davis, E. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-339. http://www.jstor.org/stable/249008 Fishbein, M. & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley. Gefen, D. & Straub, D. W. (2000). The relative importance of perceived ease of use in IS adoption: A study of e-commerce adoption. Journal of the Association for Information Systems, 1(1), 1-28. http://www.cis.gsu.edu/~dstraub/Courses/MGS%209940/2007/Tim-Mathieu- Betrice%20choice.pdf Guriting, G., Chunwen, G. & Ndu, N. N. O. (2007). Computer self-efficacy levels, perceptions and adoption of online banking. International Journal of Services Technology and Management, 8(1), 54-61. http://dx.doi.org/ 10.1504/IJSTM.2007.012218 Garrett, N. (2011). An e-portfolio design supporting ownership, social learning, and ease of use. Educational Technology & Society, 14(1), 187-202. http://www.ifets.info/journals/14_1/17.pdf Hage, J. & Aiken, M. (1970). Social change in complex organizations. NY: Random House. Hair, J. F. Jr., Anderson, R. E., Tatham, R. L. & Black, W. C. (1998). Multivariate data analysis (5th ed.). Englewood Cliffs: Prentice-Hall. Hair, J. F. Jr,, Black, W. C., Babin, B. J. & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). New Jersey: Pearson Prentice Hall. Hu, A. L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. http://dx.doi.org/10.1080/10705519909540118 Harvey, L. & Williams, J. (2010). Fifteen years of quality in higher education. Quality in Higher Education, 16(1), 3-36. http://dx.doi.org/ 10.1080/13538321003679457 Hoffer, J. A. & Alexander, M. B. (1992). The diffusion of database machines. Data Base, 23(2), 13- 19. http://dx.doi.org/10.1145/141342.141352 Hsu, M. H., Ju, T. L., Yen, C. H. & Chang, C. M. (2007). Knowledge sharing behavior in virtual communities: The relationship between trust, self-efficacy, and outcome expectations. International Journal of Human-Computer Studies, 65(2), 153-169. http://dx.doi.org/10.1016/j.ijhcs.2006.09.003 736 Australasian Journal of Educational Technology, 2012, 28(4) Ifinedo, P. (2006). Acceptance and continuance intention of web-based learning technologies (WLT) use among university students in a Baltic Country. The Electronic Journal on Information Systems in Developing Countries, 23(6), 1-20. http://www.ejisdc.org/ojs/include/getdoc.php?id=195&article=213&mode=pdf Igbaria, M. & J. Iivari (1995). The effects of self-efficacy on computer usage. Journal of Management Science, 23(6), 587-605. http://dx.doi.org/ 10.1016/0305-0483(95)00035-6 Jarvenpaa, S. L., Tractinsky, N. & Vitale, M. (2000). Consumer trust in an Internet store. Information Technology and Management, 1(1&2), 45-71. http://dx.doi.org/10.1023/A:1019104520776 Joyes, G., Gray, L. & Hartnell-Young, E. (2010). Effective practice with e-portfolios: How can the UK experience inform implementation? Australasian Journal of Educational Technology, 26(1),15-27. http://www.ascilite.org.au/ajet/ajet26/joyes.pdf Karsten, R. & Roth, R. M. (1998). Computer self-efficacy: A practical indicator of student computer competency in introductory IS courses. Informing Science, 1(3), 61-68. http://inform.nu/Articles/Vol1/v1n3p61-68.pdf Koc, M. & Bakir, N. (2010). A needs assessment survey to investigate pre-service teachers’ knowledge, experiences and perceptions about preparation to using educational technologies. The Turkish Online Journal of Educational Technology, 9(1), 13-22. http://www.tojet.net/articles/912.pdf Larsen, T. J., Sørebø, A. M. & Sørebø, Ø. (2009). The role of task-technology fit as users’ motivation to continue information system use. Computers in Human Behavior, 25(3), 778-784. http://dx.doi.org/10.1016/j.chb.2009.02.006 Lee, B. D. (2009). Teaching and learning e-portfolios. [viewed 11 May 2011] http://ge.dyu.edu.tw/981criticism/files/2009/12/eP_lee.pdf Meyer, E., Abrami, P. C., Wade, C. A., Aslan, O. & Deault, L. (2010). Improving literacy and meta- cognition with electronic portfolios: Teaching and learning with ePEARL. Computers & Education, 55, 84-91. http://dx.doi.org/10.1016/j.compedu.2009.12.005 Ministry of Education (2008). Ministry of Education teacher professional development evaluation pilot program. http://140.111.34.34/moe/common/index.php?z=436 [in Chinese; viewed 12 May 2011] Ministry of Education (2010). Ministry of Education pilot teacher professional development evaluation items. http://140.111.34.34/docdb/files/dma7da0a07101419086.pdf [in Chinese; viewed 14 May 2011] Mohrman, S. A., Cohen, S. G. & Mohrman, A. M. Jr. (1995). Designing team-based organizations: New forms for knowledge work. San Francisco: Jossey-Bass. Nicholson, B. (2004). E-portfolios for educational leaders: An ISLLS-based framework for self- assessment. Lanham, Maryland: Rowman & Littlefield. Pynoo, B., Devolder, P., Tondeur, J., van Braak, J., Duyck, W. & Duyck, P. (2011). Predicting secondary school teachers’ acceptance and use of a digital learning environment: A cross- sectional study. Computers in Human Behavior, 27(1), 568-575. http://dx.doi.org/10.1016/j.chb.2010.10.005 Ryu, M. H., Kim, S. & Lee, E. (2009). Understanding the factors affecting online elderly user's participation in video UCC services. Computers in Human Behavior, 25(3), 619-632. http://dx.doi.org/10.1016/j.chb.2008.08.013 Chou 737 Saade, R. G. & Kira, D. (2009). Computer anxiety in e-learning: The effect of computer self- efficacy. Journal of Information Technology Education, 8, 177-191. http://jite.org/documents/Vol8/JITEv8p177-191Saade724.pdf Seldin, P., Miller, J. E., Seldin, C. A. & McKeachie, W. (2010). The teaching portfolio: A practical guide to improved performance and promotion/tenure decisions. San Francisco, CA: Jossey-Bass. (pp. 62- 70) Shepherd, C. E. & Bolliger, D. U. (2011). The effects of electronic portfolio tools on online students’ perceived support and cognitive load. The Internet and Higher Education, 14(3), 142- 149. http://dx.doi.org/10.1016/j.iheduc.2011.01.002 Shroff, R. H., Deneen, C. C. & Ng, E. M. W. (2011). Analysis of the technology acceptance model in examining students' behavioural intention to use an e-portfolio system. Australasian Journal of Educational Technology, 27(4), 600-618. http://www.ascilite.org.au/ajet/ajet27/shroff.pdf Sorensen, E. K., Mathiasen, H., & Dalsgaard, C. (2009). E-learning concepts in higher education. The International Conference on E-Learning in the Workplace 2009, New York 10-12 June. [viewed 10 Sep 2011] http://person.au.dk/fil/16581526/Sorensen_Mathiasen_Dalsgaard_E- learning_concepts.pdf Sung, Y. T., Chang, K. E., Yu, W. C. & Chang, T. H. (2009). Supporting teachers' reflection and learning through structured digital teaching portfolios. Journal of Computer Assisted Learning, 25(4), 375-385. http://dx.doi.org/10.1111/j.1365-2729.2009.00312.x Susser, B. & Ariga, T. (2006). Teaching e-commerce web page evaluation and design: A pilot study using tourism destination sites. Computers & Education, 47(4), 399-413. http://dx.doi.org/10.1016/j.compedu.2004.11.006 Swan, G. (2009). Examining barriers in faculty adoption of an e-portfolio system. Australasian Journal of Educational Technology, 25(5), 627-644. http://www.ascilite.org.au/ajet/ajet25/swan.html Teo, T. (2009a). Examining the relationship between student teachers’ self-efficacy beliefs and their intended uses of technology for teaching: A structural equation modelling approach. The Turkish Online Journal of Educational Technology, 8(4), 7-16. http://www.tojet.net/articles/v8i4/841.pdf Teo, T. (2009b). Modeling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52(1), 302-312. http://dx.doi.org/10.1016/j.compedu.2008.08.006 Tsai, P. S., Tsai, C. C. & Hwang, G. J. (2011). The correlates of Taiwan teachers’ epistemological beliefs concerning Internet environments, online search strategies, and search outcomes. The Internet and Higher Education, 14(1), 54-63. http://dx.doi.org/10.1016/j.iheduc.2010.03.003 Vajargah, K. F. & Jahani, S. (2010). Application of ICTs in teaching and learning at university level: The case of Shahid Beheshti University. The Turkish Online Journal of Educational Technology, 9(2), 33-39. http://www.tojet.net/articles/v9i2/924.pdf Von Konsky, B. R. & Oliver, B. (2012). The iPortfolio: Measuring uptake and effective use of an institutional electronic portfolio in higher education. Australasian Journal of Educational Technology, 28(1), 67-90. http://www.ascilite.org.au/ajet/ajet28/vonkonsky.html Wilfong, J. D. (2006). Computer anxiety and anger: The impact of computer use, computer experience, and self-efficacy beliefs. Computers in Human Behavior, 22(6), 1001-1011. http://dx.doi.org/10.1016/j.chb.2004.03.020 738 Australasian Journal of Educational Technology, 2012, 28(4) Wong, E. M. L. & Li, S. C. (2011). Framing ICT implementation in a context of educational change: A structural equation modelling analysis. Australasian Journal of Educational Technology, 27(2), 361-379. http://www.ascilite.org.au/ajet/ajet27/wong.html Yao, Y., Aldrich , J., Foster, K. & Pecina, U. (2009). Pre-service teachers’ perceptions of an electronic portfolio as a tool for reflection and teacher certification. Journal of Educational Research & Policy Studies, 9(1), 25-43. http://www.eric.ed.gov/PDFS/EJ896277.pdf Appendix 1 Items of technological and vocational school teachers’ perceived e-portfolios acceptance and computer self-efficacy on scale for teachers’ evaluation effectiveness. Factor Compositionof scales Items Staff commitment 1. I will constantly ponder on innovative teaching materials and approaches of e-portfolios. 2. I will use multiple e-instructional approaches to develop students’ multiple creations and learning. 3. I can actively promote instructional innovative ideas and attempt to implement e-portfolios. Effort expectancy 1. E-portfolios of the school have unique innovative ideas on teacher evaluation. 2. I believe that I master e-portfolios terminology and concepts. 3. Supervisors of the school respect and support e-portfolios at work. Performance expectancy 1. I have sufficient equipment for e-portfolios teaching. 2. As long as I am in need, professional staff will effectively assist me with e-portfolios. 3. The school provides opportunities for e-portfolios. Scale of e- portfolios acceptance Technology training 1. The school financially supports technology training for e- portfolios. 2. Most teachers of the school are willing to encounter challenges of e-portfolios. 3. The school climate stimulates teachers’ e-portfolios thoughts. 4. Members of the school can understand e-portfolios training visions and goals. Playfulness 1 Computer-based instruction is more interesting. 2. Digital data is more likely to attract students. 3. E-portfolios are more interesting. Ease of use 1. For me, the computer is easy to learn. 2. I can learn how to use computers by observing others. 3. It is easy for me to write all kinds of things on the computer. Effectiveness 1. E-portfolios are more flexible. 2. E-portfolios are more likely to increase students’ learning motivations. 3. E-portfolios are more likely to increase students’ learning outcomes. Computer self-efficacy Usefulness 1. It is more useful to make teaching materials by computers. 2. It is easier to assist with students’ after-school learning by e- portfolios. 3. It is easier to deal with digital teaching materials after learning computer skills. Chou 739 Classroom management 1. I will create e-portfolios of classroom management plans in advance. 2. I will become familiar with the contents of different e-portfolios classroom management units. 3. I will prepare teaching materials and tools for different classroom management units. 4. In e-portfolios, I will change instructional activities to maintain students’ concentration of classroom management. Instructional design 1. I will grade students by multiple e-portfolios evaluations. 2. I will have discussions with students regarding their learning results and guide their future learning by e-portfolios. 3. I will adjust instructional schedules, degree of difficulty, and methods, according to e-portfolios evaluation outcomes. 4. I will collect supplementary teaching materials in order to enhance e-portfolios teaching efficacy. Dedicated attitude 1. During e-portfolios processes, I can create harmonious learning atmospheres. 2. During e-portfolios processes, I will value students’ demands. 3. During e-portfolios processes, I will interact with students and share experiences. 4. During e-portfolios processes, I can maintain order in the class. 5. During e-portfolios processes, I might do something effort while teaching with computers. Teacher evaluation effectiveness Professional development 1. I can select e-portfolios of professional development according to instructional subjects and teaching materials. 2. I will enhance the operations of e-portfolios media in order to enhance teaching activities. 3. During e-portfolios processes, I will praise and encourage myself professional development progress. 4. At-the-moment solutions while working with e-portfolios are enough for me. Author: Dr Chun-Mei Chou Graduate Institute of Vocational and Technological Education National Yunlin University of Science and Technology Yunlin, Taiwan 64002. Email: choucm@yuntech.edu.tw Please cite as: Chou, C. M. (2012). Influence of teachers’ perceived e-portfolio acceptance on teacher evaluation effectiveness in Taiwan. Australasian Journal of Educational Technology, 28(4), 719-739. http://www.ascilite.org.au/ajet/ajet28/chou- cm.html