Research and Evaluation in Education e-ISSN: 2460-6995 Research and Evaluation in Education Volume 1, Number 2, December 2015 (pages 158-174) Available online at: http://journal.uny.ac.id/index.php/reid THE EFFECTIVENESS OF MICROCONTROLLER INSRUCTIONAL SYSTEM THROUGH SIMULATION PROGRAM METHOD BY USING TRAINER KIT 1) Edidas; 2) Jalius Jama 1)2) Padang State University, Indonesia 1) edidasunp@yahoo.com; 2) jaliusjama@yahoo.com Abstract The study was to test the learning effectiveness of a program simulation method by using the trainer kit in Microcontroller System courses. The study was conducted to the students who took the microcontroller system courses in the academic year of 2013/2014. The students who took the course were divided into three groups each of which served as: (1) the experimental group, (2) control group, and (3) test instrument group. The learning outcome variables (Y2) and the awareness of thinking (Y1) served as the dependent variable; on the other hand, the motivation (X1) and the creativity (X2) served as independent variables. The data analysis was done by performing the Analysis of Variance (ANOVA) and the Multivariate Analysis of Variance (MANOVA) in order to see the differences in the learning outcomes and the level of students‟ thinking. The results of the study showed that there were significant differences in the level of metacognition and learning outcome competence between the group which conducted program simulation learning by employing the trainer kit and the group not employing it. Keywords: effective learning, learning simulation, quasi-experiment, metacognition level, learning outcome competence level Research and Evaluation in Education 159 - Volume 1, Number 2, December 2015 Introduction The Microcontroller System Course in the Electronic Engineering Study Program, Faculty of Engineering, Padang State University is one of the learning programs that belong to the group of technological and vocational education. The technological and vocational education has been offered in order to be selected only by the talented people (Prosser Theorem). Unlike the general education that equips the students with knowledge and ability to pursue higher education, the technological and vocational education equips the students with a number of specific level and competency for entering the employment. According to Presidential Regulation Number 8 Year 2012 regarding the Framework of Indonesian National Qualifi- cation (Ind.: Kerangka Kualifikasi Nasional Indonesia (KKNI)), the learning achievement of the graduates of Diploma 3 is on Level 5 and the description of Level 5, according to the national qualification, is as follows: (1) being able to perform wide-scope tasks, to select the appropriate method both from the standard- ized one and from the unstandardized one by analyzing the available data and to show performance under the measured quality and quantity; (2) being able to master the theoretical concept of certain scientific domain in general and to formulate the procedural problem solving efforts; (3) being able to manage a workgroup and to compose a written report comprehensively; and (4) being able to be responsible for his or her own job and being able to be in charge of the achievement of the workgroup. With reference to the afore-mentioned description, students who undertake a Diploma III program should really master the concept of science both theoretically and practically. In order to meet such an objective, they should practice their thinking and working capacity. Their desire to keep improving their thinking and working capacity will lead to the occurrence of creativity and innovation. The degree of qualification according to KKNI might be viewed in Figure 1. Figure1. The Degree of Qualification According to KKNI (Source: Directorate General of Higher Education (2011, p.15) Vocational education is education that aims to supply reliable and professional labors. The students who undertake vocation- al education programs are expected to be professional, responsible, quickly-finding, quickly-understanding and quickly-respoding experts toward the changes occuring in their environment. Within the learning process, they are introduced to multiple new problems and are trained in finding the solutions in order to develop their capabilities, to find their own problem-solving alternatives and to take decisions quickly. If there is a problem in their job, they should be able to find the multiple alternatives in solving the problem. In relation to the process, within the skill and expertise mastery in the Microcontroller System Course, the students are expected to be able to meet the objectives of the study program. In order to produce a graduate who has the capability of working independently, there should be a complex process. During the learning process, students are taught to master not only the vocational skills but also the entrepreneurial ability and the thinking- awareness (metacognition). Thereby, the labors who have the capability of working independently will be the graduates who have been able to combine the following three capabilites altogether at the same time namely: (1) being able to implement the skill competency for performing their tasks; (2) being able to implement the entrepreneurial skill competency for starting their business; Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 160 Edidas & Jalius Jama and (3) being able to use their ideas for developing their business, for finding opportunities and for overcoming the existing obstacles. In the Microcontroller System, there are many competencies that the students should master; in general, the competencies might be divided into two parts: (1) microcontroller system hardware-manufacturing capabiliity (microcontroller electronic combinations); and (2) micro-controller system software- manufacturing capability (application programs). The ability to manufacture the hardware should be mastered before the ability to manufacture the software. On the contrary, in order to manufacture the hardware of the micro-controller system, the students should master the microcontroller programming first. The condition of the students before the research was conducted is provided. The resume on the data of learning achievements is given in Table 1. Within the learning process for a vocational competency, the results of the learning achievements are categorized into Go (competent) and Not Go (not competent). Table 1 shows that from the two learning groups with a total of 35 members there have been only 12 people or 34% of the group members who have mastered the competency. On the other hand, there have been 23 people or 66% of the group members who have not mastered the competency. The finding has been the underlying reason for conducting the study by finding and analyzing the variables that might influence the cause and effect of such learning achievements. There might be various variables such as the aspects of teaching staffs, facilities, teaching methods, internal environments and external environments. Table 1. The Results of Students‟ Competence Achievement Score Interval Quality Level Number of Students Competence Category Number of University Students in Category Percentage Section 86186 Section 86187 81- 100 A 1 0 Competent 12 34 % 66 - 80 B 6 5 Competent 56 - 65 C 4 5 Incompetent 23 66 % 41 - 55 D 1 0 Incompetent 0 - 40 E 5 8 Incompetent NUMBER OF UNIVERSITY STUDENTS 17 18 35 100% Several problems that occur within the course are as follows: (1) the low motivation that the students have in the Microcrontroller System component manufacturing and programming; (2) the low creativity that the students have in the Microcrontroller System component manufacturing and programming; (3) low thinking awareness and understanding that the students have in the Microcrontroller System component manufacturing and programming; (4) low capability of identifying the component damage and the program error in the Microcontroller System; (5) the absence of effective learning methods for the Microcontroller System; and (6) the low capability of performing cooperation, interact- ion and tolerance that the students have in working as a group. Based on the explanation, the study is conducted under the following objectives: (1) to explain the absence of inter-independent variable interaction between the two groups, namely between the group that implements and the group that does not implement the Trainer Kit Microcontroller MCS51; and (2) to explain that the level of metacognition and Research and Evaluation in Education 161 - Volume 1, Number 2, December 2015 the level of competency, the students‟ results within the program simulation learning method, is better for the group that implements the Microcrontroller MCS51 than the group that does not implement the Microcrontroller MCS51. Simulation Learning Method Multiple methods might be performed in order to master the skills in the microcontroller system and one of such methods is simulation. A simulation learning method is a learning process that employs simulator media as the learning tools. The simulator media might be in the form of software or hardware. The simulator media in the form of software display the behavior of phenomena that have been simulated in the computer screen; on the other hand, the simulator media in the form of hardware employ the actual tools but the target of the hardware might be minimized or might be replaced by a similar one. In relation to the simulation learning, Nesbit in Joyce, Weill and Calhoun (2009, p.443) states that: „Simulation might stimulate the learning about: (1) competition; (2) cooperation; (3) empathy; (4) social system; (5) concept; (6) skills; (7) effectivenss; (8) penalty; (9) role of opportunity/chance; and (10) opportunity to perform critical thinking.‟ Learning by means of simulation method might also enrich the knowledge, the skills and the attitudes of the students. By paying attention to and performing the simulation, the students might increase their understanding, their skills and also their attitudes toward the phenomena of a program. The more the program variations are simulated by the students, the more in- depth understanding the students will gain and the more skillful the students will be in performing their job and in displaying their habits. Microcontroller MCS51 Trainer Kit Simulator In the simulation learning, the trainer kit serves as the learning media. Learning media are the components of teaching deliverance system that might be used in supporting the learning process. The development of learning media should be based on the perception that the learning process will run smoothly, effective and fun if the learning process is supported by the learning media that might draw the students‟ interest and attention. Motivation The program simulation learning by means of a trainer kit is expected to encourage the students‟ learning motivation. Motivation is defined as a power that encourages a person to do and to direct his or her activities. Motivation might come from the inside or from the outside such as from the surrounding neighborhood. Motivation that comes from the inside is defined as intrinsic motivation, while motivation that comes from the outside is defined as extrinsic motivation. The indicators of motivation from the intrinsic factors are as follows: (1) the existence of desire and expectation toward success; (2) the existence of needs and encouragement toward learning; (3) the existence of expectation and dream toward future; and (4) the existence of achievements toward the learning process. On the other hand, the indicators of motivation from the extrinsic factors are as follows: (1) the existence of appreciation toward the learning process; (2) the existence of interesting activities within the learning process; and (3) the existence of conducive learning environment (Sardiman, 2012, p.83). Handoko (1992, p.59) states that „In order to find the power of students‟ motivation people may pay attention to the following indicators: (1) the strength of willingness to do something; (2) the duration devoted to the willingness to learn; (3) the willingness to leave other duties/tasks; and (4) the dilligence in working on the assignments.‟ Another theory related to motivation is the expectancy theory of motivation. The expectancy theory of motivation defines that the intensity of tendency to perform something under certain way depends on the intensity of expectation. This theory emphasizes more on the outcomes than on the needs. Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 162 Edidas & Jalius Jama According to McClelland (1961) in his theory about the human needs that have been very vital within an organization and a company, the theory focuses on three aspects: (1) needs for achievement, namely the ability to achieve in the relationship between the job standards that have been set and the struggle for gaining the job achievements; (2) needs for power, namely the motivation to rule that makes people behave in a normal and wise way but actually they want to gain control or to be admitted in their community; and (3) needs for affiliation, namely the desire to be more friendly and to be more accustomed to the colleagues in order to cooperate and to meet the objectives that would like to be achieved. Creativity Creativity is a unique mental process that has solely been done in order to generate something new, different and original (Hurlock, 1978, p.3). A unique mental process takes the form of creating new ideas that are different from the existing ones. A student who might be considered as having a high creativity is the one who has many new ideas. The new ideas will appear from himself or herself if he or she sees a new object. The new ideas might take the form of simplification for assisting the use, the gain and even the production of the object. Piirto (2011, p.1) states that, „Creativity is simply defined here, as “to make something new,” as a prerequisite to innovation.‟ There are some people who regard that creativity only exists in art, technique, souvenir, film and alike, which are able to be manipulated or to be created. Actually, creativity is not limited to certain domains; instead, creativity might cover all of the existing domains. Explicitly, one might state that every single aspect that might create something new is a form of creativity. The 21 st century skills, one of which is the creative and innovative skill, have eight indicators as follows (Piirto, 2001, p.1): (1) being able to use a wide range of idea creation techniques such as brainstorming; (2) being able to create new and worthwhile ideas, by which a creative person will always think about new aspects in solving multiple problems; (3) being able to elaborate, refine, analyze and evaluate their own ideas in order to improve and maximize creative efforts, for which the existing new ideas need to be elaborated in order to view their strength and their weaknesses and if there are weaknesses within the new ideas there should be efforts made for eliminating the weaknesses; (4) being able to develop, implement and communicate new ideas to others effectively, by which a creative person always communicates every new idea to other people in order that the new idea might be implemented well, and communicating the new idea with other who might also improve more ideas in order to gain more power within the implementation; (5) being able to be open and responsive to new and diverse perspectives, incorporate group input and feedback into the work, a creative person will always be open and responsive toward multiple new ideas; (6) being able to demonstrate originality and inventiveness in work and understand the real world limits to adopting new ideas, with which a creative person should be able to create something new and different from something that has been in existence or should be able to create something unique and original; (7) being able to view failure as an opportunity to learn; understand that creativity and innovation is a long-term, cyclical process of small successes and frequent mistakes, the creative people will always see failures as opportunities to learn; and (8) being able to act on creative ideas to make a tangible and useful contribution to the field in which the innovation will occur, by which a creative person will always perform concrete contribution within the domain in which the innovation will be implemented. Metacognition Metacognition is a thinking awareness that a person has in understanding his or her job and the objectives of doing his or her job. Zohar and Dori (2012, p.58) state that „meta- cognition usually is subdivided into two distinct components, including knowledge of cognition and regulation of cognition.‟ In general, metacognition is divided into two Research and Evaluation in Education 163 - Volume 1, Number 2, December 2015 components: metacognitive knowledge and metacognitive regulation. Thinking awareness is an awareness that a person has regarding what he or she knows and what he or she is doing. Metacognition is a thinking awareness that a person has regarding his or her own thinking process. Flavell (1979, p.1) defines that meta- cognition has four components: (1) meta- cognitive knowledge; (2) metacognitive experiences; (3) metacognitive tasks and goals; (4) metacogntive strategies or actions. Metacognitive Knowledge Metacognitive knowledge is related to the declarative knowledge, the procedural knowledge and the conditional knowledge. Flavell provides several examples of meta- cognitive knowledge as follows: someone believes that he or she is able to learn better by listening than by reading or that he or she views his or her friends to be more socially aware than himself or herself. One‟s belief about oneself as a learner might facilitate or might inhibit his or her performance in the learning process. Metacognitive Experiences Metacogntive experiences are related to the planning skills, predicting skills, monitoring skills, and evaluating skills. Information, memories or past experiences might be recalled as a source in the problem solving process. Metacognitive experiences also include affective responses toward duties. Metacognitive Tasks and Goals Metacognitive tasks and goals are the objectives or the results expected from the cognitive efforts. The metacogntive tasks and goals include understanding, inserting facts into the mind or generating something, such as written documents or answers to mathematic problems, only improving one‟s knowledge regarding certain objects. The objective achievement is very interesting in the metacognitive knowledge and in the metacognitive experiences. Metacognitive Strategies or Actions Metacognitive strategies are designed to monitor the cognitive progress. The meta- cognitive strategies are a process that has been implemented in order to control the cognitive activity itself and in order to ensure that the cognitive objectives (for example, problem solving flowchart, syntax program writing, programming algorythm understand-ing) has been met. A person with good metacognitive skills and awareness will employ the process for monitoring the learning process itself, for planning and monitoring the cognitive activities that are in progress and for comparing the cognitive results and the internal or the external standards. An example of using metacognitive strategies is having a retrospection at the end of every learning process with an objective of improving the knowledge content or of monitoring and evaluating the new knowledge. Competencies of Learning Achievements The term competency is derived from Latin language, „competere‟, which means appropriateness or appropriateness to certain jobs. According to Spencer and Spencer (1939, p.9), a competency is an underlying characteristic of an individual that is causally related to criterion-referenced effective and/ or superior performance in a job or situation. Competency is a fundamental characteristic of an individual, namely a cause that is related to the criterion-referenced performance regard- ing the effective performance. The underlying characteristics imply that competency is an integral part of one‟s personality that has been embedded for a long time and might predict behaviors in multiple tasks and job situations. The related cause, or the causally related situation, implies that the competence causes or predicts the behavior and the performance. The criterion-referenced performance, in addition, implies that the competence actually predicts which person will do a job well or worse, as having been measured by specific criteria or standards. Competencies, thereby, are a set of characteristics that encourage an individual‟s motive and that indicate how an individual acts, thinks or generalizes situations appropriately in a long term. Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 164 Edidas & Jalius Jama The Decree of the Minister of National Education of the Republic of Indonesia Number 045/U/2002 regarding the core curricullum of higher education in verse 1 states that competencies are a set of intelligent and fully responsible actions that an individual has as a prerequisite to be considered as being able to perform tasks in certain job domains by the community. A set of intelligent actions implies that an individual who has competen- cies will be able to take actions with enough knowledge and science that he or she has gained from his or her learning achievement. An individual who performs intelligent action will not take foolish actions that might damage the tools or that might endanger himself or herself. By taking intelligent actions, an individual will be able to predict the results and the risks from the job that he or she is doing. In formal education degree, competen- cies are the minimum capability qualification that describes the mastery of knowledge, skills and attitudes that have been studded. Competencies are a set of certain jobs that will be done in order to perform a job well and rightly. Knowledge mastery is an effort to learn and understand the scientific knowledge that underlies the related jobs. The scientific concepts are the knowledge regarding the theories and the laws related to the jobs that should be done, including the knowledge about the work and environment safety. The fundamental scientific knowledge regarding a job serves as the foundation of the competencies. For example, in order to be considered as a competent person in the domain of microcontroller, he or she should understand the knowledge of the electricity and electronic engineering. The more fundamental aspect of the competencies in the domain of microcontroller includes the theories of atom, the law of Ohm, the law of Kirchoff, the electro-static and the other laws of physics that are related to the electric current and electrical resistence. The last component of competencies that should be mastered by an individual is the affective component, namely attitude. The affective component is the most important component because a good and appropriate attitude will generate job results that are useful for the humankind. The attitude of a competent individual heavily determines whether the results of his or her work will be able to improve the quality of human life conveniently without causing any harm to the environment. Other studies relevant to the recent study have been done by many researchers and some of the studies are discussed in the following sections. Saemah and Phillips (2006) studied 374 students in their second year in five colleges of Malaysia National University (University Kebangsaan Malaysia). The study has paid attention to the relation- ship between the metacognitive capability and the attitude toward the learning achievement. The results of the study show that the level of meta-cognition has a direct and positive effect on the learning achievement (β?= 0.358; t = 2.851; p < 0.05). The results of the study show that the learning achievement might be improved by improving the students‟ meta- cognition. On the other hand, in order to improve metacognition awareness, students might set the learning objectives and improve their self-efficacy. However, the component of learning motivation that has orientation toward achievement has negative effect on the improvement of students‟ thinking aware- ness. In addition, this component of learning motivation also has negative effect on learn- ing achievement. Thereby, it might be concluded that the achievement targets in the learning process might provide bad effect on learning achievement. Then, Eskrootchi and Oskrochi (2010) studies 72 students from the Northwest Middle-School Kansas City in Kansas. The results of the study show that the students who perform learning by means of the simulation method have better understanding than the others. The finding strengthens the researchers‟ belief that the simulation method has a huge effect on learning achievement. Furthermore, Liu (2010) studies 45 university students from the Public University in Taiwan. The prototype of Computer- Assisted Learning (CAL)-based simulation has been named as Simulation-Assisted Learning Statistics (SALS). The result of his study Research and Evaluation in Education 165 - Volume 1, Number 2, December 2015 shows that the learning process that implements the Simulation-Assisted Learning Statistics (SALS) has been effective in reducing the university students‟ mistakes in statistics. From the studies that have been discussed, the researchers concluded that the learning process by means of the SALS simulation method might decrease the university students‟ mistakes in statistics. These results will be a matter of comparison in the recent study. In addition, Del Populo Pablo, et al. (2012) perform an evaluation study on the IS- LM simulation program in a group of macro- economy university students in the University of Sevilla, Spain, during the academic year of 2009/2010. IS-LM is a term in the domain of economy and the term stands for Investment- Saving/Liquidity preference-Money supply. IS-LM is a fundamental material that has been implemented in the short-term macro- economy teaching. The teaching, basically, is conducted by using graphs. The IS-LM simulation program capability in overcoming the difficulties has been found in understand- ing curves because the university students are able to visualize the changes in the curve when the value parameters are changed. The variance analysis (ANOVA) from the values that all of the university students have and from several complementary statistic tests that have been performed shows the differences between the university students that use the simulator and those who do not. The results of the study show that the average scores of the university students who use the simulation program are significantly higher compared to those of the students who do not use the simulation program. The -value of the ANOVA analysis equals 1.073. Kruskal-Wallis chi-squared statistic shows a value equal to 2.51 with a p-value equal to 0.285. Alias (2012) studied 40 respondents with the background of expert, instructor and learner who actively operated Online and Distance Education. Four experts were selected, two from England and the other two from Malaysia. The instructors as well as the learners were also selected from the people who had been active in the long distance learning via the Internet. The main objective of the study was to describe the design, the development and the formative assessment toward the learning process via the Internet. The study was to answer the following questions: (1) What are the strategies that should be implemented in order to encourage the motivation of the e-learning users?; (2) How should the Web technology be implemented in order to encourage the students‟ motivation?; and (3) What is the e- learners‟ response toward the effectiveness, the practicality and the assessment of the learning consulation? The results of the study show that the learning console has potentials to reflect their learning process (M = 4.0; SD = 0,47) to encourage the students to improve their initiative (M = 4.33; SD = 0.67) and their self- efficacy (M = 4.0; SD = 0.47), to make the students keep motivated in the learning process (M=3.9; SD=0.74) and to provide the sense of achievements (M = 3.8; SD = 0.79). The university students also value that the learning console has provided opportunities for interaction between the students and the instructors (M = 3.9; SD= 0.74). On the other hand, the students value that the learning console has not had potentials for providing feedbacks (M=3.3; SD=1.25). Hung et al. (2012) conducted another study to 117 fifth grade students of an elementary school located in the southern part of Taiwan. The students were divided into two groups, namely an experimental group, consisting of 60 students (35 male students and 25 female students) and a control group, consisting of 57 students (31 male students and 26 female students). The experimental group took part in the study by performing new project-based learning in the form of digital story-telling and the control group took part in the study by performing conventional project-based learning including the project assignments and the result presentation in the groups altogether with the teachers‟ feedback and evaluation. The results of the study show that the experimental group, namely the group that took part in the study by perform- ing new project-based learning in the form of digital storytelling has been able to effectively improve their learning motivation (F = 20.38; Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 166 Edidas & Jalius Jama p <0.001). Digital storytelling is a learning approach that employs computer program application as the working guideline during their learning process. The students who were studying would follow the learning sequences by clicking the buttons according to the directions on the computer screen. The strength of digital storytelling approach is that the working guideline has been displayed not only in the texts or sketches but also in the original forms of the direction that will be clicked. Hypothesis Based on the elaboration, several hypotheses are formulated as follows: (1) the study will explain the absence of interaction between the independent variables from both groups, both the group that implements the Microcontroller MCS51 Trainer Kit simulat ion program and the group that does not implement the Microcontroller MCS51 Train- er Kit simulation program; and (2) the study will explain the level of metacognition and the level of competency within the results of the students that implement Microcontroller MCS51 Trainer Kit simulation program; the assumption is that the students who employ the trainer kit have better learning achieve- ments. Research Method The study was a quasi-experiment with a small sample and homogenous sample because the study was conducted in the class of Microcontroller System course. The approach used was the quantitative one in which the data were described quantitatively. The data were in the form of numbers and analyzed in order to describe how effective the microcontroller program learning by means of trainer kit simulation was in comparison with that without the simulation program implementation. The research design selected was the noequivalent control group design in which the experimental group and the control group were selected not randomly but based on the groups in the Microcontroller System Course and these groups in the beginning of the study seemed to have similar competence and thinking awareness. In order to find the class that might have similar condition, the researchers performed a preliminary test to all of the existing classes and performed assess- ment. Based on the results of the assessment, the researchers would decide the two classes that might have almost similar competence and thinking awareness. The study would also be developed by counting the moderator variables that might influence the results. Based on the analysis toward the results of the preliminary test toward all classes, the researchers have found the two classes that would be studied, namely: Class A (section 42951) and Class B. The site of the study would be in the Department of Electronic Engineering, the Faculty of Engineering, Padang State University within the class of Microcontroller System Course. Then, the period of the study would be from January to June 2014. The population of the study would be the students of Diploma 3 Electronic Engineering Study Program, the Department of Electronic Engineering, who has taken the class of Microcontroller System Course in January to June 2014. The numbers of the population would be 63 people, consisting of four classes. For the sample selection, the research- ers would implement the nonprobability sampling in which there has not been any similar opportunity for each of the population members to be selected as the sample because the university students have been shaped in the classes. The opportunity to be selected as the sample would fall to the classes that has existed and these classes would be selected based on the results of the preliminary test that was administered to all of the existing classes. Based on the results of the pre- liminary test, the researchers found two classes with similar capability. The sample was divided into two groups: the experimental group and the control group. The independent variables was the learning motivation (X1) and the university students‟ creativity (X2). On the other hand, the dependent variables were the students‟ metacognition (Y1) and the students‟ learning achievements (Y2). The control variables were Research and Evaluation in Education 167 - Volume 1, Number 2, December 2015 the variables that had a tendency to influence the dependent variables; therefore, the control variables were made constant during the study so that there was not anything that influence the relationship between the independent variables and the dependent variables. The moderating variables were the variables that were generated altogether with the in- dependent variables and the moderating variables influenced the dependent variables. The moderating variables were calculated because the influence of these variables was significant on the two dependent variables. The operational definition for each of the variables was as follows. (1) The learning motivation (Variable X1) is the encourage- ment to perform an activity that appears within an individual. In order to gather the data regarding the motivation, the researchers formulated the following indicators: (a) willingness to take an action; (b) time which is provid-ed for learning; (c) expectation toward appreciation (valence, expectation and also instrumentalist); (d) the motives of gaining achievement, cooperation and power; and (e) dilligence in accomoplishing the assignments. (2) The creativity (Variable X2) is a situation in which an individual will always try something new. In order to gather the data regarding the creativity, the researchers formulated the following indicators: (a) using multiple ways for generating ideas; (b) creat- ing new significant ideas; (c) having the elaborating (explaining), improving, analyzing and evaluating power toward his or her own ideas in order to increase and to maximize the creative efforts; (d) developing, implementing, and communicating the new ideas to other people effectively; (e) being open and responsive toward the new and multiple perspectives and combining the group advice and the feedback into the job; (f) showing skills and originality in the job and understanding the limitation of the actual world in order to adopt the new ideas; (g) viewing failure as an opportunity to learn since success which has been a long-term process and has been started from small achievements with many failures; and (h) taking actions based on the creative ideas in order to provide actual and useful contribut- ion to the domain in which the innovation might take place. (3) The students‟ meta- cognition (Variable Y1) is the level of students‟ metacognition namely the level of thinking awareness that the students have during the learning process. In order to gather the data regarding students‟ metacognition, the researchers formulated the following indicators: (a) having metacognitive know- ledge; (b) having metacognitive experiences; (c) having metacognitive tasks and objectives; and (d) having metacognitive strategies and actions. (4) The competencies of students‟ learning achievements (Variable Y2) are the assessment toward the level of proficiency that a student has attained after attending the learning process. In order to gather the data regarding the competency of students‟ learn- ing achievements, the researchers formulated the following indicators: (a) having meta- cognitive capability; (b) having psychomotoric capability; and (c) having affective capability. In addition to the independent variables and the dependent variables, there would also be the control variables, namely the other variables that might have great tendency to influence the dependent variables. The influence of the control variables was kept constant both in the experimental group and in the control group. Then, the control variables were as follows: lecturer or teacher preparedness, learning space comfort, learn- ing period availability and learning media. The control variables were included into the calculation of data analysis because all of the control variables were made constant to the two groups, both the experimental and control groups. The control variables whose condition was made constant was viewed in the table of control variable analysis given in the Appendix. In the table, there are 10 control variables: (1) lecturer preparedness; (2) technician preparedness; (3) space avail- ability; (4) tool availability; (5) material availability; (6) teaching set readinessness; (7) learning time sufficiency; (8) environmental support; (9) environmental inhibition; and (10) other support/inhibition. The performance variable of Micro- controller MCS51 Media Trainer that was selected for the treatment in the experimental Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 168 Edidas & Jalius Jama group was validated first. The validation of trainer performance was conducted by gather- ing information through the documentation and the questionnaire. There were three indicators validated, namely: (1) trainer design compatibility; (2) trainer implementation practicality; and (3) trainer reliability. In order to gather the data regarding the compatibility, the practicality and the reliability, there was an instrument that measured the compatibility, the practical-ity and the reliability of Micro- crontroller Kit Trainer. The instrument of MCS51 Micro- controller Trainer Kit compatibility, practical- ity and reliability measurement that was made was given to five respondents consisting of lecturers and technicians who have been used to running the MCS51 Microcontroller Train- er. The results of the measurement was written into an observational table. Then, the results of observation was analyzed in order to find its validity and reliability. If the data were dichotomous, then the analysis of MCS51 Microcontroller Trainer Kit reliability performance referred to the agreement technique by means of Cohen‟s Kappa coefficient (Wood, 2007): ... (Formula 3.1) Note: k = Inter-rater agreement coefficient o = Inter-rater assessment perfect agreement percentage E = Inter-rater disagreement percentage The summary of Cappa coefficient was turned into an assessment of trainer kit compatibility as a microcontroller system learning media. The summary is presented in Table 2. With the assistance of SPSS statistic software, the researchers were able to calculate the inter-rater reliability coefficient. Therefore, the number of raters for the indicator of trainer design compatibility as the Microcontroller System learning media is five people; as a result, there are 10 k altogether with the summary displayed in Table 2. Table 2. The Summary of Cappa Coefficient for the Assessment of Trainer Kit Compatibility as the Microcontroller System Learning Media No Cohen’s Kappa Coefficient Inter-Two Raters Agreement Note 1 k1 r1 * r2 Agreement between rater 1 and rater 2 2 k2 r1 * r3 Agreement between rater 1 and rater 3 4 k3 r1 * r4 Agreement between rater 1 and rater 4 4 k4 r1 * r5 Agreement between rater 1 and rater 5 5 k5 r2 * r3 Agreement between rater 2 and rater 3 6 k6 r2 * r4 Agreement between rater 2 and rater 4 7 k7 r2 * r5 Agreement between rater 2 and rater 5 8 k8 r3 * r4 Agreement between rater 3 and rater 4 9 k9 r3 * r5 Agreement between rater 3 and rater 5 10 k10 r4 * r5 Agreement between rater 4 and rater 5 Then, the inter-rater reliability was calculated based on the following average agreement: ... (Formula 3.2) According to Landis and Koch in Altman (1991, p.404), the meaning of k is as follows: < 0.20 = Very Bad 0.20 > 0.40 = Bad 0.41 > 0.60 = Moderate 0.61 > 0.80 = Good 0.81 > 1.00 = Very Good Since the data of practicality and reliability are ordinal data, in order to find the practicality reliability of the trainer kit media use, the researchers used Inter Class Correlation (ICC) with the following formula: ... (Formula 3.3) Note: ICC = Inter Class Coefficient Var (α) = The difference of an aspect assessment within the inter-rater assessment = K = number of rater Research and Evaluation in Education 169 - Volume 1, Number 2, December 2015 Var (β) = A rater‟s assessment average = n = number of items Var (e) = A rater‟s average error With the assistance of SPSS statistical software, the researchers could calculate inter- two raters agreement coefficient. Since the number of the raters is 5 people, there are 10 k (inter-rater reliability coefficient) and the resume is given in Table 3. Table 3. The Resume of Cappa Coefficient for the Trainer Kit Compatibility Assessment as the Microcontroller System Learning Media No Cohen’s Kappa Coefficient Inter-Two Raters Agreement k score 1 k1 r1 * r2 0.588 2 k2 r1 * r3 1.000 4 k3 r1 * r4 1.000 4 k4 r1 * r5 0.300 5 k5 r2 * r3 0.588 6 k6 r2 * r4 0.588 7 k7 r2 * r5 0.588 8 k8 r3 * r4 1.000 9 k9 r3 * r5 0.300 10 k10 r4 * r5 0.300 k total number 6.252 k average 0.625 To achieve the reliability level assessed by five raters, the researchers found the average value of the agreement among all of the raters by using Formula 3.2: According to the inter-rater agreement criteria of Cohen‟s Kappa coefficient, the figure k = 0.625 is in „Good‟ category. Thereby, the design of MCS51 Micro- controller System Trainer Kit would be compatible for implementation in the Micro- controller System laboratory practice. In order to find the reliability of the trainer kit media use practicallity by means of ordinal data, the researchers used Inter Class Correlation (ICC) by implementing Formula 3.3. By using the SPSS for analyzing the assessment reliability from the media trainer kit practicallity indicator within the use of Microcontroller System laboratory practice, the researchers found the Coefficient of Inter Class Reliability. The coefficient is displayed in Table 4. Table 4. Coefficient of Inter Class Correlation Reliability Regarding the Media Trainer Kit Use Practicality Intra-Class Correlation Coefficient Intra-Class Correlationa 95% Confidence Interval F Test with True Value 0 Lower Bound Upper Bound Value df1 df2 Sig Single Measures .665b .317 .932 10.921 5 20 .000 Average Measures .908c .699 .986 10.921 5 20 .000 Based on the coefficient value of Interclass Correlation Average Measures, the indicator of media trainer kit use practicality is 0.908 and the reliability of the media trainer kit is in „Very Good‟ category. Therefore, the MCS51 Microcontroller System Trainer media kit is practical for operation in the micro- controller system laboratory practicum. Then, the researchers performed the reliability test toward the MCS51 Micro- controller System Trainer Kit reliability during the laboratory practicum of Microcontroller System. Based on the analysis of the media trainer kit reliability, with the assistance of SPSS, the coefficient of Interclass Correlation Single Measures is equal to 0.563 and the results is displayed in Table 5. The coefficient Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 170 Edidas & Jalius Jama value states that the trainer kit has moderate reliability for the implementation in the laboratory practicum of Microcontroller System. In sum, statistically the MCS51 Microcontroller Media Trainer Kit can be implemented in the training process in the experimental group. Table 5. The Reliability Coefficient of Interclass Correlation Regarding the Trainer Reliability Intraclass Correlation Coefficient Intraclass Correlationa 95% Confidence Interval F Test with True Value 0 Lower Bound Upper Bound Value df1 df2 Sig Single Measures .563b .268 .848 7.429 8 32 .000 Average Measures .865c .647 .965 7.429 8 32 .000 Findings and Discussions The first hypothesis testing was to test whether or not there was any interaction among several independent variables both in the experimental group and control group. The first hypothesis testing was useful for investigating whether there was any interact- ion between the first and the second in- dependent variable on the dependent variable. Therefore, the hypothesis is as follows: H0 : There is no interaction between the students‟ learning motivation (X1) and the students‟ creativity (X2). H1 : There is interaction between the students‟ learning motivation (X1) and the students‟ creativity (X2). Table 6. Results of Multivariate Test toward the Experimental Group Experimental Group Multivariate Testsc Effect Value F Hypothesis Df Error Df Sig. Intercept Pillai's Trace .999 3.868e3a 2.000 5.000 .000 Wilks' Lambda .001 3.868e3a 2.000 5.000 .000 Hotelling's Trace 1.547e3 3.868e3a 2.000 5.000 .000 Roy's Largest Root 1.547e3 3.868e3a 2.000 5.000 .000 Motivation Pillai's Trace 1.099 1.830 8.000 12.000 .166 Wilks' Lambda .078 3.214a 8.000 10.000 .044 Hotelling's Trace 9.487 4.743 8.000 8.000 .021 Roy's Largest Root 9.242 13.862b 4.000 6.000 .003 Creativity Pillai's Trace 1.390 3.417 8.000 12.000 .027 Wilks' Lambda .049 4.391a 8.000 10.000 .016 Hotelling's Trace 10.425 5.212 8.000 8.000 .016 Roy's Largest Root 9.482 14.223b 4.000 6.000 .003 Motivation *Creativity Pillai's Trace .165 .493a 2.000 5.000 .638 Wilks' Lambda .835 .493a 2.000 5.000 .638 Hotelling's Trace .197 .493a 2.000 5.000 .638 Roy's Largest Root .197 .493a 2.000 5.000 .638 A. Exact Statistic B. The Statistic is an Upper Bound on F that Yields a Lower Bound on the Significance Level. C. Design: Interception + Motivation + Creativity + Motivation * Creativity Research and Evaluation in Education 171 - Volume 1, Number 2, December 2015 Based on the output of the multivariate significance test from multiple procedures (Pillai, Wilk‟s Lamda, Hoteling, and Roy‟s), as displayed in Table 6, the researchers found that the significant values in the line motivation*creativity are 0.638; 0.638; 0.638 and 0.638 respectively. All of these significance values are above 0.5 and they imply that the H0 is rejected and the H1 is accepted. Thereby, in the experimental group there is interaction between motivation and creativity. The finding shows that in the experimental group the change of meta- cognition and competence is solely determin- ed by the motivation and the creativity. Next, the first hypothesis testing is administered to the control group. Based on the output of multivariate significance test from multiple procedures (Pillai, Wilk‟s Lamda, Hoteling, and Roy‟s), as displayed in Table 7, the researchers found that the significance values in motivation*creativity line is 0.525; 0.598; 0.689 and 0.290 respectively. All of the significance values are above 0.5 and these values imply that the H0 is accepted and the H1 is rejected. Thereby, in the control group there is interaction between motivation and creativity. Table 7. Results of Multivariate Test in the Control Group Control Group Multivariate Testsc Effect Value F Hypothesis Df Error Df Sig. Intercept Pillai's Trace .994 3.095e2a 2.000 4.000 .000 Wilks' Lambda .006 3.095e2a 2.000 4.000 .000 Hotelling's Trace 154.742 3.095e2a 2.000 4.000 .000 Roy's Largest Root 154.742 3.095e2a 2.000 4.000 .000 Motivation Pillai's Trace .899 1.361 6.000 10.000 .317 Wilks' Lambda .282 1.175a 6.000 8.000 .404 Hotelling's Trace 1.898 .949 6.000 6.000 .525 Roy's Largest Root 1.457 2.428b 3.000 5.000 .181 Creativity Pillai's Trace .657 .815 6.000 10.000 .582 Wilks' Lambda .375 .844a 6.000 8.000 .570 Hotelling's Trace 1.583 .791 6.000 6.000 .608 Roy's Largest Root 1.527 2.545b 3.000 5.000 .170 Motivation * Creativity Pillai's Trace .508 .851 4.000 10.000 .525 Wilks' Lambda .538 .727a 4.000 8.000 .598 Hotelling's Trace .773 .580 4.000 6.000 .689 Roy's Largest Root .640 1.599b 2.000 5.000 .290 A. Exact Statistic B. The Statistic is an Upper Bound on F that Yields a Lower Bound on the Significance Level. C. Design: Interception + Motivation + Creativity + Motivation * Creativity The testing of second hypothesis was conducted in two stages; the first stage of the testing was to find whether there had been significant difference between the experimental group and the control group in terms of metacognition level and learning outcomes competence. Based on the results displayed in Table 8, the significance value of the group variables is 0.029. The significance value of the group variable is smaller than 0.05; therefore, the finding implies that there are differences in terms of the metacognitive capability (Y1) and the learning outcome competence (Y2) between the experimental group and the control group. Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 172 Edidas & Jalius Jama Table 8. The Significance of the Differences in Terms of the Metacognition and the Learning Achievements between the Control Group and the Experimental Group Tests of Between-Subjects Effects Dependent Variable: VALUE_Y df Mean Square F Sig. Source Type III Sum of Squares Corrected Model 2.188a 3 .729 3.407 .023 Intercept 792.616 1 792.616 3.703E3 .000 Group 1.070 1 1.070 4.998 .029 Variable_Y 1.091 1 1.091 5.094 .028 Group * Variable_Y .040 1 .040 .185 .669 Error 12.416 58 .214 Total 809.926 62 Corrected Total 14.603 61 a. R Squared = .150 (Adjusted R Squared = .106) The second stage of the testing was to test which group had bigger improvement in terms of metacognition and learning outcome competence. The results of the testing in the second stage is displayed in Table 9. The mean of metacognition variable for the control group is 3.604, while the mean of metacognition variable for the experimental group is 3.816. The two figures show that the metacognition level that the experimental group possesses is better than that of the control group. In other words, the program simulation learning by means of the trainer kit may increase the students‟ metacognition. Furthermore, the mean of learning outcome competence for the control group is 3.288 and the mean of learning outcomes competence for the experimental group is 3.608. The two figures show that the learning outcome competence level that the experimental group possesses is higher than that of the control group. Therefore, the results support the conclusion that the program simulation learning by means of trainer kit may increase the students‟ learning outcome competence. Table 9. The Statistic Description of Y Variables from the Control Group and the Experimental Group Descriptive Statistics Dependent Variable :VALUE_Y Mean Std. Deviation N Group Variable_Y Control Group Metacognition 3.603913e0 .4863139 15 Competence 3.287933E0 .4832891 15 Total 3.445923E0 .5027442 30 Experimental Group Metacognition 3.816175e0 .4981483 16 Competence 3.601381E0 .3752768 16 Total 3.708778E0 .4473527 32 Total Metacognition 3.713468E0 .4960546 31 Competence 3.449713E0 .4525148 31 Total 3.581590E0 .4892865 62 Research and Evaluation in Education 173 - Volume 1, Number 2, December 2015 Conclusion and Suggestions Conclusion There have been interactions between the class that has and the class that has not implemented the simulation program learning method by means of MCS51 Microcontroller Trainer Kit. The interactions have been marked by the interaction significance values of „motivation*creativity‟ and the values of both groups have been above 0.05. According to Pillai‟s, Trace, Wilks‟ Lambda, Hotelling‟s Trace dan Roy‟s Largest Root, the significance values for the group that has implemented the MCS51 Microcontroller Trainer Kit are 0.638; 0.638; 0.638 and 0.638, while the significance values for the group that has not implement- ed the MCS51 Microcontroller Trainer Kit are 0.461; 0.500; 0.573 and 0.182. The simulation program learning method by implementing the MCS51 Micro- controller Trainer Kit has shown better results for the experimental group rather than the control group. The evidence might be found in the results of univariate descriptive statistical text in the Descriptive Statistics table; in the table, the mean for the metacognition and the learning outcome competence for the group that has implemented the MCS51 Microcontroller Trainer Kit is higher than that of the group that has not implemented the MCS51 Micro- controller Trainer Kit. The mean for the group that has implemented the trainer kit is 3.816175, while the mean for the group that has not implemented the trainer kit is 3.603913. On the other hand, the mean of the learning outcome competence for the group that has implemented the trainer kit is 3.601381 and for the the group that has not implemented the trainer kit is 3.287933. Suggestions The results of the research show that the use of better, practical and reliable trainer kit in performing the laboratory practice might improve the students‟ creativity and, in turn, might also improve the students‟ think- ing awareness (metacognition) as well as the students‟ learning outcome competence. On the other hand, the effect of motivation and creativity individually has only been apparent on the improvement of the thinking aware- ness. Thereby, future researchers should find more tips in improving the learning process in order to improve the students‟ motivation and creativity since the improve-ment of the students‟ motivation and creavitiy will lead to the improvement of the learning outcome competence. For the teachers of micro- controller systems, it is suggested that they implement the trainer kit as the learning media because the trainer kit might improve the thinking awareness (metacognition) through the motivation and the creativity that appears during the laboratory practicum. References Alias, N. A. (2012). Design of a Motivational Scaffold for the Malaysian e-Learning Environment. Educational Technology & Society, 15(1), 137–151. Calhoun, C., & Finch, A.V. (1982). Vocational education: Concepts and operations (2 nd ed.). Belmont, California: Wadworth. Del Pópulo Pablo-Romero, M., Pozo-Barajas, R., & de la Palma Gómez-Calero, M. (2012). Evaluation of teaching the IS- LM model through a simulation program. Educational Technology & Society, 15(4), 193–204. Directorate General of Higher Education. (2011). Kajian tentang implikasi dan strategi implementasi KKNI [Study on the implication and strategy of Framework of Indonesian National Qualification]. Jakarta: Directorate of Higher Edu- cation. Eskrootchi, R., & Oskrochi, G. R. (2010). A study of the efficacy of project-based learning integrated with computer- based simulation-STELLA. Educational Technology & Society, 13(1), 236–245. Flavell, J.H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist Assosiation Inc., 34(10), 906-911. Research and Evaluation in Education The effectiveness of microcontroller instructional system... - 174 Edidas & Jalius Jama Hung, C.M., Hwang, G.J., & Huang, I. (2012). A project-based digital storytelling approach for improving students' learning motivation, problem-solving competence and learning achievement. Educational Technology & Society, 15(4), 368–379. Hurlock, E.B. (1988). Perkembangan anak jilid 2 [Children development volume 2] (M. Tjandrasa, Trans.). New York, NY: McGrow-Hill. Joyce, B., Weil, M., & Calhoun, E. (2009). Model of teaching (Model-model pembelajaran) (A. Fawaid & A. Mirza, Trans.). New Jersey: Upper Saddle River. Liu, T.C. (2010). Developing simulation-based computer assisted learning to correct students' statistical misconceptions based on cognitive conflict theory, using „Correlation‟ as an example. Educational Technology & Society, 13(2), 180–192. McClelland (1961). McClelland (Needs for affiliation, power, and achievement) Theory of Motivation. Retrieved from http://www.whatishumanresource.com /mcclelland-needs-for-affiliation- power-and-achievement-theory-of- motivation. Martin, H, (1992). Motivasi daya penggerak tingkah laku [Motivation of behavior activator]. Jakarta: Rineka Cipta. Maslow, A. (1954). Motivation and personality. New York, NY: Harper & Row. Minister of National Education. (2002). Keputusan Menteri pendidikan nasional Republik Indonesia nomor 045/U/2002 [The Decree of the Minister of National Education of the Republic of Indonesia Number 045/U/2002]. Jakarta: Minister of National Education. Piirto, J (2011). Creativity for 21 st century skill: How to emmbed creativity into curriculum. Rotterdam: Sense Publishers. President. (2012). Peraturan Presiden nomor 8 tahun 2012 tentang Kerangka Kualifikasi Nasional Indonesia (KKNI) [Presidential regulation number 8 year 2012 regarding the Framework of Indonesian National Qualification]. Jakarta: President of the Republic of Indonesia. Saemah, R., & Philips, J.A. (2006). Hubungan antara kesedaran metakognisi, motivasi dan pencapaian akademik pelajar universiti [The relationship between metacognitive awareness, motivation, and academic achievement of college students]. Jurnal Pendidikan, 31, 21-39. Sardiman, A.M., (2012). Interaksi dan motivasi belajar mengajar [Learning-teaching interaction and motivation]. Jakarta: Rajawali Press. Spencer, L.M. & Spencer, S.M. (1993). Competence at work: Models for superior performance. New York, NY: John Wiley & Sons. Wood, J.M. (2007). Understanding and computing Cohen’s Kappa: A tutorial. WebPsychEmpiricist. Retrieved from http://wpe.info/vault/wood07/wood0 7ab.html/. Zohar, A & Dori, Y.J. (2012). Metacognition in science education: Trends in current research. New York, NY: Springer Science + Business Media B.V. http://www.whatishumanresource.com/mcclelland-needs-for-affiliation-power-and-achievement-theory-of-motivation http://www.whatishumanresource.com/mcclelland-needs-for-affiliation-power-and-achievement-theory-of-motivation http://www.whatishumanresource.com/mcclelland-needs-for-affiliation-power-and-achievement-theory-of-motivation http://www.whatishumanresource.com/mcclelland-needs-for-affiliation-power-and-achievement-theory-of-motivation http://wpe.info/vault/wood07/wood07ab.html/ http://wpe.info/vault/wood07/wood07ab.html/