This study is intended to understand teaching quality of English student teachers when they conduct their teaching practicum. Teaching quality is conceptualized based on the principles of effective teaching resulted by teacher effectiveness studies. Thes IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 407 Experts’ Agreement of the Personalized m-Learning Curriculum Model Based on Fuzzy Delphi Method MOGANADASS RAMALINGAM 1 , SITI HAJAR BINTI HALILI 2 , AND SAEDAH BINTI SIRAJ 3 Abstract The purpose of this study was to identify the experts‟ agreement on the suitability of the personalized m-learning curriculum model. A total of 25 experts were selected to evaluate the model. A Fuzzy Delphi Method was used to identify suitability of the elements in personalized m-learning model. Based on the overall findings, all the items have met the requirements needed in the triangular fuzzy number and meet the deffuzification process (more than the Alpha α - cut value of 0.5). This revealed that all the experts consensually agreed with all questionnaire items. Hence, the proposed personalized m-learning curriculum implementation model is suitable to serve as a guideline for the instructor in implementing personalized m-learning. The new personalized m-learning elements for another course or programme could be determined based on the experts‟ opinions. This will enable the development of personalized m-learning model that could be implemented for any course or programme. Keywords Curriculum, experts, fuzzy delphi method, model, personalized m-learning 1. Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia; moganmmu@yahoo.com 2. Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia; siti_hajar@um.edu.my 3. Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia; saedah@um.edu.my mailto:moganmmu@yahoo.com mailto:siti_hajar@um.edu.my mailto:saedah@um.edu.my IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 408 Introduction Over the years, the technology used to support m-learning has evolved. Learning content that made available was not able to cater to a variety of mobile devices and many types of learners‟ need. Aspects like content reusability and content adaptation become more and more critical. Learning platforms or learning environment should better support variety of mobile devices and many needs of the learners will motivate the learners to have a better understanding of the content and acquire the knowledge required (Firas, Noraidah, & Tengku, 2019). M-learning attained more importance in supporting and complementing the formal and informal learning, especially among the young generation today. Extraordinary flexibility regarding time and location is the main characteristic of mobile devices which allow the learners to learn anywhere at any time without the restriction of time and space. Loidl-Reisinger and Paramythis (2003) describe m-learning setting as "anywhere, anytime, any data and any device", such as learning content can be retrieved, viewed or repeated via an arbitrary device from an arbitrary place. In order for a learner to access any data on any device, content adaptation and customization plays a major part. These adaptation and customization are needed for a learner with different devices and different preferences. The first step is the process of adaptation and customization is by making the learning content reusable, interoperable, more easily accessible and more durable (Svensson, 2001). The evaluation by the experts is based on the question: “What is the experts‟ agreement on the suitability of the personalized m-learning curriculum implementation model in the teaching and learning of Food and Beverage Service course in the hospitality programme?” Literature Review M-learning is not just about learning using portable devices but learning across contexts (Joorabchi, Mesbah, & Kruchten, 2013). So, any device m-learning is what we are looking at and moving towards. Beside mobile device, the content adaptation also will be based on the learners‟ preferences and learners‟ surroundings. With this, personalisation is added to mobile learning. To utilize the full potential of personalized m-learning, the application designers face many challenges and one of the major challenges is on how should content be converted and adapted for delivering to mobile devices with limited hardware, software, and communication capabilities without changing the actual content of the material and on top of that based on learners‟ preferences (Franz & Holger, 2002). Many technological challenges, such as small display size, low resolution, less memory power, low bandwidth, and input technology also become an obstacle in creating effective learning in a mobile environment. Personalized learning happens both in school and out of school. While personalized classroom learning dictates by the teacher's choice of new tech tools, the learner has their personal tech tools when they are out of school. Mobile devices are the popular choice for IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 409 out of classroom learning because mobile devices are very personal to the learner and almost every learner carry minimum one of these devices (Issham, Siti, & Thenmolli, 2016). Personalized m-learning is teaching a learner where they are (regardless of location), what they need in order to understand (preferences) and with the device of their choice. Traditional classroom learning to personalized learning is a shift on what the learners do and their attitudes towards the learning process. Most learning system usually ignores personalization feature, such as learning styles, learner's ability (knowledge level), learner's current need and learner's learning environment (network). They tend to deliver the same learning content to every learner. Delivering personalized content to the learner's need (learner's specific domain) is the main purpose of this study. Learners learn best when they are engaged with the content. In order to do that, learning content must be delivered according to their needs and preferences. The personalized content gives the learners the "RIGHT" content. By doing this, learning becomes flexible because they believe that they can manipulate the content. Personalized m-learning help learning becomes alive and real in time when learners exchange ideas with their peers and instructor (Kok, 2013). The ultimate goal of this personalized m-learning approach is to create a similar situation on how would learners respond to a one-to-one session with their peers and teacher/professional educator. This is likely a strategy to achieve learning goals for all type of learners (Knowles, 1984). Every learner is different. They have a different ability in learning a particular subject. Given the same content, some learners may easily understand it by reading but some learners may need illustration or example to understand it. On top of that, every learner has their own preferences in learning such as learning at a quiet environment, learning by playing games, learning by communicating or cooperating with other people, and others. When the learners carry out the learning process with their own preferences, they will understand the content easily and may extend or apply the knowledge they have learned (Quinn, 2011). Hence, content that is personalized to learners is needed to enhance the learning efficiency and effectiveness of the learners. Mobile phones or cell phones, or just mobiles for short, facilitate voice conversations as well as text messaging. Today's mobile devices are work like multi-functional but tiny computer capable of hosting a broad range of applications for both business and consumer use. The ever-growing category of mobile devices with vast capabilities allow the users to access the Internet for e-mail, instant messaging, text messaging and Web browsing, as well as work documents, contact list and more. For students, mobile devices are often seen as an extension of their desktop, even laptop computers. For them, work can be done anywhere and at any time even though there are away from the classroom that later can be synchronized with their desktop computers. With mobile technology changing almost daily, the users have various options to select their personal mobile devices based on their requirement (Keskín & Metcalf, 2011). With the correct choice, the users can have a productive day while being away from the classroom. A smart mobile technology choice enables the learners to carry around their classroom with them. Mobile device blends the functionality of a desktop computer, cell-phone, email access and web browser into one device. IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 410 Chen, Lee and Tsai (2011) presented ontology in mobile phone domain for the construction of a knowledge base. The concept of a mobile phone is divided into seven parts: model, hardware, software, standard, brand, shape and color. There are different elements that can be considered in m-learning domain. Zhang (2003) proposed a generic model for delivering personalized content to mobile users based on user profiles. Al-Hmouz (2012) proposed personalized model for mobile learners was based on three aspects which are context-based, content-based, and learner-based. Methodology The study was focused on identifying the suitability of personalized m-learning curriculum implementation model to a specific course, Food and Beverage Service, for diploma students in a private higher education institution in Malaysia. The fuzzy Delphi technique is used in this study to evaluate the model. These methods are primarily based on experts‟ opinion. Hence, the outcome of the study, which is the curriculum implementation model, was solely depending on the experts‟ selection and their opinion. The Delphi method is a forecasting process framework based on the results of multiple rounds of questionnaires sent to a panel of experts. This method was originally developed as a systematic, interactive forecasting method which relies on a panel of experts. This method was introduced by Dalkey and Helmer (1963). It is a survey method with three features: anonymous response, iteration and controlled feedback and finally statistical group response (Hsu, Lee, & Kreng, 2010). An expert is a person with extensive knowledge or ability based on research, experience, or occupation and in a particular area of study and as in the context of this study, mobile learning field. It is based on four „expertise‟ requirements: 1) knowledge and experience with the issue under investigation; 2) capacity and willingness to participate; 3) sufficient time to participate in the study; and, 4) effective communication skills in both written and in expressing priorities through voting procedure (Adler & Ziglio, 1996). This is to ensure the model is suitable to guide in the implementation of personalized m-learning as learning support for student enroll in this course. This model will serve as a guide in the effective incorporation of personalized m-learning in formal education. A group of specifically selected experts are used to evaluate the model. Experts were asked to look into the suitability of the elements in personalized m-learning, the relationship among these elements, the classification of these elements and the suitability of the model in the teaching and learning of a hospitality programme. The evaluation by the experts is based on the question: “What is the experts‟ agreement on the suitability of the personalized m-learning curriculum implementation model in the teaching and learning of Food and Beverage Service course in the hospitality programme?” Research design-fuzzy delphi method The Delphi panel consisted of experts from different geographic regions of the world. Kaufmann and Gupta (1988) introduced the Fuzzy Delphi method. It consists of - IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 411 fuzzy set theory and Delphi techniques (Murray, Pipino, & Van Gigch, 1985). This is an analytical method used in decision-making process which incorporates fuzzy theory in the traditional Delphi method. The Delphi technique is a method for the systematic solicitation and collection of judgments on a particular topic through a set of carefully designed sequential questionnaires interspersed with summarized information and feedback of opinions derived from earlier responses (Delbecq, Van de Ven, & Gustafson, 1975). It is a time- and cost-efficient method to obtain opinions from experts without physically bringing them together for a face-to-face meeting. One of the major advantages of the Delphi technique is anonymity which removes common biases occurring in face-to-face group settings (Listone & Turoff, 1975). This Delphi study method is able to overcome implicit weaknesses in group communication, such as confrontation, argumentation, or dominance by a few individuals. The experts, who are anonymous and independent, are free to express their own ideas without direct communication with each other. According to Linstone and Turoff (2002), the Delphi method involves several rounds of questionnaire surveys to elicit experts‟ opinion on an issue being investigated. This method is also known as consensus approach or inner-opinions consensus of a group of selected experts or Delphi polls of experts. RAND Corporation ("Research and Development") or just RAND is an American non-profit global policy think tank created in 1948 by Douglas Aircraft Company to offer research and analysis to the United States Armed Forces. According to its report in 1953, the Delphi technique was originally intended to solve the problems of the military (Dalkey & Helmer, 1963). Later it has evolved into a variety of disciplines that can be found on various articles and journals. Just to name a few, this method has been used in the field of education by Baggio (2008), in management area by Schmiedel and Brocke (2013), in sports by Eberman and Cleary (2011) and in a banking sector by Bradley and Stewart (2002). The Delphi technique is an expert opinion survey method with the following features:  Anonymous response: Experts has no knowledge of the identity of the other experts involved in the panel. According to Armstrong (1985), the relationship among samples does not exist and their opinions are classified but their ideas are integrated in the analysis of data. The advantage of this anonymity is that the experts would not face any pressure or influence in responding to their questionnaire.  Controlled feedback: Experts would be given the main ideas constructed from the group in the subsequent rounds of a questionnaire which allows the exports to re-evaluate their judgment and submit their responses again to the group.  Statistical: The experts‟ feedbacks are analyzed statistically which result in a splines graph. The top part of the graph indicates the experts' consensus opinion (50% of experts), which represent the overall exports' consensus opinion.  Convergence: The result will be determined as a result that converges after multiple rounds of feedback from the experts. Through this, the Delphi method archive its aim to make a decision based on consensus on a particular study. The method allows the integration of opinions that is gained independently IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 412 from each expert through multiple cycles of questionnaires for prediction outcomes. However, there are also weaknesses in this method where the process become more costly and the repetition of the research cycle is time-consuming as it needs repetitive surveys to allow forecasting values to converge (Hwang & Lin, 1987; Ishikawa, Amagasa, Shiga, Tomizawa, Tatsuta, & Mieno, 1993). Besides this, since people use linguistic terms such as „good‟ or „very good‟ to reflect their preferences (Hsu et al., 2010), the experts‟ judgments cannot be properly reflected in quantitative terms. Ambiguity might happen due to the differences in the meanings and interpretations of the expert‟s opinions. This can be overcome by combining fuzzy set theory and Delphi, which was proposed by Murray, Pipino, and Gigch (1985) and was named the Fuzzy Delphi Method (FDM). Participants In this phase, a panel of experts selected through purposive sampling to evaluate the model as described in the modified FMD above. In Delphi method, selection of experts is the most important step because it affects the quality of the result of the study (Jacobs, 1996). The technique of selecting the appropriate sample in the FDM is not a non-probability sampling (Hasson, Keeney and McKenna, 2000). This is because the samples were not selected randomly since they were chosen based on their knowledge and experience in the field of the study. A lecturer who has experience of more than five years is classified as an expert since they have experience in teaching and managing an ongoing basis (Berliner, 2004). According to Akbari and Yazdanmehr (2014), the term expert in the field of education refers to an individual who has more than five years based on their specific experience. In order for the study to reach its specific objectives, Linstone and Turoff (2002) suggested the panel of experts is from 5 to 10. Okoli and Pawlowski (2004) suggested from 10 to 18 experts to validate the model. According to Gordon (2009), the usual numbers of experts selected are between 15 to 35 experts to guarantee for comprehensive and reliable research findings. After considering the related factors, the number of experts selected to evaluate and validate the model was set to 25. Data collection and analysis The instrument used was a set of evaluation survey questionnaire which consisted of 28 questions. This questionnaire divided into three parts: 1) Experts‟ personal details; and 2) Experts‟ use of technologies; and 3) Experts' views of the model. The first part of the questionnaire was to elicit the experts' background information. The second part was to elicit the experts' use of mobile technologies. The third part was to elicit experts‟ view on the usability of the model using a 5-point linguistic scale as follows: 1 – Strongly Disagree, 2 – Disagree, 3 – Neutral, 4 – Agree, and 5 – Strongly Disagree. As the study employed FDM to evaluate the model, the procedures for this phase are as follow: IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 413 Selection of experts to evaluate the model; based on the experts' selection criteria explained in the previous section, a total of 25 experts were selected to evaluate the model. All the communication with the experts was done via email. Determine the linguistic scale based on triangular fuzzy; in order to address the issue of fuzziness among the experts‟ opinion, a linguistic scale was used to frame the experts' feedback. This linguistic scale is very much alike to the Likert scale with an additional of fuzzy numbers given to the scale of responses based on a triangular fuzzy number. For every response, there are three fuzzy values given to consider the fuzziness of the experts‟ opinions. The three values (Figure 1) consist of three levels of fuzzy value: minimum value (m1), most plausible value (m2), and maximum value (m3). Figure 1. Three fuzzy values Figure 1. Three fuzzy values The linguistic scale is used to change the linguistic variable to fuzzy numbers. The level of agreement scale should be in odd numbers, usually in 3, 5 or 7 points of linguistic scale. A higher scale would indicate that the response analysis is more accurate. Table 1 shows an example of a 5-point linguistic scale. Table 1. Sample of linguistic scale 5 Point Linguistic Scale Fuzzy Scale Strongly Agree 0.60 0.80 1.00 Agree 0.40 0.60 0.80 Moderately Agree/Neutral 0.20 0.40 0.60 Disagree 0.10 0.20 0.40 Strongly Disagree 0.00 0.10 0.20 Based on the example scale above, we could observe that the fuzzy numbers are in a range of 0 to 1. In this study, this 5-point linguistic scale was used as the fuzzy numbers for the responses. Calculating the average for fuzzy responses of experts; responses from the experts for each questionnaire item on their view of the model and their correspondent fuzzy number scales IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 414 were then inserted into an excel spreadsheet. The purpose of this is to get the average for m1, m2 and m3. This procedure is known as identifying the average responses for each fuzzy number (Benitez, Martín, & Román, 2007). They were calculated using the following formula: Identify the threshold value; in the next step, the difference between the experts‟ evaluation data and the average value of each item were calculated to determine the threshold value, „d‟. The formula was used to calculate the threshold value: In reference to the formula above, m₁, m₂ and m₃ are the average values for all the experts‟ opinions while n₁, n₂ and n₃ are fuzzy values for all three values for every user. The threshold value is essential in determining the consensus level among the experts. Cheng and Lin (2002) stated that all the experts are considered to have reached a consensus when the threshold value is less than or equal to 0.2. The overall group consensus should exceed 75% and any value less than this required a second round of fuzzy Delphi. Determine the percentage agreement; the overall consensus for all the items was determined based on the threshold value for each item. Murray et al. (1985) highlighted that the percentage agreement of all experts must be equal to or greater than 75%. Defuzzification process; defuzzification process is the final step in the evaluation phase. The data were analyzed using the average of fuzzy numbers. The defuzzification value, also known as fuzzy scores, (A) for each questionnaire item, was calculated using the following formula: A = 1/3 * (m₁ + m₂ + m₃). The calculation of defuzzification was used to identify which questionnaire items were agreed upon in evaluating the personalized m-learning curriculum implementation model. The data were later analyzed using a template Microsoft Excel. Findings The evaluation questionnaire divided into three parts. Thus, the findings of this evaluation phase will be presented in three parts. The first part of the survey questionnaire is about the experts' background information. The experts' background information is used to validate their expertise in evaluating the model. The second part presents the experts‟ use of mobile technologies in their daily life. This part will reveal how good they are with mobile technologies and how they use these technologies. The third part presents the experts‟ views on the suitability of the personalized m-learning curriculum implementation model. Their views and opinions will be used as a guideline for the instructor in implementing personalized m-learning to support formal classroom learning. IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 415 Background information of the experts A total of 25 experts were selected for this evaluation phase to evaluate the model, which was developed in phase 2 of the study. Table 2 shows the findings of the experts‟ background information. Table 2. Experts’ background information Item Frequency Percentage Gender Male 14 56.0 Female 11 44.0 Teaching/Working Experiences Below 5 years 0 0.0 5 - 10 years 12 48.0 11 - 20 years 10 40.0 Above 20 years 3 12.0 Highest Qualification PhD 8 32.0 Master 11 44.0 Degree 6 24 Diploma/Certificate 0 0.0 Field of work/expertise Education (m-learning/online learning) 13 52.0 Education (Science and Engineering) 6 24.0 Mobile Technologies and Interface Design 4 16.0 Information System/Technology 2 8 Table 2 reveals the background information of the 25 experts involved in this survey questionnaire. Based on the table, a total number of male and female experts who participate in this study represents 56% and 44% respectively. The findings show that the majority of the experts have teaching and working experience between 5 to 10 years (48%, n = 12). Very close to this number are experts from 11 to 20 years of experience (40%, n = 10). There are also experts with more than 20 years of experience (12%, n = 3). In terms of their academic qualification, majority of the experts (44%, n = 11) possessed masters as their highest qualification, 32% (n = 8) with PhD, and balance 24% (n = 6) with a basic degree (Table 2). In terms of field of expertise, out of the four major categories given, majority of the experts were from the field of education with a specialized area in m-learning and online learning (52%, n = 13) and (24%, n = 6) are specialized in science and engineering. Whereas, (16%, n = 4) experts were expertise in mobile technologies and interface design. The rest of the experts were from the information system and/or technology field. Use of mobile technologies This part will reveal how good the experts are with mobile technologies and their technical skills with mobile devices. Table 3 shows the findings in the aspect of experts‟ IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 416 mobile device related skills. The findings reveal that 64% (n = 16) of the experts claimed that they were moderate in terms of their mobile device-related skills and 36% (n = 9) of them claimed that they were skillful. Table 3. Experts’ mobile device-related skills Items Frequency Percentage Skillful (Develop and managing website or/and blogs, content creation for an online system) 9 36.0 Moderate (Able to communicate through social networks like Facebook, Twitter, Instagram, etc.) 16 64.0 Low skilled (Browse and search for information on the Internet; use of office tools such as spreadsheets, words, powerpoint; receive and sending emails) 0 0.0 None 0 0.0 Total 25 100.0 Table 4 shows the findings in terms of experts‟ mobile device technical skill level. The findings indicate that majority of the experts (60%, n = 15) claimed that they were highly skilled, whereas the remaining 40% (n = 10) of the experts indicates that they have average technical skill with mobile devices. Table 4. Experts’ mobile device technical skill level Items Frequency Percentage High 15 60.0 Average 10 40.0 Low 0 0.0 Total 25 100.0 Based on the analysis results from all the three tables above (Table 2, Table 3 and Table 4), the selected survey questionnaire participants fit the description as experts in evaluating the model in this phase. According to Pawlowski, Suzanne, and Okoli (2004), experts selected for a specific Delphi study should have some background or experience in the related field of study, to be able to contribute their opinions to the needs of the study, and willing to revise their initial judgment to reach consensus among other experts in the team. In terms of experts' background experience and academic qualification in the related study field, the findings showed that majority of the participants specialize either from m-learning or online learning in the field of education. This indicates that these selected experts were suitable to evaluate the personalized m-learning curriculum model of the study. Besides this, the experts also have some mobile device-related skills and the majority of them claimed that they have high technical skill with mobile devices. These criteria are an added advantage in evaluating the model. Thus, based on the findings in this part, the selected respondents were qualified as experts in this phase. IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 417 Experts’ views on the suitability of the personalized m-learning curriculum implementation model The responses of the participants (experts) to the evaluation survey questionnaire were based on the five-point linguistic scale. Based on the responses collected from participants, the threshold value „d‟ was calculated for all the questionnaire items. This is to determine the level of consensus among experts for each item in the questionnaire. The process of calculating the threshold value „d‟ was based on this formula: The overall threshold value „d‟, was calculated as: [475 (total experts‟ responses) – 48 (total responses more than 0.2) ÷ 960] x 100%= 89.89%. This indicates that the threshold value „d‟, has exceeded 75%. This means that the experts have reached the required consensus in their views for all questionnaire items of the evaluation survey questionnaire in evaluating the personalized m-learning curriculum implementation model for Food and Beverage course in the hospitality program. Discussion The analysis of questionnaire findings for FDM is based on the requirements contained in the triangular fuzzy number and deffuzification process. The triangular fuzzy number take into consideration of the threshold value „d‟ and the percentage of the experts‟ consensus for each item. The threshold value „d‟ for each item measured must be less than or equal to 0.2 to indicate the expert's consensus with other experts for a particular item in the survey questionnaire. Whereas, the percentage of agreement of the experts must be more than or equal to 75%. As for the deffuzification process, there is only one condition which is the Fuzzy Score (A) must be greater than or equal to the α-cut value of 0.5. Based on the overall findings, all the items have met the requirements needed in the triangular fuzzy number (less or equal to 0.2 for threshold value, d, and more than 75% for the percentage of experts' consensus) and deffuzification process (more than the Alpha α - cut value of 0.5). This revealed that all the experts consensually agreed with all questionnaire items. Hence, according to the experts, the proposed personalized m-learning curriculum implementation model is suitable to serve as a guideline for the instructor in implementing personalized m-learning to support formal classroom learning. The experts were consensually agreed that the model shows a clear guide on how personalized m-learning could be conducted in complementing the conventional face-to-face classroom learning. The experts also consensually agreed to the model that shows clearly on the elements that need to be considered before designing a curriculum to implement a personalized learning for mobile devices in order to provide personalized learning experience. The model also shows clearly on how elements from different domain could merge to offer a holistic learning experience for the students. The experts also consensually IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 418 agreed to the model, which clearly personalized m-learning elements are connected to each other in aiding the learners in achieving the course's learning objectives. Findings also revealed that the model could be used to assist planning of course unit lessons by the lecturer in facilitating students‟ in personalized m-learning, and the model could be used as an example to develop other curriculum implementation models for other courses. Based on the threshold value 'd' and the defuzzification values, the findings conclusively suggest that the experts have consensually agreed to all the items in the evaluation aspects of the model. This concluded that the experts consensually agreed that the proposed personalized m-learning model is suitable to be used as a guide for the lecturers to implement personalized m-learning as learning support in teaching and learning process. Conclusion and Recommendations The study was focused on designing personalized m-learning curriculum model to a specific course, Food and Beverage Service, for diploma students in a private higher education institution in Malaysia. But the study through the development of the model and the result of this study will also be able to contribute to the implementation of personalized m-learning for other courses as well. This pedagogical model can also be used to support formal classroom teaching and learning. Through this learning method of personalized m-learning, the students can explore the new way of learning which gives them control over the learning materials that they want to receive. This will bring excitement to the students and they will be able to keep their attention focused on their own adopted materials. The new personalized m-learning elements for another course or programme could be determined, if any, based on the opinions from selected panel of experts. This will enable the development of personalized m-learning model that could be implemented for any course or program. Disclosure statement No potential conflict of interest was reported by the authors. References Al-Hmouz, A. (2012). An adaptive framework to provide personalisation for mobile learners. Retrieved from https://ro.uow.edu.au/cgi/viewcontent.cgi?referer=https://www.google.com/&htt psredir=1&article=4467&context=theses Akbari, R., & Yazdanmehr, E. (2014). A critical analysis of the selection criteria of expert teachers in ELT. Theory and Practice in Language Studies, 4(8), 1653-1658. Armstrong, J. S. 1985. Long-range forecasting (2 nd ed.). New York: Wiley. Chen, B., Lee, C.Y., & Tsai, I.C. (2011). Ontology-based e-learning system for personalized learning. International Conference on Education, Research and Innovation, 38-42. IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 419 Baggio, B. (2008). Integrating social software into blended-learning courses: A Delphi study of instructional-design processes. (Doctoral dissertation, Capella University). Retrieved from ProQuest Dissertations Publishing, 3297938. Benitez, J., Martín, J., & Román, C. (2007). Using fuzzy number for measuring quality of service in the hotel industry. Tourism Management, 28(2), 544-555. Bradley, L., & Stewart, K. (2002). A Delphi study of the drivers and inhibitors of Internet banking. International Journal of Bank Marketing, 20(6), 250-260. Cheng, C., & Lin, Y. (2002). Evaluating the best main battle tank using fuzzy decision theory. European Journal of Operational Research, 142(1), 174–186. Dalkey, N., & Helmert, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458-467. Delbecq A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: a guide to nominal group and Delphi process. Glenview, IL: Scot, Foresman and Company. Eberman, L., & Cleary, M. (2011). Development of a heat-illness screening instrument using the Delphi panel technique. Journal of Athletic Training, 46(2), 176-184. Firas, L.K., Noraidah, S. A., & Tengku, S.M. (2019). An empirical study on gamification for learning programming language website. Jurnal Teknologi, 81(2), 151-162. Franz, L., & Holger, N. (2002). The role of mobile devices in e-learning – first experience with a wireless e-learning environment, Wireless and mobile technologies in education. Proceedings. IEEE International Workshop on Wireless and Mobile Technologies in Education, 103 –106. Gordon, T. J. (2009). The Delphi method. Futures Research Methodology, 3(0), 1–29. Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015. Hsu, Y.-L., Lee, C.-H., & Kreng, V. B. (2010). The application of Fuzzy Delphi method and Fuzzy AHP in lubricant regenerative technology selection. Expert Systems with Applications, 37(1), 419–425. Hwang, C., & Lin, M. (1987). Decision making under multiple criteria. Berlin: SpringerVerlag. Issham, I., Siti, N., A., & Thenmolli, G. (2016). Mobile learning in Malaysian universities: Are students ready? International Journal of Interactive Mobile Technologies, 10(3), 17– 23. Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993). The max-min Delphi method and fuzzy Delphi method via fuzzy integration. Fuzzy Sets and Systems, 241–253. Jacobs, J. M. (1996). Essential assessment criteria for physical education teacher education programs: A Delphi study (Unpublished doctoral dissertation). West Virginia University, Morgantown. Joorabchi, M. E., Mesbah, A., & Kruchten, P. (2013). Real challenges in mobile app Development. Retrieved from http://www.ece.ubc.ca/~amesbah/docs/mona-esem13.pdf Kaufmann, A., & Gupta, M. M. (1988). Fuzzy mathematical models in engineering and management science. Elsevier Science Inc. New York, NY, USA. IRJE | Vol. 3 | No. 2| Year 2019 |E-ISSN: 2580-5711 420 Keskín, N., & Metcalf, D. (2011). The current perspectives, theories and practices of mobile learning. Turkish Online Journal of Educational Technology, 10(2), 202-208. Kingsley, O. (2018). A systematic review of process modeling methods and its application for personalized adaptive learning systems. Journal of International Technology and Information Management, 27(3), 23-46. Knowles, M. (1984). The adult learner: a neglected species, (3rd ed.). Houston: Gulf Publishing. Kok, A. (2013). How to manage the inclusion of e-learning in learning strategy. International Journal of Advanced Corporate Learning, 6(1), 20-27. Linstone, H. A., & Turoff, M. (2002). The Delphi method techniques and applications. London: Addison-Wesley. Loidl-Reisinger, S. & Paramythis, A. (2003). Distance education - A Battlefield for Standards. In The Quality Dialogue. Integrating quality cultures in flexible, distance and elearning; Proceedings of the EDEN Annual Conference, 89-94. Murray, T. J., Pipino, L., & Van Gigch, J. P. (1985). A Pilot Study of Fuzzy Set Modification of Delphi. Human Systems Management, 6–80. Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29. Quinn, C. N., (2011). Designing m-learning: Tapping into mobile revolutions for organizational performance. San Francisco: Pfeiffer. Svensson, M. (2001). E-learning standards and technical specifications. Technical report, Luvit AB. Ting, Y. L. (2013). Using mobile technologies to create interwoven learning interactions an intuitive design and its evaluation. Computers & Education, 60(1), 1- 13. Zhang, D. (2003). Delivery of personalized and adaptive content to mobile devices: a framework and enabling technology, Communications of the Association for Information Systems, 12(1), 13. Biographical notes MOGANADASS RAMALINGAM is a postgraduate student in Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia. His research interest is in personalised m-learning curriculum and TVET education. SITI HAJAR BINTI HALILI is a senior lecturer in Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia. Her research interest is in flipped learning, instructional design and technology and education 4.0. SAEDAH BINTI SIRAJ is a professor in Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia. Her research interest is in future studies, curriculum development and instructional design and technology.