Australasian Journal of Educational Technology, 2021, 37(3). 71 Influence of the NAO robot as a teaching assistant on university students’ vocabulary learning and attitudes Hossein Banaeian, Ilkay Gilanlioglu Eastern Mediterranean University, North Cyprus This study investigated how the NAO robot as a teaching assistant affects the way university students learn vocabulary and their attitudes towards it. A mixed method approach was followed to gather both quantitative and qualitative data. A quasi-experimental design, including a pre-test and a post-test, was employed to explore the impact of the NAO robot on students’ vocabulary learning. Moreover, a questionnaire and an interview were used to identify the attitudes of the participants towards the NAO robot. The participants, who were taking a vocabulary course for language improvement as part of the regular curriculum of the program during the experiment, were freshman students of the English language teaching undergraduate program at a higher education institution in North Cyprus. Of the two groups, the control group performed slightly better than the experimental group even though the analyses did not show any statistically significant difference between them. Descriptive data analysis showed that most of the students liked the NAO robot and its abilities. However, related findings from the qualitative data were mixed. Most participants liked the NAO robot and thought that the robot helped them to learn the new words, while some thought that the technology needed to be improved. Implications for practice or policy: Evidence of vocabulary gains and students’ generally positive attitudes mark the NAO robot as a promising tool in language learning and teaching. The benefits and drawbacks of the NAO robot should be taken into account prior to its integration into second language vocabulary learning and teaching. In alignment with the continuous assessment of developments in robot-assisted language learning research, teachers should be provided with professional training for using the NAO robot effectively with various learners. Keywords: NAO robot, RALL, computer-assisted language learning (CALL), attitudes, vocabulary learning and teaching, mixed method Introduction A sufficient command of vocabulary is an important component of any language, enabling people to communicate. In other words, communicating in a particular language is almost impossible without sufficient lexical knowledge, which involves receptive and productive aspects of words (Nation, 2001). Vocabulary has gained more prominence as we have come to know more about its nature and behaviour as an outcome of the employment of the computer in language data analysis. The availability of the computer and modern software revolutionised the ways in which corpora is lexically analysed. Lexical analysis of large texts changed radically, that is, from manual analysis to digital analysis (Kennedy, 2014), and from intuition-driven analysis to systematic data-driven analysis (Hunston, 2002).. As a result, the frequencies of different meanings of polysemous words, concordances and collocations became available, which increased lexical awareness and led to endeavours ranging from lexicography to pedagogy. From this renewed perspective, corpus-based dictionaries (e.g., Collins COBUILD English Language Dictionary – Sinclair, 1987) and dictionaries of collocations (e.g., Longman Collocations Dictionary and Thesaurus – Cleveland-Marwick, 2013) have been produced. This linguistic evidence has impacted pedagogical practices in vocabulary teaching; that is, the choice of which vocabulary to teach and how to teach it (Cehan, 2014; Willis, 1990). Computer technology advances have offered various digital devices, which recently have become even more commonly employed, particularly in language learning and teaching. As an application for vocabulary, Dalton and Grisham (2011) suggested 10 ways (known as eVoc) of integrating technology with learning and teaching vocabulary (p. 306). Australasian Journal of Educational Technology, 2021, 37(3). 72 Computers as digital devices have been used to make vocabulary learning easier and more effective. Some studies within the realm of computer-assisted language learning in the last few decades (e.g., Hung et al., 2016; Nejati et al., 2018; Yanguas, 2009) have reported that computer use can positively impact language learning, and vocabulary learning in particular. Portable devices as derivatives of computer technology soon succeeded the computer and made their way into our lives as well as education. The use of mobile devices for language learning is known as mobile-assisted language learning. Similar to the evidence in computer-assisted language learning, many studies (e.g., Browne & Culligan, 2008; Motallebzadeh & Ganjali, 2011) found that mobile devices positively affect vocabulary learning. More recently, a new type of computer technology has been introduced – robots. With their intelligence, mobility and high capability, robots have even featured in catering (e.g., ASIMO – the Japanese robot; Chen et al., 2010). Robots have also been used for educational purposes. However, using robots in educational contexts does not have a long history. Papert (1993) was the first person to use an educational robot. Robots are usually used to teach science, mathematics and languages. Employing robots in facilitating language instruction is known as robot-assisted language learning (RALL; Han, 2012). So far, RALL research has looked into the impact of the robot on young learners (e.g., Aidinlou et al., 2014; Hong et al., 2016); however, the influence of this technology on adult language learners and their attitudes towards it remains unexplored but for a few exceptions (e.g., Rosenthal-von der Pütten et al., 2016; Schodde et al., 2017). Hence, this study was conducted to determine the possible effect of a humanoid robot on university students’ vocabulary learning, specifically word meaning, and their attitudes towards it. The study sought to answer the following research questions: Could a robot as a teaching assistant aid students’ vocabulary learning? What are the students’ attitudes towards the robot and using it for vocabulary learning? Literature review Robot use and vocabulary learning The importance of vocabulary has been demonstrated by many scholars (e.g., Nation, 2001; Read, 2000; N. Schmitt, 2000; N. Schmitt et al., 2001). Knowing a word means being aware of its productive and receptive aspects, which are related to writing and speaking skills, and listening and reading skills, respectively (Nation, 2001). In order for learners to master both productive and receptive aspects, they should be exposed to words through diverse channels (Meghdari et al., 2013). In other words, learners should receive input from various sources in different forms, which reinforces the vocabulary in the mind of the learner. Thus, including several types of stimuli in the vocabulary learning and teaching process is of great importance. Moreover, it is well known that learners have different learning styles, adding to the significance of utilising a variety of stimuli. Therefore, using robots such as the NAO robot, which are able to perform various functions (e.g., native speaker accent, gestures, body movements and actions), can be more effective in comparison with previous technologies. Transferring the newly learned word from short-term to long-term memory is a crucial step in the vocabulary learning process. Robots can be used to reinforce this transfer as they have attributes such as interaction, repeatability, humanoid appearance, intelligence and adaptability (Meghdari et al., 2013). To provide theoretical justification for using the humanoid robot in language learning, and specifically vocabulary learning, we will draw on the two most relevant features. Interaction enables learners to engage in meaningful exchanges, which promotes language use and therefore facilitates language acquisition. It has been claimed that conversational interaction is necessary for second language learning (Gass, 1997; M. Long, 1996; M. H. Long, 1983; Pica, 1994). The processes of modified interaction (M. H. Long, 1983) and negotiation for meaning and corrective feedback are essential in second language acquisition (M. Long, 1996). Student-robot interaction approximates to human-human interaction in that it offers opportunities for asking and answering questions, giving examples, repeating utterances, checking comprehension and correcting themselves. Thus, a robot appears to be a useful conversational partner for interaction, which can facilitate vocabulary learning in particular and language learning in general. Repeatability can be employed to foster vocabulary learning through repeated exposure (Hiebert & Kamil, 2005; Nation, 2001; Australasian Journal of Educational Technology, 2021, 37(3). 73 Thornbury, 2002) and recycling (Cameron, 2001). The NAO robot possesses all the above-mentioned characteristics, providing an opportunity for learners to assimilate new words and incorporate them into their lexical knowledge. Robot use may also help reduce learners’ anxiety and increase their motivation. Many studies (e.g., Aidinlou et al., 2014; Alemi et al., 2017; Shin & Shin, 2015) have revealed that robot use can not only reduce the anxiety of the learners but can also incentivise them, which may lead to better language learning. Such findings are in line with Krashen’s (1982) affective filter hypothesis, which is concerned with the reduction of students’ anxiety and stress. Robots can make vocabulary learning tasks more interesting and enjoyable. This means that the tasks will be catchier; thus, students will be more engaged, and in this way active inquiry will be promoted (McClanahan, 2014). Particularly for adult learners of English as a foreign language, learning vocabulary through interaction with a robot with a native-speaker accent may be a great source of motivation. We can conclude from the discussion in this section that the NAO robot possesses many distinctive characteristics befitting both theory and research, well justifying its deployment. In the following sections, we will look at RALL and some studies that have integrated robots into language learning. RALL Robots have more capabilities in comparison with computers and mobiles. Robots are intelligent and able to work autonomously. They can act like a person and may have a personality and a history. These devices are able to listen, talk and walk. There are two major types of robots – hands-on and educational. The first type is used in science and engineering to increase the students’ creativity; the second type is more appropriate for tasks such as language learning and teaching. The main characteristic of educational robots is artificial intelligence, which enables them to act independently or be operated remotely (for a review of different types, see Aidinlou et al., 2014). Educational robots can play a variety of roles in the classroom to facilitate the learning process. These roles are diversified with some degree of overlap by scholars. According to Mubin et al. (2013), robots can play the role of a tool, a peer or a tutor. Aidinlou et al. (2014) suggested a set of three categories: learning material, learning companion/pets and teaching assistant. Within these roles, robots can perform several activities, such as storytelling, question and answer and cheerleader (p. 16). There are many factors, such as task type and students’ needs, that can affect the role that robots play. In the light of the literature, the NAO robot’s capabilities and the context of this study were carefully considered, and it was decided that the NAO robot be used as a teaching assistant. This particular role was thought to be most suitable for the given context, in which adults are taught vocabulary interactively. The NAO robot was seen as a complementary tool in teaching rather than as a substitute for the teacher. It was programmed to facilitate vocabulary learning by providing word meaning, examples and feedback in interaction with students. Used in its full capacity, the NAO robot was intended to boost vocabulary learning in several respects. First of all, the robot represented another example of diverse channels, through which students were exposed to word meaning in addition to the coursebook and the teacher. The students also had the chance to negotiate for meaning. During this interaction, multiple exposures to word meaning were available. They could ask the robot to repeat the answers again and again. Lastly, as also discussed in the literature, robot use can decrease learner anxiety and increase motivation, which may lead to better vocabulary learning. In brief, the presence of the robot used in its predefined role provided a context for students where they could learn vocabulary better through interaction, negotiation for meaning and repeated exposure to various stimuli. Related studies RALL research is varied in language focus, learner characteristics, robot features and method of use. So far, it has mostly focused on young language learners with few exceptions, in which certain language skills and language areas have received varying attention. There has also been some research into the impact of the robot on learners’ attitudes and motivation. These studies differed in terms of robot type, role employed and method of use. Australasian Journal of Educational Technology, 2021, 37(3). 74 Concerning reading skills, robots may help young learners as either an assistant or a tutor (e.g., Hsiao et al., 2015; Hyun et al., 2008). Regarding speaking, however, the findings seem to be mixed (van den Berghe et al., 2019). Although some studies have reported a positive effect on the speaking skills of school-aged language learners (e.g., Lee et al., 2011; Wang et al., 2013), others have not found such an impact on the same age group (e.g., Hong et al., 2016; In & Han, 2015). Interestingly, the studies conducted by Lee et al. and In and Han took place in the same context (South Korea) with learners of the same age group, but they produced conflicting results. On the other hand, studies concerned with the same language skill (i.e., speaking) where adult learners were involved revealed positive and more consistent results (e.g., Iio et al., 2018). In the latest literature, two studies have so far explored the impact of a humanoid robot on the grammar learning of school-aged language learners, both of which reported a positive influence (e.g., Herberg et al., 2015; Kennedy et al., 2016). With regards to vocabulary learning, research has produced mixed results. While some (e.g., Gordon et al., 2016; Movellan et al., 2009) reported small or no gains by young learners, substantial gains were found by others (e.g., de Wit et al., 2018; Tanaka & Matsuzoe, 2012). A similar discrepancy exists between studies targeting school-aged participants and adults. More specifically, regarding school-aged language learners, some studies (e.g., Meiirbekov et al., 2016) reported small vocabulary gains (3 out of 10 words), while others (e.g., Alemi et al., 2014; Eimler et al., 2010) found substantial gains. For adults, substantial vocabulary gains were reported; however, no significant difference was found between the study groups (Schodde et al., 2017). Another related line of research into the relationship between robot use and affective factors, such as motivation and attitudes, depicts the favourable effects of robots on students’ engagement, confidence and motivation (e.g., Alemi et al., 2017; Hsiao et al., 2015; Shin & Shin, 2015); however, students’ satisfaction and how this technology affects their motivation and attitude need to be further explored. Finally, it should be noted that these mixed findings may be rooted in different types of robots, the role of the robots and the methods of use employed by the researchers. In terms of method of use, for instance, some studies, including the present one, have used the robot autonomously (e.g., Alemi et al., 2014; de Wit et al., 2018), while others have used the robot in a tele-operated fashion (e.g., van den Berghe et al., 2018). As a consequence, there is a clear need for systematic studies on the possible impact of robot use on adults’ vocabulary learning and learners’ attitudes. The current study attempts to fill this gap. Method Aims of the study This study aimed to determine whether the NAO robot as a teaching assistant has an effect on adult language learners’ vocabulary learning and to identify their attitudes towards this technology and its use. Setting and participants This study was conducted in an English-medium higher education institution in North Cyprus. The participants were freshman students of the English language teaching undergraduate program taking a vocabulary course – one of the compulsory language improvement courses in the curriculum – and two instructors teaching the course. Participation was voluntary, and written consent was obtained in advance from all participants – instructors and students. This study was conducted in the spring semester of the 2017–2018 academic year. Prior to the study, a total of 65 students had been divided by the department administration into two fairly homogenous classes according to their proficiency test scores (33 students in one class and 32 in the other). A combination of convenience sampling and purposive sampling techniques was applied (Creswell, 2009). We had convenient access to the two classes of students but were not granted the opportunity for randomisation. Accordingly, a quasi-experimental research design was employed, where one class was designated as the experimental group (henceforth referred to as the RALL group) and the other as the control group (henceforth referred to as the non-RALL group). Furthermore, purposive sampling was used to make the sample as representative as possible. Based on the pre-test results, 18 students from each class qualified to be participants in the study. Those eliminated continued to attend their regular class in order to sustain integrity and to avoid disruption; however, their vocabulary learning performances were not considered. Australasian Journal of Educational Technology, 2021, 37(3). 75 Nevertheless, as the RALL group experienced the whole process and were exposed to the NAO robot, they were eligible to participate in the questionnaire on attitudes. The RALL group consisted of 6 males and 12 females, with an average age of 20.44 (SD = 2.03). The non- RALL group was composed of 4 males and 14 females, with an average age of 19.61 (SD = 1.94). Twenty- nine of the 33 students in the RALL group volunteered to fill in the questionnaire designed to determine students’ attitudes. Ten of those who completed the questionnaire participated in the interview. Three of the interviewees were male, and the rest were female. It should also be noted that two of them (both females) were native speakers of English. Therefore, the interviewees were divided into native participants (NP) and non-native participants (NNP). Instruments The NAO robot The NAO robot, designed by the Aldebaran Robotics company in 2008 in France (taken over by SoftBank Robotics in 2012), is a humanoid social robot which can work autonomously or be tele-operated. This makes it an appropriate option with high flexibility which can be used in different learning contexts. It is 58 centimetres tall and weighs 5.6 kilograms. An ATOM Quad core processor with 4GB RAM enables it to process the programs quickly and act accordingly. It can be programmed to speak about 20 languages with a native accent. The NAO robot is equipped with an audio system which includes voice recognition. Another feature is its face detection system using two digital cameras installed on its face. Thanks to this facility, the robot is able to detect human faces and make eye contact. Mobility is another important characteristic of the NAO robot. With the help of flexible joints, it moves like the human body does (i.e., walks around, bends, dances and gets back on its feet when it falls down). These systems and functions enable the robot to keep track of its interactants even while they are moving around (for further details on the robot, refer to http://www.softbankrobotics.com/emea/en/nao). Similar to the human body, the vital parts (such as the processor, audio system and the face detection system) are located in the head of the NAO robot. Figure 1 illustrates the NAO robot’s technical features. Figure 1. Technical features of the NAO http://www.softbankrobotics.com/emea/en/nao Australasian Journal of Educational Technology, 2021, 37(3). 76 The pre-test A pre-test took place in a formal class session to check the familiarity of learners with the 88 words taken from their coursebook Focus on Vocabulary 2: Mastering the Academic Word List (D. Schmitt & Schmitt, 2011), which had been selected by the department. This vocabulary test is an adapted version of a task in the coursebook. This task was adapted by asking students to provide proof of their knowledge of the word by giving an example or its meaning. The test is a reliable tool for checking learners’ familiarity with words (Nation, 2001). The purpose of this test was to identify a list of words which were completely unknown to the participants; therefore, it was chosen to eliminate the chance factor. The words were selected in consultation with course instructors from upcoming chapters. Only those words which were marked as completely unknown at the receptive level (options 1 & 2 of the pre-test) were selected as the target words (23 out of 88). The results also reduced the number of participants from 65 to 36, three of whom were native speakers of English. Figure 2 presents a snapshot of the pre-test with a sample item with instructions. Figure 2. Sample item of pre-test with instructions The post-test A vocabulary test was adapted and administered in a formal class session to examine the students’ vocabulary knowledge after the treatment. A matching test was designed following N. Schmitt et al. (2001), based on the target words. Figure 3 presents a sample item with instructions. Australasian Journal of Educational Technology, 2021, 37(3). 77 Figure 3. Sample item with instructions from the post-test, adapted from N. Schmitt et al. (2001, p. 82) This test is known to be a reliable instrument for examining vocabulary knowledge (particularly word meaning) at the receptive level. The test was piloted with two adult native speakers of English. We were assured that there was no ambiguity within the test. Additionally, an inter-rater reliability check between two qualified English teachers was applied. The result was found to be kappa = 1.00 (p < 0.05), indicating almost perfect agreement. Table 1 below illustrates how the kappa value can be interpreted: Table 1 Interpretation of Cohen’s kappa (McHugh, 2012, p. 279) Value of kappa Level of agreement % of data that are reliable 0–.20 None 0%–4% .21–.39 Minimal 4%–15% .40–59 Weak 15%–35% .60–.79 Moderate 35%–63% .80–.90 Strong 64%–81% Above .90 Almost perfect 82%–100% The questionnaire Lee et al. (2011) used a questionnaire in English with a 4-point Likert scale to check the students’ satisfaction with the robots (i.e., Mero & Engkey). The Cronbach alpha reported in their study was .73. We adapted this questionnaire by inserting a fifth option – neutral – in the scale and modifying a few items to suit the context of the study. The Cronbach alpha of the adapted questionnaire used in the study rose to .88. The interview A semi-structured interview consisting of 9 open-ended questions was designed to find out what students really thought about the NAO robot and its use. Interview sessions were conducted in English and lasted 10 to 15 minutes. The interview technique was used to collect qualitative data which could be triangulated with the experimental and the questionnaire data. Interviews were recorded with the interviewees’ permission and later transcribed. Procedures The following section outlines the procedures followed in the three main phases: pre-experiment, while- experiment, and post-experiment. Australasian Journal of Educational Technology, 2021, 37(3). 78 The pre-experiment phase The pre-experiment phase involved a series of pre-planned stages followed systematically. Firstly, the robot was programmed using the Choregraphe NAO software 1.14.5 (for more details, please refer to http://doc.aldebaran.com/1-14/software/choregraphe/index.html) to function as a teaching assistant in autonomous mode, providing answers to the students’ questions about the target words in two similar ways. Students could ask the meaning of a target word in several ways. The student-robot interaction process is illustrated in Figure 4. Secondly, a meeting was held with the instructors of the two groups where the NAO robot’s abilities were demonstrated. Within this session, the method of using the robot was discussed. The instructors were briefed on the methodology of teaching vocabulary with and without the robot. For the sake of uniformity, the method of teaching, the definitions and examples provided for the learners were all the same in both groups, except that the RALL group was exposed to the robot. In addition, the RALL group instructor was trained in using the robot appropriately in her classroom. Thirdly, the pre-test was administered. According to the results of the pre-test, participants and target words were selected. Both groups were given the test in a formal class session simultaneously. The final stage comprised the introduction of the robot to the participants in the RALL group. This session aimed to familiarise the participants with the robot’s capabilities, control for the novelty effect of the robot and help reduce possible resistance. Studies have highlighted proper introduction of the robot (Vogt et al., 2017; Westlund et al., 2016). Students interacted with the robot and asked the NAO robot some questions. Additionally, some of the robot’s abilities, such as walking and dancing, were demonstrated to the learners. Figure 4. Student-robot interaction flowchart The while-experiment phase The experiment consisted of four sessions over 2 weeks. The participants in the RALL group had the chance to interact with the robot. In each session, approximately 20 students out of 33 engaged in student-robot interaction, involving dialogues of varying lengths. Here is an interactive set of exchanges between a student (S1) and the robot (NAO) on the target word “integral”: http://doc.aldebaran.com/1-14/software/choregraphe/index.html Australasian Journal of Educational Technology, 2021, 37(3). 79 S1: What is the meaning of the word integral? NAO: Integral is an adjective. Integral means important and necessary as a part of whole. For instance, he’s an integral part of the team and we can’t do without him. [first definition and example] Is it clear? [comprehension check] S1: No, it is not. Can you explain more? [negotiation for meaning] NAO: Okay, let me explain it in another way. Integral means essential. Here is another example: Bars and cafes are integral to the social life of the city. [second definition and example] Is it clear now? [comprehension check] S1: Yes, thank you. NAO: Great! The instructors performed vocabulary teaching in their classes as planned. That is, the target words were presented to the learners in context (i.e., through a text), and definitions and examples of those words were provided either by the robot (for the RALL group) or by the class teacher (for the control group). To provide a richer image of the classroom, it should be noted that whenever students came across a new word (target vocabulary), they were encouraged to ask about its meaning. Here, they had the chance to interact with the robot and negotiate for the meaning. The NAO robot would provide meaning, examples, comprehension checks and feedback. Once the meaning of the target word was established, the process would continue. The classes of both groups were synchronised; that is, they were held at the same time on the same day of the week. All of the learners had the chance to interact with the robot and ask about the meaning of the words. Even though some did not ask about the meaning on some occasions, they were still exposed to answers provided by the robot in response to questions asked by others in the RALL group, and by the teacher in the non-RALL group. The post-experiment phase Upon completion of the experiment, students were given the post-test. The RALL group also filled in the questionnaire on the same day. Subsequently, 10 of those students volunteered to be interviewed. All in all, the systematic procedure, which lasted 1 month, is outlined in Figure 5 in the form of a flowchart covering the main stages involved. Figure 5. Flowchart of the procedure PROGRAMMING MEETING WITH INSTRUCTORS (DISCUSSION AND TRAINING) PRE-TEST INTRODUCTION OF THE NAO ROBOT TO RALL GROUP CONSENT FROM PARTICIPANTS EXPERIMENT POST-TEST QUESTIONNAIRE INTERVIEW Australasian Journal of Educational Technology, 2021, 37(3). 80 Data analysis and results The quantitative data were analysed using SPSS version 22. The independent samples t test was used to compare the results of the post-test to those of the pre-test. Descriptive statistics were obtained by analysing the questionnaire data. Interviews were transcribed and qualitatively analysed to identify the major themes reflecting the RALL group’s reactions to the use of the NAO robot. The results are presented in two parts with reference to the research questions. The NAO robot’s impact on vocabulary learning Since the target words were all unknown to the learners, the score of the pre-test for all of the participants was zero. Quantitative data showed some level of vocabulary gain in both groups. The mean and standard deviation for the control and experimental groups were 16.55 (4.51) and 15.22 (4.69), respectively. Interestingly, the control group scored slightly higher than the experimental group. As can be seen in Table 2, the significance value is p = .391, which indicates that the difference between groups was not significant (p > .05). Table 2 Independent samples t test Levene’s test for equality of variances t test for equality of means Sig. Sig. (two-tailed) Equal variances assumed .721 .391 Equal variances not assumed .391 Students’ attitudes towards the NAO robot and its use The results of the questionnaire showed that students liked most of the NAO robot’s abilities and thought that the robot was appropriate for the learning context. The rate of agreement was more than 60% for almost all items (see Table 3 below). Table 3 Summary of questionnaire descriptive analyses Item Agree % (n) Neutral % (n) Disagree % (n) The robot looks smart. 69 (20) 17.2 (5) 13.8 (4) The robot can watch you. 72.4 (21) 10.3 (3) 17.2 (5) The robot can listen to your song and speech. 75.9 (22) 17.2 (5) 6.9 (2) The robot can come to you. 44.8 (13) 44.8 (13) 10.3 (3) The robot’s appearance looks comfortable for learning. 65.5 (19) 17.2 (5) 17.2 (5) The robot’s body looks comfortable for moving around in a classroom. 62.1 (18) 13.8 (4) 24.1 (7) The robot’s facial expression looks comfortable to you. 58.6 (17) 27.6 (8) 13.8 (4) The robot’s compliment is pleasing to you. 65.5 (19) 24.1 (7) 10.3 (3) The robot’s voice is pleasing. 65.5 (19) 20.7 (6) 13.8 (4) The robot seems secure. 72.4 (21) 24.1 (7) 3.4 (1) Although the questionnaire data revealed positive attitudes towards the robot, a few interviewees expressed mixed feelings about the robot and its use. For ease of interpretation, the findings are divided into five categories. The robot’s intelligence and safety Statistical analysis showed that almost 70% of participants believed that the robot is an intelligent, safe and non-threatening device. The safety of the NAO robot was also supported by interviewees’ comments. For example, one reported: “I was not afraid of it, he was like my friend; it was like asking a friend a question” (NNP 1). Overall, it can be said that students were satisfied with these features. Australasian Journal of Educational Technology, 2021, 37(3). 81 The NAO robot’s walking and watching abilities There were two items in the questionnaire that explored students’ opinions about the robot’s walking ability. Almost 45% of the students agreed that the robot could walk (item 4) while the same percentage of them were neutral. The rate of agreement was higher on the second walking-related item (item 6): 62.1% of the participants believed that the robot’s body was comfortable to walk. Comments from the interviews showed that they were aware of the walking ability. One participant commented: “The movements were great; the robot could walk and talk” (NNP 7). In addition, around 72% of the students agreed that the robot could watch them. Generally, it can be interpreted that students were aware of these abilities and had no problem with being watched by the robot. The NAO robot’s appearance and facial expressions Approximately 66% of the participants believed that the NAO robot’s appearance was suitable for learning, which was supported by comments such as “cool and interesting” (NP 1) and “attractive” (NNP 4). NNP4 further said: “The robot looks like so cute, that’s why I felt so comfortable; I don’t [feel] I am under pressure.” Another participant shared a similar feeling by saying “I see the robot for the first time in my life, I liked when it moves or dancing, it was interesting” (NNP 8). Moreover, about 55% of the students liked the facial expressions of the robot. In general, it can be said that most of the students liked the NAO robot’s appearance and facial expressions. Listening and speaking abilities of the NAO robot Although 76% of the participants stated that the robot could hear them, interviewees reported that this feature had some limitations and problems. Sometimes the robot could not understand students since their voice was not loud enough or their pronunciation was not correct. One participant suggested: “I know just one thing will be improving the robot ability to hear, receive sound because sometimes it has problems, I mean voice recognition” (NP 1). Similarly, another participant stated: “Only thing that I had stress about was the sound, because when I went to the robot and I asked something, it couldn’t hear me well, and teacher said ‘Speak louder.’ but I couldn’t as required, I am shy” (NNP 7). Some suggested using the robot in smaller groups in order to overcome this problem. For instance, the same participant reported: “I prefer small groups; I think it can be more helpful in small groups, as much as I see the abilities”. It can be inferred from the above that the robot’s voice recognition system is a technical issue especially for learners who characteristically speak softly or have lower self-esteem. Regarding the robot’s speech, 65.5% of the participants liked its voice and the responses it used as positive reinforcement. However, some of the interviewees stated that the NAO robot spoke fast. Indeed, there were somewhat contradictory comments about the speech rate of the robot. Some thought that the speech rate was fine and understandable while others thought that the robot was talking too fast to understand. One of the participants commented: No emotion and goes like a robot which talks really fast, for me it’s like ok I can hear it but most of my classmates is like really difficult to understand [be]cause they are not used to that kind of fast English. (NNP 6) Another participant was more specific about the source of the problem: In order to acquire some knowledge from that robot you have to know English like very well. Some sentences or the listening skill must be some advanced. I mean if you are advanced in listening skill you won’t have any problem with it. (NNP 3) Comments seem to reflect that comprehending the robot was, at least for some, a challenge arising from the perceived high speech rate and inadequate listening skills. Advantages of the NAO robot According to the interviewees, the robot use facilitated vocabulary learning. Specifically, it helped them, even the native speakers, to learn new words. One native speaker participant reported: “It [the robot] helped me to learn a few words that I didn’t know, for example, ‘albeit’” (NP 2). As expected, the interaction was found to be beneficial in vocabulary learning, as one participant stated: “We interact with it; that's why I learned vocabulary” (NNP 4). Another positive aspect of the robot use was the provision of examples in context. Here are two similar comments on the contextual examples provided: Australasian Journal of Educational Technology, 2021, 37(3). 82 When I asked it questions, it gave so many sentences; as I mentioned before it doesn’t give you just a meaning but also in context so this is the best way that I can learn the language, I mean words. (NNP 3) Repeating examples was helpful because I have problems making a sentence and the robot gave some ideas for making sentences to me, it helped me. (NNP 1) Repetition was also seen as an advantage and referred to by most of the participants. A case in point is NNP1’s statement above. Another participant supported the idea of repetition by saying: “The robot explains again and again so the repetition was helpful” (NNP 4). Yet another benefit of the robot use was its capacity to provide immediate answers and multiple examples. One of the participants reported: “He [the robot] answers immediately and if you don’t understand, you ask again and it gives another example” (NNP 8). Likewise, another participant thought that this ability is really helpful and can help her when she becomes a teacher. She elaborated: I really use it as an assistant because you know personally I’m a native speaker. I know what a word means, I know synonyms but synonym is hard for me to explain it clearly or think of an example. So, I see it’s actually helpful for me as a teacher if I have a trouble in explaining a word. I need it to give some examples to my students I teach and they can ask virtually. (NP 1) All in all, the reported reactions to the use of the NAO robot in the classroom were predominantly positive. That is, participants reported more advantages than disadvantages of robot use (i.e., eight participants noted advantages while two reported disadvantages). Discussion With respect to vocabulary learning, the current study did not find a significant difference between the performances of the two groups. Nevertheless, both groups made substantial vocabulary gains, which is consistent with previous studies such as that by Schodde et al. (2017). However, the control group performed slightly better as far as the mean scores are concerned, which differs from the results of Alemi et al.’s (2014) study, which employed the same robot and role but with a school-aged group. This outcome may be due to several factors. First of all, the age of the participants in the present study is different from that of the participants in other studies (e.g., Eimler et al., 2010; de Wit et al., 2018). The focus of those studies was on younger learners, while the current study focused on adult language learners. It could be that adult language learners appear more cautious about welcoming such technology into their learning environment. The second factor which may explain this inconsistency could be the role and function of the robot within the classroom. In this study, the NAO robot was used as a teaching assistant that answered the students’ questions about the meaning of the target words. For some interviewees, the predefined role and function of the robot did not work out smoothly. For instance, some interviewees (e.g., NNP 6 above) stated that they were not used to fast English. In other words, they most probably had difficulty understanding the explanations and examples provided by the robot, which may stem from a lack of adequate listening skills. The final major factor could be that the NAO robot is a fairly new technology that most participants had not seen before. The only exposure they had was the demonstration session. The technology was attractive, constructive and helpful in vocabulary learning for some of the learners. For instance, one participant said: “It was motivating because it was a robot like something that every people haven't seen around the world; Yes, it grabs the student’s attention” (NNP 3), while for others it was disruptive so they could not get used to it. For example, another participant reported: It was distractive after a while, I couldn’t get used to the robot; my attention was on the robot itself rather than English, maybe if I get used to the robot in the future I may benefit, but at the moment I didn’t benefit that much. (NNP 5) Australasian Journal of Educational Technology, 2021, 37(3). 83 This issue might hold true for other participants as well. It can be considered as a factor that hindered the vocabulary learning of students in the RALL group. They might have performed better if they had spent more time with the robot and had the chance to get used to it. With regard to attitudes towards RALL, some studies have quantitatively investigated students’ attitudes, but few have looked into it both quantitatively and qualitatively. The questionnaire results, showing high degree of agreement for almost all the items ranging from 59% to 76%, are in line with previous studies, such as that by Lee et al. (2011). This indicates that students were satisfied with the robot and its abilities. Interview data also supported the questionnaire data in most cases. The students were mostly positive about this technology and its use in teaching vocabulary. Most of them believed that it was beneficial. The NAO robot was reported to be motivating and attractive by some participants, which is in line with the findings of previous studies (e.g., Alemi et al., 2014; Hong et al., 2016; Lee et al., 2011). Almost all of the interviewees believed that the NAO robot helped them to learn the new words. According to their comments, it facilitated their vocabulary learning by giving them the chance to interact. For instance, one participant said: “We interact with it; that's why I learned vocabulary” (NNP 4). Similarly, another participant said: “We ask and then he [the robot] explains; when someone explains, it [meaning] sticks better in your head rather than reading it online or in a dictionary” (NNP 6). This was not the only advantage: the NAO robot provided immediate answers, context and a chance to be exposed to meanings and examples repeatedly. Strikingly, almost all the participants humanised the robot by referring to it as “he” or “my friend”, which distinguished the robot from other technologies. For instance, one participant described her experience with the robot: “He was like my friend; it was like asking a question from a friend” (NNP 1). This indicates that the NAO robot was received as an acceptable conversational partner. Based on its perceived qualities, the NAO robot is arguably superior to other technologies, such as virtual applications; therefore, it may have greater potential for facilitating learning, which, in turn, makes it a more attractive technological choice for educational institutions to invest in. However, several interviewees reported that they could not focus on language learning due to the novelty of the technology. They reported that they had paid more attention to the robot itself rather than the language or the words they were supposed to learn. Nevertheless, those interviewees also came up with some recommendations to overcome this distraction: spending more time with the NAO robot (NNP 1); getting used to its functions (NNP 5); and using it in smaller groups (NNP 7). The robot’s voice was also considered emotionless and robotic by several participants. These comments show that the participants did not share the same opinions regarding the NAO robot. In other words, while some believed that the robot incentivised them and affected their vocabulary learning positively, a few others reported contradictory ideas. Clearly, more research is needed to further explore adult learners’ attitudes towards the RALL. Conclusions and implications This study attempted to determine the possible impact of robot use on adult language learners’ vocabulary learning and their attitudes towards using such technology. Therefore, both quantitative and qualitative data were collected. The quantitative data analysis did not reveal any statistically significant differences between groups, as the participants of both groups made similar gains in vocabulary learning. Several crucial factors may account for this outcome: the novelty of the technology introduced, the participants’ limited prior contact with the robot, reported problems with voice recognition and speech rate, students’ inadequate listening skills and individual differences. These factors might have led to differing responses to the technology. As this study was limited to only one experiment, further studies are needed to help better understand how various factors may translate into successful vocabulary gains. In terms of learners’ attitudes, the results of the analyses were mixed. The quantitative data collected through the questionnaire indicated that adult learners generally responded positively to the NAO robot’s capabilities and were attracted by the robot itself. On the other hand, the interview data revealed that learners had varied feelings about the robot and its use. While some found this technology constructive, helpful and beneficial, others thought it had its limitations and needed to be improved. Continuous assessment of student attitudes in reference to the developing features of the NAO robot seems to be essential for integrating such technology into the classroom. Australasian Journal of Educational Technology, 2021, 37(3). 84 Considering the positive reactions, but bearing in mind the criticisms as well, the use of the NAO robot in the classroom is promising. By promoting its acceptance as a conversational partner (as revealed in this study), it can function more effectively as a teaching assistant. The teacher can teach new linguistic features (e.g., vocabulary) with the assistance of the robot. Furthermore, the use of the robot can be a source of motivation both for students and teachers, which can facilitate language learning in an enjoyable manner. In particular, the interview data appears convincing enough for us to argue positively for a robot-assisted component in the language education curriculum, which necessitates teachers’ background knowledge and skills to integrate the NAO robot into the classroom as well as teacher trainers’ perspective on RALL and their preparedness for providing practical professional training for teachers. Limitations and further research directions One limitation of this study was the number of participants and target words. Since we could not make any changes to the groups, the number of participants was limited. Similarly, a finite number of target words (23 academic words) had to be used. It should be noted that we did our best to include all available words unknown to all of the participants. The findings of this study could have been more comprehensive if the sample size and the number of target words of the study had been greater. Although we trained the instructors to use a common method to teach target words for consistency, the inability to use a common instructor for both groups might have had an effect on the results. Additionally, due to time limitations, the instructors’ opinions were left unexplored. Researchers could replicate the current research in multiple experiments with larger numbers of participants and target words. The robot could be used for a longer period of time and its long-term effect on vocabulary learning could be investigated. Adding a delayed post-test could facilitate greater understanding of the effect of the robot on language learners’ vocabulary retention. Further research could explore adult learners’ attitudes towards robots and their use in different contexts. Additionally, instructors’ opinions about the use of robots could be explored. The effect of training in how to use robots for teaching purposes may also be a fruitful line of research. Such studies will undoubtedly help us discover the ways in which robots can be used more effectively for language learning and teaching. Acknowledgements We would like to acknowledge the technical support and encouragement provided by the Engineering Faculty at the institution where the study was conducted. Our special thanks go to the dean of the faculty and the chair and staff of the Electrical and Electronic Engineering Department. We would also like to thank the participants and the two instructors involved in the study for their cooperation. Last but not least, we wish to express our great appreciation to the two anonymous reviewers and Dr Anita L. Akkas for their comments and constructive feedback. References Aidinlou, N. A., Alemi, M., Farjami, F., & Makhdoumi, M. (2014). Applications of robot assisted language learning (RALL) in language learning and teaching. International Journal of Language and Linguistics, 2(3-1), 12–20. https://doi.org/10.11648/j.ijll.s.2014020301.12 Alemi, M., Meghdari, A., & Ghazisaedy, M. (2014). Employing humanoid robots for teaching English language in Iranian junior high-schools. International Journal of Humanoid Robotics, 11(3), Article 1450022. https://doi.org/10.1142/S0219843614500224 Alemi, M., Meghdari, A., & Haeri, N. S. (2017). Young EFL learners’ attitude towards RALL: An observational study focusing on motivation, anxiety, and interaction. In A. Kheddar, E. Yoshida, S. S. Ge, K. Suzuki, J.-J. Cabibihan, F. Eyssel, & H. He (Eds.), Proceedings of the International Conference on Social Robotics (pp. 252–261). Springer. https://doi.org/10.1007/978-3-319-70022- 9_25 Browne, C., & Culligan, B. (2008). Combining technology and IRT testing to build student knowledge of high frequency vocabulary. The JALT CALL Journal, 4(2), 3–16. https://journal.jaltcall.org/storage/articles/JALTCALL%204-2-3.pdf Cameron, D. (2001). Working with spoken discourse. Sage. https://doi.org/10.11648/j.ijll.s.2014020301.12 https://doi.org/10.1142/S0219843614500224 https://doi.org/10.1007/978-3-319-70022-9_25 https://doi.org/10.1007/978-3-319-70022-9_25 https://journal.jaltcall.org/storage/articles/JALTCALL%204-2-3.pdf Australasian Journal of Educational Technology, 2021, 37(3). 85 Cehan, A. (2014). English pedagogic lexicography: A few milestones. The Journal of Linguistic and Intercultural Education, 7, 69–80. http://jolie.uab.ro/upload/12_126_5cehan_anca.pdf Chen, C., Gao, Q., Song, Z., Liping, O., & Wu, X. (2010). Catering service robot. In Proceedings of WCICA 2010: 8th World Congress on Intelligent Control and Automation (pp. 599–604). IEEE. https://doi.org/10.1109/WCICA.2010.5553843 Cleveland-Marwick, K. (Ed.). (2013). Longman collocations dictionary and thesaurus. Pearson Education. Creswell, J. W. (2009). Research design: Qualitative and mixed methods approaches. Sage Publications. Dalton, B., & Grisham, D. L. (2011). eVoc strategies: 10 ways to use technology to build vocabulary. The Reading Teacher, 64(5), 306–317. https://doi.org/10.1598/RT.64.5.1 de Wit, J., Schodde, T., Willemsen, B., Bergmann, K., de Haas, M., Kopp, S., Krahmer, E., & Vogt, P. (2018). The effect of a robot’s gestures and adaptive tutoring on children’s acquisition of second language vocabularies. In Proceedings of the 2018 ACM/IEEE International Conference on Human- Robot Interaction (pp. 50–58). Association for Computing Machinery. https://doi.org/10.1145/3171221.3171277 Eimler, S., von der Putten, A., & Schachtle, U. (2010). Following the white rabbit—A robot rabbit as vocabulary trainer for beginners of English. In G. Leitner, M. Hitz, & A. Holzinger (Eds.), HCI in work and learning, life and leisure (pp. 322–339). Springer. https://doi.org/10.1007/978-3-642-16607- 5_22 Gass, S. M. (1997). Input, interaction, and the second language learner. Erlbaum. Gordon, G., Spaulding, S., Westlund, J. K., Lee, J. J., Plummer, L., Martinez, M., Das, M., & Breazeal, C. (2016). Affective personalization of a social robot tutor for children’s second language skills. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (pp. 3951–3957). Association for the Advancement of Artificial Intelligence. https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/download/11759/12184 Han, J. (2012). Emerging technologies: Robot assisted language learning. Language Learning & Technology, 16(3), 1–9. http://llt.msu.edu/issues/october2012/emerging.pdf Herberg, J. S., Feller, S., Yengin, I., & Saerbeck, M. (2015). Robot watchfulness hinders learning performance. In Proceedings of the 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (pp. 153–160). IEEE. https://doi.org/10.1109/ROMAN.2015.7333620 Hiebert, E. H., & Kamil, M. L. (2005). Teaching and learning vocabulary: Bringing research to practice. Routledge. https://doi.org/10.4324/9781410612922 Hong, Z. W., Huang, Y. M., Hsu, M., & Shen, W. W. (2016). Authoring robot-assisted instructional materials for improving learning performance and motivation in EFL classrooms. Journal of Educational Technology & Society, 19(1), 337–349. https://www.jstor.org/stable/10.2307/jeductechsoci.19.1.337 Hsiao, H. S., Chang, C. S., Lin, C. Y., & Hsu, H. L. (2015). “iRobiQ”: The influence of bidirectional interaction on kindergarteners’ reading motivation, literacy, and behavior. Interactive Learning Environments, 23(3), 269–292. https://doi.org/10.1080/10494820.2012.745435 Hung, H. T., Yeh, H. C., & Chiang, C. W. (2016). Computer assisted vocabulary learning: Examining English language learners' vocabulary notebooks. In T. Matsuo (Ed.), Proceedings of the 5th IIAI International Congress on Advanced Applied Informatics (pp. 381–385). IEEE. https://doi.org/10.1109/IIAI-AAI.2016.158 Hunston, S. (2002). Corpora in applied linguistics. Cambridge University Press. https://doi.org/10.1017/CBO9781139524773 Hyun, E. J., Kim, S. Y., Jang, S., & Park, S. (2008). Comparative study of effects of language instruction program using intelligence robot and multimedia on linguistic ability of young children. In Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication (pp. 187–192). IEEE. https://doi.org/10.1109/ROMAN.2008.4600664 Iio, T., Maeda, R., Ogawa, K., Yoshikawa, Y., Ishiguro, H., Suzuki, K., Aoki, T., Maesaki, M., & Hama, M. (2018). Improvement of Japanese adults’ English speaking skills via experiences speaking to a robot. Journal of Computer Assisted Learning, 35(2), 228–245. https://doi.org/10.1111/jcal.12325 In, J., & Han, J. (2015). The acoustic-phonetics change of English learners in robot assisted learning. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts (pp. 39–40). Association for Computing Machinery. https://doi.org/10.1145/2701973.2702003 Kennedy, G. (2014). An introduction to corpus linguistics. Routledge. https://doi.org/10.4324/9781315843674 http://jolie.uab.ro/upload/12_126_5cehan_anca.pdf https://doi.org/10.1109/WCICA.2010.5553843 https://doi.org/10.1598/RT.64.5.1 https://doi.org/10.1145/3171221.3171277 https://doi.org/10.1007/978-3-642-16607-5_22 https://doi.org/10.1007/978-3-642-16607-5_22 https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/download/11759/12184 http://llt.msu.edu/issues/october2012/emerging.pdf https://doi.org/10.1109/ROMAN.2015.7333620 https://doi.org/10.4324/9781410612922 https://doi.org/10.1080/10494820.2012.745435 https://doi.org/10.1109/IIAI-AAI.2016.158 https://doi.org/10.1017/CBO9781139524773 https://doi.org/10.1109/ROMAN.2008.4600664 https://doi.org/10.1111/jcal.12325 https://doi.org/10.1145/2701973.2702003 https://doi.org/10.4324/9781315843674 Australasian Journal of Educational Technology, 2021, 37(3). 86 Kennedy, J., Baxter, P., Senft, E., & Belpaeme, T. (2016). Social robot tutoring for child second language learning. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (pp. 231–238). IEEE. https://doi.org/10.1109/HRI.2016.7451757 Krashen, S. (1982). Principles and practice in second language acquisition. Pergamon Press. http://www.sdkrashen.com/content/books/principles_and_practice.pdf Lee, S., Noh, H., Lee, J., Lee, K., Lee, G. G., Sagong, S., & Kim, M. (2011). On the effectiveness of robot-assisted language learning. ReCALL, 23(1), 25–58. https://doi.org/10.1017/S0958344010000273 Long, M. H. (1983). Native speaker/non-native speaker conversation and the negotiation of comprehensible input. Applied Linguistics, 4(2), 126–141. https://doi.org/10.1093/applin/4.2.126 Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of second language acquisition (Vol. 2, pp. 413–468). Elsevier. McClanahan, L. (2014). Training using technology in the adult ESL classroom. Journal of Adult Education, 43(1), 22–27. https://doi.org/10.11613/BM.2012.031 McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031 Meghdari, A., Alemi, M., Ghazisaedy, M., Taheri, A. R., Karimian, A., & Zandvakili, M. (2013). Applying robots as teaching assistant in EFL classes at Iranian middle-schools. In P. Dondon (Ed.), Proceedings of the International Conference on Education and Modern Educational Technologies (pp. 67–73). Institute for Natural Sciences and Engineering. http://www.inase.org/library/2013/venice/EMET.pdf Meiirbekov, S., Balkibekov, K., Jalankuzov, Z., & Sandygulova, A. (2016). “You win, I lose”: Towards adapting robot’s teaching strategy. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (pp. 475–476). IEEE. https://doi.org/10.1109/HRI.2016.7451813 Motallebzadeh, K., & Ganjali, R. (2011). SMS: Tool for L2 vocabulary retention and reading comprehension ability. Journal of Language Teaching & Research, 2(5), 1111–1115. https://doi.org/10.4304/jltr.2.5.1111-1115 Movellan, J. R., Eckhardt, M., Virnes, M., & Rodriguez, A. (2009). Sociable robot improves toddler vocabulary skills. In Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction (pp. 307–308). Association for Computing Machinery. https://doi.org/10.1145/1514095.1514189 Mubin, O., Stevens, C. J., Shahid, S., Al Mahmud, A., & Dong, J. J. (2013). A review of the applicability of robots in education. Technology for Education and Learning, 1, 1–7. https://doi.org/10.2316/Journal.209.2013.1.209-0015 Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge University Press. https://doi.org/10.1017/CBO9781139524759 Nejati, E., Jahangiri, A., & Salehi, M. R. (2018). The effect of using computer-assisted language learning (CALL) on Iranian EFL learners' vocabulary learning: An experimental study. Cypriot Journal of Educational Sciences, 13(2), 351–362. https://doi.org/10.18844/cjes.v13i2.752 Papert, S. (1993). The children’s machine: Rethinking school in the age of the computer. Basic Books. Pica, T. (1994). Research on negotiation: What does it reveal about second‐language learning conditions, processes, and outcomes? Language learning, 44(3), 493–527. https://doi.org/10.1111/j.1467- 1770.1994.tb01115.x Read, J. (2000). Assessing vocabulary. Cambridge University Press. https://doi.org/10.1017/CBO9780511732942 Rosenthal-von der Pütten, A. M., Straßmann, C., & Krämer, N. C. (2016). Robots or agents–neither helps you more or less during second language acquisition. In D. Traum, W. Swartout, P. Khooshabeh, S. Kopp, S. Scherer, & A. Leuski (Eds.), Proceedings of the International Conference on Intelligent Virtual Agents (pp. 256–268). Springer. https://doi.org/10.1007/978-3-319-47665-0_23 Schmitt, D., Schmitt, N. (2011). Focus on vocabulary 2: Mastering the academic word list. Pearson Education. Schmitt, N. (2000). Vocabulary in language teaching. Cambridge University Press. Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55–88. https://doi.org/10.1177/026553220101800103 https://doi.org/10.1109/HRI.2016.7451757 http://www.sdkrashen.com/content/books/principles_and_practice.pdf https://doi.org/10.1017/S0958344010000273 https://doi.org/10.1093/applin/4.2.126 https://doi.org/10.11613/BM.2012.031 https://doi.org/10.11613/BM.2012.031 http://www.inase.org/library/2013/venice/EMET.pdf https://doi.org/10.1109/HRI.2016.7451813 https://doi.org/10.4304/jltr.2.5.1111-1115 https://doi.org/10.1145/1514095.1514189 https://doi.org/10.2316/Journal.209.2013.1.209-0015 https://doi.org/10.1017/CBO9781139524759 https://doi.org/10.18844/cjes.v13i2.752 https://doi.org/10.1111/j.1467-1770.1994.tb01115.x https://doi.org/10.1111/j.1467-1770.1994.tb01115.x https://doi.org/10.1017/CBO9780511732942 https://doi.org/10.1007/978-3-319-47665-0_23 https://doi.org/10.1177/026553220101800103 https://doi.org/10.1177/026553220101800103 Australasian Journal of Educational Technology, 2021, 37(3). 87 Schodde, T., Bergmann, K., & Kopp, S. (2017). Adaptive robot language tutoring based on Bayesian knowledge tracing and predictive decision-making. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 128–136). Association for Computing Machinery. https://doi.org/10.1145/2909824.3020222 Shin, J., & Shin, D. H. (2015). Robot as a facilitator in language conversation class. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts (pp. 11–12). Association for Computing Machinery. https://doi.org/10.1145/2701973.2702062 Sinclair, J. (1987). Collins COBUILD English language dictionary. HarperCollins. Tanaka, F., & Matsuzoe, S. (2012). Children teach a care-receiving robot to promote their learning: Field experiments in a classroom for vocabulary learning. Journal of Human-Robot Interaction, 1, 78–95. https://doi.org/10.5898/JHRI.1.1.Tanaka Thornbury, S. (2002). How to teach vocabulary. Longman. van den Berghe, R., van der Ven, S., Verhagen, J., Oudgenoeg-Paz, O., Papadopoulos, F., & Leseman, P. (2018). Investigating the effects of a robot peer on L2 word learning. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (pp. 267–268). Association for Computing Machinery. https://doi.org/10.1145/3173386.3176986 van den Berghe, R., Verhagen, J., Oudgenoeg-Paz, O., van der Ven, S., & Leseman, P. (2019). Social robots for language learning: A review. Review of Educational Research, 89(2), 259–295. https://doi.org/10.3102/0034654318821286 Vogt, P., De Haas, M., De Jong, C., Baxter, P., & Krahmer, E. (2017). Child-robot interactions for second language tutoring to preschool children. Frontiers in Human Neuroscience, 11, 1–7. https://doi.org/10.3389/fnhum.2017.00073 Wang, Y. H., Young, S. S. C., & Jang, J. S. R. (2013). Using tangible companions for enhancing learning English conversation. Journal of Educational Technology & Society, 16(2), 296–309. https://www.jstor.org/stable/10.2307/jeductechsoci.16.2.296 Westlund, J. M. K., Martinez, M., Archie, M., Das, M., & Breazeal, C. (2016). Effects of framing a robot as a social agent or as a machine on children's social behavior. In Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (pp. 688–693). IEEE. https://doi.org/10.1109/ROMAN.2016.7745193 Willis, D. (1990). The lexical syllabus. Collins. Yanguas, I. (2009). Multimedia glosses and their effect on L2 text comprehension and vocabulary learning. Language Learning & Technology, 13(2), 48–67. http://llt.msu.edu/vol13num2/yanguas.pdf Corresponding author: Hossein Banaeian, h.banaeian1@gmail.com Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Banaeian, H., & Gilanlioglu, I. (2021). Influence of the NAO robot as a teaching assistant on university students’ vocabulary learning and attitudes. Australasian Journal of Educational Technology, 37(3), 71–87. https://doi.org/10.14742/ajet.6130 https://doi.org/10.1145/2909824.3020222 https://doi.org/10.1145/2701973.2702062 https://doi.org/10.5898/JHRI.1.1.Tanaka https://doi.org/10.1145/3173386.3176986 https://doi.org/10.1145/3173386.3176986 https://doi.org/10.3102/0034654318821286 https://doi.org/10.3389/fnhum.2017.00073 https://doi.org/10.1109/ROMAN.2016.7745193 http://llt.msu.edu/vol13num2/yanguas.pdf mailto:h.banaeian1@gmail.com https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6130