Available online at www.HighTechJournal.org HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 179 A Wearable MYO Gesture Armband Controlling Sphero BB-8 Robot T. S. Chu a, b, A. Y. Chua b, E. L. Secco a* a Robotics Laboratory, School of Mathematics, Computer Science & Engineering, Liverpool Hope University, United Kingdom. b Mechanical Engineering Department, De La Salle University, Manila, Philippines. Received 09 August 2020; Revised 17 October 2020; Accepted 20 October 2020; Published 01 December 2020 Abstract In this paper, we present the development and preliminary validation of a wearable system, which is combined with an algorithm interfacing the MYO gesture armband with a Sphero BB-8 robotic device. The MYO armband is a wearable device, which measures real-time EMG signals of the end user’s forearm muscles as they execute a set of upper limb gestures. These gestures are interpreted and transmitted to the computing hardware via the Bluetooth Low Energy IEEE 802.15.1 wireless protocol. The algorithm analyzes and sorts the data and sends a set of commands to the Sphero robotic device while performing navigation movements. After designing and integrating the software and hardware architecture, we have validated the system with two sets of trials involving a series of commands performed in multiple iterations. The consequent reactions of the robots due to these commands, were recorded and the performance of the system was analyzed in a confusion matrix to obtain an average accuracy of the system outcome vs. the expected and desired actions. Results show that our integrated system can satisfactorily interface with the system in an intuitive way, with an accuracy rating of 85.7% and 92.9% for the two tests, respectively. Keywords: Wearable Interface; Human-Robot Interface; EMG; Gesture Recognition. 1. Introduction Implementation interactive systems have undergone improvements over the years due to various reasons. One of them is the desire for convenience. Such inspiration has driven the integration of interactive systems into fields such as medical, military, and research. Focusing on the medical application, interactive systems have aided medical professionals in examining and analyzing their patients' conditions through various medical tests. The ElectroMyoGraphy (EMG) is a diagnostic test conducted on patients to determine the muscle and nerve condition of a particular limb. This is achieved by capturing the electrical signals that are generated by muscle contractions. Invasive procedures and non-invasive procedures are the two primary methods of conducting this test. An invasive procedure requires a surgical operation to be performed on the patient to implant a sensor chip that captures the electrical signals generated by the muscle and transmits this information to computing hardware. The non- invasive procedure makes use of the concept of surface EMG (sEMG), where electrode stickers are attached to the skin of the patient. This method obtains the electrical signals by which the muscle generates when it contracts; however, this method may exhibit less accuracy in comparison to the invasive procedure. Previously, this test was done inside medical facilities and often required multiple hardware systems. Due to developments in hardware components, commercial products that are portable and capable of conducting EMG tests are currently being developed. * Corresponding author: seccoe@hope.ac.uk http://dx.doi.org/10.28991/HIJ-2020-01-04-05  This is an open access article under the CC-BY license (https://creativecommons.org/licenses/by/4.0/). © Authors retain all copyrights. https://creativecommons.org/licenses/by/4.0/ http://orcid.org/0000-0002-3269-6749 HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 180 The application of this concept was expanded by researchers for various applications. The study conducted by Coban and Gelen (2018) [1] utilized the MYO Gesture Armband to control a robotic arm and hand respectively. The study employed a wearable bracelet comprised of multiple sEMG sensors and a 9 Degree of Freedom IMU. The researcher used a mixture of arm movements with hand gestures to control a robotic arm. In Kurniawan and Pamungkas study [2], the same armband was used to replace computer peripherals as a race car game controller. The study made a user-experience comparison between the traditional controls and MYO gesture controls. The experimentation of the study comprised of 2 categories of participants, experienced and inexperienced MYO users; the study noted that even with little experience with the MYO Armband, participants were able to adapt to it fairly well, but errors were obtained in the experiment when participants forget which gesture corresponds to a particular action. In the study conducted by Ploengpit and Phienthrakul (2016) [3], researchers creatively used the technology in a game application between 2 players which involves hand gestures. The experiment required 2 participants to play a game of rock-paper-scissors while wearing the MYO armband. This allowed the MYO to obtain data from the users, analyze the data to detect the gesture of each participant through a decision tree algorithm, and then determine the winner of the round. Therefore, according to the aforementioned projects, the MYO has a wide range of interesting research applications. Previous researches have validated the effectivity of the use of the MYO Armband in obtaining sEMG signals from a user’s forearm, which may be very useful in field such as rehabilitation and health. Another implementation is the Human-Robot Interaction (HRI); HRIs are robots that socially interact with humans and aid in day to day tasks, offer companionship, and offer assistance in health care and therapy [4]. An HRI robot called Pepper developed by SoftBank Robotics which falls under the category of Socially Assistive Robots (SAR) was evaluated in the study conducted by Barakeh et al. (2019) [5]; and it was found that the implementation of an HRI robot, particularly Pepper, in airports, and hospitals were found to be more socially acceptable as compared to malls, and banks by people through a survey. The main objective of this type of robot is to offer aid to humans when necessary [6]. It was also observed that HRI robots, when given enough social and emotional interaction, can significantly improve one’s capability in coping with stress [7]; additionally, there have been cases where HRI robots were used for entertainment purposes [8]. In the study conducted by Sathiyanarayanan and Rajan (2016) [9], it evaluated the viability of utilizing MYO in the medical field. The survey conducted in the study, where its respondents were medical doctors and students, showed that the respondents were satisfied when using the armband daily and that the armband can be easily adapted, though some respondents suggested that the component was unnecessarily complex for the function they use. HRI robots are emerging to supplement human lifestyle as it offers physical and emotional support. A particularly interesting function of HRI robots is that they are mostly designed to assist people in managing difficult situations, allowing people to overcome current circumstances. Combining the effectivity of the MYO Armband for sEMG detection and its potential application in rehabilitation, together with HRI robots offers a creative approach in health. For example, a physician may prescribe a certain set of exercises, to be observed by the MYO Armband. Adding interaction with an HRI robot in the process may motivate the patient in doing the prescribed exercises. In this context, this paper aims at presenting a novel system framework that allows the interaction between an sEMG wearable device and an HRI robot, specifically the MYO Gesture Armband and the Sphero BB-8 robot, respectively. 2. Framework 2.1. Software Algorithm The software architecture which has been developed in this research is written in the Python Programming Language. A library of specific arm gestures have been initially defined before starting the design: for practical convenience, we used a pre-defined set of gestures according to what has been proposed by Thalamic Labs (Figure 1) [10]. These gestures are defined by a unique set of vectors, where each vector represents the muscle activity when the specified gesture is executed by a human end-user, in a vectorized format. Figure 1. The 5 pre-defined Hand Gestures [10] HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 181 Following the definition of the gesture parameters, each gesture is then tied to a corresponding rolling function of the BB-8 robotic device, as it is shown in Table 1. To monitor the wireless communication between the wearable armband and the robot, feedback systems have been incorporated within the system. The bracelet is tasked to vibrate for a few seconds upon reading EMG information from the user’s arm. This information is sent to the computing hardware and the Python User Interface (UI) reflects the obtained information and then sends the data to the robot. In addition to the movement commands, LED color indicators are also used to consolidate the feedback form a visual point of view and to furtherly confirm that the robot has received the instructions from the computing hardware. Table 1. The User’s Active Gestures vs. the Corresponding Robot’s Movements Human Hand Gesture Robot’s Movement Fingers Spread Move Forward Fist Move Backward Wave In Move Right Wave Out Move Left Double Tap Make a square shape 2.2. Experimental Framework Figure 2 offers a visualization of how the sEMG concept is being utilized together with the algorithm employed to control the Sphero robot. The sEMG signals from a user’s forearm are obtained using the MYO bracelet, where the processing of the raw information is also performed (i). This information is communicated to the computing hardware through a Bluetooth connectivity implemented within the armband; upon receiving the information, the algorithm sorts out the obtained information, takes the necessary information, such as the vectorized sEMG signals to distinguish the gesture that is being executed by the user (ii). The algorithm then sends commands via Bluetooth connectivity as well to the robot for movement execution (iii). Finally, the robot moves according to the gesture executed by the user (iv). Figure 2. The system architecture (i.e. Experimental Framework) 3. Materials and Methods 3.1. MYO Gesture Armband The MYO Gesture Armband often referred to as MYO, is a commercial bracelet developed by Thalamic Labs used to capture the EMG signals along a user’s forearm [11]. The armband is one example of a Human-Computer Interface (HCI) machine that allows the user to control computer peripherals for various applications such as gaming and presentations using PowerPoint [10]. The MYO consists of 8 independent stainless-steel EMG sensors and employs an ARM Cortex M4 processor, allowing the bracelet to effectively capture the electric signal behavior occurring in the muscles while being positioned on along the skin and process the raw signals. The function of this EMG sensors is primarily programmed to detect the 5 pre-defined hand gestures as shown in Figure 3. It is also comprised of a 9-axis HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 182 IMU, 3-axis accelerometer, 3-axis gyroscope, and 3-axis magnetometer. This allows the bracelet to detect its orientation based on arm movements. It is equipped with several accessories as well such as the vibratory motor for user feedback, a Bluetooth LE transmitter for communication, and a socket for charging its batteries. Figure 3. The MYO Gesture Armband [11] 3.2. Sphero BB-8 Robot The Sphero BB-8 is an interactive robot toy developed by Sphero company. It is an 11.4 cm tall robot equipped with a Bluetooth Low Energy module for communication, and it possesses motors inside the main sphere, allowing the robot to move and turn. The motors are connected to a spherical body, so when the motor rotates, it consequently rotates the sphere, thus allowing BB-8 to roll in a particular direction. Turning function is also achieved when each motor rotates in opposite directions. Its head possesses small wheels, causing the head to be unaffected when the body starts to roll; and, the head is magnetically connected to the body, allowing it to be independent of the body. The robot can maintain its balance as it possesses a low center of gravity. It mainly uses a mobile app for interaction with users; the users get to move BB-8 around, speak with BB-8, and draw a trajectory that BB-8 will follow. For this research, only the BB-8 hardware will be used. Figure 4. The Sphero BB-8 Droid [12] 3.3. Bluetooth 4.0 Low Energy Dongle To effectively communicate with the Sphero BB-8 Robot, a Bluetooth 4.0 Low Energy dongle is used. The dongle operates in the same manner as any traditional Bluetooth dongles; however, the Low Energy specification implies that it operates on lower power input. This allows the Bluetooth technology to be implemented and is usually implemented in small robots. The Bluetooth dongle also has a maximum effective range of up to 50 meters. 3.4. Experimental Methodology The goal of this research is to determine how effective the algorithm is in interfacing the MYO with the Sphero BB-8 robot. When the program is initiated, it starts to communicate with the robot first before communicating with MYO armband. Once the robot’s internal LED flashes white, the computing hardware then communicates with the MYO armband. The user needs to calibrate the armband by holding the wave out gesture along the neutral axes until HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 183 the vibration feedback stops; neutral axes are defined as the position where the arm forms an “L” shape with the user’s forearm is pointing to the front. Figure 5. Finger Spread gesture activates Forward robot motion The LED indicator on the MYO should not blink nor should it fade in and out. When these two conditions are met, this indicates that both the MYO and the BB-8 robot are ready and that the user just needs to execute a double-tap gesture to unlock the MYO and start the experiment. The user simply has to execute the gesture of the desired command, this will then be transmitted to the Bluetooth dongle connected to the computing hardware. Figure 6. Fist gesture activates Backward robot motion The computing hardware analyzes the data it receives, and create commands corresponding to the received input to be implemented by the robot. The commands are then transmitted with another Bluetooth dongle to the robot for execution. This experiment is considered successful if the BB-8 can make movements such as forward, back, move left, and move right, and both vibratory feedback from the MYO, and LED indicator feedback are functioning. To determine the effectivity of the algorithm, a confusion matrix is used to monitor the number of correct and incorrect commands, and the experiment will be conducted twice. The average accuracy is obtained in both experiments and serves as the parameter for determining the effectiveness of the algorithm. The researcher employs a sequence of events, forward, backward, right, and left, and executed the sequence in 7 iterations. While the robot moves, the python UI will also reflect the command the computer received and the movement that the robot will execute. Picture references for the actual experiment are shown in Figures 5 to 8. 4. Results and Discussion With the given hand gestures, the BB-8 robot was capable of moving accordingly. Table 1 represents the gestures and their corresponding movements. The robot was able to execute the order of the movements most of the time. Errors were found when the robot did not perform the command corresponding to the gesture held by the researcher. HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 184 Additionally, delays were observed in the detection of new commands. The inaccuracy lies within the MYO armband detection, primarily with the usage of the researcher. Figure 7. Wave In gesture activates RightMotion of the robot Whenever the researcher would hold a particular gesture, the MYO would detect a different gesture, for example, when the user makes the fingers spread gesture, the MYO detects a fist gesture. The issue is caused due to the over- exertion of muscle flexion which exhibited a signal behavior similar to a fist, despite the fingers spread gestures that the researcher made. With the over-exertion of flexion actions, the user’s muscles took time to relax, consequently affecting the detection of the right gesture in the succeeding commands. The second test was designed and performed according to the previous findings, leading to a more relaxed approach in experimentation. The results of these 2 tests are shown in Tables 2 and 3, respectively. Table 2. Confusion Matrix, First Test, Inexperienced User Successful Trials (out of 7 trials) Forward Backward Right Left Total Forward 4 3 0 0 4 Backward 0 7 0 0 7 Right 0 1 6 0 6 Left 0 0 0 7 7 Average Accuracy (%) 57.1 100 85.7 100 85.7 Figure 8. Wave Out gesture activates Left Motion of the robot HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 185 The accuracy in the detection of hand gestures showed improvements in the second test. Thus, inferring that the user experience with the MYO will greatly improve the performance of the setup; however, inexperienced users may still use the setup as the robot can execute the needed commands at an acceptable accuracy. This research is successful as the robot was able to execute the commands with 85.7% accuracy for the first test and 92.8% accuracy for the second test. Table 3. Confusion Matrix, Second Test, Experienced User Successful Trials (out of 7 trials) Forward Backward Right Left Total Forward 5 0 2 0 5 Backward 0 7 0 0 7 Right 0 0 7 0 7 Left 0 0 0 7 7 Average Accuracy (%) 71.4 100 100 100 92.8 5. Conclusion The research aimed to determine the effectiveness of the algorithm for interfacing the MYO Gesture Armband with the Sphero BB-8 robot. The MYO utilized the concept of sEMG to detect muscle activity on the forearm of its user upon executing defined hand gestures, and the obtained electrical signals were then processed and transmitted to the computing hardware. The algorithm, written in the Python programming language, analyzed the data and communicated with the robot to execute movements that corresponded to the gesture executed by the user. The experiment required the user to execute a series of hand gestures and observe the movements made by the robot; this experiment was iterated seven times to obtain data to be used in a confusion matrix. During the experimentation, problems such as gesture misclassification and delays in gesture detection were observed. However, this issue can be addressed by fine-tuning the algorithm with machine learning algorithms such as Support Vector Machines. Machine learning implementation of this research should show accuracy improvements and minimization of gesture detection delays. With accuracy ratings of 85.7% and 92.8%, the algorithm was able to satisfactorily interface the MYO Gesture Armband with the Sphero BB-8 robot. The proposed system could be further validated with more laboratory tests and in-field trials [13-15]. However, current preliminary results suggest that the architecture is reliable and robust, with foreseen possible applications in gaming, ambient assisted living, and rehabilitation, provided that proper clinical validation will be performed. 6. Acknowledgements This work was presented in project & coursework form in fulfillment of the requirements for the MSc in Robotics Engineering for the student Timothy Chu under the supervision of E. L. Secco from the Robotics Laboratory, School of Mathematics, Computer Science and Engineering, Liverpool Hope University. 7. Institutional Review Board Statement Not applicable. 8. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 9. References [1] Coban, M., and Gelen, G. (2018). Wireless teleoperation of an industrial robot by using myo arm band. 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), Malatya, Turkey. doi:10.1109/idap.2018.8620789. [2] Kurniawan, S. R., & Pamungkas, D. (2018). MYO Armband sensors and Neural Network Algorithm for Controlling Hand Robot. 2018 International Conference on Applied Engineering (ICAE), Batam. doi:10.1109/incae.2018.8579153. [3] Ploengpit, Y., & Phienthrakul, T. (2016). Rock-paper-scissors with Myo Armband pose detection. 2016 International Computer Science and Engineering Conference (ICSEC), Chiang Mai. doi:10.1109/icsec.2016.7859949 [4] Doisy, G., Meyer, J., & Edan, Y. (2014). The Impact of Human–Robot Interface Design on the Use of a Learning Robot System. IEEE Transactions on Human-Machine Systems, 44(6), 788–795. doi:10.1109/thms.2014.2331618. HighTech and Innovation Journal Vol. 1, No. 4, December, 2020 186 [5] Barakeh, Z. A., Alkork, S., Karar, A. S., Said, S., & Beyrouthy, T. (2019). Pepper Humanoid Robot as a Service Robot: a Customer Approach. 2019 3rd International Conference on Bio-Engineering for Smart Technologies (BioSMART), Paris, France. doi:10.1109/biosmart.2019.8734250. [6] Feil-Seifer, D., & Mataric, M. J. (2005). Defining Socially Assistive Robotics. 9th International Conference on Rehabilitation Robotics, Chicago, IL, USA. ICORR 2005. doi:10.1109/icorr.2005.1501143. [7] Lohani, M., Stokes, C., McCoy, M., Bailey, C. A., & Rivers, S. E. (2016). Social interaction moderates human-robot trust- reliance relationship and improves stress coping. 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, 471-472. doi:10.1109/hri.2016.7451811 [8] Linert, J., & Kopacek, P. (2018). Humanoid robots Robotainment. IFAC-PapersOnLine, 51(30), 220–225. doi:10.1016/j.ifacol.2018.11.290. [9] Sathiyanarayanan, M., & Rajan, S. (2016). MYO Armband for physiotherapy healthcare: A case study using gesture recognition application. 8th International Conference on Communication Systems and Networks (COMSNETS), Bangalore, India. doi:10.1109/comsnets.2016.7439933. [10] Rawat, S., Vats, S., & Kumar, P. (2016). Evaluating and exploring the MYO ARMBAND. 2016 International Conference System Modeling & Advancement in Research Trends (SMART), Moradabad, India. doi:10.1109/sysmart.2016.7894501. [11] Eadicicco, L. (2016). “Review: Myo Armband Enables Gesture-Controlled Computing,” Time, 20-Jan-2016. Available online: https://time.com/4173507/myo-armband-review/ (accessed on 26 May 2020). [12] Sphero Legacy Products. (2020). Sphero made innovative robots for Star Wars, Marvel, and Pixar. Available online: https://images-na.ssl-images-amazon.com/images/I/51EY9Kq7nnL._AC_SY606_.jpg (accessed on 26 May 2020). [13] Secco, E.L., Moutschen, C., Tadesse, A., Barrett-Baxendale, M., Reid, D., Nagar, A. (2016). Development of a sustainable and ergonomic interface for the EMG control of prosthetic hands, 6th EAI International Conference on Wireless Mobile Communication and Healthcare, November 14–16, 2016, Milano, Italy. [14] Secco, E. L., Moutschen, C., Maereg, A. T., Barrett-Baxendale, M., Reid, D., & Nagar, A. K. (2017). Development of a Sustainable and Ergonomic Interface for the EMG Control of Prosthetic Hands. Wireless Mobile Communication and Healthcare, 321–327. doi:10.1007/978-3-319-58877-3_41. [15] Brown, K., Secco, E. L., & Nagar, A. K. (2019). A Low-Cost Portable Health Platform for Monitoring of Human Physiological Signals. Lecture Notes in Electrical Engineering, 211–224. doi:10.1007/978-3-030-02242-6_16. https://images-na.ssl-images-amazon.com/images/I/51EY9Kq7nnL._AC_SY606_.jpg