Instruction FACTA UNIVERSITATIS Series: Electronics and Energetics Vol. 30, N o 1, March 2017, pp. 137 - 144 DOI: 10.2298/FUEE1701137K MODIFIED INTERNAL MODEL CONTROL FOR A THERAPEUTIC ROBOT  Miloš D. Kostić 1 , Miroslav R. Mataušek 2 , Dejan B. Popović 2,3 1 Tecnalia, San Sebastian, Spain 2 University of Belgrade, Faculty of Electrical Engineering, Belgrade, Serbia 3 Institute of Technical Sciences of the Serbian Academy of Sciences and Arts, Belgrade, Serbia Abstract. We present the use of the modified internal model controller (MIMC) and the “Probability Tube” (PT) action representation for robot-assisted upper extremities training of hemiplegic patients. The robot-assisted training session has two phases. During the first "demonstration" phase the robot learns from the therapist the target path through examples. In the second "exercise" phase the robot assists a patient to follow the target path. During this process, the control limits the interface force between the robot and the hand to be below the preset threshold (F = 50 N). The system allows the assessment of the range of movement, the positional error between the target and the reached position, the amount of added assistance (the interface force between the hand and the robot). We demonstrate the operation in two hemiplegic patients. The patients and therapist suggested after the tests that the new system is straightforward and intuitive for clinical applications. Key words: stroke, disability, assistant robot, modified internal model control, assessment 1. INTRODUCTION Intensive repetition of functional movements is proven to be an efficient method of motor control relearning during the neurorehabilitation process [1]. Robotic devices are inherently well suited for repetitive tasks as well as for providing the quantitative assessment of performed movements, which is why they are becoming the preferred tools to support such therapeutic modality [2]. Two types of robot assistants are dominantly used for intensive exercise: 1) devices that assist the end-point movement of the arm and interface the patient at hand (e.g., MIT-Manus [3], Braccio di Ferro [4]) and 2) exoskeleton robots that assist individual arm joints and interface the arm at multiple points (e.g., ARMin [5], Cozens arm robot [6]).  Received May 24, 2016; received in revised form June 15, 2016 Corresponding author: Dejan B. Popović Institute of Technical Sciences of SANU, Kneza Mihaila 35, 11000 Belgrade, Serbia (E-mail: dbp@etf.rs) 138 M. D. KOSTIĆ, M. R. MATAUŠEK, D. B. POPOVIĆ Depending on the chosen therapy modality the robotic device can support, assist, resist or even perturb the movement of the arm/hand. To do so, robot assistants apply sophisticated methods for actuator control in position, velocity or impedance space. The rehabilitation gain is maximized when the device adapts to the patient’s performance in a manner which encourages the efforts, e.g., by providing "assistance-as-needed" or "faded guidance" [5 - 9]. To implement these complex assistance schemes the “haptic” approach, where the device acts on the patient with the force determined by the computer model, is frequently employed in the control of robot assistants [4, 5, 10, 11]. The essential elements of haptic control that are used in current robot assistants can be described with the following two equations: motor intrinsic hapticT (t) = T (q,q,q,p) + T (t) (1) T haptic hapticT (t) = J(q) F (t) (2) where [ q,q,q  ] are kinematic variables and p is a set of unknown parameters in the nonlinear model of intrinsic torque, Tintrinsic [4]. This torque relates to inertia, dissipative friction, and external forces (i.e., gravity). J(q) is the Jacobean of the device's geometry, and Fhaptic is the targeted interface force between the arm and the apparatus. The application of such a system requires an adequate nonlinear model and experimental assessment of unknown parameter p for an extensive range of operating conditions. A difficulty is that on-line compensation of the intrinsic dynamics is highly complex [4]. Another major practical problem for the implementation is the selection of the target trajectory for the hand that the robot needs to assist. We show here one possible method for solving two problems: 1) How to select a target trajectory which is suited to the current patient needs, and dynamically changing abilities; and 2) How can this trajectory be translated to the controller of a robot to is used in daily clinical work? We demonstrate a solution for both tasks in the case of point-to-point movements. The demonstration is presented with a new 3D robot prototype (R3-BEG), shown in Fig. 1. The assumed principle for the operation of the R3-BEG is: "teach-and-repeat" scenario [2, 7, 12], which is adopted in current clinical practice and present in some commercial devices [13]. The "teach-and-repeat" consists of the "demonstration" phase, in which the therapist and patient hold the endpoint of the robot, and the therapist selects a target trajectory based on heuristics; and the "exercise" phase, in which the robot assists the arm to move along the preferred trajectory with the force constraint (threshold maximum force) [13]. The following elements of the system are new: 1) The interface between the therapist, the patient and the robot used during the demonstration phase; 2) The action representation which translates the captured kinematics to the controller; 3) Integration of the natural variability of the therapist’s movements into the target trajectory [14, 15]; 4) Two-level control comprising at higher level velocity the set points selection in each movement phase, based on the “Probability Tube” (PT) action representation and at the low-level control implementation of the modified internal model control (MIMC) [16, 17] to ensure offset-free following of the set point; and 5) Motivating feedback based on the online assessment of the patient’s performance in the “exercise phase” (Fig. 1). The presentation Modified Internal Model Control for A Therapeutic Robot 139 starts with the description of the robot and controller, and continues to the presentation of tests in two post-stroke patients. 2. THE R3-BEG ROBOT ASSISTANT The R3-BEG combines a two-segment planar manipulandum (arm) and a vertical slider (Fig. 1). Following the analogy with the patient’s arm, the joints were named shoulder (S) and elbow (E). Fig. 1 The R3-BEG robot for the arm exercise (left panel). The sketch of the robot arm showing the task (top left panel) and feedback presented to the therapist (bottom right panels). The handle (Fig. 1) is instrumented by a set of force transducers allowing the estimation of the size and direction of the force acting at the handle in the plane orthogonal to the handle. This handle serves as the interface between the patient and the robot. The top part (extension) of the same handle is the interface between the therapist and the robot. This configuration allows the therapist to set the target trajectory by moving the end-point of the robot while the patient is holding the same handle. The force sensor is used in the second phase as the source of feedback for controlling the maximum assistive force constraint and for assessment of the added amount of assistance. High level control is based on methods described in [18], which suggested high rehabilitation potential, but required a sophisticated haptic platform. Here the PT is used as a lookup table to determine velocity set point, based on current movement phase and performance. This can be presented as: 1k),i, k )i),t(v(PT1 )i),t(v(PT( 1 PT)t(refv      (3) 140 M. D. KOSTIĆ, M. R. MATAUŠEK, D. B. POPOVIĆ where v(t) is current acceleration and i current phase. The factor k determines the level of allowed variability and is set up by the therapist. The low level control is based on two single-input-single-output MIMC linear digital controllers [17] to control the shoulder and the elbow of the system. The essential characteristics of the MIMC design and tuning concept from [18] are: it is well suited to exploit the benefits of prior knowledge and experience gained from the open-loop dynamics of the plant; the control system structure is directly obtainable from the model used to approximate process dynamics; a small number of tuning parameters, with clear meaning, followed by simple tuning rules, enough easy to apply. This concept also allows scalability of the presented solution, as it is suitable for designing multiple-input multiple- output (MIMO) neural network (NN) digital controllers [19]. Measured variables on the plant are the elbow and shoulder positions, pE(t) [rad] and pS(t) [rad], however, the controlled variables consist of the velocity of the elbow vE(t) [rad/s] and the velocity of the shoulder vS(t) [rad/s], which are obtained from E s E s E s sv (kT )=(p (kT ) p ((k 1)T ))/T  (4) S s S s S s sv (kT )=(p (kT ) p ((k 1)T ))/T  (5) Their dynamic characteristics are defined by the elbow velocity model GmvE(s) and the shoulder velocity model GmvS(s), which are obtained from open loop step response test. Models GmvE(s) and GmvS(s)are defined by equations 6 and 7: 1sTζ2sT eK (s)G EE 22 E sL E mvE e    , (6) 1sTζ2sT eK (s)G SS 22 S sL S mvS S    (7) where KE = 0.00024, KS = 0.00023, LE = 0.07, LS = 0.1, TE = 0.04, TS = 0.08, ζE = ζS = 0.7. Fig. 2 MIMC controller block diagram, modified from Fig. 2 in [17] The velocity models GmvE(s) and GmvS(s) were used to design and tune MIMC velocity controllers, defined by the structure presented in Fig. 2, modified from [17]. The elbow MIMC velocity controller is defined by: 2 4 rE E LE 0.4z F (z)º1, F (z) = , G (z) = z z-0.6       (8) Modified Internal Model Control for A Therapeutic Robot 141 -1 2 m0E 2 P (z) 1 z 1.3205z + 0.4966 = z 0.00024 0.1761z  (9) where z -1 represents the unite delay operator, z -1 = e− sTs . The shoulder MIMC velocity controller is defined by 2 5 rS S LS 0.2z 0.2z F (z) = , F (z) = , G (z) = z z 0.8 z-0.8        (10) -1 2 m0S 2 P (z) 1 z 1.6522z + 0.7047 = z 0.00023 0.0525z  (11) Both MIMC controllers are implemented with the sample time Ts = 0.02 s. We validated linear models of R3-BEG joints. The parameters of the models were estimated based on recordings of the open-loop step responses. The set-points to the elbow and shoulder controllers of the R3-BEG are defined in the phase-plane by the procedure described in Kostić et al. [14, 15]. However, to test the closed-loop tracking performance of the low level control (MIMC controllers - Equations 8-11), without the influence of higher level control algorithm, sinusoidal set-points defined in time were applied to the shoulder and the elbow control systems. Results presented in Fig. 3 were obtained for the control system defined in the loop with the MIMC elbow velocity controller by Equations 8 and 9. Fig. 3 Closed-loop responses for the elbow in the loop with MIMC elbow controllers (8) and (9): model (red line), plant (black line) and set-point (blue line). 3 IMPLEMENTATION OF THE R3-BEG Two hemiplegic patients signed the informed consent approved by the local ethics committee of the Clinic for Rehabilitation "Dr Miroslav Zotović", Belgrade, Serbia. Patient P1 had a small range of movement and was highly spastic while the patient P2 had a larger range of motion and less pronounced spasticity. The level of disability was assessed by an experienced clinician before the beginning of the tests (the Ashworth spasticity scale (AS), the action research arm test (ARAT), and the Fugl-Meyer (FM) motor test for upper extremities). 142 M. D. KOSTIĆ, M. R. MATAUŠEK, D. B. POPOVIĆ The session with R3-BEG followed the previously described two-phase procedure. In the "demonstration phase", the therapist "presented" the movement to the patient and the robot by manipulating the handle while the patient held the instrumented handle and was instructed not to resist the imposed movement between the starting and end points. The robot was passive (decoupled motors), and sensors captured movement kinematics and interface force. Each movement was repeated several times to create the action representation using a procedure described in detail elsewhere [Argall et al., 2009). The obtained PT provided set-points to the elbow and shoulder MIMC controllers of the R3- BEG in the phase-plane while the maximal force of assistance was defined as maximal interface force exerted by the therapist. In the "exercise phase", the robot assisted a patient to perform the desired point-to- point movement. There were two different movements, one in the ipsilateral direction and one in the contralateral direction. The starting position and the target were marked with a green and a red circle (diameters D = 4 cm), respectively. The handle was instrumented with a laser pointer which projected the position of the handle to allow the patient to know the position of the handle. Data presented in Figure 4 illustrate the performance of patients P1 and P2, respectively. The efficacy of the robotic intervention is documented by two objective measures: 1) the Euclidian distance between the reached position and the target point, which relates to the range of movement, and 2) the interface force between the hand and the R3-BEG, compared to the amount of provided assistance. These metrics were selected based on the recommendations of the European scientific community [19]. Fig. 4 Trajectories achieved by the patients P1 (severe spasticity - left panels) for the two target points. F is the force. D is distance between the end point of the trajectory and the target T. Right panels show the performance of Patient P2 (mild spasticity) As shown in Fig. 4 (left panels), the patient P1 was not able to completely perform the task and could not reach the target point in the case in which the handle needed to be moved to the contralateral side of his body (the distance between the endpoint of the movement and the target was 9.6 cm). However, he encountered fewer problems with the radial movement in the ipsilateral direction (D = 2.9 cm). The interface force indicates that the robot was assisting the movement all along the trajectory. The robot assisted the movement with significant force during the last 25 % of the movement (F ≈ 30 N). The force was gradually increasing to about 10 N during the first 75% of the movement. This result is by the patent’s impairment (constraints introduced by spasticity and decreased the range of movement) Fig. 4 (right panels) illustrates the Modified Internal Model Control for A Therapeutic Robot 143 performance of the patient P2 characterized with mild spasticity. In this case, the interface force was substantially smaller compared with the interface force estimated during the tests with patient P1. The distance between the endpoint and the target was only 2 cm and an interface force never reached F = 15 N. This indicates that the patent used the robotic guidance to compensate for the lack of motor control, rather than the compromised range of motion, which supports the reported patient impairment. 4 CONCLUSIONS We developed a control method for a rehabilitation robot. The new system was proved to be simple for tuning and implementation in the clinical environment. The novel "teach- and-repeat" method for high-level control, described in [14,15] implemented in this scenario was found to be useful for translating the therapist's skills and experience to the robot-assisted therapy. The signals from sensors used for control allow direct assessment of the differences between passive and active arm movements (range and smoothness of the movement and required force assistance). The force controlled interface (haptics) also allows the setup of the tasks that need to be trained to improve the performance. Acknowledgement: The work on this project was partly supported by the Project No RS35003, Ministry of Education, Sciences and Technological Development of Serbia. REFERENCES [1] G. Kwakkel, "Intensity of practice after stroke: More is better", Schweizer Archiv für Neurologie und Psychiatrie, vol. 160.7, pp. 295-298, 2009. [2] T. Nef, M. Mihelj, and R. Riener, "ARMin: a robot for patient-cooperative arm therapy", Medical & Biological Engineering & Computing, vol. 45(9), pp. 887-900, 2007. [3] N. Hogan, H. I. Krebs, J. Charnnarong, P. Srikrishna and A. Sharon, "MIT-MANUS: a workstation for manual therapy and training I", In Proceedings of the IEEE International Workshop Robot and Human Communication, 1992, pp. 161-165. [4] M. Casadio, V. Sanguineti, P. G. Morasso, and V. Arrichiello, "Braccio di Ferro: a new haptic workstation for neuromotor rehabilitation", Technology and Health Care, vol. 14(3), pp. 123-142, 2006. [5] T. Nef, and R. Riener, "ARMin-design of a novel arm rehabilitation robot", In Proc. of the 9 th IEEE International Conference on Rehabilitation Robotics, 2005, pp. 57-60. [6] J. A. Cozens, "Robotic assistance of an active upper limb exercise in neurologically impaired patients", Rehabilitation Engineering, IEEE Transactions on, vol. 7(2), pp. 254-256, 1999. [7] L. Marchal-Crespo, and D. J. Reinkensmeyer, "Review of control strategies for robotic movement training after neurologic injury", Journal of NeuroEngineering and Rehabilitation, vol. 6(1), pp. 20, 2009. [8] M. Casadio, P. Giannoni, L. Masia, P. G. Morasso, G. Sandini, V. Sanguineti, V. Squeri, and E. Vergaro, "Robot therapy of the upper limb in stroke patients: rational guidelines for the principled use of this technology", Functional neurology, vol. 24 (4), pp. 195-202, 2009. [9] H. I. Krebs, J. J. Palazzolo, L. Dipietro, M. Ferraro, J. Krol, K. Rannekleiv, B. T. Volpe, and N. Hogan, "Rehabilitation robotics: Performance-based progressive robot-assisted therapy", Autonomous Robots, vol. 15 (1), pp. 7-20, 2003. [10] R. Q. van der Linde, P. Lammertse, E. Frederiksen, and B. Ruiter, "The HapticMaster, a new high-performance haptic interface", In Proc. Eurohaptics, 2002, pp. 1-5. [11] R. Loureiro, F. Amirabdollahian, M. Topping, B. Driessen, and W. Harwin, "Upper limb robot mediated stroke therapy: GENTLE/s approach", Autonomous Robots, vol. 15 (1), pp. 35-51, 2003. 144 M. D. KOSTIĆ, M. R. MATAUŠEK, D. B. POPOVIĆ [12] J. L. Emken, S. J. Harkema, J. A. Beres-Jones, C. K. Ferreira, and D. J. Reinkensmeyer, "Feasibility of manual teach-and-replay and continuous impedance shaping for robotic locomotor training following spinal cord injury", Biomedical Engineering, IEEE Transactions on, vol. 55 (1), pp. 322-334, 2008. [13] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "A survey of robot learning from demonstration", Robotics and Autonomous Systems, vol. 57 (5), pp. 469-483, 2009. [14] M. D. Kostić, M. B. Popović, and D.. B. Popović, "A Method for Assessing the Arm Movement Performance: Probability Tube", Medical & Biological Engineering & Computing, vol. 51 (12), pp. 1315-1323, 2013. [15] M.D. Kostić, ;M. D. Popović, and D. B. Popović, "The robot that learns from the therapist how to assist stroke patients", New Trends in Medical and Service Robots. Springer International Publishing, pp. 17- 29, 2014. [16] M. R. Mataušek and D. M. Stipanović, "Modified nonlinear internal model control", Control and intelligent systems, vol. 26 (2), pp. 57-63, 1998. [17] M.R. Mataušek, A. D. Mićić, and D. B. Dacić, "Modified internal model control approach to the design and tuning of linear digital controllers", International Journal of Systems Science, vol. 33 (1), pp. 67-79, 2002. [18] M. R. Mataušek, D. M. Miljković, and B. I. Jeftenić, "Nonlinear multi-input-multi-output neural network control of DC motor drive with field weakening", IEEE Transactions on Industrial Electronics, vol. 45 (1), pp. 185-187, 1998. [19] "COST Action TD1006", http://www.rehabilitationrobotics.eu/2013.