عبد السلام وغيث Al-Khwarizmi Engineering Journal Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) Design of Robotic Arm Control System Mimics Human Arm Motion A. Salam Al-Ammri* Ghaith .A .Taki** *,**Department of Mechatronics Engineering/ Al-Khawarizmi College of Engineering/ University of Baghdad *Email: asalamalammri@yahoo.com **Email: ghaith.taki@gmail.com (Received 14 May 2012; accepted 6 November 2012) Abstract This paper presents a control system to make the robotic hand mimic human hand motion in real time and offline mode. The human hand tracking system is a wearable sensing arm (potentiometers) used to determine the position in space and to sense the grasping task of human hand. The maskable sensing arm was designed with same geometrical arrangement of robotic hand that needs to be controlled. The control software of a robot was implemented using Visual Basic and supported with graphical user interface (GUI). The control algorithm depends on joint to joint mapping method to match between the motions at each joint of portable sensing arm with corresponding joint of a robot in order to make the robot mimic the motion. Keywords: Robotic Arm, Control, human hand tracking. 1. Introduction Teleporting of a robot through human hand motion plays an important role in many fields such as medicine, space exploration, military, nuclear environment, construction and many other fields. A large number of interfaces concerning robot control using hand motion such as vision based system by tracking markers placed at a dorsal side of human hand using cameras to estimate the 3D position of hand to control the robot were proposed [ 1, 2]. EMG (electromyography) signals which are measured using non-invasive electrodes are placed at specific locations of human arm and represent the activity of human arm muscles due to motion to drive the robot [3]. Micro-Electro-Mechanical Systems (MEMS) was used to estimate the 3D position and orientation of human hand by integrating inertial sensors such as gyroscope, accelerometer and magnetometer [4]. Electro- mechanical devices (potentiometer and metal parts), are worn by the operator arm to sense the motion of arm [5]. This paper presents a method to control the robot motion using a built sensing arm that can be gripped by human hand to determine the position and orientation of human hand and to sense the grasping task performed by human hand finger. 2. The Experimental Work The system consists of four parts: sensing arm, interface circuit, robotic arm and control software, Fig.1. A. Sensing Arm A wearable arm Fig.2, is used to measure the 3D position of human hand (Within the workspace of sensing arm), and to sense the grasping of a hand finger. The sensing arm has the same degrees of freedom DOF of robotic arm, which is 4 DOF that needs to be controlled. It was made from four metal parts and four potentiometers (1M Ω) sensing arm joints, base, shoulder, elbow, wrist, and push-button switch for finger joint. mailto:asalamalammri@yahoo.com mailto:ghaith.taki@gmail.com A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 10 The principle of operation of each sensing arm joint (except finger joint) depends on the voltage divider. So, when the operator mask the sensing arm and move his hand, this motion will make each joint (except finger joint) rotate with a specific motion. Therefore, the resistance of each joint varies, so, the output voltage of each potentiometer will change according to the voltage divider principle and represents the measuring of joint rotation. Finger joint was implemented using a push-bottom switch which is connected with the ground. Fig. 1. Control System Block Diagram. Fig. 2. Sensing Arm B. Interface Circuit The analog voltage of each sensing arm joint (potentiometer) must be converted to the digital form before sending them to the computer via parallel port. Therefore, the interface circuit ,Fig.3, consists of four ADC IC’s (0804LCN) [6], one for each sensing arm (except finger joint) joint. Additionally, four Buffer IC’s (74Ls245) [7] are used to protect the computer from any harmful signals. The finger joint of sensing arm is connected directly to the parallel port without using ADC and buffer IC. The analog voltage of sensing arm base joint is converted to the digital voltages of 4-bits and are sent via pin(2),pin(3),pin(4),pin(5) of parallel port. The analog voltage of sensing arm shoulder joint is converted to the digital voltages of 3-bits and are sent via pin (6), pin (7), pin (8) of parallel port. The analog voltage of sensing arm elbow joint is converted to the digital voltages of 3-bits and are sent via pin (15), pin (13), pin (12) of parallel port. The analog voltage of sensing arm wrist joint is converted to the digital voltages of 2-bits and are sent via pin (10), pin (11) of parallel port. While, the push-button switch of a finger joint of sensing arm is connected between the ground and pin (9) of parallel port. The LEDs are used as an indicator to simplify a programming task. Computer Visual Basic Software Joint to joint motion mapping O,Robotic Arm Sensing Arm (potentiometers) Interface Circuit (ADC IC’s, buffer IC’s) Parallel Port A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 11 Fig. 3. Interface Circuit C. Robotic Arm The PhantomX Pincher AX-12 Robot Arm, Fig.4&5, consists of four joints plus a gripper. There are seven servo motors (AX-12 Dynamixel) used to actuate robot joints; one servo for base joint, two servos for shoulder joint, two servos for elbow joint, one servo is for wrist joint and one servo for gripper joint. Each servo motor has its own ID number and can rotate (300° through 1024 steps). The robot is supported with USB2Dynamixel interface circuit to connect the robot with computer through USB port, additionally, it is supported with SDK software to use it with different programs such as MATLAB, Visual Basic, LabVIEW , etc. Fig. 4. Robotic Arm Fig. 5. AX-12 Dynamixels Network D. Control Software The control algorithm of a robot reads the decimal value of status and data bus of parallel port and extracts the binary value of pin(2,3,4,5), pin(6,7,8) ,pin(9) pin(9 , pin(10,11) and pin(15,13,12) . The control algorithm uses (IF. .Else) instructions to match between the digital values of sensing arm joints with corresponding joint's motion of a robot in a manner to make the robot mimic the motion of sensing arm. For example, when the base joint of sensing arm rotates (90º) , the base joint of a robot also rotates (90°) to mimic this motion, and when the operator pushes the switch of a finger joint of sensing arm, the robot’s two fingers are closed to perform the grasping task. The software was supported with offline control to repeat the motions of a robot (during the real time control), Fig.6. A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 12 Fig. 6. (GUI) of Control Software. 3. Results When the sensing base joint rotates (CW) with angle (-90º) with respect to the reference point, the digital output of this motion is (1111b ) . In the same manner, when sensing arm rotates (CCW) with angle (90º) with respect to the reference point, the digital output of this motion is (0001b).Calculating the step of rotation for each digital output within (0111 b to 1111 b) is shown in table 1,and Fig.7 & 8: Өs= ° ° = 11.25° , and Өr= =~39 step ( for robot) And to calculate the step of rotation for each digital output within (0111 b to 0001 b): Өs= ° ° = 15°, and Өr= =51 step ( for robot). Table 1, Sensing Arm and Robot Base Joint Motion. Ө Sensing Base Digital Output Sensing Base Rotation Angle Robot Base Motion Ө1 0111 b 0° 512+(0*51)=512 Ө2 0110 b 15° 512+(1*51)=563 Ө3 0101 b 30° 512+(2*51)=614 Ө4 0100 b 45° 512+(3*51)=665 Ө5 0011 b 60° 512+(4*51)=716 Ө6 0010 b 75° 512+(5*51)=767 Ө7 0001 b 90° 512+(6*51)=~820 Ө8 0000 b 105° 512+(7*51)=869 Ө9 1000 b -11.25° 512-(1*39)=473 Ө10 1001 b -22.5° 512-(2*39)=434 Ө11 1010 b -33.75° 512-(3*39)=395 Ө12 1011 b -45° 512-(4*39)=356 Ө13 1100 b -56.25° 512-(5*39)=317 Ө14 1101 b -67.5° 512-(6*39)=278 Ө15 1110 b -78.75° 512-(7*39)=239 Ө16 1111 b -90° 512-(8*39)=204 A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 13 Fig. 7. Range of Sensing Arm and Robot Base Joint Motion. Fig. 8. Sensing Arm and Robot Base Angles. For shoulder joint, the digital value (001) was chosen as a reference point, so, the rotation angle for this position is considered (0°). Now, to rotate down with rotation angle (-90º), the digital output of sensing shoulder will be (101) . To calculate the rotation angles within (111 - 000) ), table 2 and Fig.9 &10: Өs= ° ° = 22.5° and Өr= =67 step (for robot). Table 2, Sensing Arm Shoulder Joint Motion. Ө Sensing Shoulder Digital Output Sensing Shoulder Rotation Angle = − . , = , , Robot Shoulder Motion + , = − , ,1 Ө1 000 b 22.5º 512-(1*67)=445 Ө2 001 b 0º 512+(0*67)=512 Ө3 010 b -22.5º 512+(1*67)=579 Ө4 011 b -45º 512+(2*67)=646 Ө5 100 b -67.5º 512+(3*67)=713 Ө6 101 b -90º 512+(4*67)=780 Ө7 110 b -112.5º 512+(5*67)=847 Ө8 111 b -135º 512+(6*67)=914 Step 512,0º Step 204,-90° Step 820,90º Ө16 Ө15 Ө14 Ө13 Ө12 Ө11 Ө9 Ө10 Ө2 Ө3 Ө1 Ө4 Ө5 Ө6 Ө7 Ө8 -150 -100 -50 0 50 100 150 -150 -100 -50 0 50 100 Sensing base angles Robot base angles A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 14 Fig. 9. Range of Sensing Arm and Robot Shoulder Joint Motion. Fig.10.Sensing Arm and Robot Shoulder Angles. There are eight digital outputs due to sensing arm elbow joint (000 to 111). The digital output (001) was chosen as a reference point with a rotation angle (0°) and when the joint rotates up with angle (90º), the digital output is (101),table 3 and Fig .11 &12. Өs= ° ° = 22.5° and Өr = =77 step (for robot). Table 3, Sensing Arm and Robot Elbow Joint Motion. -120 -100 -80 -60 -40 -20 0 20 40 -150 -100 -50 0 50 Robot shoulder angle Sensing shoulder angle Ө Sensing elbow digital output = − . , = , , Sensing elbow rotation angle − , = , − , − Robot wrist Motion Ө1 111 135° 512-(6*77)=50 Ө2 110 112.5° 512-(5*77)=127 Ө3 101 90° 512-(4*77)=204 Ө4 100 67.5° 512-(3*77)=281 Ө5 011 45° 512-(2*77)=358 Ө6 010 22.5° 512-(1*77)=435 Ө7 001 0° 512+(0*77)=512 Ө8 000 -22.5° 512+(1*77)=589 Θ2 Θ3 Θ1 Θ4 Θ5 Θ6 Θ7 Θ8 Step 512 Step 780 A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 15 Fig. 11. Range of Sensing Arm and Robot Elbow Joint Motion. For wrist joint, there are four digital outputs (00b to 11b). The digital value (10b) was chosen as a reference point, below are the results of motion, table 4 &5 and Fig 12 &13 : Table 4, Sensing Arm and Robot Wrist Joint Motion. Table 5, Sensing Arm And Robot Two Fingers. Sensing switch Robot fingers motion Robot fingers status Pressed (0b) 700 Close Released (1b) 600 open Ө Sensing wrist digital output Sensing wrist rotation angle Robot wrist motion Ө1 10 b 0° 512 Ө2 11 b -45° 666 Ө3 01 b 45° 358 Ө4 00 b 90° 204 Θ7 Θ 6 Θ5 Θ4 Θ1 Θ2 Θ 3 Θ8 Step 512 Step 204 A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 16 Fig. 12. Range of Sensing Arm and Robot Wrist Joint Motion Fig. 13.Sensing Arm and Robot Wrist Angles. 4. Conclusion This paper presents a real time and offline (teaching) control of robotic hand. The control system depends on (If. Else) , instruction to map directly between sensing arm and robot joints,Fig.14. The system achieved a good response time of robot control (~ 1 second) and high repeatability. The dimensions were chosen to cope with geometrical dimensions of a real robot because they affect on the results of motion. The type of potentiometer affects on the analog to digital conversion performance through the noise that may occur due to bad manufacturing of a potentiometer. -100 -80 -60 -40 -20 0 20 40 60 -100 -50 0 50 100 Robot wrist angles Sensing wrist angles Ө 3 Ө 2 Ө 4 Ө 1 Step 204 Step 512 A. Salam Al-Ammri Al-Khwarizmi Engineering Journal, Vol. 9, No. 1, P.P. 9-18 (2013) 17 Fig. 14.Samples of Robot Motion. 5. References [1] Jonathan and Siddharth ,” Teleportation of a Robot Manipulator Using a Vision- Based Human –Robot Interface” , IEEE Transaction on Industrial Electronics, VOL.52, NO.5, 2005. [2] Chris Yu-liang LIU,” Three-Dimensional Hand Tracking and Surface – Geometry Measurement for a Robot-Vision System”, university of Waterloo Canada,2008 [3] Jorn and Patrick , “ EMG-based Teleportation and Manipulation with the DLR LWR-III “ , IEEE/RSJ International Conference on Intelligent Robots and Systems,2011 [4] Farrukh Iqbal Sheikh , “ Real-Time Human Arm Motion Translation for the WorkPartner Robot “, Helsinki University of Technology,2008 [5] Gupta and Demidenko , “ Master-Slave Control of a Teleported Anthropomorphic Robotic Arm with Gripper Forces Sensing “ , IMTC,2005 [6] “ ADC0804 LCN Data sheet” [7] “ 74LS245 Data sheet “ )2013( 9- 18، صفحة 1، العدد9مجلة الخوارزمي الھندسیة المجلدعبد السالم عبد العباس العامري 18 ذراع االنسان حركة حاكي یانسان الي ذراع لتصمیم نظام تحكم **غیث عبد الودود تقي* السالم عبد العباس العامريعبد جامعة بغداد/ كلیة الھندسة الخوارزمي/ قسم ھندسة المیكاترونكس** ،* asalamalammri@yahoo.com :البرید االلكتروني* ghaith.taki@gmail.com :البرید االلكتروني** الخالصة متابعةنظام اللقد تم تصمیم وبناء .غیر متصلالالوضع بالوقت الحقیقي وحركة ذراع اإلنسان لمحاكاة ،ذراع إنسان آليلنظام تحكم البحث ھذا رضعی التحسس نظام تصمیم لقد استند في . االلتقاط لید اإلنسانالمسك وعملیة كذلك لحركة ید االنسان من خالل نظام تحسس لتحدید الموقع واالتجاه في الفضاء و واجھة بفیجوال بیسك بواسطة برنامج ألُنِفذت خوارزمیة التحكم بذراع اإلنسان اآللي .المراد السیطرة علیھا متابعة حركة ذراع اإلنسانلالھندسي الھیكل على وذراع االنسان لیتم محاكاتھا ذراع التحسس كل مفصل للمطابقة حركة اعتمدت خوارزمیة السیطرة على طریقة خارطة مفصل لمفصل . رسومیة للمستخدم . ذراع اإلنسان من قبل mailto:asalamalammri@yahoo.com mailto:ghaith.taki@gmail.com