Al-Khwarizmi Engineering!!! Journal Al-Khwarizmi Engineering Journal, Vol. 6, No. 1, PP 31- 47 (2010) Bionics-Based Approach for Object Tracking to Implement in Robot Applications Hussam K. Abdul-Ameer Department of biomedical Engineering/ Al-Khwarizmi College of Engineering/ University of Baghdad (Received 7 December 2009: accepted 17 February 2010) Abstract In this paper, an approach for object tracking that is inspired from human oculomotor system is proposed and verified experimentally. The developed approach divided into two phases, fast tracking or saccadic phase and smooth pursuit phase. In the first phase, the field of the view is segmented into four regions that are analogue to retinal periphery in the oculomotor system. When the object of interest is entering these regions, the developed vision system responds by changing the values of the pan and tilt angles to allow the object lies in the fovea area and then the second phase will activate. A fuzzy logic method is implemented in the saccadic phase as an intelligent decision maker to select the values of the pan and tilt angle based on suggested logical rules. In smooth pursuit phase, the object is kept in the center area of the field of the view by smooth adjusting the values of the pan and tilt angles where this stage is analogue to putting the observing object in the fovea centralis of the oculomotor system. The proposed approach was implemented by using (Camera-Pan / Tilt) configuration and showed good results to improve the vision capability of the humanoid robots. Keywords: Tracking, bionics, and fuzzy decision maker. 1. Introduction Human vision systems have an appropriate ability to interpret nature scenes in real time, despite the limited speed of the biological architecture available for such tasks. The cerebral control system that directs the eye toward an object for viewing is as significant as the system that interprets the visual signals received from the eye. In the same way as the cerebral control system, the most important role of the active vision system is to direct an artificial visual receptor, such as a CCD camera toward an interesting target object in the visual field as in a human vision system [1]. This redirection is usually referred to as object tracking, where it is the problem of following image elements moving across a video sequence automatically. Object tracking is an essential building block for vision systems addressing robotic tasks like visual servoing, pipe and cable inspection, and metrology [2]. Many studies considered the object tracking problems. Some of the most recent and related papers to the present paper are reviewed. In [3], [4], [5], and [6] object tracking algorithms for mobile robot are developed, where [3] suggested a control technique that end-effecter of manipulator can chase an object using an image gained from a stereo camera installed in a robot platform. The adaptive color matching and Kalman filter were used in the proposed algorithm in [4]. In [5], the developed algorithm is based on sensor network, where the occlusion problem is assumed and solved. 3D-object tracker using a new Kalman- filter based framework that carries out fast and accurate rigid object tracking was introduced in [6]. The authors of [7] presented a grasp tracking method to be used in eye-in-hand robot configuration, where it consists of three stages, grasp synthesis, grasp transfer, and grasp analysis. An object tracking method based on image This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 32 segmentation and rough set minimal rule set was presented in [8], where the rough set theory is used to control the camera pose. In [9], the problems of tracking human face using active camera that installed on 2-DOF (pan-tilt) platform was discussed, and the kinematics model between object in motion and image of the camera is achieved. In [10],[11],and [12], fuzzy technique is adopted in the developed tracking algorithms, where the author of [10] applied fuzzy method in the CIElabcolor space to model target color in a target recentering task using an active camera. Fuzzy control system and adaptive Rao-Black partial filter to track object in 3D space were used in [11]. In [12], texture segmentation approach using both wavelet transform and fuzzy grammar as classifier was introduced. The references [13] and [14] demonstrated multi-sensor fusion in object tracking algorithms, where in [13], the cooperation method of cameras for object tracking was introduced, while [14], presented the localization and object tracking problems using audiovisual measurements, where two microphones are used beside CCD camera to achieve target localization. Bionics studies simulating the principles of biologic system principles to build advanced technology systems or enable the artificial technology systems have some characteristics or functions of biologic system [15]. In the following, some of the recent studies that simulating the biological oculomotor system to use in visual tracking applications will be presented. Fortunato Flores Ando and Alfredo Weitzenfeld Ridel [16] introduced a model of the oculomotor system involved in the generation of saccadic and its implementation in a real robot used to keep track of visual input lost due to reorientation. Reference [17] presented a new method for visual scanning and target tracking by means of independent pan-tilt cameras, which mimic chameleon visual system. In [18], a discussion of how human eyes track an object by moving the optical axis is presented. Reference [15] introduced a visual tracking system for unmanned aerial vehicle (UAV) where the principle of biomimetic eye is used. In this paper, an approach for object tracking based on bionics of human eye movements has been introduced and verified experimentally. The proposed tracking method is divided into two stages, saccade and pursuit stages. In the saccade stage, fast searching and tracking algorithm for the observing object is developed, where fuzzy technique is used as decision maker to predicate the proper pan and tilt angles that are needed to direct the camera platform to allow the tracked object to be within the field of camera view. In the second stage, pursuit-tracking algorithm is proposed where a smooth object tracking will occur. The present paper layout is as follows: a description of the human oculomotor system is presented in section two. The proposed tracking approach is illustrated in section three. Experimental setup and results with their discussions are shown in sections four and five. Finally, conclusions are drawn in section six. 2. Human Oculomotor System The ocular motor control system is an effective device for capturing an object in the central pits (fovea) of the retinas. Human eyes always move their optical axes to keep the image from leaving the central pit, i.e. eyeballs never let an image move freely over the retina. When the fovea of the retina catches a moving target, the oculomotor control system performs different kinds of eye movements according to the position and velocity of the target and head. These movements are produced by contractions of the extrinsic eye muscles, innervated by neurons originating in the brain [18],[19],[20]. Figure (1) shows the fovea centralis and the internal anatomy of the eyeball. There are three types of eye movements coordinated by the brain [20]: Saccade eye movements are very high velocity movements, 400 to 800 degree per second, of both eyes that target an image on the fovea centralis. Smooth pursuit movements are slower (up to 30 degree per second), and match the speed of the moving objects to keep their images at or near the fovea. Vergence movements (30 to 150 degree per second) cause the eyes to converge so that an image of an object is brought to the fovea of both eyes, allowing the object to be seen more clearly three-dimensionally. The saccadic movements occurred when an object enters the filed of view, where the projected image of the object will be lied on the periphery area of the retina. This area contains a type of photoreceptors, which allowed the visual cortex to diagnose a non-accurate shape and position of the observing object. The brain will send impulse signals to the eye muscles to actuate the eyeball position to a new position allowing the projected image of the object laying on the fovea, This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 33 which is a pit in the retina containing a rich amount of cones photoreceptors that can give a clear view and accurate location of the seen objects. When the image of object is projected on fovea, a smooth pursuit movement will take place. This movement is coordinated by the brain when the eyes track a moving target smoothly. The developed tracking approach used both saccadic and smooth pursuit principles to achieve object-tracking process. The vergence movements will not considered in this research because a monocular vision system is assumed. Fig.1. The internal Anatomy of the Eyeball [20]. 3. Tracking Approach In this section, an overview of the developed approach will be presented. The tracking approach assumed a monocular active camera, which is installed on pan/tilt platform. This structure refers to artificial eyeball, where the acquired images from the active camera will process through artificial central processing unit it gives the proper decisions to pan/tilt platform form of angle values and then the platform motors will be actuated based on these values. The process of acquiring and manipulation of the images and giving the instructions to the platform is referred to as visual feedback and it is used widely in the applications of robot tracking. Two stages of the tracking method is assumed, fast or saccadic and smooth pursuit. Since the human oculomotor system reflects the object image in tracking task on the retina into two regions, periphery and fovea, image plane will be divided into two regions also, periphery and fovea regions, where fast tracking will occur in the periphery region and the smooth tracking will take place at the fovea region. In the saccadic stage, a developed searching algorithm will be applied on the saccadic region to check if the object entered the field of the view. If so, a developed fuzzy decision maker will select the pan and tilt angle values to redirect the artificial eye system in order to put the object within the fovea region. If the object enters the fovea region, the second stage of the tracking approach will be activated. In this stage, the tracked object will be kept in the fovea region by adjusting the pan and tilt angle values of the platform, where a search algorithm based on object centroid is used. Figure (2) presents a block diagram of the developed approach. In the following subsections, complete descriptions of saccadic phase, the used fuzzy technique, and smooth pursuit phase will be illustrated. 3.1. Saccadic Phase In human oculomotor system, saccadic is small jerking movements that rapidly bring eye from one fixation point to another and allow search of the visual field [21]. During a saccade, the oculomotor system operates in an open-loop mode This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 34 because the information from the retina and muscle proprioceptors is not transmitted quickly enough during the eye movement for use in altering the control signal [22]. For that, open- loop mode will be considered in the saccadic phase of the tracking approach. Fig.2. The Proposed Approach for Object Tracking. Digital Video Camera Two successive images are captured Image values are binaries The binaries images will be subtracted Weight Factors are calculated for the resulted image The Proposed FDM Pan and Tilt angle values will obtain If WFn≠0 Is motion detected in fovea Image is captured and binaries The edges of the observing object are detected The object COP is calculated If abs(COPx,y− COFx,y) ≠ 0 Pan and Tilt angle values will calculate Control software of Pan/Tilt unit Parallel Interface Card (Pan/Tilt) Unit Saccadic Pursuit Control part Image is captured and binaries This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 35 The image plane, artificial retina, is divided into two regions, saccadic and fovea, where at saccadic region the saccadic tracking phase will be applied, while the smooth pursuit-tracking phase will occur in the fovea region. Figure (3) presents the proposed divisions of the image plane, where the saccadic region is divided into four zones. These zones are analogue to the periphery area of retina. Weight factor will be introduced at each zone to give an indication about the amount of the object that is entered in each zone. This weight factor can mathematically be presented as the ratio of the summation of the white pixels, object pixels, in each zone to the total number of pixels in the zone. Equation (1) presents this relation: n zz n Z yxI WF  ),( …(1) where: ),( zz yxI is the pixel value at each zone and the subscript (n) refer to zone number. Z is the total number of pixels at each zone. Fig.3. The Proposed Divisions of Image Plane. The weight factor may have any value between 0 to 1, where zero value means no object entered the zone, while one value points to the object that entered the whole zone. Interference between object and background pixels may occur. To minimize the effect of the interference, a subtraction process between successive frames will be adopted and the gained image will be used to calculate the weight factors. This process can be expressed as follow: ),(),(),( ''' yxIyxIyxI  …(2) where the super scribe presents the image function after a certain period. The weight factors for the saccadic zones will be used to give the proper pan and tilt angle values that are used to drive the camera platform to a new location. The selection of the pan and tilt angle is based on logical statements, where table (1) shows the required logical values of pan and tilt when the object enters saccadic zones individually. To clarify this, assume the weight factor of the first zone indicates a value, so, to redirect the camera platform to allow the object to be in the fovea region, positive pan and negative tilt angle values are needed. The amounts of these values depend on the weight factor value, where a large weight factor means a large increase in angles value. Table 1, The Required Logical Values of Pan and Tilt when the observing Object Enters the Saccadic Zones Individually. Weight Factor Weight factor value Pan value Tilt value WF1 Nil Nil Nil 0.25 Positive small Negative small 0.5 Positive middle Negative middle 0.75 Positive large Negative large WF2 Nil Nil Nil 0.25 Negative small Negative small 0.5 Negative middle Negative middle 0.75 Negative large Negative large WF3 Nil Nil Nil 0.25 Negative small Positive small 0.5 Negative middle Positive middle 0.75 Negative large Positive large WF4 Nil Nil Nil 0.25 Positive small Positive small 0.5 Positive middle Positive middle 0.75 Positive large Positive large When two weight factors have values; i.e. the object enters two zones; new logical statements -T ilit − Tilt (CW) +Pan (CCW) − Pan (CW) + Tilt (CW) This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 36 are needed to be added. If three weight factors indicate certain values, zoom out is required because the object is large. However, this case is limited because the large object will lie on the fovea region; therefore, there is no need to the saccadic phase. Since, the selection of the pan and tilt angle values needs a combination between two or more of the logical statements when the object enters more than one zone, fuzzy logic will be used as a decision maker to select the proper pan and tilt values. 3.2. Fuzzy-based Decision Making (FDM) The FDM was used to choose the pan and tilt angle values based on the weight factors of the saccadic zones. Figure (4) shows a block diagram of the FDM, where four linguistic variables (LVs) were used as input to the FDM and two (LVs) were used as output. The input (LVs) are the weight factors and the output (LVs) are Pan and Tilt angles. Both input and output (LVs) used five different fuzzy sets for each input and output. These sets are Out, In Small, In Middle, In Large, and In for the input (LVs) and Negative Large (NegL), Negative Small (NegS), Zero, Positive Small (PosS), and Positive Large (PosL) for the output (LVs). In figure (5), the membership functions of the input and output (LVs) are illustrated, where triangular shape was adopted to present the membership function. The developed rule base consists of 81 rules, where table (2) shows sparse rules of the developed rules. The singleton technique was used for fuzzification and the adopted fuzzy inference method is Mamdain’s, while the defuzzification is the center of gravity method. Fig.4. Block Diagram of the FDM. Fig.5. The Proposed Membership Functions of the Input and Output Linguistic Variables. Weight Factor (WF1) Weight Factor (WF2) Weight Factor (WF3) Weight Factor (WF4) F u zz if ic at io n (S in g le to n ) Inference mechanism (Mamdain’s) Rule-base D ef u zz if ic at io n (C O G ) Pan Angle (Deg.) Tilt Angle (Deg.) This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 37 Table 2, Sparse Rules of the Developed FDM. Input of FDM Output of FDM WF1 WF2 WF3 WF4 Pan Tilt In Middle Nil Nil Nil PosS NegS In Small In Middle Nil Nil NegS NegS Nil In Small In Large Nil NegS PosS Nil In Middle In Middle Nil NegS Nil Nil Nil In In Small NegL PosL Nil Nil In Large In Middle NegS PosS In Large Nil Nil In Large PosL Nil In Nil Nil In PosL Nil 3.3. Smooth Pursuit Phase This phase will be activated when the object enters the fovea region, where fovea in human eyeball is a pit in retina that contains a dense amount of photoreceptors. The important task here is to keep the object in the fovea region otherwise saccadic phase will be reactivated. The developed algorithm for this phase is based on detecting the object edges and then locating the center of the perimeter (COP) of it. This process will be repeated for the next frame. The path between COP and the center of fovea region (COF) will give the orientation of the pan and tilt angles, where figure (6) presents the logical orientation of them based on determining the direction of the path between COP and COF. Since the orientations of the angles are obtained, the values of pan and tilt are needed to be determined, where these values depend on different parameters. The most significant parameters are the velocity and the acceleration of the object. In addition, the extrinsic parameters of camera, such as resolution and focal length, can play an important role in this phase. In terms of mathematics, the angles value can be expressed as follow: ),( camo fVfangle  …(3) where: oV is the object velocity camf is the camera extrinsic parameters. Equation (3) is an empirical function and depends on the selection of camera specifications, boundary conditions of the fovea region, lighting conditions, image digitization, and environmental noise. However, in this paper fixed and controlled conditions will be assumed in this phase. Fig.6. The Logical Orientation of the Pan and Tilt Angles in Fovea Region. 4. Experimental Setup An experimental setup was developed to verify the proposed approach. It consists of Pan / Tilt unit type (Luntai - PTS 302), where it has two rotational joints, one perpendicular to the other in order to simulate the pan and tilt motions of a real eye. Digital video camera type (Panasonic NV- GS25) is used as an image acquiring device, where it is held on the pan/tilt unit and linked with PC computer using hi speed USB link. The images are digitized at 720×576 and the fovea region is selected as 240×192. Interface card between the pan/tilt unit and PC is built in order to control the pan/tilt unit via MATLAB environment. Figure (7) presents a photograph of the developed experimental setup. 5. Results and Discussions In this section, the proposed method for object tracking is verified using the developed experimental setup. The algorithm of the tracking approach was written using (MATLAB-Ver. 7.3) environment. Each phase of the approach is verified separately. In the following subsections, the results and their discussions will be presented. COF If COP in (1) quarter then -Pan (CW) -Tilt (CW) If COP in (4) quarter then -Pan (CW) +Tilt (CCW) If COP in (3) quarter then +Pan (CCW) +Tilt (CCW) If COP in (2) quarter then +Pan (CCW) -Tilt (CW) This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 38 Control apparatus on (Pan\Tilt) unit – Consists of interface- card and power supply. Digital video camera (Pan / Tilt) Unit The observing object Fig.7. Photograph of the Developed Experimental Setup. 5.1. Saccadic Phase Eight different cases are adopted to investigate this phase. In each case, part of the observing object is appeared in the image plane. Figure (8) shows the adopted cases. A circular plane object is used as the observing object where its diameter is 8 cm. To scrutinize the effect of object enlargement on the proposed approach, three different locations are used for the (Camera/(Pan- Tilt unit)) (CPT) configuration, where the distance between each location is 10 cm. Fig.8. The selected Cases to Investigate Saccadic Phase. Case1: In this case, a part of the observed object is laid in the upper left corner of the image plane. This area belongs to the first saccadic zone and the proposed approach will respond by giving the proper pan and tilt angle values to put the object in fovea region. Figure (9) shows the output results of the saccadic phase for the three different locations of the CPT configuration, where it presents the relation between the pan and tilt values with frame number. From figure (9), the absolute values of pan and tilt angles of the first frame are matched, while the sign of tilt angle is negative and it shows that the object is laid in the first zone of the saccadic region. This result conforms the operation of developed FDM. Figures (9 - b and c) show different behavior form figure (9-a), where this is related to different reasons. The first one is object enlargement that results from decreasing the distance between the object and the CPT configuration, which may cause the weight factors of the saccadic zones to increase. This increasing will have an effect on the outputs, pan and tilt angles, of the FDM. The second reason is the initial values of the pan and tilt angles, where these values may cause transferring the object form a certain zone to another or may cause the object to lie between two zones that make the FDM give diverse outputs for each case. In addition, noise that comes from light condition and object projection on the image plane may increase or decrease the Fovea Region Case 1 Case 6 Case 7 Case 5 Case 8 Case 2 Case 3 Case 4 Zone 1 Zone 2 Zone 3Zone 4 This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 39 weight factors of the saccadic zones and this will change the outputs of the FDM. However, the total net values of the pan and tilt angles has nearly similar values for the cases (a), (b), and (c) of figure (9) and that indicates the robustness of the developed approach to object enlargement problem. It can be seen from figure (9) that the number of frames that are required to put the object in fovea region does not exceed four frames for all location cases. This result indicates the acceptable response of the developed approach. Fig.9. Results of Case 1-Saccadic Phase. Case2: In this case, part of the observed object is laid in two zones (1 and 2) of the saccadic region. Figure (10) shows the obtained results of this case, where the optimal response of this case can be considered as changing in tilt values and that is presented in figure (10-a). In other figures (10-b and c), varying in pan values can be recognized. The enlargement of the object causes this varying where the weight factors of the zones (1 and 2) are increased and the rules of the FDM will add an additional amount of the pan angle value. However, the total net values of the pan angle for each location case of case 2 is close to zero or can be neglected. Fig.10. Results of Case2 –Saccadic Phase. Case 3: The object in this case is laid in the upper-right side of the image plane. Figure (11) shows the experimental results of this case, where it can be observed that in the first frame the values of the pan and tilt angles are identical. This is caused by laying the object in the second zone of the saccadic region. The FDM responded ideally in this case. In figure (11-c), fluctuation in the angle values could be recognized and the number of frames that are required to put the object in fovea region is increased from that which is stated in figures (11-a and b). This behavior can be explained as follows: the first angle values are large because the influence of the object enlargement. These large values will put the object in the third zone of the saccadic region. Then the object will lie in the first saccadic zone because of the pan and tilt angle values, as shown in figure (11-c). This object fluctuation between the first and third zones will continue until the object is laid completely in fovea region. 12.23 6.23 0.06 0.02 0.01 0 0 -12.23 6.23 -0.29 -0.13 -0.01 0 0 -15 -10 -5 0 5 10 15 1 2 3 4 5 6 7 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt 5.69 9.63 8.88 0 0 0 0 0 -5.69 -9.63 8.88 0 0 0 0 0 - 15 - 10 - 5 0 5 10 15 1 2 3 4 5 6 7 8 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt 8.91 11.54 -0.4 0 0 0 0 0 0 -8.91 -4.56 6.45 0 0 0 0 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 7 8 9 Frame Number A n g le V a lu e ( D e g .) Pan Tilt (a) 50 cm far from object (c) 30 cm far from object (b) 40 cm far from object 0 0 0 -6.12 -3.88 0 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 Frame N umber A n g le V a lu e ( D e g .) Pan Tilt (a) 50 cm far from object -9.66 4.69 0.17 0.12 0.02 0 -12.02 3.15 -0.17 -0.12 -0.02 0 -15 -10 -5 0 5 10 1 2 3 4 5 6 Frame Number A n g le V a lu e ( D e g .) Pan Tilt (c) 30 cm far from object -3.47 2.92 0 0 0 0 -6.75 -2.92 0 0 0 0 -8 -6 -4 -2 0 2 4 1 2 3 4 5 6 Frame N umber A n g le V a lu e ( D e g .) Pan Tilt (b) 40 cm far from object This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 40 Fig.11. Results of Case3 –Saccadic Phase. Case4: In this case, part of the object is placed in the middle-left side of the image plane. The ideal response of the FDM to this case is by giving a value to the tilt angle only. From figure (12), it can be seen that in the first case the tilt angle is less than 1 degree. However, the FDM gives a value to the tilt angle for other location cases that are presented in figures (12- b and c) and in the next frames in figure (12-a). This manner of acting is caused due to unavailable knowledge of the object size that is similar to human response in the saccadic movement. The overall tilt values for all cases presented in figure (12) are less than 1.5 degree and that shows the adaptability of the developed approach to the object size. Case 5: This case is the mirror to case 4, where the object is laid in the second and third zones of the saccadic region. From figure (13), the response of the FDM is ideal, where the maximum tilt value does not exceed 2.2 degree and the overall tilt values for each location case of figure (13) is less than 0.4 degree. Fig.12. Results of Case 4 –Saccadic Phase. Fig.13. Results of Case5 –Saccadic Phase. -12.31 -7.35 0.82 0 -0.01 0 -12.31 7.35 -3.59 0 -0.01 0 -15 -10 -5 0 5 10 1 2 3 4 5 6 Fr am e Num ber A n g le V a lu e ( D e g .) Pan T ilt -9.48 - 6.76 0 0 0 0 -9.48 1.56 0 0 0 0 -12 -10 - 8 - 6 - 4 - 2 0 2 4 1 2 3 4 5 6 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (b) 40 cm far from object (a) 50 cm far from object -15.74 -14.86 19.87 -13.05 4.26 0.23 0.09 0.07 0.02 -15.74 16.48 -19.87 12.98 -3.87 0.24 0.09 0.07 0.02 -30 -20 -10 0 10 20 30 1 2 3 4 5 6 7 8 9 Fr ame Number A n g le V a lu e ( D e g .) Pan Tilt (c) 30 cm far from object Zone 1 Zone 3 Zone 1 Zone 3 5.26 8.87 9.21 -6.04 0 0 0 0 2.27 -6.66 9.21 -6.04 0 0 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 7 8 Fr am e Num ber A ng le V al u e (D e g .) Pan Tilt (b) 40 cm far from object 9.39 12.08 -7.26 14.8 -16.57 17.43 -12.79 2.84 -0.92 0-1.2 4.39 -8.37 14.8 -16.57 17.43 -12.79 3.99 -1.5 0 -20 -15 -10 -5 0 5 10 15 20 1 2 3 4 5 6 7 8 9 10 Fr am e Num ber A ng le V a lu e ( D eg .) Pan Tilt (c) 30 cm far from object 7.41 10.19 6.58 -5.19 0.54 0.11 0 0 0-0.92 4.29 - 6.58 5.19 - 0.75 -0.11 0 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 7 8 9 Frame N umber A n g le V a lu e ( D e g .) Pan Tilt (a) 50 cm far from object -5.65 -7.26 -5.27 0 0 0-0.21 1.09 -1.74 0 0 0 -8 -6 -4 -2 0 2 1 2 3 4 5 6 Fr am e Num ber A ng le V a lu e (D e g. ) Pan Tilt (b) 40 cm far from object -5.65 -13.66 -1.47 0 0 0 -1.01 2.1 -1.47 0 0 0 -15 -10 -5 0 5 1 2 3 4 5 6 Fr am e Num ber A ng le V al ue ( D eg .) Pan Tilt (c) 30 cm far from object -6.83 -9.96 -1.39 0 0 0 -0.75 2.19 -1.37 0 0 0 -12 -10 -8 -6 -4 -2 0 2 4 1 2 3 4 5 6 Fr am e Num ber A ng le V a lu e (D eg .) Pan Tilt (a) 50 cm far from object This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 41 . Fig.14. Results of Case6 –Saccadic Phase. Fig.15. Results of Case7 –Saccadic Phase. Cases 5, 6, and 7: These cases are similar to cases (1,2, and 3), where the same behavior in general can be noticed as shown in figures (14,15, and 16) respectively. However, figure (16-c) presents the outputs of the third location case of case 7; the FDM failed to put the object in the fovea region. This miss tracking can relate to the effect of object enlargement. However, the tracking of large objects in human oculomotor system will be in fovea and no saccadic movements are necessary. Fig.16. Results of Case8 –Saccadic Phase. 5.2. Smooth Pursuit Phase This phase is verified using the developed experimental setup. The proposed algorithm is based on minimizing the difference between the center of object perimeter (COP) and the center of fovea region (COF). In mathematic expression: 0COFCOP ,,  yxyx …(4) The acceptable error value of equation 4 will be selected to 10 pixels. In the adopted tests, 10 pixels are less than 4 mm in real environment; i. e., if the object moves more than 4 mm, the CPT (c) 30 cm far from object 10.15 11.46 0 0 0 0 0 10.15 -0.77 0 0 0 0 0 -5 0 5 10 15 1 2 3 4 5 6 7 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt 4.83 14.89 3.78 0 0 0 0 0 4.83 14.89 -9.47 0 0 0 0 0 -15 -10 -5 0 5 10 15 20 1 2 3 4 5 6 7 8 Fr am e Num ber A n g le V a lu e ( D e g .) P an Tilt (a) 50 cm far from object 11.78 7.74 -4.36 2.23 0 0 0 0 0 11.78 -7.74 6.92 -2.23 0 0 0 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 7 8 9 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (b) 40 cm far from object -4.35 0.73 0.56 0.3 0 0 10.25 0.53 -0.09 -0.06 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 Frame N umber A n g le V a lu e ( D e g .) Pan Tilt (c) 30 cm far from object -7.67 1.34 0.47 0 0 0 0 9.76 0.78 -0.34 0 0 0 0 -10 -5 0 5 10 15 1 2 3 4 5 6 7 Frame Number A n g le V a lu e ( D e g .) Pan Tilt (b) 40 cm far from object -7.42 2.45 0.01 0.02 0.01 0.02 0.01 9.91 0.12 0.01 0.02 0.01 0.02 0.01 -10 -5 0 5 10 15 1 2 3 4 5 6 7 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (a) 50 cm far from object -9.55 -7.62 -6.81 6.87 -4.74 3.14 0 0 0 9.55 3.45 -7.05 6.87 -5.87 3.14 0 0 0 -15 -10 -5 0 5 10 15 1 2 3 4 5 6 7 8 9 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (b) 40 cm far from object -12.14 -14.34 16.4 -25.17 -9.86 -30 12.14 -12.38 16.4 -25.17 -9.86 -30 -40 -30 -20 -10 0 10 20 1 2 3 4 5 6 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (c) 30 cm far from object -10.25 -8.74 0 0 0 0 10.25 0.46 0 0 0 0 -15 -10 -5 0 5 10 15 1 2 3 4 5 6 Fr am e Num ber A n g le V a lu e ( D e g .) Pan Tilt (a) 50 cm far from object This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 42 configuration needs to change its pan and tilt angles in order to keep equation (4) within 10 pixels error. The same circular object that is used in saccadic phase is adopted in this phase. The directions of the pan and tilt angles are selected depending on figure (6). The absolute values of the pan and tilt angles are calculated based on calibration process between visual domain and real world domain. The same locations of the CPT configuration used in saccadic phase are used in this phase. To verify the robustness of the developed algorithm, the CPT configuration will be moved manually in two directions. The first direction is from left to right and from right to left, while the second one is from down to upward and from up to downward. The total distance that the CPT configuration will move is about 30 cm for each case. The COFx and COFy are 120 and 96 pixels respectively. Left – Right case: Figure (17) shows the relation between pan and tilt angles with frame number for different locations of the CPT configuration. In addition, it presents the relation between the object COP and frame number. It can be seen from figure (17-a), the initial values of the pan and tilt are small and can be neglected because the object is laid in the fovea region and the difference value between the COP of the object and the COF is less than 10 pixels. However, in figures (17-b and c), the location of the object needs an adjustment because the difference value is larger then 10 pixels. The CPT configuration is moved to the right and the proposed algorithm will respond by changing the values of pan angle to prevent the object from leaving the fovea region, where the net variation of the pan angle is positive. After that, the CPT configuration moves backward to the initial location, where the average change in the pan values is negative. The observed object is always in fovea region during this verification, where the influence of location of the CPT configuration is limited on the approach as shown in figure (17). However, ripples in the angles values can be recognized, where the manual movement of the CPT configuration causes these ripples. Up – Down case: In this case, the CPT configuration will be moved in an upward direction until it reaches 15 cm from the initial position and then back to its initial location. Figure (18) shows the verification results of this case, where tilt values are changed in the same behavior that pan angle was changed in the previous case. From figure (18), it can be noticed that the pan values are changed while the ideal situation assumed no change. This is because of the manual movement of the CPT configuration that makes the moving track not straight, where the pan angle is needed to correct such movements. 6. Conclusions In this paper, an approach for object tracking was proposed and verified experimentally. The main conclusions of this research are: 1. Both tracking phases showed beneficial results and robust response to the object enlargement and digital noise. 2. The use of FDM in the saccadic phase is beneficially and it accelerates the response of the approach to meet its goal in few frames. However, the adjustment of the output membership functions may reduce the effect of the object enlargement. 3. The smooth pursuit phase showed efficient smooth tracking of the observing object, however, calibration process between the real and visual domains is necessary. In general, the proposed approach can be used with confidence in robot applications. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 43 Fig.17. Results of Right-Left case-Pursuit Phase. 0 50 100 150 200 1 34 67 100 133 166 199 232 Frame Number P ix e l COPx COPy P ix el Frame Number -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 1 30 59 88 117 146 175 204 233 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al u e (D eg .) Frame Number 0 50 100 150 200 1 59 117 175 233 291 349 407 465 Frame Number P ix e l COPx COPy Frame Number P ix el -3 -2 -1 0 1 2 3 1 56 111 166 221 276 331 386 441 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al ue ( D eg .) Frame Number 0 50 100 150 200 1 29 57 85 113 141 169 197 225 Frame Number P ix e l COPx COPy Frame Number P ix el -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 1 28 55 82 109 136 163 190 217 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al ue ( D eg .) Frame Number (b) 40 cm far from object (c) 30 cm far from object (a) 50 cm far from object This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 44 Fig.18. Results of Up-Down case-Pursuit Phase. -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 1 33 65 97 129 161 193 225 257 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al ue ( D eg .) Frame Number 0 20 40 60 80 100 120 140 160 1 35 69 103 137 171 205 239 Frame Number P ix e l COPx COPy Frame Number P ix el (a) 50 cm far from object -1.5 -1 -0.5 0 0.5 1 1.5 2 1 21 41 61 81 101 121 141 161 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al u e (D eg .) Frame Number 0 20 40 60 80 100 120 140 160 1 23 45 67 89 111 133 155 177 Frame Number P ix e l COPx COPy Frame Number P ix el (c) 30 cm far from object -2 -1.5 -1 -0.5 0 0.5 1 1.5 1 32 63 94 125 156 187 218 249 Frame Number A n g le V a lu e ( D e g .) Pan Tilt A n g le v al ue ( D eg .) Frame Number 0 20 40 60 80 100 120 140 160 180 1 32 63 94 125 156 187 218 249 Frame Number P ix e l COPx COPy Frame Number P ix el (b) 40 cm far from object This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 45 7. References [1] Won-Cheol Kim, Jung-Hwan Kim, Minho Lee, Jang-Kyoo Shin, Hyun-Seung Yang, and Hiroo Yonezu, “Smooth Pursuit Eye Movement System Using Artificial Retina Chip and Shape Memory Alloy Actuator,” IEEE SENSORS J., vol. 5, no. 3,pp. 901-509, June 2005. [2] Emanuele Trucco and Konstantinos Plakas, “Video Tracking: A Concise Survey,” IEEE J. OF OCEANIC ENGINEERING, vol. 31, no. 2, pp. 520-528, APRIL 2006 [3] Kwang-Hee Lee, Do-Eun Kim, Sang Hwa Lee, Young-Ho Lee, and Tae-Yong Kuc, “Visual Servoing of 8-DOF Arm for Mobile Robot Platform,” Int. Conf. on Control, Automation and Systems, in COEX, Seoul, Korea, Oct. 14-17, 2008, pp. 2176-2180. [4] Lin Rui, Du Zhijiang, He Fujun, Kong Minxiu, and Sun Lining, “Tracking a Moving Object with Mobile Robot Based on Vision,” IEEE/WCCI, Int. Conf. on Neural Networks, 1-8 June 2008, pp. 716 – 720. [5] J. Takahashi, M.D. Rokunuzzaman, K. Sekiyma, and T. Fukuda, “Mobile Robotic Sensor Network Using Vision Tracking,” IEEE/SICE Int. Sympo. on System Integration, 4-4 Dec. 2008, pp. 117 – 122. [6] Youngrock Yoon, Akio Kosaka, and Avinash C. Kak, “A New Kalman-Filter-Based Framework for Fast and Accurate Visual Tracking of Rigid Objects,” IEEE TRANS. ON ROBOTICS, vol. 24, no. 5, pp.1238- 1251, Oct. 2008. [7] Gabriel Recatal´a, Raffaella Carloni, Claudio Melchiorri, Pedro J. Sanz, Enric Cervera, and A´ ngel P. del Pobil, ”Vision-Based Grasp Tracking for Planar Objects,” IEEE TRANS. ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS”, vol. 38, no. 6, pp. 844-849, Nov. 2008. [8] Shijie Dai, Weiguo Ren, Fenglong Gu, He Huang, and Shuying Chang, “Implementation of Robot Visual Tracking System Based on Rough Set Theory” IEEE/FSKD - Fifth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 2, 18-20 Oct. 2008, pp. 155 – 160. [9] Diansheng Chen, Feng Bai, Yi Xie, and Tianmiao Wang, “Tracking System of Motion Object for Scout Robot,” IEEE/ROBIO Int. Conf. on Robotics and Biomimetics, 22-25 Feb. 2009, pp.1111 – 1116. [10] Francisco J. Montecillo-Puente, Víctor Ayala-Ramírez, Arturo Pérez-García and Raú 1 E. Sánchez-Yáňez, “Fuzzy color tracking for robotic tasks,” IEEE Int. Conf. on Systems, Man and Cybernetics, vol. 3, 5- 8 Oct. 2003, pp. 2769 – 2773. [11] Suk-Bum Kang and Tae-Kyu Yang, “Tracking for Moving Object Using Adaptive Rao-Blackwellized Particle Filter and Fuzzy Control System,” IEEE/ ICCAS Int. Conf. on Control, Automation and Systems, 17-20 Oct. 2007, pp. 420 – 424. [12] Manuel J. Ferreira, Cristina P. Santos, and Joao Monteiro, “Texture Cue Based Tracking System Using Wavelet Transform and a Fuzzy Grammar,” IEEE Int. Conf. on Industrial Informatics, vol. 1, 23-27 June 2007 pp. 393 – 398. [13] Kazuyuki Morioka, Szilveszter Kovacs, Joo-Ho Lee, Peter Korondi, and Hideki Hashimoto, “Fuzzy-Based Camera Selection for Object Tracking in a Multi- Camera System,” IEEE Int. Conf. on Human System Interactions, 25-27 May 2008, pp.767 – 772. [14] Huiyu Zhou, Murtaza Taj, and Andrea Cavallaro, “Target Detection and Tracking with Heterogeneous Sensors,” IEEE J. OF Selected Topics in Signal Processing, vol. 2, no. 4, pp. 503-513. Aug. 2008. [15] Hairong Zou, Zhenbang Gong, Shaorong Xie, and Wei Ding, “A Pan-tilt Camera Control System ofUAV Visual Tracking Based on Biomimetic Eye,” IEEE Int. Conf. on Robotics and Biomimetics, 17 – 20 Dec. 2006, pp.1477-1482. [16] Fortunato Flores Ando and Alfredo Weitzenfeld Ridel,“Visual Input Compensation using the Crowley-Arbib Saccade Model,” IEEE Int. Conf. on Advanced Robotics, 8-20 July 2005, pp. 476 – 483. [17] Ofir Avni, Francesco Borrelli, Gadi Katzir, Ehud Rivlin, and Hector Rotstein, “Scanning the Environment with Two Independent Cameras - Biologically Motivated Approach,” IEEE/ RSJ Int. Conf. on Intelligent Robots and Systems, 9 - 15 Oct. 2006, pp. 5297-5302. [18] Xiaolin Zhang, “An Object Tracking System Based on Human Neural Pathways of Binocular Motor System,” IEEE Int. Conf. on Control, Automation, Robotics and Vision, 5-8 Dec. 2006, pp. 1 – 8. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Hussam K. Abdul-Ameer Al-Khwarizmi Engineering Journal, Vol.6, No.1, PP 31 -47 (2010) 46 [19] Hairong Zou, “Modeling and Simulation of the Oculomotor Control System,” IEEE Int. Conf. on Bioinformatics and Biomedical Engineering, 16-18 May 2008 pp. 853 – 856. [20] S. I. Fox, Human Physiology.9th edition, McGrawHill, 2006. [21] A. Vander, J. Sherman, and D. Luciano, Human Physiology. 8th edition, McGrawHill, 2001. [22] Joseoh D. Bronzino, The Biomedical Engineering Handbook. 2nd edition, CRC PRESS, 2000. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ )2010( 47 - 31فحة ، ص1، العدد 6مجلة الخوارزمي الھندسیة المجلد حسام كاظم عبد األمیر 47 لتتبع األجسام لالستخدام في) Bionics(طریقة مقترحة باعتماد تطبیقات اإلنسان اآللي حسام كاظم عبد األمیر جامعة بغداد/ كلیة الھندسة الخوارزمي /لطب الحیاتياقسم ھندسة ةصالخال ا زئیین , تناول البحث اقتراح طریقة لتتبع األجسام مستوحاة من نظام الرؤیا البشري و التحقق منھا عملی ى ج ة إل ة المقترح مت الطریق ث قس و , حی األول ھ ).Smooth pursuit(و الثاني المالحقة الھادئة ) Saccadic phase(التتبع السریع زئیین اطق ) Saccadic region(األول , قسم مستوى الصورة و الذي یماثل شبكیة العین البشریة إلى ج ذه المن اطق وھ ھ من ى أربع م إل دوره قس ذي ب و ال ة . حیث یتم فیھ الجزء الثاني من عملیة التتبع) Fovea region(في شبكیة العین و الثاني ) Periphery(تقابل منطقة م منطق دخل الجس دما ی ) Saccadic(عن بب ) Tilt(و) Pan(من خالل تغییر قیم زوایا ) Fovea(یبدأ النظام المعد بتتبع الجسم و إدخالھ إلى منطقة ق المض تخدام المنط الل اس ن خ ر م و یتم ھذا التغیی ن ) Fovea(المقترحة ینشط عند دخول الجسم منطقة الجزء الثاني من الطریقة . كأسلوب اتخاذ قرار ذكي حیث اعد مجموعة شروط منطقیة لذلك ھ م و خروج م و ) Tilt(و) Pan(حیث یتم الحفاظ على الجسم داخل تلك المنطقة من خالل تغییر قیم زوایا ) Saccadic(منطقة یط الجس ز مح وفق طریقة الفروق بین مرك ة ز منطق ام ). Fovea(مرك ى نظ ا عل ة عملی ة المقترح ت الطریق ا ) Camera-Pan/Tilt(طبق ة الرؤی ین قابلی دة لتحس ائج جی رت نت رض و أظھ ذا الغ د لھ اع . ة لإلنسان اآللي المماثل لإلنساناالصطناعی This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/