 Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Point-Structured Human Body Modeling Based on 3D Scan Data Ming-June Tsai 1,* , Hsueh-Yung Lung 1 1 Dep artment of M echanical Engineering, National Cheng Kung University , Tainan, Taiwan. Received 05 June 2017; received in revised form 13 Sept ember 2017; accept ed 20 Sept ember 2017 Abstract A novel p oint-structured geometrical modellin g for realistic human body is introduced in this p ap er. This technique is based on the feature extraction from the 3D body scan data. Anatomic feature such as the neck, the arm p its, the crotch p oints, and other major feature p oints are recogn ized. The body data is then segmented into 6 major p arts. A body model is then constructed by re-samp ling the scann ed data to create a p oint-structured mesh. The body model contains body geodetic landmarks in latitudinal and lon gitudinal curves p assing throu gh those feature p oints. The body model p reserves the p erfect body shap e and all the body dimensions but requires little sp ace. Therefore, the body model can be used as a mannequ in in garment industry , or as a manikin in various human factor designs, but the most imp ortant app lication is to use as a virtue character to animate the body motion in mocap (motion cap ture) sy stems. By adding suitable joint freedo ms between the segmented body links, kinematic and dy namic p rop erties of the motion theories can be ap p lied to the body model. As a result, a 3D virtual character that is fully resembled the original scann ed individu al is vividly animatin g the body motions. The gap s between the body segments due to motion can be filled up by skin blending technique usin g the characteristic of the p oint -structured model. The model has the p otential to serve as a standardized dataty p e to archive body information for all custom-made p roducts. Keywor ds: 3D body model, body motion animation, feature recognition 1. Introduction Hu man body modeling is the act that creates a proper shape description for a specific human body. The shape description is a geo metrical representation in the 3D co mputer environ ment and, usually, co mb ined with skin colo ring and te xture mapping to perform more rea listic results. A convenient way to obtain a rea listic representation of human body in the virtual world is using 3D dig itizing technique [1]. Recently, a number of 3D scanning systems with the ability to quickly and easily obtain digital human body model are now co mme rcia lly availab le [2-5]. Ho wever, the scanned point cloud is commonly scattered and unorganized, and only very little semantic information involved. Processing the raw data, therefore , is a cruc ial task in the e xtendable utilit ies of the human body scans. A number of data processing for human body scans, such as the data segmentation for significant parts, the indication of landmarks and feature points, and the dimension measurements, have now been proposed [6-9]. It is obvious that the key point to turn the raw scanned data into useful informat ion should be by means of feature recognition, data extraction, and symbolization. The feature e xt raction plays an important ro le not only in the description of hu man body, but also in the anthropometric analysis of human factor designs. Recently, a novel method of body feature e xtract ion fro m a ma rke r -less scanned body was presented [10-11], in which the description of human body features are interpreted into logica l mathemat ical defin itions. Tsai and Fang [12] patented a feature based data structure for co mputer manikin. Accordingly, the body scanned d ata are segmented into 6 six parts: head, torso, two arms and two legs. Each part can be encoded into a range image format, and then the featur e points and curves are recognized according to the gradient of the gray scale in the range image format. A point -structured * Corresponding author. E-mail address: mjtsai@mail.ncku.edu.tw Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 2 geometric model is constructed by re-sampling the original 3D scanned data into an interweaving geodetic latitudinal and longitudinal curves, and then achieving an accurate model to describe the human body . There is also an increasing demand for realistic human animation models in various applications such as interactive computer ga mes, virtual reality, and movie a musement industry. Although an artificia l mode l is able to serve as useful animation tool to control articu lated mot ions and body surface deformat ions according to different body postures, it is still required a large degree of skill and manual intervention for more rea listic. However, it rese mbles to nobody. Currently, a lo t of studies on human motion capture and analysis have been presented [13-16]. So far, there is no direct way to use the personalized body model to animate his own body motion. It is persuasive that using the body model created fro m the scanned human body data to animate his own personalized body motions will yie ld a more rea listic result without laborious motion retargeting intervention because it is a d irect and an e ffic ient way to e xhib it prec ious body motions. By appropriate ly addin g joint freedo ms between the body lin ks to cons truct a kine matic model, the point-structured geometric mode ls obtained by scanned data can naturally and exactly be animated. In this paper, we have also shown that the skin gaps between the body segments due to motion can be filled up by dual quaternion linear blending using the characteristic of the point -structured model. 2. Point-Structured Geometric Model The surface informat ion of hu man body can be obtained quickly and easily by using 3D scanning technology. However, the scanned data points without further processing are un-organized and are too huge to be directly used in practice. Therefore, a proper way of the data processing with the ability to reduce the a mount of c loud points while e xt racting the significant characteristics of human body will b e e xtre me ly beneficia l for further applicat ions. Such a process can be achieved by employing the computational geometry and image processing techniques on the scanned point cloud. According to the geometric characteristics of human body, it is general to divide the body into six topological parts: head, torso, two arms and two legs. The arms can be seg mented fro m the body by the armp it, and the legs can be segmented by the crotch. In the following, we only illustrate the method of point -structured modeling for the torso. 2.1. Feature recognition The outside contour of a human body is a very comple x smooth surface, and it is difficult to be described and represented clearly without the help of some body significant features. By the way, the dimensional measure ments of the body used in garment design and anthropometric surveys are a lways dependent on the feature points and feature lines. However, it seems that there is no unanimity on the definit ions of these specific body features in the literatures [17-18]. In order to e xt ract these feature lines and feature points fro m the 3D scanned point cloud automatica lly, the se mantic definit ions of body features are needed to be interpreted into the mathemat ical definit ions. A number of methods based on the image processing techniques, computational geometry and computer graphics have been used to identify these features from the 3D scanned points. And the developed algorithms fo r searching these features automatically have been presented in [10-12]. The body feature lines searching result by [11] is shown in Fig. 1(a). A point-structured model of the torso is developed using the concept of geodetic coordinate, which is similar to the longitudinal and latitudinal lines of the Earth. It means that the longitudinal cu rves include all the feature curves in the vertical direction of the torso, whereas the latitudinal curves contain all the feature girth lines of the body. Therefore, all the significant features can be preserved we ll in such a point -structured representation. The girth lines in horizontal a re important in the description of body curve. Consequently, a total of 60 sections are arranged in the representation as shown in Fig. 1(b). Exc ept Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 3 for the a lready e xt racted feature g irths wh ich have been assigned as sp ecific orders, the others between each interval are needed to be re-sampled by the method of interpolation fro m the origina l 3D scanned point cloud. The interpolation is conducted with a uniform distribution within each interval. Front centerline Upper neck line Lower neck line Shoulder line Shoulder girth Armpit girth Bust girth Under bust girth Waist girth Mid-waist girth Hip girth Crotch girth Side line Side line Armhole Armhole Princess lines Princess lines Shoulder line Back centerline 6 0 G ir th s (a) Body feature lines searching result by [11] (b) Arrangement of structured girth lines Fig. 1 Body feature lines obtained from the 3D body scanned point cloud and the re -sampled girth lines 2.2. Structured points In general, there a re a total of 80 structured points (0~79) used to characterize the circu mference of each latitudinal g irth line. So me specific structured points are needed to correspond to some feature points, respective. It means that they are assigned to locate on the longitudinal feature lines. For e xa mp le, the structured points along the armpit girth a re illustrat ed in Fig. 2. Therefo re, each structure point has a body geodesic coordinates (BGC) (g, h). Where g designates the number of the girth that the point is located; and h is the order of the point on the girth, which also denotes the number of the longitudinal curve. For e xa mp le, the two po ints with numbers (38, 10) and (38, 70) represent the left and right bust points, which are very important land marks of the human body. The BGC are standardized and norma lized in our body model regardless of the gender, age, shape and race of the hu man. Similarly, the structured points between two adjacent feature points are a lso generated by interpolation with a uniform distribution. It is clear that the point densities of these intervals are different. The reason is for the curve section with a larger curvature to have more points used for proper representation. Fig. 2 Structured points of the armpit girth line [12] 2.3. Processing result The process to obtain the point-structured model of hu man body is very sophisticated. For the torso, a lot o f significant features with special geometric propert ies are needed to be recognized. Based on these features, the concept of longitude and latitude are applied to construct the point-structured torso model. As a result, a feature-based BGC data structure is included in the point-structured model. Such a concise representation, as shown in Fig. 3, contains all the body features, anthropometric data, and body shape in the torso, it just like a body atlas that can be readily extracted as needed . Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 4 Fig. 3 Point-structured torso model and its polygonal mesh In contrast to the torso, to obtain the point -structured models of the t wo arms and two legs are simple r. The a rm can be divided into upper arm and lowe r arm by the elbow girth. Due to the cylindrica l shape of arm, its point -structured model can be constructed by using several girth lines distributed along the cylindrica l a xis. Like wise, the point -structured model of the leg can be conducted by the same way. A mo re detailed statement of the model construction is shown in [19]. Fina lly, a fu ll 3D point-structured body model consisting of only 10,214 structured points is obtained and is shown in Fig. 4. The body model is also called the body geometric model (BGM). (a) Point cloud (b) Point-structured model (c) Meshed by triangulation Fig. 4 A 3D BGM 3. Animation Model It is demonstrated in this section how to converse the BGM into a kine matic model that can be used for animation purpose. Before the 3D body scanned points can be used for animation, the scanned points should be put in well-organized order, this is our BDM. However, the BGM still cannot be animated because it has no joint freedo m. It means that the segmentation of the body scanned data should be according to the anatomical structure for jo int arrange ment. By the way , a joint model with high degrees of freedom is required for perfo rming much more hu man -like motion. With the kine mat ic analysis based on the joint model, it is not difficult to converse the BGM into an articulated kine mat ic model (BKM ). Fortunately, the BGM has been segmented according to the anatomic feature at some joint positions. We just need to add appropriate joint freedoms between the adjacent body links. A realistic human kinematic body can be built for motion animation . 3.1. Segmentation A virtual hu man model to perform much more hu man -like mot ion is generally dependent on how many degrees of freedom (DOFs) it has. In order to achieve highly realistic human animation, the whole BGM is further divided into 23 parts as Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 5 shown in Fig. 5. These parts will be vie wed as the body links, and a suitable nu mber of joint freedoms are assigned to each of adjacent links. Then, the joint-link kine matic model (BKM) is constructed for hu man mot ion animat ion. The jo int -link hierarchica l structure is modeled by five kine matic chains. The first chain consists of hip, waist, chest, neck, and head. The hip is the base fra me in this hie rarchica l structure. Cha ins 2 and 3 are co mposed of the scapula, upper arm, lo wer arm, hand and fingers on the left and right, res pectively. The left and right thighs, lower legs, feet and toes are the links of chain 4 and 5, respectively. Please note all of these links co me fro m the point -structured BGM which is constructed by the 3D scanned data of a specific person. So fa r, we have finished a compact static BGM with personal shape and appearance. For hu man animat ion, the only require ment is to use the BKM of the specific person and then animate these segmented lin k mode ls by applying the motion data. Left toe Left foot Left lower leg Left thigh Left fingers Left hand Left lower arm Left upper arm Left scapula Head Neck Right toe Right foot Right lower leg Right thigh Right fingers Right hand Right lower arm Right upper arm Right scapula Chest Waist Hip Fig. 5 Segmentation for articulated human model 3.2. Kinematic model According to the anatomical structure of human body, the Type -1 BKM in this study is conducted by 5 kine matic chains with a total of 48 degrees of freedo m, not counting the 6 DOF in th e pe lvis. The pelvis is considered as the based lin k of the body, which has 6 DOF with respect to the fix coordinate system, and all the other joint freedo ms are co mputed based on this lin k. Ho wever, human body has many more jo int freedo ms that performs ve ry co mple x body motion. It is fa milia r that more DOFs will lead to a more realistic motion. Ho wever, too many DOFs also involve a rise in comple xity and computational cost. It is a trade-off that which kind of kine matic mode l is good enough for what kind o f body motion. In our study, it is believed that 48 joint freedoms are suitable for animating most of the body motions . (a) Segmented body model (b) BKM with 48 DOFs (c) BKM with 31 DOF Fig. 6 Joint arrangement of the articulated models [20] Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 6 As shown in Fig. 6, there are 20 jo ints arranged in the different locations on the BKM. The four jo ints at L_Grasp, R_ Grasp, L_Toe and R_Toe each has one DOF only. There are 8 jo ints with two DOFs, i.e . L_ Elbow, L_Wrist, L_Knee, L_Ankle , R_ Elbow, R_Wrist, R_Knee and R_Ankle. The three-DOF joints are L_ Shoulder, L_ Leg, R_Shoulder and R_Leg. All of these joints are regarded as revolute jo ints. Subsequently, the four joints (Nec k, Waist, L_Scapula and R_Scapula) are modeled as four-DOF joints composed of three revolute freedoms and one pris matic freedo m sliding a long the last rotational a xis. Consequently, the model possesses totally 48 DOFs in the 20 jo ints. Other kine matic mode ls with different number of joint freedoms can be created, e.g. the Type-2 BKM shown in Fig. 6(c ), which has 31 DOF to simulate a conventional humanoid robot. It is difficu lt to locate the jo int a xes at e xact position and orientation since the real hu man joints are not simp ly the rev olute or pris matic. If the hu man move ments can be captured by a precision motion trac ker, it is possible to locate the joint a xes accurately. However, using a good BKM with enough joint freedoms, people can replicate the mot ion without proble m. That is why the realistic human animation can be carried out by using the motion capture system. There fore, the motion data recorded fro m a mot ion capture system p lays an important role to produce highly realistic hu man animations. Based on the BGM and BKM, Tsai and Lung [20] e mp loys a self-made mot ion capture system to acquire the space information of a ll lin ks, and then use the method of two-phased optimization to solve the joint angles via inverse kine matics. Besides, an intelligent Body Motion Processing System (iBMPS) has also been developed by Tsai and Lung [21 ]. The result shows that highly realistic human animat ions can be achieved using the iBMPS. While apply ing the original motion data to BGM and the joint angles to the BKM, the compa rison for four postures is shown in Fig. 7, in wh ich all of these body segments are displayed without any blending function for the joint deformations . Fig. 7 Motion captured model compared with the model after optimization [21] 3.3. Sk in blending Because the segmented body parts are viewed as the lin ks wh ich are usually treated as rig id bodies, it is in ev idence the relative move ment of two ad jacent lin ks will cause some kinds of splits or penetrations on the joint position, which are in-appropriate for looking in an animation system. In order to have a more authentic appearance, blending function is emp loyed on the structure points to overcome this proble m. Fig . 8 illustrated the effect of the blending function. It is obvious that the gaps at the joints (between the adjacent links) have been filled s moothly. The skin b lending is fu lfilled by e mp loying ScLERP (quaternion Spherica l Linear interpolation) by using co nstant volume of the links as the constraint before and after blending. As a result, it yie lds a highly realistic representation with the blending. Since the BGM contains the information of skin location, we will know how to move the structure point by studying the skin deformat ion during a body movement. This is another benefit of using the point-structured BGM. Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 7 Fig. 8 The effectiveness of blending function 4. Conclusions Hu man body modeling is now a very hot research topic. To produce highly rea listic 3D hu man an imation is very useful for many applications. Reality is always dependent on both of the realistic appearance and move ments. In this paper, we use the point-structured modeling method to ma intain the real boy shape and reduce the amount of data points considerably. We also construct appropriate BKM to achieve the highly authentic body motion replication. It de monstrated that such a point structured human body model is very useful in the dig ital hu man body modeling and realistic body motion animat ion. The body model can also be used as a mannequin in ga rment industry, or as a manikin in v arious human factor design since it contains all the body dimensions, shape, as well as all body features . Acknowledgements This research was supported by the project of National Sc ience Council, (project number: NSC 99-2221-E-006-018-M Y3), and Ministry of Sc ience and Technology of Taiwan (pro ject number: MOST 103-2221-E-006-024) which are greatly appreciative. References [1] M. Petrov, A. Talapov, T. Robertson, A. Lebedev, A. Zhilyaev, and L. Polonskiy, “ Optical 3D dig itizers: Bring life to the virtual world,” IEEE Computer Graphics and App lication, vol. 18, no. 3, pp. 28-37, May 1998. [2] Cyberware, http://www.cyberware.com, 2012. [3] TC2, http://www.tc2.com, 2012. [4] Vitus, http://www.vitronic.de, 2012. [5] Creaform, http://www.creaform3d.com, 2012. [6] J. H. Nurre , J. Connor, E. A. Le wark, and J. S. Collie r, “On segmenting the three -d imensional scan data of a hu man body,” IEEE Transactions on Medical Imaging , vol. 19, no. 8, pp. 787-797, August 2000. [7] X. Ju, N. Werghi, and J. P. Siebert, “Automatic segmentation of 3D human body scans,” Proc. of IASTED Int. Conf. on Comp. Graphics and Imaging, 2000. [8] K. Robinette, M. Boeh mer, and D. Burnsides, “3-D landma rk detection and identification in the CA ESA R pro ject,” Proc. 3rd Int. Conf. on 3-D Digital Imaging and Modeling, pp. 292-298, 2001. [9] R. P. Pargas, N. J. Staples, and J. S. Davis, “Automatic measurement e xtract ion for apparel fro m a three -dimensional body scan,” Optics and Lasers in Engineering, vol. 28, no. 2, pp. 157-172, September 1997. [10] M. J. Tsai, Z. P. Chen, and Y. S. Liu, “A study on the automatic feature search fro m 3D body scanners,” Proc. of the 20th National Conference, Society of Chinese Mechanical Engineering, Taipei, Dec ember 2003. Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 8 [11] I. F. Leong, J. J. Fang, and M. J. Tsai, “Automatic body feature extract ion from a marker-less scanned human body,” Computer-Aided Design, vol. 39, no. 7, pp. 568-582, July 2007. [12] M. J. Tsai and J. J. Fang, Feature based data structure for computer manikin, US Patent, 7,218,752 B2, May 15 2007. [13] L. A. Wang, W. M. Hu, and T. N. Tan, “Recent development in human motion analysis,” Pattern Recognition, vol. 36, no. 3, pp. 585-601, March 2003. [14] J. K. Aggarwal and Q. Cai, “Hu man motion analysis: A revie w,” Co mputer Vision and Image Understanding, vol. 73 , no. 3, pp. 428-440, March 1999. [15] T. B. Moeslund and E. Granu m, “A survey of co mputer vision -based human motion capture,” Co mputer Vision and Image Understanding, vol. 81, no. 3, pp. 231-268, March 2001. [16] R. Poppe, “ Vision-based human mot ion analysis: An overvie w,” Co mputer Vision and Image Understanding, vol. 108, no. 1-2, pp. 4-18, October-November 2007. [17] International Organization for Standardization, Garment construction and anthropometric surveys – body dimensions, Reference No. 8559-1989, Switzerland: ISO, 1989. [18] ASTM, Standard termino logy relat ing to body dimensions for apparel siz ing, ASTM Designation: D 5219-99, USA, 1999. [19] M. J. Tsai, H. W. Lee, and H. Y. Lung, “Feature-based data structure for digital manikin,” U.S. Patent , US 20130069936 A1, March 21, 2013. [20] M. J. Tsai and H. Y. Lung, “Two-phase optimized inverse kinematics for motion replication of real human mod els,” Journal of the Chinese Institute of Engineers, vol. 37, no. 7, pp. 899-914, April 2014. [21] M. J. Tsai, J. H. Chao, and T. W. Yang, “ Construction of a general motion editing system for hu man body and humanoid robots,” Proc. ASME 2014 Int. Design Engineering Technical Conf. & Computers and Information in Engineering (IDETC/CIE 2014), ASME Press, August, 2014, pp. V01AT02A072.