 Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Point-Structured Human Body Modeling Based on 3D Scan Data Ming-June Tsai 1,* , Hsueh-Yung Lung 1 1Department of Mechanical Engineering, National Cheng Kung University , Tainan, Taiwan. Received 05 June 2017; received in revised form 13 September 2017; accepted 20 September 2017 Abstract A novel point-structured geometrical modelling for realistic human body is introduced in this paper. This technique is based on the feature extraction from the 3D body scan data. Anatomic feature such as the neck, the arm pits, the crotch points, and other major feature points are recognized. The body data is then segmented into 6 major parts. A body model is then constructed by re-sampling the scanned data to create a point-structured mesh. The body model contains body geodetic landmarks in latitudinal and longitudinal curves passing through those feature points. The body model preserves the perfect body shape and all the body dimensions but requires little space. Therefore, the body model can be used as a mannequin in garment industry, or as a manikin in various human factor designs, but the most important application is to use as a virtue character to animate the body motion in mocap (motion capture) systems. By adding suitable joint freedoms between the segmented body links, kinematic and dynamic properties of the motion theories can be applied to the body model. As a result, a 3D virtual character that is fully resembled the original scanned individual is vividly animating the body motions. The gaps between the body segments due to motion can be filled up by skin blending technique using the characteristic of the point-structured model. The model has the potential to serve as a standardized datatype to archive body information for all custom-made products. Keywords: 3D body model, body motion animation, feature recognition 1. Introduction Human body modeling is the act that creates a proper shape description for a specific human body. The shape description is a geometrical representation in the 3D computer environment and, usually, combined with skin coloring and texture mapping to perform more realistic results. A convenient way to obtain a realistic representation of human body in the virtual world is using 3D dig itizing technique [1]. Recently, a number of 3D scanning systems with the ability to quickly and easily obtain digital human body model are now commercially availab le [2-5]. However, the scanned point cloud is commonly scattered and unorganized, and only very little semantic information involved. Processing the raw data, therefore, is a crucial task in the extendable utilit ies of the human body scans. A number of data processing for human body scans, such as the data segmentation for significant parts, the indication of landmarks and feature points, and the dimension measurements, have now been proposed [6-9]. It is obvious that the key point to turn the raw scanned data into useful informat ion should be by means of feature recognition, data extraction, and symbolization. The feature ext raction plays an important ro le not only in the description of human body, but also in the anthropometric analysis of human factor designs. Recently, a novel method of body feature extract ion from a marker-less scanned body was presented [10-11], in which the description of human body features are interpreted into logical mathemat ical defin itions. Tsai and Fang [12] patented a feature based data structure for computer manikin. Accordingly, the body scanned d ata are segmented into 6 six parts: head, torso, two arms and two legs. Each part can be encoded into a range image format, and then the feature points and curves are recognized according to the gradient of the gray scale in the range image format. A point -structured * Corresponding author. E-mail address: mjtsai@mail.ncku.edu.tw Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 2 geometric model is constructed by re-sampling the original 3D scanned data into an interweaving geodetic latitudinal and longitudinal curves, and then achieving an accurate model to describe the human body . There is also an increasing demand for realistic human animation models in various applications such as interactive computer games, virtual reality, and movie amusement industry. Although an artificial model is able to serve as useful animation tool to control articu lated mot ions and body surface deformat ions according to different body postures, it is still required a large degree of skill and manual intervention for more realistic. However, it resembles to nobody. Currently, a lo t of studies on human motion capture and analysis have been presented [13-16]. So far, there is no direct way to use the personalized body model to animate his own body motion. It is persuasive that using the body model created from the scanned human body data to animate his own personalized body motions will yield a more realistic result without laborious motion retargeting intervention because it is a d irect and an efficient way to exhib it precious body motions. By appropriately addin g joint freedoms between the body links to cons truct a kinematic model, the point-structured geometric models obtained by scanned data can naturally and exactly be animated. In this paper, we have also shown that the skin gaps between the body segments due to motion can be filled up by dual quaternion linear blending using the characteristic of the point-structured model. 2. Point-Structured Geometric Model The surface informat ion of human body can be obtained quickly and easily by using 3D scanning technology. However, the scanned data points without further processing are un-organized and are too huge to be directly used in practice. Therefore, a proper way of the data processing with the ability to reduce the amount of cloud points while ext racting the significant characteristics of human body will be extremely beneficial for further applicat ions. Such a process can be achieved by employing the computational geometry and image processing techniques on the scanned point cloud. According to the geometric characteristics of human body, it is general to divide the body into six topological parts: head, torso, two arms and two legs. The arms can be segmented from the body by the armpit, and the legs can be segmented by the crotch. In the following, we only illustrate the method of point-structured modeling for the torso. 2.1. Feature recognition The outside contour of a human body is a very complex smooth surface, and it is difficult to be described and represented clearly without the help of some body significant features. By the way, the dimensional measurements of the body used in garment design and anthropometric surveys are always dependent on the feature points and feature lines. However, it seems that there is no unanimity on the definit ions of these specific body features in the literatures [17-18]. In order to ext ract these feature lines and feature points from the 3D scanned point cloud automatically, the semantic definit ions of body features are needed to be interpreted into the mathemat ical definit ions. A number of methods based on the image processing techniques, computational geometry and computer graphics have been used to identify these features from the 3D scanned points. And the developed algorithms fo r searching these features automatically have been presented in [10-12]. The body feature lines searching result by [11] is shown in Fig. 1(a). A point-structured model of the torso is developed using the concept of geodetic coordinate, which is similar to the longitudinal and latitudinal lines of the Earth. It means that the longitudinal cu rves include all the feature curves in the vertical direction of the torso, whereas the latitudinal curves contain all the feature girth lines of the body. Therefore, all the significant features can be preserved well in such a point-structured representation. The girth lines in horizontal are important in the description of body curve. Consequently, a total of 60 sections are arranged in the representation as shown in Fig. 1(b). Exc ept Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 3 for the already ext racted feature g irths which have been assigned as specific orders, the others between each interval are needed to be re-sampled by the method of interpolation from the original 3D scanned point cloud. The interpolation is conducted with a uniform distribution within each interval. Front centerline Upper neck line Lower neck line Shoulder line Shoulder girth Armpit girth Bust girth Under bust girth Waist girth Mid-waist girth Hip girth Crotch girth Side line Side line Armhole Armhole Princess lines Princess lines Shoulder line Back centerline 6 0 G ir th s (a) Body feature lines searching result by [11] (b) Arrangement of structured girth lines Fig. 1 Body feature lines obtained from the 3D body scanned point cloud and the re-sampled girth lines 2.2. Structured points In general, there are a total of 80 structured points (0~79) used to characterize the circumference of each latitudinal g irth line. Some specific structured points are needed to correspond to some feature points, respective. It means that they are assigned to locate on the longitudinal feature lines. For example, the structured points along the armpit girth are illustrat ed in Fig. 2. Therefore, each structure point has a body geodesic coordinates (BGC) (g, h). Where g designates the number of the girth that the point is located; and h is the order of the point on the girth, which also denotes the number of the longitudinal curve. For example, the two po ints with numbers (38, 10) and (38, 70) represent the left and right bust points, which are very important landmarks of the human body. The BGC are standardized and normalized in our body model regardless of the gender, age, shape and race of the human. Similarly, the structured points between two adjacent feature points are also generated by interpolation with a uniform distribution. It is clear that the point densities of these intervals are different. The reason is for the curve section with a larger curvature to have more points used for proper representation. Fig. 2 Structured points of the armpit girth line [12] 2.3. Processing result The process to obtain the point-structured model of human body is very sophisticated. For the torso, a lot o f significant features with special geometric propert ies are needed to be recognized. Based on these features, the concept of longitude and latitude are applied to construct the point-structured torso model. As a result, a feature-based BGC data structure is included in the point-structured model. Such a concise representation, as shown in Fig. 3, contains all the body features, anthropometric data, and body shape in the torso, it just like a body atlas that can be readily extracted as needed . Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 4 Fig. 3 Point-structured torso model and its polygonal mesh In contrast to the torso, to obtain the point-structured models of the two arms and two legs are simpler. The arm can be divided into upper arm and lower arm by the elbow girth. Due to the cylindrical shape of arm, its point -structured model can be constructed by using several girth lines distributed along the cylindrical axis. Likewise, the point -structured model of the leg can be conducted by the same way. A more detailed statement of the model construction is shown in [19]. Finally, a fu ll 3D point-structured body model consisting of only 10,214 structured points is obtained and is shown in Fig. 4. The body model is also called the body geometric model (BGM). (a) Point cloud (b) Point-structured model (c) Meshed by triangulation Fig. 4 A 3D BGM 3. Animation Model It is demonstrated in this section how to converse the BGM into a kinematic model that can be used for animation purpose. Before the 3D body scanned points can be used for animation, the scanned points should be put in well-organized order, this is our BDM. However, the BGM still cannot be animated because it has no joint freedom. It means that the segmentation of the body scanned data should be according to the anatomical structure for jo int arrangement. By the way , a joint model with high degrees of freedom is required for performing much more human-like motion. With the kinemat ic analysis based on the joint model, it is not difficult to converse the BGM into an articulated kinemat ic model (BKM). Fortunately, the BGM has been segmented according to the anatomic feature at some joint positions. We just need to add appropriate joint freedoms between the adjacent body links. A realistic human kinematic body can be built for motion animation . 3.1. Segmentation A virtual human model to perform much more human-like mot ion is generally dependent on how many degrees of freedom (DOFs) it has. In order to achieve highly realistic human animation, the whole BGM is further divided into 23 parts as Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 5 shown in Fig. 5. These parts will be viewed as the body links, and a suitable number of joint freedoms are assigned to each of adjacent links. Then, the joint-link kinematic model (BKM) is constructed for human mot ion animat ion. The jo int -link hierarchical structure is modeled by five kinematic chains. The first chain consists of hip, waist, chest, neck, and head. The hip is the base frame in this hierarchical structure. Chains 2 and 3 are composed of the scapula, upper arm, lower arm, hand and fingers on the left and right, respectively. The left and right thighs, lower legs, feet and toes are the links of chain 4 and 5, respectively. Please note all of these links come from the point-structured BGM which is constructed by the 3D scanned data of a specific person. So far, we have finished a compact static BGM with personal shape and appearance. For human animat ion, the only requirement is to use the BKM of the specific person and then animate these segmented link models by applying the motion data. Left toe Left foot Left lower leg Left thigh Left fingers Left hand Left lower arm Left upper arm Left scapula Head Neck Right toe Right foot Right lower leg Right thigh Right fingers Right hand Right lower arm Right upper arm Right scapula Chest Waist Hip Fig. 5 Segmentation for articulated human model 3.2. Kinematic model According to the anatomical structure of human body, the Type-1 BKM in this study is conducted by 5 kinematic chains with a total of 48 degrees of freedom, not counting the 6 DOF in the pelvis. The pelvis is considered as the based link of the body, which has 6 DOF with respect to the fix coordinate system, and all the other joint freedoms are computed based on this link. However, human body has many more jo int freedoms that performs very complex body motion. It is familiar that more DOFs will lead to a more realistic motion. However, too many DOFs also involve a rise in complexity and computational cost. It is a trade-off that which kind of kinematic model is good enough for what kind o f body motion. In our study, it is believed that 48 joint freedoms are suitable for animating most of the body motions . (a) Segmented body model (b) BKM with 48 DOFs (c) BKM with 31 DOF Fig. 6 Joint arrangement of the articulated models [20] Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 6 As shown in Fig. 6, there are 20 jo ints arranged in the different locations on the BKM. The four jo ints at L_Grasp, R_Grasp, L_Toe and R_Toe each has one DOF only. There are 8 jo ints with two DOFs, i.e . L_Elbow, L_Wrist, L_Knee, L_Ankle, R_Elbow, R_Wrist, R_Knee and R_Ankle. The three-DOF joints are L_Shoulder, L_Leg, R_Shoulder and R_Leg. All of these joints are regarded as revolute jo ints. Subsequently, the four joints (Neck, Waist, L_Scapula and R_Scapula) are modeled as four-DOF joints composed of three revolute freedoms and one pris matic freedom sliding along the last rotational axis. Consequently, the model possesses totally 48 DOFs in the 20 jo ints. Other kinematic models with different number of joint freedoms can be created, e.g. the Type-2 BKM shown in Fig. 6(c), which has 31 DOF to simulate a conventional humanoid robot. It is difficu lt to locate the jo int axes at exact position and orientation since the real human joints are not simply the rev olute or pris matic. If the human movements can be captured by a precision motion tracker, it is possible to locate the joint axes accurately. However, using a good BKM with enough joint freedoms, people can replicate the mot ion without problem. That is why the realistic human animation can be carried out by using the motion capture system. Therefore, the motion data recorded from a mot ion capture system p lays an important role to produce highly realistic human animations. Based on the BGM and BKM, Tsai and Lung [20] employs a self-made mot ion capture system to acquire the space information of all links, and then use the method of two-phased optimization to solve the joint angles via inverse kinematics. Besides, an intelligent Body Motion Processing System (iBMPS) has also been developed by Tsai and Lung [21 ]. The result shows that highly realistic human animat ions can be achieved using the iBMPS. While apply ing the original motion data to BGM and the joint angles to the BKM, the comparison for four postures is shown in Fig. 7, in which all of these body segments are displayed without any blending function for the joint deformations . Fig. 7 Motion captured model compared with the model after optimization [21] 3.3. Skin blending Because the segmented body parts are viewed as the links which are usually treated as rig id bodies, it is in ev idence the relative movement of two ad jacent links will cause some kinds of splits or penetrations on the joint position, which are in-appropriate for looking in an animation system. In order to have a more authentic appearance, blending function is employed on the structure points to overcome this problem. Fig . 8 illustrated the effect of the blending function. It is obvious that the gaps at the joints (between the adjacent links) have been filled s moothly. The skin b lending is fu lfilled by employing ScLERP (quaternion Spherical Linear interpolation) by using constant volume of the links as the constraint before and after blending. As a result, it yields a highly realistic representation with the blending. Since the BGM contains the information of skin location, we will know how to move the structure point by studying the skin deformat ion during a body movement. This is another benefit of using the point-structured BGM. Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 7 Fig. 8 The effectiveness of blending function 4. Conclusions Human body modeling is now a very hot research topic. To produce highly realistic 3D human an imation is very useful for many applications. Reality is always dependent on both of the realistic appearance and movements. In this paper, we use the point-structured modeling method to maintain the real boy shape and reduce the amount of data points considerably. We also construct appropriate BKM to achieve the highly authentic body motion replication. It demonstrated that such a point structured human body model is very useful in the dig ital human body modeling and realistic body motion animat ion. The body model can also be used as a mannequin in garment industry, or as a manikin in v arious human factor design since it contains all the body dimensions, shape, as well as all body features . Acknowledgements This research was supported by the project of National Science Council, (project number: NSC 99-2221-E-006-018-MY3), and Ministry of Science and Technology of Taiwan (pro ject number: MOST 103-2221-E-006-024) which are greatly appreciative. References [1] M. Petrov, A. Talapov, T. Robertson, A. Lebedev, A. Zhilyaev, and L. Polonskiy, “Optical 3D dig itizers: Bring life to the virtual world,” IEEE Computer Graphics and Application, vol. 18, no. 3, pp. 28-37, May 1998. [2] Cyberware, http://www.cyberware.com, 2012. [3] TC2, http://www.tc2.com, 2012. [4] Vitus, http://www.vitronic.de, 2012. [5] Creaform, http://www.creaform3d.com, 2012. [6] J. H. Nurre , J. Connor, E. A. Lewark, and J. S. Collier, “On segmenting the three-dimensional scan data of a human body,” IEEE Transactions on Medical Imaging, vol. 19, no. 8, pp. 787-797, August 2000. [7] X. Ju, N. Werghi, and J. P. Siebert, “Automatic segmentation of 3D human body scans,” Proc. of IASTED Int. Conf. on Comp. Graphics and Imaging, 2000. [8] K. Robinette, M. Boehmer, and D. Burnsides, “3-D landmark detection and identification in the CAESAR pro ject,” Proc. 3rd Int. Conf. on 3-D Digital Imaging and Modeling, pp. 292-298, 2001. [9] R. P. Pargas, N. J. Staples, and J. S. Davis, “Automatic measurement extract ion for apparel from a three -dimensional body scan,” Optics and Lasers in Engineering, vol. 28, no. 2, pp. 157-172, September 1997. [10] M. J. Tsai, Z. P. Chen, and Y. S. Liu, “A study on the automatic feature search from 3D body scanners,” Proc. of the 20th National Conference, Society of Chinese Mechanical Engineering, Taipei, December 2003. Advances in Technology Innovation, vol. 3, no. 1, 2018, pp. 01 - 08 Copyright © TAETI 8 [11] I. F. Leong, J. J. Fang, and M. J. Tsai, “Automatic body feature extract ion from a marker-less scanned human body,” Computer-Aided Design, vol. 39, no. 7, pp. 568-582, July 2007. [12] M. J. Tsai and J. J. Fang, Feature based data structure for computer manikin, US Patent, 7,218,752 B2, May 15 2007. [13] L. A. Wang, W. M. Hu, and T. N. Tan, “Recent development in human motion analysis,” Pattern Recognition, vol. 36, no. 3, pp. 585-601, March 2003. [14] J. K. Aggarwal and Q. Cai, “Human motion analysis: A review,” Computer Vision and Image Understanding, vol. 73 , no. 3, pp. 428-440, March 1999. [15] T. B. Moeslund and E. Granum, “A survey of computer vision -based human motion capture,” Computer Vision and Image Understanding, vol. 81, no. 3, pp. 231-268, March 2001. [16] R. Poppe, “Vision-based human mot ion analysis: An overview,” Computer Vision and Image Understanding, vol. 108, no. 1-2, pp. 4-18, October-November 2007. [17] International Organization for Standardization, Garment construction and anthropometric surveys – body dimensions, Reference No. 8559-1989, Switzerland: ISO, 1989. [18] ASTM, Standard termino logy relat ing to body dimensions for apparel sizing, ASTM Designation: D 5219-99, USA, 1999. [19] M. J. Tsai, H. W. Lee, and H. Y. Lung, “Feature-based data structure for digital manikin,” U.S. Patent , US 20130069936 A1, March 21, 2013. [20] M. J. Tsai and H. Y. Lung, “Two-phase optimized inverse kinematics for motion replication of real human models,” Journal of the Chinese Institute of Engineers, vol. 37, no. 7, pp. 899-914, April 2014. [21] M. J. Tsai, J. H. Chao, and T. W. Yang, “Construction of a general motion editing system for human body and humanoid robots,” Proc. ASME 2014 Int. Design Engineering Technical Conf. & Computers and Information in Engineering (IDETC/CIE 2014), ASME Press, August, 2014, pp. V01AT02A072.