9 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 © Global Society of Scientific Research and Researchers http://asrjetsjournal.org/ Research on Application of Cognitive-Driven Human- Computer Interaction Qianwen Fu a *, Jian Lv b a,b Key Laboratory of Advanced Manufacturing Technology, Ministry of Education, Guizhou University, Guiyang 550025, China) a Email: 854907975@qq.com b Email: jlv@gzu.edu.cn Abstract Human-computer interaction is an important research content of intelligent manufacturing human factor engineering. Natural human-computer interaction conforms to the cognition of users' habits and can efficiently process inaccurate information interaction, thus improving user experience and reducing cognitive load. Through the analysis of the information interaction process, user interaction experience cognition and human- computer interaction principles in the human-computer interaction system, a cognitive-driven human-computer interaction information transmission model is established. Investigate the main interaction modes in the current human-computer interaction system, and discuss its application status, technical requirements and problems. This paper discusses the analysis and evaluation methods of interaction modes in human-computer system from three levels of subjective evaluation, physiological measurement and mathematical method evaluation, so as to promote the understanding of inaccurate information to achieve the effect of interaction self-adaptation and guide the design and optimization of human-computer interaction system. According to the development status of human-computer interaction in intelligent environment, the research hotspots, problems and development trends of human-computer interaction are put forward. Keywords: Human-computer interaction mode; user experience; design cognition; analysis and evaluation; information interaction Introduction. ------------------------------------------------------------------------ * Corresponding author American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 10 1. Introduction Human-Computer Interaction (HCI) is a technical discipline that studies the interaction between users and equipment and machines in the working environment. Its goal is to enable machines to help users complete tasks and requirements with comfort, high efficiency and security [1,2,3]. Facing the demand for natural human- computer interaction in the fields of intelligent manufacturing, medical rehabilitation, national protection and safety, The study of interactive mode is interdisciplinary, The aim is to make research progress in cognitive- operational coupling design, multi-modal environment perception and human-computer interaction adaptive cooperation, swarm intelligence and distributed human-computer operating system [4,5], The object of human- computer interaction has also changed from emphasizing three-dimensional sensory modeling design to emphasizing immaterial emotional design of human-computer interaction and user experience, which in turn has changed the diversity of human-computer interaction media and interaction modes. In recent years, with the in- depth research of facial recognition, speech recognition, gesture motion recognition, posture recognition and other technologies in the field of artificial intelligence [6,7], Traditional human-computer interaction can no longer adapt to the efficient transmission of information between human and computer in multi-modal intelligent environment, so scholars begin to seek new interaction methods to adapt to the development of new technologies in unstructured environment. The current background and development trend show the importance of man-machine interaction, There is a lack of research on the form of human-computer interaction in the system. Based on the analysis of the research results and development status of human-computer interaction at home and abroad, This paper discusses the related concepts of man-machine interaction mode, Based on the analysis of the cognitive behavior of users in the process of human-computer interaction, Establishing the human-computer interaction information processing model, understanding the function of each interaction form in the human-computer interaction system, summarizing the user's experience and cognition in the interaction process and analyzing the interaction principles in the application of human-computer interaction mode through the analysis of cognitive load and cognitive emotion-driven interaction. This paper expounds different types of human-computer interaction modes, and analyzes their application conditions and technologies. From the two levels of subjective evaluation and psychological measurement, the analysis and evaluation methods of interaction mode in man-machine system are analyzed to guide the design and optimization of man-machine interaction system. Finally, the problems, research hotspots and development trends of human-computer interaction are put forward. 2. Information Transmission of Human-Computer Interaction Driven by Cognition 2.1. Human-computer interaction information transmission process The process of human-computer interaction information transmission can be summarized into three stages: input, processing and output. In the three stages, what users need to actively participate in is the information input stage. Whether the human-computer interaction mode of information input is easy to learn and use, and whether it conforms to people's cognitive mode, thus determining the naturalness of user interaction experience and ensuring that users have lower cognitive load in the interaction process. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 11 People's perception system can be divided into vision, touch, hearing, smell and taste. Through input equipment, information input channels can be divided into categories of information input equipment [8], and their corresponding relationships are such as vision-camera, touch-pressure sensor, hearing-microphone, smell-smell sensor, taste-taste sensor, etc. The response of the machine system to the user's input is the information output, and the user can judge whether the interaction is effective, accurate and natural according to the system output. The modes of interactive system output information include visual output (such as display screen), auditory output (such as sound) and CPU direct response (such as turning on the light). In the study of interactive system media, Scholars regard the degree of information feedback as a classification dimension, That is, according to the "degree of closeness between input points and output points", Four interactive feedback states are summarized: a, complete feedback: the input device is the output device: b, nearby feedback, the output device is near the input device: c, environmental feedback, the output can change the environment around the user: d, remote feedback, the output is in other positions, such as using remote controller to turn on TV, air conditioner, etc. The human-computer interaction information transmission process model is shown in Figure 1 below. Figure 1: Human-computer interaction information transmission process model 2.2. Information Transfer User Experience (1) Cognitive load Cognitive load theory holds that the capacity of human cognitive resources is limited [9]. The main purpose of studying cognitive load is to control the capacity of cognitive load that affects work performance to the greatest extent in cognitive tasks, so as to optimize users' cognitive state and work conditions, enable users to properly and accurately use cognitive resources, and improve job performance and security of interactive systems. As for the design research of human-computer interaction cognitive load, scholars at home and abroad have considered cognitive mechanism, task environment, information coding and design evaluation. Scholar Cheng Shiwei [10] and others proposed a resource model based on distributed cognition to complete the explicit and characterization of user cognition and reduce the cognitive overload of users in interactive operation scenarios. Lu Lu [11] and other scholars proposed a multi-channel information cognitive processing model integrating touch, hearing and vision based on the multi-channel interaction research related to touch. In NASA-TLX American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 12 situational cognition method, its task load index is a popular technology used to measure subjective psychological load [12]. (2) Cognitive-emotional drive In the process of human-computer interaction, users not only emphasize the final function and ease of use of the machine, but also involve more profound connotations such as emotion and culture. Among them, the interactive form acts on the user's perception and cognitive level of multiple levels of information analysis. These levels will be influenced by many contents such as visual analysis habits, values, psychological cues, etc. when controlling users' decisions [13]. At the macro level, users' needs are analyzed according to Masno's needs level. At the micro level, human behavior is affected by emotional factors. Scholars divide the responses to products derived from emotional factors into the following levels; Instinctive level reaction, behavioral level reaction C, reflective level reaction [14]. At the 2019 China Digital Expo, Google's Shadow Art will bring gesture interaction to the level of reflection on users' emotional factors, combine technology with traditional Chinese culture, and correspond to the shadow play image responded in the gesture library through gesture movements. Then the image will enter the digital screen and begin to perform. Figure 2: Google Shadow Art Gesture Interaction Process (3) Task Scenario Drive Task context drive is an important impression factor in human-computer interaction information transmission. Good and continuous situational awareness drive helps users to make rapid and accurate situational assessment, thus enabling efficient decision-making. There are three levels of situational awareness drive: perception, comprehension and prediction. Situational perception is directly related to working memory, and people are more sensitive to emotional changes. 2.3. User-centered Human-Computer Interaction Principle At the level of human-computer interaction principles, the following four interaction principles are summarized in combination with scholars' research [15-21]. (1) Principle of capability level The general principle of taking the user's background as the design basis conforms to the user's ability level. That is, the perception ability, understanding ability, operation ability and working environment of the user American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 13 group in the process of man-machine operation; Task Requirements, etc. (2) Feedback principle This principle is manifested in the effectiveness of feedback information, so that users know the results of their operations and the current state of the system. That is, the user can know the current operation position and have instant feedback, so that the user can know whether the current indication is effective and save the waiting time of the user. (3) The principle of user control interaction process This principle is manifested in the user's physiology and psychology, which meets the user's experience needs. Provide users with sufficient freedom of operation so that they can control according to their own wishes; Provide user permission to modify; It provides a multi-channel interactive communication layer and provides a variety of choices for users at different levels. (4) Principles of effectiveness and safety. This principle is embodied in helpful design, error-proof design and privacy protection design. Reduce users' cognitive resources, provide help prompt information, instruction guidance, difficult help information, etc. The human-computer interaction process should include appropriate confirmation prompt information, status prompt before return and cancellation, etc. The privacy of user information is considered in many aspects, and effective protection mechanisms are provided, such as digital password, fingerprint identification password mode, etc. 2.4. User-centered Human-Computer Interaction Principle The key problem of natural human-computer interaction is the understanding and modeling of human-machine- environment interaction [6]. The human-computer interaction information processing model is based on the human-computer-environment interaction, with the interaction design principle as the limiting factor, and integrates the human-computer interaction process, user experience and cognition to guide. The user information processing model is expressed as the user processes information and makes feedback actions through the information perception system, the user cognition system and the user response system after receiving information stimulation. As shown in the figure, in the human-computer interaction information processing model, the information perception system is to store the stimulation signal by short-term memory and make preliminary judgment after the output information is perceived by the human body through perception channels such as vision and hearing. User cognitive system provides design criteria for human-computer interaction system design. For example, the information conveyed should be classified and organized according to relevant relationships. The user response system requires that the design should minimize the cognitive load caused by the conveyed information on people, such as reducing the excessive switching frequency of keyboard and mouse keys and reducing useless visual movement of users. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 14 Figure 3: User cognitive information process 3. Types of Human-Computer Interaction Modes and Their Applications The interaction modes in natural human-computer interaction can be divided into the following categories: voice interaction, gesture interaction, somatosensory interaction, multimodal interaction, line-of-sight tracking interaction, brain-computer interaction. Figure 4: Proportion of Research Categories on Human-Computer Interaction at Home and Abroad in Recent Ten Years In order to make human-computer interaction closer to human-to-human interaction, human-to-object interaction, voice recognition, touch interface, posture interaction, line-of-sight tracking and other interaction technologies in the real world, users can obtain more natural and smooth interaction experience and higher work efficiency. The above four interactive feedback states are analyzed to classify the outputs of various interactions. Table 1 summarizes some representative interactions. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 15 Table 1: Representative interaction mode Name Input Output Function Settings iOS Siri Speech Synthetic Speech Screen Text Through the microphone voice input, through the processor analysis, the real user's original sentence or answer on the screen, sends out the synthetic voice and executes the instruction Z-Gloves Hand posture Tactile vibration synthetic music Screen display Operating gloves are equipped with sensors to sense changes in posture and tactile feedback vibrators, which can be used to evaluate hand function. Kinect Whole body posture Screen display Through wide-angle camera, infrared architecture optical system to capture the user's whole body movement Macbook Touchpad Touch Screen display The tactile sensor can sense the number of contacts and touch actions. Infrared eye movement instrument Line of sight Screen display Hawking's glasses contain tactile sensors at the top right, which can detect the movement of their eyeballs. BCI Brain wave Action or Screen Text The user manipulates an entity through visual stimuli or ideas 3.1. Voice Interaction Speech interaction depends on speech recognition technology. Speech recognition applications include call routing, speech to text, machine manipulation, etc. Guzman and his colleagues [22] believed in 2018 that the introduction of voice assistants on mobile devices provides individuals with the first usable and meaningful form of artificial intelligence interaction. At present, voice interaction is mainly applied to voice listening level and voice control system level. The application fields at the phonetic dictation level include medical records, legal and commercial records, and general word processing. Voice interactive input can help some people with physiological defects to better input words or other operations. The voice control system level can be used in industrial manufacturing control, smart home, voice dialing, voice control equipment and many other aspects. Graeme [23] and his colleagues 2019 propose that home voice assistants can obtain social benefits from user interaction, helping users complete tasks, find information, seek help and process orders. Mccallum M C and his colleagues [24] proposed through the "Wizard of Oz" vehicle-mounted interface experiment in 2004 that voice interaction can focus the driver's attention and reduce the cognitive load of interaction compared with manual operation interface. 3.2. Body posture interaction American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 16 Body posture is regarded as a common way of communication. The goals of gesture interaction can be divided into two categories: improving user autonomy and initiative; Provide intelligent service of the system. Previous studies have taken posture as one of the means of human-computer interaction. From the perspective of human- computer interaction, posture classification is shown in Table 2. Table 2: Human-Computer Interaction Posture Semantics Posture Behavior Description Cases Indication Pointing action used to identify objects or determine spatial orientation. Put "that" in "where" Control Motion closely associated with the object being manipulated The traffic police directed the road conditions. Signal Signal language system using flags, lights or arms The traffic police motioned pedestrians to pass through. Sign Language A gesture system based on natural language with specific vocabulary and grammar. The Chinese "Fuel Ceremony" shows respect. Nice to meet you Gesture Gesture movement used to assist oral expression. In the study hall, the teacher put his index finger in front of his mouth to express his silence. Carrier technologies used in posture interaction are divided into two categories: wearable/mobile sensing technology and computer vision technology. Wearable/mobile sensing technology is better applied in controllable environment because its data acquisition is more direct and easier to process. Its carrier equipment is such as sensing tie, pendant, bracelet, glove, Google Glass, etc. The sensing glove is shown in Fig. 5. In its posture interaction, three types of information need to be sensed: shape change, translation and rotation motion, azimuth and distance. Computer vision technology is that the system perceives body movements through electromagnetic waves. The key step is motion capture, and the principles used can be divided into three categories: acoustics, electromagnetism and optics. The cost of the acoustic system is relatively low, the electromagnetic motion capture technology is relatively mature, and it can capture six dimensions and all-round motion information. The optical system can capture three free dimensions of motion information, while the rotation and orientation of shoulders, elbows, knees and other joints need to be calculated from the position data of multiple labels. Rafik Gouiaa and his colleagues [25] proposed a shadow-based multi-view system for human posture recognition in 2016, using cameras and infrared light sources to recognize human posture. Boulay B and other scholars [26] proposed to use three-dimensional human body model to recognize the user's posture and extract feedback information. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 17 Figure 5: Sensing Gloves and Computer Motion Capture Recognition 3.3. Gesture interaction Gesture recognition system can convert user behavior actions into information instructions. Gesture recognition technology has changed the form of human-computer interaction in terms of physical and mechanical properties, so that there is no need for transmission media in the process of human-computer interaction, and it has the operational characteristics of 3D, liberalization, visualization, etc. [27]. Kim H and his colleagues [28] mentioned in a 2005 study that the interface of human-computer interaction gestures is more intuitive than that of mouse operation, and allows inexperienced users to operate directly without too much training. Rahman A and his colleagues [29] believe that interaction with the comfort function of the car can be realized without moving the line of sight away from the road. Barros and his colleagues [30] believe that gestures can be used to help elderly people interact with electronic devices more easily. There are two types of work classifications in gesture interactive multi-touch: one is to obtain multi-touch signal at the same time; The other is to judge the conveying meaning of each point signal. Through the action combination of multiple fingers on the touch screen, interactive operation that makes it quick and conforms to the user's habit tendency can be realized, as shown in FIG. 6. Figure 6: Finger movement combination The development trend of gesture recognition technology has the following two points: more miniature and convenient motion capture equipment; More accurate motion information can be effectively sensed and recognized. The development of Kinect and LEAP sensors [31,32] has realized a simpler implementation. Details such as learners' finger motion tracks will be accurately sensed and effectively recognized. Scholar American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 18 Bruce Lee used Kinect equipment to realize gesture recognition and Unity 3D software to complete 3D modeling, and developed a teaching system of virtual human anatomy [33]. 3.4. Tactile perception interaction The principle of tactile perception interaction is to make interaction more natural and tend to the best experience cognition of users by simulating human body mechanism, force response information and confrontation force with entity interaction. Skinput, developed by Carnegie Mellon University and Microsoft, uses a wearable projection device for human arms to project an operation keyboard or screen image onto the arms, uses a tactile sensing system to sense vibrations of different frequency levels, and converts the vibration frequencies into operation information [34]. Carnegie Mellon University and Disney developed TeslaTouch system, which brings users the cognitive experience of touching real objects based on the principle of electrical vibration. When users manipulate objects through different behaviors and actions on the touch screen, the system generates tactile effects such as convex, concave and texture imitating the solid surface [35]. When applied to barrier-free design, tactile interaction can bring convenience to users with mobility disorders. Professor Lu Xiaobo of Tsinghua Academy of Fine Arts, starting from the blind's demand for electronic tactile images, summarized and put forward the design criteria for tactile images for the blind [36]. The practicality of tactile cane is its widely practical mobile device [37]. In the field of safety prevention, Rosario HD and his colleagues [38] applied tactile pedal design to the driving simulation cabin in 2010, with warning of vibration and tactile frontal collision. After tests, it is found that tactile stimulation is more effective than visual signals and can be used to develop effective and safe warning systems in vehicles. 3.5. Line of sight tracking and input The interaction between line of sight tracking and input is also called eye-machine interaction. This technology is a man-machine interaction mode based on bioelectric signals. The technical theory is based on visual buffer processing theory and eye movement theoretical model. The three basic modes of eye movement are gaze, saccade and follow [39]. Eye posture recognition technology is an important link in eye-machine interaction technology. The basic eye movement modes of users are eye movement modes (up, down, left, right), blinking mode, staring mode and scanning mode. Technologies used to detect eye movements include search coils, infrared eye vision, video-based eye movement tracking and electroophthalmography (EOG), among which EOG system is non-invasive, easier to operate, and more accurate and reliable in collecting signals [40]. Eye movement recognition technology is used in the notebook computer configuration system jointly developed by Tobii and Lenovo. The system combines infrared light and camera to capture the reflected light of user's vision, calculates the region of interest being watched through software, and uses the feedback information to input instructions for control. At the barrier-free design level, Julius Sweetland has developed computer software controlled by eye movements, which enables users with physical disabilities or language disorders to control computers and input characters. Soltani S and his colleagues [56] developed an EOG-based wearable system in 2016, which enables people with dyskinesia to operate computers and type through eye movements, and developed a typing system with an interface as shown in Figure 7 [41]. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 19 Figure 7: Eye Movement Typing System Interface 3.6. Brain-computer interaction BCI in brain-computer interaction can accept the user's cranial nerve signals and use algorithm programs to convert the feedback signals into actions, that is, cognition controls the user's behavior. The flow of BCI interactive information transmission is: acceptance, decoding, coding and feedback. The specific operation is to convert the information into machine actions by receiving signals from functional corresponding neurons and using computer chips and programs. At present, the non-invasive BCI model changes the traditional BCI model of the mechanical device implanted in the brain and controls the device as the natural part of its body representation. The non-invasive BCI mode maps the EEG signal of the user at the visual stimulation or motion imagination level stage into the control information of the external device. The University of Minnesota research team uses non-invasive BCI to enable users to control entities through cognition, such as manipulating robotic arms to grab and place entities [42]. Scholars Fu Yunfa and his colleagues [43] BCI based on SSVEP or mixed with motion imagination is applied to fine control of robot direction, and lays a certain foundation for direct BCI control of robot technology to be put into practical application. Scholars Zhang Yi and his colleagues [44] extracted the user's control intention through EEG signals in barrier-free design, and proposed an intelligent wheelchair human-computer interaction method based on EEG/wave. EEG signals are used to operate the wheelchair to move forward, and EEG signals control the intelligent wheelchair to turn left and right, thus realizing the control of the wheelchair's movement direction due to dyskinesia. 3.7. Multimodal interaction Multimodal interaction is also called multi-channel interaction. The multi-channel perceptual coupling mechanism follows the natural interaction principle of "user-centered". As the information input of a single channel will lead to the problems of fuzzy information received by sensors and inaccurate identification, multi- channel fusion will increase the accuracy of information conveyed by users [45]. When performing multi-mode interaction, users can use two or more coordinated combined input modes for input, such as hearing + vision [46,47], vision + touch [48,49], vision + smell [50], etc. Yongda D and his colleagues [51] conducted a multi- modal human-computer interaction research based on voice and gesture in 2018, and designed a robot control command system in the interface, which can convert the user's voice and gesture into commands that the robot can execute. Marcos S and his colleagues [52] proposed an interactive virtual head for human-computer interaction and social robots in 2010. It adopts multi-modal interaction method, integrates speech synthesis and face visual recognition and tracking functions, and is used in the electromechanical head of social robots. The application fields of multi-mode system are very wide, including environmental space, mobile computing, virtual American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 20 environment, art, applications for the disabled and applications in public/private space [53]. Lamia Gaouar and his colleagues [64] developed a three-dimensional interactive virtual reality environment with natural interactive interface in 2018, and moved the 3D Graph Explorer plug-in to realize multi-modal interactive scenes. Fernandez Montenegro J M and his colleagues [55] improved the cognitive ability of patients with Alzheimer's disease and effectively delayed Alzheimer's disease by studying multi-modal human-computer interactive puzzle games of vision, hearing and touch. Wang Yanxin and others designed a human-robot interaction system based on mixed line of sight-brain-computer interface and shared control in 2018, So that users can continuously control the motion of the robot end in two-dimensional space through visual stimulation and motion imagination, and obtain the assistance of machine intelligence in avoiding obstacles and completing the target with high precision. 4. Methods of analysis and evaluation Effectiveness evaluation of human-computer interaction mode can be studied from user cognition and work efficiency evaluation. A large number of researches have been done at home and abroad. Human-computer interaction usability evaluation methods can be divided into subjective evaluation method, physiological measurement method and mathematical evaluation method. 4.1. Subjective evaluation method Subjective evaluation indicators include user comprehensibility, learnability, user satisfaction, etc. Mathematical methods and models are supplemented in the process of determining the weight of evaluation indexes, such as grey theory analysis, cluster analysis, fuzzy mathematics and analytic hierarchy process. In the subjective evaluation method, scholar Tao Nin [56] designed and evaluated the interface graphic menu of interactive gestures. Scholars Hayrettin and others evaluated the interactive experience and methods of users in the game. Scholars Yan Shengyuan and others [57] optimized the usability of human-computer interaction interface and proposed a subjective evaluation method based on grey system theory. 4.2. Physiological measurement method At present, there are two physiological evaluation methods for the usability evaluation of human-computer interaction mode: eye movement tracking and EEG. These two technologies have become the focus and hotspot of research at home and abroad. The usage scenario of the equipment is shown in Figure 9, with EEG experiment on the left and eye movement experiment on the right. In eye movement technology, the evaluation of human-computer interaction interface needs to go through the following four steps: interactive task information extraction, eye movement signal information collection, processing detection data, and usability feature analysis. Scholars Christer Ahlstrom and others used eye movement technology to test drivers' drowsiness while driving, thus improving the alarm. In EEG detection technology, New Delhi IT Institute has proposed a method to measure users' cognitive load by using EEG signal power spectrum. Its purpose is to coordinate feedback information in human-computer interaction with barrier-free operation of demand and matching resources, and to comprehensively analyze the internal and external representations of cognitive American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 21 process. Scholar Shen Qingyang [59] established a human-computer interaction design method based on ERP threshold by using EEG scientific research, which provides scientific and reliable theoretical support for design researchers to carry out interface design. Figure 8: EEG equipment and eye movement equipment 4.3. Mathematical evaluation method In the mathematical evaluation method of human-computer interaction mode, scholars at home and abroad have carried out a large number of scientific researches and made theoretical support. Common mathematical evaluation methods include: artificial neural network method, Delphi method, fuzzy theory method, analytic hierarchy process, grey correlation analysis method, grey system theory, principal component analysis method and cluster analysis method. Scholar Su Weihua [60] made a comprehensive analysis and summary of these mathematical evaluation methods. Scholars Negri P and his colleagues [61] used genetic algorithm to design bare hand posture recognition algorithm in VR human-computer interaction system. Scholar Yan Shengyuan [57] used particle swarm optimization algorithm to optimize the layout of man-machine interface in the actual evaluation of interface and products by mathematical methods. Due to the different research contents of different methods, the qualitative and quantitative methods should be adopted to evaluate the practical effect of human-computer interaction mode, so as to reduce the low accuracy of subjective evaluation and ensure high reliability and validity in the selection of human-computer interaction mode. 5. Recommendations According to previous studies, suggestions are made to conduct in-depth discussions on the following issues (1) User Behavior Level People's actions are more casual and have vague meanings, so the recognition of actions and inaccurate information by computer vision are not accurate enough, and the workload of correcting, processing and restoring massive data in the later period is also large. (2) Interactive technology American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 22 At present, the natural interaction mode is highly dependent on the performance of each sensor. The accuracy rate of information fusion of sensors is low. The tracking error and interaction delay of information greatly affect the interaction performance. The interaction technology needs to be further improved to solve the efficiency of information recognition processing. In the user experience, wearable devices will make the virtual tactile perception have a certain gap from the real tactile perception. (3) Security issues The output of human behavior information needs to be further guaranteed at the security level. For example, in speech recognition, personal privacy issues need to be screened for security processing. 6. Conclusion The development trend of human-computer interaction is gradually moving towards the level of human- computer integration. First of all, machines and equipment can naturally interact with the environment and users and independently adapt to complex dynamic working environment and cooperate with each other. Secondly, the demand for human-computer natural interaction is gradually increasing in the fields of intelligent manufacturing, medical rehabilitation and national defense security. Therefore, it is necessary to carry out research on human-computer interaction theory and design methods that integrate human-computer- environment. Finally, human-computer interaction needs to make innovative information research progress in multi-modal environment perception, human-computer mutual adaptation and cooperation, swarm intelligent interaction operating system, etc. through multi-disciplinary cross-integration. At the theoretical level, researchers need to invest more efforts in the thinking factors, design principles and usability under the multi- modal perception and situational understanding, and the behavioral intention and human-computer natural interaction based on biological signals. At the technical level, aiming at the demand of large and complex component manufacturing in key industries for production mode and equipment innovation, the information drive of human-computer interaction design perception and multi-sensory channel is studied. It is convenient for the design of human-computer integrated robots, solves the natural interaction problems of living body drive, perception, natural human-computer interaction optimization, neural perception and control, and innovates the application of polymorphic distributed human-computer interaction operating systems. Human-computer interaction research has been exploring more natural and efficient interaction methods. After years of research, remarkable progress has been made and shows a vigorous development trend. We believe that the development of new human-computer interaction will bring more new ideas and appear in front of people in a brand-new, more natural and efficient way. References [1] Mapping the field of human-computer interaction (HCI)[J]. Proceedings of the American Society for Information Science and Technology, 2006, 43(1):1-7. [2] Seffah A，Gulliksen J ,Desmarais M C . [Human-Computer Interaction Series] Human-Centered American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 23 Software Engineering — Integrating Usability in the Software Development Lifecycle Volume 8 || [J]. 2005, 10.1007/1-4020-4113-6.] [3] Adamczyk P D , Twidale M B . Supporting multidisciplinary collaboration:requirements from novel HCI education[C]// Conference on Human Factors in Computing Systems. DBLP, 2007. [4] Kuo-Yi L . User Experience–Based Product Design for Smart Production to Empower Industry 4.0 in the Glass Recycling Circular Economy[J]. Computers & Industrial Engineering, 2018:S036083521830295X-. [5] Hanington B . Chapter 6 – Design and Emotional Experience[J]. Emotions and Affect in Human Factors and Human-Computer Interaction, 2017:165-183. [6] Bunt H , Beun R J , Borghuis T . Multimodal Human-Computer Communication, Systems, Techniques, and Experiments[C]// Multimodal Human-computer Communication, Systems, Techniques, & Experiments. Springer-Verlag, 1998. [7] Quek F , Mcneill D , Bryll R , et al. Multimodal human discourse: Gesture and speech[J]. ACM Transactions on Computer-Human Interaction, 2002, 9(3):171-193. [8] Benoit A, Bonnaud L, Caplier A, et al. Multimodal signal processing and interaction for a driving simulator: Component-based architecture[J]. Journal on Multimodal User Interfaces, 2007, 1(1):49-58. [9] Chen F , Ruiz N , Choi E , et al. Multimodal behavior and interaction as indicators of cognitive load[J]. ACM Transactions on Interactive Intelligent Systems, 2012, 2(4):1-36. [10] CHENG Shiwei, SUN Shouqian.Human-computer interaction resource model based on distributed cognition [J]. Computer Integrated Manufacturing System, 2008 (9): 1683-1689. [11] Lu Lu, Tian Feng, Dai Guozhong, Wang Hongan.A multi-channel cognitive and interactive model integrating touch, hearing and vision [J]. Acta Computer Aided Design and Graphics, 2014, 26 (04): 654-661. [12] Pavlovic VI , Sharma R , Huang T S . Visual interpretation of hand gestures for human-computer interaction: a review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7):0-695 [22] Guzman A L. Voices, in, and, of, the machine: Source orientation toward mobile virtual assistants[J]. Computers in Human Behavior, 2018:S0747563218303844-. [13] Erden M S , Tomiyama T . Human-Intent Detection and Physically Interactive Control of a Robot Without Force Sensors[J]. IEEE Transactions on Robotics, 2010, 26(2):370-382. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 24 [14] Shackel B. The psychology of everyday things : Donald A. Norman, New York, Basic Books Inc. 1988, ISBN 0-465-06709-3[J]. Acta Psychologica, 1992, 79(3):275-278. [15] Cockburn A , Quinn P , Gutwin C . The Effects of Interaction Sequencing on User Experience and Preference[J]. International Journal of Human-Computer Studies, 2017, 108:89-104. [16] Norman, Draper D A , Stephen W . User Centered System Design : New Perspectives on Human- Computer Interaction[J]. The American Journal of Psychology, 1986, 101(1). [17] Mccarthy J. User experience and the idea of design in HCI[C]// International Conference on Interactive Systems: Design. 2005. [18] NetScan, Magazine A I. User Experience Research Group[J]. [19] Bateman A, Zhao O K, Bajcsy A V, et al. A user-centered design and analysis of an electrostatic haptic touchscreen system for students with visual impairments[J]. International Journal of Human-Computer Studies, 2018, 109:102-111. [20] Eskridge T C, Still D, Hoffman R R. Principles for Human-Centered Interaction Design, Part 1: Performative Systems[J]. IEEE Intelligent Systems, 2014, 29(4):88-94. [21] Khatib O , Demircan E , Sapio V D , et al. Robotics-based synthesis of human motion[J]. Journal of Physiology - Paris, 2009, 103(3-5):211-219. [22] Guzman A L. Voices, in, and, of, the machine: Source orientation toward mobile virtual assistants[J]. Computers in Human Behavior, 2018:S0747563218303844-. [23] Hey Alexa examine the variables influencing the use of artificial intelligent in-home voice assistants Computers in Human Behavior, Volume 99, October 2019, Pages 28-37 [24] Mccallum M C, Campbell J L, Richman J B, et al. Speech Recognition and In-Vehicle Telematics Devices: Potential Reductions in Driver Distraction[J]. International Journal of Speech Technology, 2004, 7(1):25-33. [25] Gouiaa R , Meunier J . Human posture recognition by combining silhouette and infrared cast shadows[C]// International Conference on Image Processing Theory. IEEE, 2016. [26] Boulay B , Francois Brémond, Thonnat M . Applying 3D human model in a posture recognition system[J]. Pattern Recognition Letters, 2006, 27(15):1788-1796. [27] Lou Zehua, Yin Jibin.Analysis of gesture design principles in human-computer interaction [J]. Software Guide, 2018, 17 (04): 19-24 +35. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 25 [28] Kim H , Albuquerque G , Havemann S , et al. Tangible 3D: Hand Gesture Interaction for Immersive 3D Modeling[C]// Workshop on Int Workshop on Immersive Projection Technology. Eurographics Association, 2005. [29] Rahman A , Saboune J , El Saddik A . Motion-Path Based in Car Gesture Control of the Multimedia Devices[C]// Acm International Symposium on Design & Analysis of Intelligent Vehicular Networks & Applications. ACM, 2011. [30] Barros A C D , LeitO R , Ribeiro J . Design and Evaluation of a Mobile User Interface for Older Adults: Navigation, Interaction and Visual Design Recommendations[J]. Procedia Computer Science, 2014, 27:369-378. [31] Shotton J, Kipman A, Kipman A, et al. Real-time human pose recognition in parts from single depth images[J]. Communications of the Acm, 2013, 56(1):116-124. [32]Taylor J , Shotton J , Sharp T , et al. The Vitruvian Manifold: Inferring Dense Correspondences for One- Shot Human Pose Estimation[C]// IEEE Computer Vision and Pattern Recognition. IEEE, 2012. [33] Bruce Lee. Design and Implementation of Virtual Human Anatomy Teaching System Based on Kinect Gesture Recognition [D]. Beijing: Beijing University of Technology, 2014. [34] Chun-Cheng Chang, Linda Ng Boyle, John D. Lee, James Jenness Using tactile detection response tasks to assess in-vehicle voice control interactions Transportation Research Part F: Traffic Psychology and Behaviour, Volume 51, November 2017, Pages 38-46 [35] Harrison C， Tan D， Morris D. Skinput: Appropriating the Skin as an Interactive Canvas [J]. Communications of the ACM ， 2011 （8）: 111- 118. [36] Jiang Ning, Lu Xiaobo, Li Yuan, Xu Yingqing.Graphic display design method for blind people and its user experience research [J]. Journal of Computer Aided Design and Graphics, 2011, 23 (09): 1539- 1544. [37] Dakopoulos D , Bourbakis N G . Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey[J]. IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews), 2010, 40(1):25-35. [38] Rosario H D , Louredo M , I?aki Díaz, et al. Efficacy and feeling of a vibrotactile Frontal Collision Warning implemented in a haptic pedal[J]. Transportation Research Part F Traffic Psychology & Behaviour, 2010, 13(2):0-91. [39] Lippiello V , Siciliano B , Villani L . Interaction Control of Robot Manipulators Using Force and Vision[J]. INTERNATIONAL JOURNAL OF OPTOMECHATRONICS, 2008, 2(3):257-274. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 26 [40] Rahman M M, Ikeura R, Mizutani K. Control characteristics of two humans in cooperative task and its application to robot control[C]// Conference of the IEEE Industrial Electronics Society. 2000. [41] Soltani S , Mahnam A . A practical efficient human computer interface based on saccadic eye movements for people with disabilities[J]. Computers in Biology and Medicine, 2016, 70:S0010482516300026. [42] Wolpaw J R， Birbaumer N， Heetderks W J， et al. Brain-Computer Interface Technology: A Review of the First International Meeting[J]. IEEE Transactions on Rehabilitation Engineering， 2000 （2）: 164-173. [43] Fu Yunfa, Guo Yanlong, Li Song, Xiong Xin, Li Bo, Yu Zhengtao.Research on direction and speed of direct brain-controlled robot based on SSVEP [J]. Acta Automata Sinica, 2016, 42 (11): 1630-1640. [44] Zhang Yi, Luo Mingwei, Luo Yuan, Xu Xiaodong.Human-computer interaction in intelligent wheelchair based on EEG/wave [J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2013, 41 (07): 109-114. [45] Prats M Sanz P J , Pobil A P D . Vision-tactile-force integration and robot physical interaction[C]// IEEE International Conference on Robotics & Automation. IEEE Press, 2009. [46] Falchier A , Clavagnier S , Barone P , et al. Anatomical Evidence of Multimodal Integration in Primate Striate Cortex[J]. Journal of Neuroscience, 2002, 22(13):5749. [47] Leitao J , Thielscher A , Werner S , et al. Effects of Parietal TMS on Visual and Auditory Processing at the Primary Cortical Level - A Concurrent TMS-fMRI Study[J]. Cerebral Cortex, 2013, 23(4):873-884. [48] Vasconcelos N , Pantoja J , Belchior H , et al. Cross-modal responses in the primary visual cortex encode complex objects and correlate with tactile discrimination[J]. Proceedings of the National Academy of Sciences, 2011, 108(37):15408-15413. [49] Zhou W , Zhang X , Chen J , et al. Nostril-Specific Olfactory Modulation of Visual Perception in Binocular Rivalry[J]. JOURNAL OF NEUROSCIENCE, 2012, 32(48):17225-17229. [50] Powers A R , Hevey M A , Wallace M T . Neural Correlates of Multisensory Perceptual Learning[J]. Journal of Neuroscience, 2012, 32(18):6263-6274. [51] Yongda D , Fang L , Huang X . Research on multimodal human-robot interaction based on speech and gesture[J]. Computers & Electrical Engineering, 2018, 72:443-454. [52] Cutug no F, Leano V A, Rinaldi R, et al. Multimodal framework for mobile interaction[J]. 2012. [53] Marcos S , Jaime Gómez-García-Bermejo, Zalama E . A realistic, virtual head for human–computer American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 64, No 1, pp 9-27 27 interaction[J]. Interacting with Computers, 2010, 22(3):176-192. [54]Lamia Gaouar, Abdelkrim Benamar, Olivier Le Goaer, Frédérique Biennier,HCIDL: Human-computer interface description language for multi-target, multimodal, plastic user interfaces,Future Computing and Informatics Journal,Volume 3, Issue 1,2018,Pages 110-130,ISSN 2314-7288 [55] Fernandez Montenegro J M, Argyriou V. Cognitive evaluation for the diagnosis of Alzheimer's disease based on Turing Test and Virtual Environments[J]. Physiology & Behavior, 2017, 173:42-51. [56] Nin T, Bowman D A, North C, et al. 2011. Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures [J]. International Journal of Human-Computer Studies. 69:551- 562. [57] YAN Shengyuan.2013. Design and Evaluation of Human-Machine Interface [M]. Beijing: National Defense Industry Press [58] Naveen K, Jyoti K. Measurement of cognitive load in HCI systems using EEG power spectrum: an experimental study [J]. Procedia Computer Science, 2016, 84 (27): 70-78. [59] SHEN Qing-yang, XUE Cheng-qi, NIU Yafeng.Research on the design method of human-computer interaction interface based on ERP [J]. Design, 2018 (6): 134-135. [60] Su Weihua.Research on the Theory and Method of Multi-index Comprehensive Evaluation [D]. Xiamen University, 2000. [61] Negri P, Omedas P, Chech L, et al. Comparing Input Sensors in an Immersive Mixed-Reality Environment for Human-Computer Symbiosis[M]// Symbiotic Interaction. 2015.