Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 2, No. 1, 2022 1 Hardward Design and Implementation of Blind Guide for Intelligent Traffic Scene Identification and Ranging Based on HiSilicon Taurus Yongfeng Huang, Weizhi Liu, Shengwei Lu, Yuzhong Li School Comuputer Science and engineering of Huizhou University, Huizhou, 516007, China Abstract: In this paper, according to the characteristics of the low-power hardware design of the combination of HiSilicon Hi3861V100 chip and HiSilicon Hi3516DV300 chip, which cooperates with a series of peripherals, in order to design a Blind guide that can recognize and deal with complex traffic conditions and can broadcasts dangerous situations of the surrounding environment in time. At the same time, it combines 4G module to complete audio and video data transmission and one key for SOS function. After the service gets the data of device terminal for unified processing and management. On the other side, APP links the service to realize the function of information and data transfer, map positioning, real-time dibble seeding and full- duplex intercom. Keywords: Low-power consumption, AV processing, Radio communications, HiSilicon chip, Blind guide instrument. 1. Introduction According to the seventh national population census, the number of visually impaired people in China has exceeded 17.3 million, ranking first in the world. Blind people have difficulties in all aspects of life, including travel, which often requires a person to accompany them. Although guide dogs can solve the inconvenience of travel, the cost of training a guide dog is too high to meet the needs of most blind people. In recent years, with the attention of the society to the blind group, the technology of guide equipment is more and more mature. For example, the multi-functional guide device designed by students of Nanjing Institute of Technology [1] and the wearable guide device designed by students of Wuhan University of Technology [2]. In addition to optimizing its own structure, the guide robot can also combine artificial intelligence, high-speed mobile communication, cloud platform, edge computing and other technologies to improve the overall service ability and better serve the visually impaired and the society [3]. Therefore, some people have designed a visual assist system for the blind using artificial intelligence and APP in this way [4], and there have been 5G guide glasses using 5G high-speed and stable network combined with powerful edge cloud computing capability [5][6]. By virtue of the advantages of HiSilicon’s intelligent engine NNIE and references, we designed a blind guide based on the image acquisition chip scheme of HiSilicon to realize intelligent traffic scene identification and ranging. The purpose is to reduce the pressure of caregivers, facilitate the travel of the blind, and guide the blind to avoid obstacles. 2. Architectural Design 2.1. Requirement Analysis According to our investigation and requirement analysis, the following design contents are obtained: First, traffic condition collection and recognition. On the one hand, the collected video can be accurately identified and marked after AI recognition, and broadcast in time. On the other hand, the video should be clear and smooth; Second, necessary measurement of peripheral data. This includes using GPS to locate the device; Use ultrasonic module to avoid obstacles and pits; Rely on three axis acceleration module to realize fall alarm; Third, HiGV's human-computer interaction adaption. Different user requirements are different, so the use of the guide instrument is also different. It includes displaying peripheral information, modifying the hazard distance threshold, and setting emergency contacts. Fourth, the APP remote viewing. The caretaker can checks the condition of the device through the APP. In addition, it can secure the location of the device, check the danger messages encountered by the device in the recent period of time, as well as real-time video on demand and audio communications. 2.2. System Total Design According to the above system requirements, we designed the overall structural diagram of the system as shown in Figure 1. The Pegasus module, a HiSilicon’s embedded master control chip solution, which is mainly used to manage peripherals. Taurus, a HiSilicon’s image acquisition chip solution, displays the collected peripheral data on the interface of HiGV, which also provides Settings for different people to adapt to the device. The Taurus also features AI recognition, voice broadcasting, audio and video streaming and data uploading. In order to facilitate device data transmission and server management, we make a series of data of peripherals, state and control interact with the server using MQTT, and the push stream of audio and video data is realized by TCP Socket. 2.3. System Module Design Pegasus connects GPS module, ultrasonic module and triaxial acceleration sensor in turn. After it completes the system initialization, we first create the task of serial communication module to prevent peripheral data from transmitting data before creating a good serial communication task, thus leading to system errors. Then the GPS positioning module task, ultrasonic ranging module task and fall detection module task are created respectively. 2 Figure 1. The overall structural diagram of the blind guide system According to the requirements of the system, the discernable categories of the blind guide are divided into three categories as shown in Table 1, namely pedestrian traffic lights, vehicle traffic lights and things that the blind should pay attention to when walking. When identifying the zebra crossing, it also need to judge whether to cross the zebra crossing for improving the safety of the equipment. When the X-axis coordinate interval of traffic light identification is within the X-axis coordinate interval of zebra crossing, the zebra crossing can be judged to meet the crossing requirements. In addition, as shown in Figure 2, we divided several subsystems on the Taurus system based on the Hi3516DV300 chip characteristics for our development and testing. The system flow chart of APP is shown in Figure 3. Table 1. The discernable categories of the blind guide Types of detection Vehicle green light Vehicle red light Vehicle side-looking light Pedestrian green light Pedestrian red light Pedestrian side-looking light Zebra crossing Road shoulder Bus station Figure 2. The partition map of Taurus subsystem 3 device management page open software function selection add/remove device ID add/delete data in local database exit software device page device online? device detail page function selection hide/show map turn on/off intercom see the list of messages. receive message send the message of cancel on- demand send the message of request on- demand position information update location live streaming yes no Figure 3. The system flow chart of Harmony APP 3. Detailed Design 3.1. Detailed Design of Peripheral Hardware Module The general process of each peripheral module we designed is to first initialize the GPIO pins of the sensor and reuse them into the corresponding mode. After obtaining the data, the serial communication module will package and send them. Figure 4 is ultrasonic ranging module flow chart, and figure 5 is the flow chart of fall detection module, we only read the data from the z-axis, it is because according to the GPIO multiplexing relational tables, in addition to the already used, used for debugging, and used with Taurus the pin end to a serial port communication, the rest of pins can't meet the read to three axes at the same time. Therefore, in the limited conditions, we choose to read only the z-axis data to judge whether the blind person falls. Figure 4. Ultrasonic ranging module flow chart Figure 5. Fall detection module flow chart 3.2. Detailed Design of Communication Module Pegasus and Taurus can be connected through serial port, USB virtual serial port, and WiFi based on the reference solution. The only USB port that can be output of Taurus is used by LTE communication module, and the WiFi interconnection scheme has poor anti-interference capability, so serial interconnection is used here. In this solution, in order to facilitate the communication and control between Taurus and Pegasus, the designers have defined a set of HiSignalling control protocol. So we need to write and modify according to this protocol. LTE communication module, after the success of dial-up Internet access, mainly realizes the functions of audio and video streaming, message uploading, provides emergency dialing and sending SMS for help. 3.3. Detailed Design of Media Processing Platform The media processing platform provided by HiSilicon not 4 only shields the underlying processing of the chip, but also directly provides the interface completion function. On this basis, we use the FFmpeg to package and upload the video and audio streams together after the device has acquired them. Figure 6 and Figure 7 are flow charts of pushing stream of audio module and video module push stream respectively. start set AI attributes and enable it, and open AI_ CHN channel set the encoding attribute of AENC and enable it, as well as open the AENC_ CHN channel make AI_CHN and AENC_CHN bind together open microphone video and audio stream are converted and packaged into TS stream through FFmpeg interface push TS stream to streaming media server endaudio PCM stream is converted to AAC stream through FFmpeg interface Figure 6. Flow chart of audio module streaming Figure 7. Flow chart of audio module streaming 4. Results Show According to the above design scheme, we designed the prototype of the whole system. After our test operation, the system can normally acquire peripheral data, recognize traffic lights, upload audio and video and call for help. Figure 8 shows the prototype of the image acquisition system of the blind guide. It can be seen that the device can collect and display peripheral data in real time at the upper left corner of the display screen. And after the test, the device recognition effect can reach 17 frames per second after NNIE accelerates. After the device runs normally and we open the Harmony APP we designed, we can view the data collected by the 5 device by using the self-built server. Figure 9 shows the display effect of mobile video stream based on Harmony system. The APP is displaying the video stream collected from the device in real time, as well as the current position of the device. Figure 8. The prototype of the image acquisition system of the blind guide Figure 9. The display effect of mobile video stream based on Harmony system 5. Summary In order to make it more convenient for the blind to travel and take into account the safety of the blind, this project is to realize a blind guide that can identify traffic scenes and ranging intelligently by combining Taurus development kit based on HiSilicon with Pegasus development kit. In terms of function, AI recognition uses NNIE to accelerate the recognition of zebra crossing, traffic lights, bus stations and other information, and improves the safety of instrument in guidance through ultrasonic ranging module. At the same time, GPS module and LTE communication module are used to enable the family of the blind to remotely view and care about the situation of the blind. At present, the project has completed the design and implementation of the whole system prototype. However, it is only one of the basic solutions of all blind guide instruments, and it will need to increase its functions and productized in the future. References [1] Aiping Wu. Design of Multi-functional Blind Guide[J]. Industrial Control Computer. Vol. 34 (2021) No. 04, p. 136- 137. [2] Shiyun Li. A Navigation System for Blind and Visually Impaired People[J]. Industrial Design. (2018) No. 03, p. 102- 103. [3] Zhaohan Wu, Xuewen Rong, Yong Fan. Survey On Research Status of Blind-Guiding Robots[J]. Computer Engineering and Applications. Vol. 56 (2020) No. 14, p. 1-13. [4] Bo Xu: Visual Aided Navigation System Design for the Blind[D] (University of Chinese Academy of Sciences, China 2020). [5] Anonymity. 5G Guide Glasses, Which Allow the Blind to Listen to the World[J]. Information Construction. (2020) No. 04, p. 24. [6] Anonymity. 5G Guide Glasses for Blind People to Listen to the World[J]. Information Construction. (2021) No. 06, p. 26-27.