BCP Education & Psychology CEPC 2022 Volume 8 (2023) 71 Research on Music Visualization in New Media Feifan Chen*, † and Chengfei Diao† School of Humanities, Xi'an Conservatory of Music, Xi’an, Shaanxi, China *Corresponding author: 2020011003@stu.xacom.edu.cn †These authors contributed equally to this work Abstract. With the current development of science and technology, technology provides a new carrier for music, and the form and quantity of music are constantly expanding. The way people come into contact with music is no longer just radio, but also TV, mobile phones, computers, etc. The number of music and video bundling is increasing, and the amount of music in independent art forms is decreasing. The integration between industries has made the combination of vision and hearing closer, music visualization has become more mature, and the art emerging in this form has also become the mainstream of music art. This paper studies music visualization through the case study method, content analysis method and literature analysis method to analyze the current development status of music visualization from two aspects: streaming media and games. On this basis, this paper studies the development trend and prospects of the future music and music industry, providing reference suggestions for the future development of the music industry. Keywords: Music; Music visualization; New media; Game music. 1. Introduction In the process of the development of music industry, the application of technology has always been one of the main driving forces for the development of the music industry. Since the era of vinyl records, the music industry has experienced the era of tape and CD records to today's digital music era. Every large-scale change in the music industry is related to the universal application of new technologies. The emergence and application of new technologies affect the development of the music industry and the commercialization of music as an art. At the same time, technological progress and application have also prompted new media forms to enter the current market, and gradually promoted changes in relevant social relations with new media as the core while new media enter the market. As an artistic form of information, music will also be affected by the transformation of new media. In the current information society, with the progress of information transmission technology, information acquisition is becoming more and more convenient, and the information obtained also shows the characteristics of diversification and composition. In addition, with the progress of personal information reception terminals, the forms in that information can be presented are gradually moving towards the superposition of multiple forms, and the music industry has also seen different previous changes in this change, that is, music visualization. This paper focuses on the performance of music visualization in different new media as a research theme, and explores the specific artistic expressions of music in the direction of visualization, including the combination of music and streaming media, the combination of music and games, and the application of virtual reality technology in music in the future. In the current market environment, the probability of music appearing in the form of independent art is gradually decreasing. More and more art forms have begun to merge with music, and music has gradually become a member of a comprehensive form of information dissemination. In view of this situation, this paper adopts the research method of literature analysis. Through the study of music visualization articles and the investigation of the content containing music elements in various platforms, this paper comprehensively analyzes the embodiment of music visualization in different types of media, and summarizes them. At the same time, this paper combines the perspective of communication, studies the influence of music visualization on music communication and the music industry in detail, and summarizes the differences between music information dissemination BCP Education & Psychology CEPC 2022 Volume 8 (2023) 72 and music information interpretation of different types of music visualization. In addition, the paper also predicts the future development direction of the music industry in combination with the current cutting-edge technology development situation. At the same time, combined with the current situation of music visualization in the music industry, it looks forward from the present to the future from a comprehensive perspective, in order to serve as a reference for the future development direction of the music industry. 2. Definition 2.1 New Media New media is a communication carrier with computer technology as the core. It mainly refers to optical fiber cables, large database communication systems, communication satellite and satellite live TV systems, high-definition television, and the Internet, multimedia, etc., which grew rapidly in the 1990s [1]. The Internet is its main body. The new media combines text, graphics, sound and images, in contrast to traditional media, that is, print media (newspapers, magazines, books), sound media (radio and recording), and image media (film and television). The rise and development of new media is a new breakthrough at the media level in human communication since the mid-20th century. It makes the information industry the most prominent industry in social development. Knowledge and information have become the basic elements and main resources of the development of productive forces. There is an informatization trend in human society, and the emergence of globalization is also related to new media. The generation of the introduction is inextricably linked. 2.2 Music Visualization Music visualization is a branch of information visualization. The creative purpose of music visualization may be an artistic expression or simple information visualization. The creative method is to use the visual form as the expression means, with the help of multimedia platforms as the expression carrier, and display music information through visual graphics, images, film and television and other forms to help the audience and listeners appreciate, and analyze the information contained in the music [2]. 3. Streaming Media Streaming media refers to the continuous time-based media transmitted by streaming technology on the network, such as audio, video, animation or other multimedia files [3]. The main feature of streaming media is the use of streaming data transmission in the process of data transmission, compressing media data files to the network server in a cache manner, and then gradually transferring data to the user terminal, so that users can browse data without downloading. At present, there are two main streaming media transmission strategies: CDN (Cont Distributed Network) and P2P (Peer to Peer) [4]. Judging from the current market situation, streaming media technology has been widely used in the overall Internet industry environment, especially in the fields of music and video. As Figure 1 shows the Global Music Industry Report 2022 released by the International Recording Association (IFPI), people can see that the global record market grew strongly in 2021, with an increase of 18.5% over 2020, with a total revenue of $29.5 billion [5]. At the same time, streaming media has once again proved to be the main driving force and dominant format in the world. Streaming media revenue increased by 24.3% in 2021, up 5 percentage points from the 19.1% growth rate last year. Overall streaming media revenue accounts for 65.0% of global record revenue, leading the whole music industry. BCP Education & Psychology CEPC 2022 Volume 8 (2023) 73 Fig. 1 Global Music Industry Report 2022 [5] While streaming music is warmly welcomed by users, people have to pay close attention to the strong rise of streaming short video platforms. Short videos are still very popular. According to a report released by Cloudflare, a cybersecurity service company, TikTok has replaced Google and topped the Internet traffic list in 2021. In addition, ByteDance also announced that it will try to develop its own music streaming service in 2022. Through data comparison, it can be found that the music industry will embark on a new development path in the future, that is, music visualization. 3.1 Three Manifestations of Music Visualization The authors have explained the definition of music visualization above. In this part, the authors will focus on the three manifestations of music visualization in streaming video platforms, namely, the combination of music and film and television images, MV, and music and graphic images. 7.1.1 The combination of music and film and television images As one of the most frequent forms of information dissemination in life, film and television has the characteristics of narrative, art and graphics. These characteristics make film and television the most common form of expression in music visualization. Film and television are deeply integrated with music in the process of music visualization. This new fusion makes up for the shortcomings of the two forms of information transmission, hearing and vision, and forms a multi-sensory form of communication. The definitions and types of film and television are too complicated, and they also show multiple types in the fusion with music, which can be divided into two parts according to the different content subject. The first is the work with film and television as the content subject, and the other is the work with music as the content subject. In the work with film and television as the content subject, the content description of the work and the picture performance of the work have become the most important parts of the whole work. Among them, music is mainly divided into rendering atmosphere, depicting the environment and mobilizing emotions. In the whole work, music is regarded as a potential background element compared with narrative and picture. With the help of film and television, the main content is transmitted to the audience in an obscure and non-direct form. Although this form is also recognized as a part of music visualization, music is not in the subjective sense and is a weakened expression of music visualization. Unlike works with film and television as the content subject, works with music as the main body usually use film and television as a supplement and explanation of music content, interpret music through direct and specific information transmission methods such as narrative and pictures, or express music further. In it, music has become the main body of the narrative and the object that needs to be expressed in the picture. In this kind of work, music is dominant, and all pictures and narrative expressions are based on music works. According to the author's understanding of music, BCP Education & Psychology CEPC 2022 Volume 8 (2023) 74 this kind of work usually has a strong author's subjective tendency, which can well show the author's interpretation and portrayal of musical works. 7.1.2 MV MV is an art form that takes the theme and structure of music as the source of the creative concept, film and television technology and digital media as the means, and the combination of music and pictures as the presenting carrier [6]. It should be noted here that the MV in this paper refers to the visualization of music created by the official to better promote the dissemination, circulation and publicity of music works. The second creations carried out collectively or by individuals by folk creators are not within the scope of the concept. At the beginning of MV's birth, it was to better publicize commercial pop music, intuitively show the theme content of music by video, and use film shooting techniques to put music, literature, art and other art forms to penetrate an entire object into a whole style [7]. However, MV unlike movies, the creation of MV works is based on music rather than specific stories. MV does not have a specific narrative process like movies, but adopts illusory emotional artistic expression like music, often adopts unique self-created methods in shooting, and often attracts audiences in a short time. Fierce conflicts and high contrast and fantasy-style colors and decoration are often used. The story is often secondary. The expression of emotions is the main focus of the MV's portrayal. 7.1.3 The combination of music and image Unlike the above two forms of music visualization, the combination of music and image is a more concise and clear expression of music. The combination of music and image is more like a digital and dynamic score, mainly the time value and pitch of musical notes are expressed in the form of images, which are mostly used for pure tones. This kind of music visualization often rarely involves emotional changes in music, or leads to the simple use of color and shape changes to express musical emotions. This form of expression is mainly to enable the audience to understand the music score that professionals can understand in a simpler way or simply show the melody and emotional changes of music for the audience. It is a means of popularization of music and a relatively common and simple music visualization work. In the process of music visualization on streaming platforms, music works have been better popularized, and the understanding of audience music works has been enhanced. However, the visual expression method strengthens memory and embodiments music, but also limits the emotional appeal and narrative variability of music. The means of visualization binds illusory music expression. 4. Game Music With the development and innovation of science and technology, the Internet has become the most important media technology in today's society. The visual effect of the network has also gradually changed from the original text, black and white images to dynamic and color pictures [8]. On this basis, the video game industry has made rapid progress. First of all, in terms of vision, video games have changed from the original simple black and white lines to realistic color pictures; secondly, in terms of hearing, it is no longer the original simple prompt sound effect, but has added a more complete music art. With this change, the game has gradually become an independent and complete world, in which music has also become an indispensable factor. As the art of sound, music has long had a scientific, independent and complete theoretical and practical system [9]. When music enters the game field, it not only plays a role in setting off the atmosphere and creating a realistic game environment; more importantly, music has special effects in enhancing the artistic appeal of the game, promoting the development of the game plot, deepening the theme, etc., and even plays a vital role in the process of experiencing the game. The authors will take Genshin Impact as an example to explain how game music combines vision and hearing to create a more immersive world. BCP Education & Psychology CEPC 2022 Volume 8 (2023) 75 4.1 Music Visualization in the Game's Featured Areas The background music used in different game map scenes is very different. At present, Genshin Impact has developed four regions, and the background music in the four regions is different. The first is the Mondstadt, which is designed against the background of medieval Germany. Visually, the building refers to a large number of medieval German architectural styles, and people can see a large number of buildings based on German red tile skeleton houses on the map. In terms of hearing, the music in the region adopts the Lydia style, and the soundtrack is based on traditional orchestral music, and the Irish whistle, guitar and other plucking instruments with Nordic national characteristics are added. The music of the region has obvious European characteristics, and the melody is full of ancient fantasy [10]. Next is the Liyue area, which is designed with ancient China as the background. Visually, large areas of bamboo forests, terraces and steep peaks can be seen in the game, and the architecture adopts the style of ancient Chinese architecture. Chinese-style stone lion statues, inns and other characteristic buildings can be seen everywhere in the game. In terms of hearing, the music in the region adopts the Chinese style. In the soundtrack, the composer adds Chinese national musical instruments such as Guzheng, Pipa, Erhu and Flute, making this part of the score more distinctive Chinese classical characteristics. The elegance and crispness of the flute, the softness and gentleness of the erhu, and the noisiness of the pipa are woven into classical scrolls full of the Chinese style in the interweaving and integration with Western orchestral music [10]. The third is the Inazuma area, which is designed in Japan. Visually, the building adopts the architectural style of the Japanese shogunate period. The courtyard adopts the Japanese courtyard- style withered landscapes. There are not too many decorations and trees in the courtyard. It only uses simple elements such as white sand, rocks, green moss, etc., and has a strong Japanese style. In terms of hearing, Japanese musical instruments with Japanese characteristics such as Chiba, Japanese kite and three strings are added to the music. Against the background of the whole orchestra, it was a little more melodious, as if people could raise their eyes and see the cherry blossoms of the whole rice wife city flying all over the sky [10]. Finally, there is the Sumeru area, which refers to the culture of Arabia, ancient Egypt, India and other regions. Visually, people can see a large number of rainforests, deserts, architecture in Kerala, India, Arab patterns, etc [11]. Based on the hearing, Neidi, Udeqin, Mandolin... dozens of musical instruments are added to the music, which together play exotic rhyme. The combination of the soundtrack and visual styles in four different regions shows that game music plays an important role in describing the characteristics of different regions. The combination of vision and hearing also gives players a more immersive experience. 4.2 Music Visualization in Different Game Times There is a time system in the game Genshin Impact, which is mainly divided into four time periods: dawn, day, dusk and night. The composer of Genshin Impact Music has created different music for different periods of time based on the characteristic background music of the whole region, so that the music of each period has different changes, but the music of each period of time still maintains a unified melodic tone. During the game, as time changes from day to night, the sun and moon in the sky will move with time; roadside lights and shops will also light up with the change of time. At the same time, music will also change from lively and light during the day to quiet and soft. So when people listen to music at different times in the same region, they will feel obvious differences. Taking the city "Mondstadt" as an example, the composer wrote Bustling Afternoon of Mondstadt, The Dusk of Mondstadt, Mondstadt Under the Starlight, Another day of Mondstadt and many other musics. In music depicting the day, a lively and relaxed melody is played by playing and sweeping strings with plucking instruments, while playing lively and relaxing melodies with traditional orchestral instruments and Irishs [10]. From the music, people can hear that people in Mondstadt are living a busy and happy life. When night falls, the music loses the strong rhythm. A piano has been added to the soundtrack, and the sound of the piano always reveals a little comfort and leisure, BCP Education & Psychology CEPC 2022 Volume 8 (2023) 76 reflecting the stable and happy life of people after being busy. By dawn, the soft melody in the music seemed to depict Mondstadt City in silence. The long night was over, dawn was approaching, and a new day would come in peace. From the changes in "Mondstadt" urban music, people can feel that the music changes in time can provide players with a strong sense of immersion in the game. Music plays a crucial role in describing changes in time, which shows that the visualization of music in the time system of Genshin Impact is very prominent. 4.3 Music Visualization in Different Game Scenes There are many different game scenes in Genshin Impact, which are roughly divided into two scenes: ordinary exploration and combat scenes. Music in ordinary exploration is the basic scene music, and after entering the battle scene, the music becomes more intense and hot-blooded. While the music rings, players can quickly experience the intensity of the battle, which greatly enhances the player's immersion. Alex Moukala, a music producer who participated in the music production of Avengers, also analyzed the battle music with the boss Azhdaha in Genshin Impact, and also gave high praise. In the whole process of the boss battle, the battle music has also changed. The music of each battle stage has different battle music, so that players can well feel the battle rhythm and changes in the battle stage in the boss battle. In this way, players have a strong sense of immersion in the battle process. The music of the first stage tells that the Azhdaha is strong and has a cruel personality, mortals can't match him. This Azhdaha has brought a lot of trouble to Liyue and spent a lot of effort to seal it. He must be extremely careful when fighting. In the second stage, the style of music changed from a ferocious, horrible and oppressive tune to a hot-blooded melody. It seems to be saying Azhdaha was not like this in fact. He was once the most reliable ally of the Liyue people. He was once a powerful and gentle god like the emperor, although the heroes of the past fell. It looks like this, but he still has goodwill toward Liyue and defeats him with respect. With the change of music in the battle scene, people can feel that the combination of game graphics and music complements them. It is very important to describe the role played by music in battle. Players can also experience the same changes in music while experiencing the visual changes of the boss. The music created according to the boss’s background also has a prominent performance in music visualization. 5. Conclusion Through the above research on music visualization and the analysis of music visualization case data, it can be found that in the social communication environment of new media with the Internet as the main body, music is no longer consumed and appreciated as an independent art form, and music has begun to merge with other similar art forms and feel more. The form of officialization appears in the new Internet consumer market. Music visualization not only makes up for the shortcomings of music in figurative narrative and picture description to a certain extent, and enhances the emotional rendering ability of music, but also limits the illusory and subjective aesthetic image of music freely associated with the spiritual world to a certain extent. In addition, music visualization, a multi-sensory artistic expression, also enhances the audience's memory of music and video content, transmitting music art in a more understandable and profound way. Through the study of current music visualization, the authors find that the road of music visualization will further move towards the road of immersion and realization in the future. The development of virtual reality technology and the popularization of related consumer equipment have begun to promote the development of related fields in the direction of virtual reality. At the same time, music visualization will further improve itself on the virtual reality side. In addition to the research results, this study also has many shortcomings, such as being too monotonous in case analysis and not summarizing a large number of case data. At the same time, the BCP Education & Psychology CEPC 2022 Volume 8 (2023) 77 research on the social impact of music visualization is unequal. The impact of music visualization is not analyzed from the social level, and there are also gaps in research in the frontier field of music visualization. In the future, the research of this project will continue to carry out in-depth research in several fields, such as case analysis summary, social research and future industrial development direction, to obtain more detailed and objective research reports. References [1] Denis McQuail. McQuail's Mass Communicaiton Theory. Tsinghua university press, 2006. [2] Huang Xiaozhong, Yang Xiaohong, Feng Gang. Interpretation of key terms in communication. Sichuan University Press, 2005. [3] Ma Jie, Tian Jingwen, Liu Jian. Streaming media technology and its file format. Computer Engineering and Applications | Comput Eng Appl, 2003(23): 49-52. [4] Yang Ge, Liao Jianxin, Zhu Xiaomin, Fan Xiumei. Overview of key technologies in streaming media distribution system. Acta Electronica Sinica, 2009, 37(01): 137-145. [5] IFPI. Global Music Report 2022 - State of The Industry. [6] Zhao Bo. The historical trajectory, classification and social factors of MV's birth. Music Communication, 2015(02): 78-86. [7] Hua Jing. Audiovisual artistic characteristics of MV. Communication and Education, 2012(03): 40-43. [8] Cai Jianchun, Chen Xun. Interaction between game music and video games. Song of the Yellow River, 2019(21): 144. [9] Quan Weihua. The fusion of online games and background music. Private Science and Technology, 2012(11): 232. [10] Di Dapeng,Yao Siwei. The depiction function of map music in RPG games - Take Genshin Impact as an example. Cultural Monthly, 2022(01): 145-147. [11] Xuan Jin. The "Genshin Impact" game concert set off a new trend of "going to sea" and led players around the world to walk around the cultural "Silk Road". Wen Hui Daily, 2022-09-06 (005).