BCP Education & Psychology ESPS 2022 Volume 4 (2022) 64 Research on the Online Classroom Mode of Chinese Teaching Based on Multimodal Discourse Theory Chengyue Liu School of Literature and Journalism, Yantai University, Yantai 264000, China ddsdbd@163.com Abstract. Under the background of the global COVID-19 pandemic and the limited entry of international students, the number of overseas students in China has decreased significantly, and the teaching Chinese as a second language (TCSL) has entered a slow development stage. Due to the low learning efficiency of online classroom teaching and the lack of support for the target- language environment, online TCSL cannot be popularized and promoted. To solve the drawbacks and develop a new way of online teaching, this paper combines multimodal theory with the online class of TCSL to explore the new teaching model. Keywords: Multimodal Discourse; TCSL; Online teaching. 1. Introduction In recent years, due to the popularization of information technology, people's communication modes are increasingly diversified. New modes of video conversation and image-text sharing have gradually evolved from the traditional verbal and writing communication. Because of this transformation, the non-verbal forms, such as images and videos, which were originally in a subordinate position, get wider attention as the objects of scientific research for the first time. Therefore, the multimodal discourse theory emerged. Multimodal discourse refers to the phenomenon that multiple modalities (vision, hearing, touch, smell and palate) cooperate with each other to complete meaning construction in the process of communication. Many situations in life are realized by the cooperation of multiple modalities, among which the two closely related to discourse analysis are audio modality and visual modality. In most occasions, people must rely on the joint operation of two or more modalities to complete the reception and the output of information. For example, when communicating with people, in addition to receiving language information (audio modality), sometimes we need to combine the other's body language or gestures (visual modality) to understand the contents of conversation; when watching movies, we need not only to watch the pictures (visual modality) on the big screen but also listen to the actors’ lines (audio modality) to help us understand the plot; when teaching in the classroom, teachers often repeat the contents on the PPT orally (audio modality) to strengthen students' memory while displaying PPT (visual modality). Kress, G. and van Leeuwen, T. (1996) first put forward the concept of multimodal discourse back in the 1980s and published a related book Multimodal Discourse: The Modes and Media of Contemporary Communication in 2001. They believed that each modality could construct its own grammatical system like language, and proposed theories such as the grammar of color and visual design; R. Barthes, also one of the earliest researchers, published his paper Rhetoric of the Image in 1997, which explained the interaction between images and language, pioneering the study of multimodal discourse theory. The multimodal discourse has absorbed the relevant theories of Hallidayan Systemic Functional Linguistics (SFL), Semiotics and Critical Discourse Analysis (CDA), with remarkable interdisciplinary characteristics. With the help of the theoretical framework of SFL, it is mainly composed of five levels (Martin 1992) namely (1) the level of content of cultural: including ideology and language structure genre as cultural context; (2) the level of content: including the context of situation consisting of field, tenor and mode of discourse; (3) the semantic level: including discourse meaning consisting of conceptual meaning, interpersonal meaning and layout meaning; (4) BCP Education & Psychology ESPS 2022 Volume 4 (2022) 65 the formal level: including different modalities used to express communicative meaning, and their grammatical systems, as well as the relationship generated by the interaction between different modalities, which can be divided into two categories, complementary and non-complementary; (5) the level of media: it is the final realization form of discourse in the real world, including two categories of language (accompanied language and pure language) and non-language (body and non- body). In teaching, it is mainly the content of form and media that has a direct impact on the class, which is also the focus of this paper. In recent years, the related literature on multimodal research in China is increasing day by day, and people pay more and more attention to the practical application of this theory. Many scholars have conducted a lot of analysis and discussion according to their experience and opinions. Zhu Yongsheng (2007) analyzed the theoretical basis and methodology of multimodality; Li Zhanzi (2003) illustrated the multimodal theory from the perspective of SFL; Zhang Delu (2010) advocated the combination of multimodality and foreign language teaching in our country, proposed the multimodal selection program, and advocated the application of modal synergy and media technology to the classroom teaching, which broke the traditional teaching models; Jiang Yanyan and Li Dengqiao (2016) discussed the possibility of introducing multiple modalities into the classroom of TCSL, and conceived the models under the multimodality. At this stage, the concept of multimodality has developed from theoretical research to a new stage of practical application, which provides a solid basis for discourse analysis and classroom teaching in the multimedia network era. At present, the research on how to apply multimodal theory to foreign languages and TCSL in China has achieved certain results, with a significant impact on the actual classroom teaching. However, due to the influence of the global COVID-19 pandemic in the past two years, the development of offline TCSL in China has slowed down, and a large number of colleges and universities have turned to online teaching. Nevertheless, because the online teaching model is still in its primary stage, problems occur such as students' poor efficiency, insufficient interest, blocked interaction, lack of target-language environment. Therefore, this paper tries to take the multimodal discourse theory as a framework and combine it with online teaching, a new class type of TCSL, to solve problems in the teaching process. 2. Multimodal Discourse Theory and Online TCSL The traditional TCSL is mainly based on offline classes. Teachers can have face-to-face communication with students, track students' class performance in real-time, understand their classroom learning and interact effectively with them. Additionally, teachers have more options for the application of teaching methods. In an offline teaching environment with a small space, they can use body gestures and sign language, or create context with the help of multimedia to assist in teaching. However, when classes are shifted to online teaching, great changes will be made to teachers' teaching models. 2.1 Problems in Online TCSL The offline class has formed a relatively complete teaching process in many practices. However, under the background of the pandemic where “teachers can't get out, students can't get in", it is difficult to carry out offline teaching. The online teaching model has developed rapidly and become the current mainstream with the help of online teaching platform. However, in the process of practical operation, many problems arise from online teaching. After our observations, from perspectives of teacher and students, the following problems were identified: (1) Teachers perspective: limited interaction and unable to get timely feedback. Teachers cannot interact directly with students face-to-face, nor can they observe students' reactions while giving lectures. Especially in pre-recorded video lectures, teachers cannot communicate with students, which widens the estrangement between both sides, resulting in reduced teaching effect. BCP Education & Psychology ESPS 2022 Volume 4 (2022) 66 (2) Student perspective: Inefficiency and lack of interest in learning. Due to the inadequate simulation context of online courses and the lack of the contextual background of the target-language country, the international student community generally repels online teaching, which is reflected in the dramatic decline of the number of international students and obvious reduction in teaching efficiency. At the same time, students cannot engage in class with rigid and monotonous teaching methods and will lose interest. 2.2 Online Class and Multimodal Discourse Theory of TCSL The multimodal discourse theory provides the theoretical basis for the reform of the offline class teaching model, which transfer it from a single model to the co-existence of multiple modalities. Through verbal media and non-verbal media, with the help of PPT, video, animation, online resources, a more vivid communication environment has been created for language learning, which has achieved remarkable results in strengthening knowledge learning and attracted students' attention. With the successful experience of offline education, it is imperative to combine multimodal discourse theory with online teaching. 2.3 The Construction Principle of Multimodal Discourse Theory in Online TCSL Given the many problems in online teaching, this paper holds that the following four principles should be followed in multimodal online teaching: Utilize multimodal collaboration to drive teaching and create a better language learning environment. In online teaching, audio and visual modalities play an absolute leading role. Therefore, the importance of images and video should be emphasized, and teachers should select information materials that fit the real environment of the target-language country to make up for the lack of context. Strengthen exercise links, enhance interaction and improve efficiency. In the online environment, teachers' writing is greatly limited, and multimodality can save the time on blackboard writing; at the same time, the combination of each mode can provide more exercise methods in a more flexible way, so that teachers and students can interact. Enhance students' interest in learning and stimulate learning motivation. Compared with the traditional blackboard writing + textbook-based teaching model, multimodality can provide more flexible, vivid and interesting learning methods, so that students can think outside the box, become interested in the target-language culture and integrate into the environment of the target-language country. Separate teaching from exercise. Teaching is mainly based on pre-recorded video lectures while exercise is mainly based on the platform-assisted exercise module to maintain teaching consistency. Considering the chaotic speech, frequent noise, poor network connection and other conditions in online teaching exercise, as well as the time difference restriction, teachers can develop teaching classes with the help of recording function of major teaching platforms, and organize interactive exercise after the teaching class. 3. Design of Online Model of TCSL Based on Multimodal Discourse Theory The realization of online teaching is based on the development of multimodality and network communication. However, the main problem of online teaching at present is that it cannot use multiple modalities reasonably to effectively teach courses, resulting in teachers' rejection and students' low acceptance. The class design of this paper aims to solve the above two problems and create a universal teaching framework suitable for various platforms. Nowadays, the pre-recorded video lecture is the most popular online teaching method, so this paper only discusses the online teaching model based on asynchronous teaching. The following are the specific teaching steps. BCP Education & Psychology ESPS 2022 Volume 4 (2022) 67 3.1 Preparations before the Class Hardware Facilities Online teaching platform: The teaching platform referred to here can be divided into professional and non-professional platforms. The professional platform refers to the online platform specially used for TCSL, which has complete functions and is easy to use; the non-professional platform refers to the educational platform that can carry out the teaching of multiple disciplines, such as Tencent courses and APPs developed by major schools. Pre-recorded Classroom: refers to a multimedia classroom specially used for recording lectures. Teachers can directly use computers as teaching tools or use professional microteaching classrooms to record lectures by using large screens, cameras, video recorders and other equipment. Presentation of Online Teaching Modal Types According to the theory of multimodal discourse analysis, the modal types presented in an online class can be divided into five main modules: Multimedia module: including PPT slides, images, audio, interactive videos, remote interactive dialogues, virtual scenes, online platform exercises, games, supervision and other modes; Body movement module: including teachers' expressions, intonation, speed, gestures, movements and other modes; Discourse module: including Q&A, repetition, narration and other modes; Object module: including the mode that teachers display objects in class; Blackboard writing module: including the mode that teachers simulate the blackboard writing in the form of typing and other modes. Selection of Teaching Materials and Lesson Preparations When traditional offline teaching materials are used for online teaching with a short teaching period, it is likely to separate texts and knowledge points. Therefore, teachers should choose the appropriate teaching materials. When preparing lessons, teachers can do it collectively or individually, determining teaching contents, key teaching points, difficult points and teaching methods according to the types of students. In addition, teachers should prepare relevant hardware equipment to keep the environment quiet and ensure network connection. Finally, teachers should plan class time for students in advance, require them to complete preview and homework tasks, and keep order in class. 3.2 Design of Multimodal Online Model of TCSL As stated above, it is impossible to solve many problems in online teaching which are different from offline teaching by simply applying traditional teaching theory to teach. At the same time, whether online or offline, the most important purpose of language learning is to realize communication. In online teaching, we should emphasize communicative intention, with a focus on “exercise” and supported by “teaching”. At the same time, for online teaching, due to the lack of teachers' supervision, students are easily distracted in class, so the traditional 90-minute continuous class is not suitable for unsupervised online pre-recorded video lectures. From the author’s experimental results, we can set the teaching time to 45 minutes per session, with a 45-minute platform exercise. In this way, teachers can effectively improve students’ class efficiency. The specific teaching steps of online teaching are designed as follows: Lead-in of New Lesson (5 minutes) Have a simple review by PPT, supported by blackboard writing and teachers’ body language. Teachers can make students review the last lessons with the help of the blackboard writing outline summarized in the last class, or teachers can connect the knowledge points in series under a context. Since it is a pre-recorded video lecture, teachers cannot ask students questions, but only review the knowledge of the previous class. BCP Education & Psychology ESPS 2022 Volume 4 (2022) 68 After that, teachers can introduce the new lesson based on the contents of the previous lesson. At the same time, before playing the video, teachers should first ask questions about relevant knowledge points, so that students can watch the animation with questions and arouse their thinking. Playing the Video (5 minutes) When preparing lessons, teachers should first choose words with strong relevance and incorporate them into situation animation. Before explaining words and grammar one by one, they should play the animated videos containing the knowledge points students will learn, so that students can have a certain understanding of the contents to be learned in a scene reconstruction form, and can also master the usage and context of vocabulary and grammar. Words and Grammar Explanation (20 minutes) Compared with offline classes, the knowledge points in online classes should be carefully selected due to time constraints. The main teaching contents for each class should be 4-5 words and a specific grammar point. Knowledge points should be split and taught in separate classes, focusing on quality instead of quantity. In the teaching process, teachers should fully mobilize students' visual and auditory systems and sense of participation. They can use multimedia technology to create virtual characters, sitcoms, audio, image and physical object display to reproduce the real context so that students can quickly integrate and master the knowledge they need to learn. At the same time, when preparing lessons at the beginning, teachers can set a complete storyline for all the class contents, create virtual protagonists and character relationships, and take the life of protagonists as the theme of each lesson. This practice can make teaching more practical in daily communication, and can also stimulate students' interest. When teaching nouns or adjectives with actual meanings, teachers can use physical object or image display; while teaching words, such as verbs and adverbs, which are not easy to be presented directly, they can use short videos or action demonstrations, and create scenes orally to explain. When explaining grammar, the teacher can make sentences according to the scenes in the animation, and summarize the complete usage of sentence patterns according to a specific application. When learning vocabulary and grammar, students need a lot of example sentences to support them. Teachers can choose example sentences from video animation, and also give example sentences according to images to enrich the corpus. Teachers should repeat the sentences when explaining, slow down the pace of speech, deepen students' impressions, and ask students to read after them in front of the screen. With the methodology of “approaching parts from the whole", students can have a comprehensive understanding of knowledge points, helping them to review and connect the knowledge points. Moreover, it takes important and difficult knowledge points into consideration and takes them down one by one. Learning State Supervision (3 minutes) To make students concentrate and effectively supervise students' learning status, settings can be made on the platform to randomly give exercises related to the knowledge learned after pre-recorded video lectures are played for a certain period of time. Only when students answer the questions correctly can they continue to watch the lecture. The success rate of each student's answers will be recorded and reported to the teacher. Lesson Exercise (10 minutes) This link is to consolidate the new knowledge points. In pre-recorded exercise class, teachers can enhance students' memory of the new things by means of interactive video. The so-called interactive video is a new video mode of human-computer interaction. Students can choose from the options provided in the video uploaded by teachers, triggering different sub-line plots. Teachers can compile the vocabulary and grammar points learned in this lesson into videos and make them into multiple-choice questions. When students answer questions correctly and complete the test, it can promote the development of the plot. In addition to questions and answers, teachers can also set the voice mode in which the sub-line plot can be triggered only when the specified words are BCP Education & Psychology ESPS 2022 Volume 4 (2022) 69 pronounced accurately. This method can remotely check the mastery of knowledge and interact with students. Meanwhile, the interesting narrative video can arouse students' interest and help them concentrate on knowledge points. Table 1. Visual Design Table of Multimodal Chinese Online Class Framework Teaching Link Teaching Contents Modality Used Teaching Time Lead-in of New Lesson Review the knowledge points of the last class; Introducing new lesson; Put forward 3-4 questions related to video knowledge points. PPT modality Blackboard writing modality 5min Playing Video Play the new lesson video； Pay attention to the new words and grammar points in the video. Audio modality 5min Words and Grammar Explanation Explain words and grammar knowledge points one by one. PPT modality Virtual Scene Modality Physical Object Modality Body Movement Modality Image Modality Discourse Modality 20min Student State Supervision Supervise students' status in class; Put forward 1-3 questions involving knowledge points. PPT Modality Supervision Modality 3min Lesson Exercise Practice and consolidate the words and grammar you have learned Interactive Video Modality Game Modality 10min Summary and Homework Summarize the knowledge framework of the new lesson; Preview the contents of the next class; Submit homework to the platform for correction PPT Modality Blackboard Writing Modality Audio Modality 2min Platform Exercise After Class Complete homework after class; Rely on platform resources for extended exercise; Share learning experience and Q&A Multimedia Platform Modality Discourse Modality 45min In addition, games can also be used as an exercise method. It is often used when learning nouns, numerals and other words with specific meanings. For example, when learning directional adverbs, teachers may have students to drag objects with the mouse according to the prompts and put them in the correct directional position; drag the correct costumes from the wardrobe with the mouse to match them according to the needs of virtual characters, and other game forms. Summary of New Lessons and Assignment of Homework (2 minutes) Teachers can play multimedia animation again to briefly review the knowledge they have learned, or list the words and grammar systems in the form of blackboard writing to help students build a knowledge framework. At the same time, teachers can assign the homework, preview the new lesson, and let the students finish the homework and upload it to the designated place on the platform. Online Platform Exercise (45 minutes) BCP Education & Psychology ESPS 2022 Volume 4 (2022) 70 In addition to the online teaching function, the online teaching platform should also set up a special exercise module. Teachers should arrange platform homework and exercise of corresponding duration for students after the pre-recorded video lecture to achieve the purpose from input to output, from learning to communication application. The online platform exercise module can be divided into three parts: Curriculum internal resources section This section is only open to students and teachers in fixed courses. Teachers can share courseware or videos, publish exercises, correct homework and other operations; students can also view the courseware here for preview and review, and submit their homework for teachers to correct. Extracurricular learning section This section is open to all users of the platform so that students can get more extracurricular learning materials. In addition to Chinese books, songs, interesting popular science and other corpus, the platform should set up five sections of listening, speaking, reading and writing and vocabulary memory exercises (or allow teachers to upload corresponding training) to help students improve their communicative competence simultaneously. Teacher-student Interactive Q&A Forum The platform can also set up a BBS where students can share their learning insights, learning experiences and difficult problems, enhancing the teacher-student and student-student interactions. Table 2. Flow Chart of Multimodal Online Class of TCSL Process Lead-in of New Lesson Playing Video Words and Grammar Explanation Student Status Supervision Lesson Exercise Summary and Homework Platform Exercise After Class Time 5min 5min 20min 3min 10min 2min 45min In a word, by using multimodal teaching theory to guide online teaching, giving full play to new Internet technologies such as multimedia audio, virtual characters, animation and interactive video, we can solve the problem of the time difference, building a professional asynchronous teaching platform for TCSL. Thanks to its flexible time and the diverse methods, this online class can also overcome the disadvantages of online teaching and improve learning efficiency. In this way, China can acquire more overseas learners for online class of TCSL, promote the communication of Chinese culture and language with other nations’ culture and language, and achieve the goal of strengthening the country by culture. 4. Conclusion With the advent of the Internet + era, new media technology is changing with each passing day and developing rapidly, which provides technical support for the emergence of new teaching models. However, in the practice of online TCSL at the present stage, due to poor timeliness, weak interaction and insufficient construction of related platforms, online TCSL has not gained enough attention. To get rid of the realistic background that online teaching has long been attached to off-line teaching, and to enable overseas students to have access to high-quality Chinese learning class, this paper combines multimodal teaching theory with online class, gives full play to the advantages of online teaching platform, designs a two-modality teaching model of "pre-recorded video lecture and platform exercise". We comprehensively use new Internet technologies, animation videos, interactive videos, practice games, virtual scenes and other multi-modal in the online class. It committed to creating a good online Chinese cultural context for students, so that students can get rid of time, space and geographical restrictions, and realize the flexible learning form of "learn anytime, anywhere you want". BCP Education & Psychology ESPS 2022 Volume 4 (2022) 71 Online learning is in an emerging stage. This paper is an exploration and assumption of the online teaching model based on multimodal discourse theory, which inevitably has shortcomings. It is hoped that more experiences and lessons can be learned in future teaching practice, and receive more criticism for improvement. In the future, we will continue to improve the online class construction of TCSL, keep up with the development of the times, and actively explore new models, promoting Chinese and Chinese culture to go abroad and share with the people of the world. References [1] Chen Xin. Design and Research of Teaching Mode of Chinese Audio-visual Course Based on Multi- modal Theory Framework [J]. Journal of Yunnan University (Natural Sciences Edition), 2020, 42(S1): 116-122. [2] Fu Ruolan. Knowledge Crowdsourcing of Interactive Videos: A Research on the Knowledge Community of the Bilibili Website [J]. Journal of Southwest University (Social Sciences Edition), 2021, 47(06): 190- 199+260. DOI: 10.13718/j.cnki.xdsk. 2021.06.020. [3] Jiang Yanyan and Li Dengqiao. On Application of Multimodal Discourse in TCSL--A Case Study of "Ba" Sentence [J]. Research in Teaching, 2016, 39(04):91-93. [4] Li Zhanzi. Social Semiotic Approach to Multimodal Discourse [J]. Foreign Language Research, 2003(05):1-8+80. [5] Ma Lifang. Research on The Teaching Mode under the Multimodal Discourse Framework [J]. Theory and Practice of Education, 2016,36(01):57-60. [6] Zhang Delu and Li Yuxiang. Modality Synergy in Multimodal Classroom Discourse [J].Foreign Languages and Their Teaching, 2012(01):39-43.DOI:10.13458/j.cnki.flatt.000461. [7] Zhang Delu and Wang Lu. The Synergy of Different Modes in Multimodal Discourse and Their Realization in Foreign Language Teaching [J]. Foreign Language Research,2010(02):97-102. [8] Zhang Delu. On A Synthetic Theoretical Framework for Multimodal Discourse Analysis［J］．Foreign Languages in China, 2009(1) : 24-30． [9] Zhang Delu. Multimodal Discourse Theory and Its Application to Foreign Language Teaching with Modern Media Technology ［J］．Foreign Language Education, 2009(4):15-20. [10] Zhu Yongsheng. Theory and Methodology of Multimodal Discourse Analysis [J]. Foreign Language Research,2007(05): 82-86.DOI: 10.16263/j.cnki.23-1071/h.2007.05.034. [11] Kress, G. and van Leeuwen, T. Reading Images: The Grammar of Visual Design [C]. London: Routledge, 1996. [12] Kress, G.& van Leeuwen, T. Multimodal Discourse: The Modes and Media of Contemporary Communication [C]. London: Arnold, 2001. [13] Martin, J.R. English Text: System and Structure [M]. Amsterdam: John Benjamins, 1992. [14] Royce, T. Multimodality in the TESOL classroom: Exploring visual-verbal synergy [J]. TESOL QUARTERLY, 2002, 36(2). 191-205. 1. Introduction 2. Multimodal Discourse Theory and Online TCSL 2.1 Problems in Online TCSL 2.2 Online Class and Multimodal Discourse Theory of TCSL 2.3 The Construction Principle of Multimodal Discourse Theory in Online TCSL 3. Design of Online Model of TCSL Based on Multimodal Discourse Theory 3.1 Preparations before the Class 3.2 Design of Multimodal Online Model of TCSL 4. Conclusion References