Linguistic, English Education and Art (LEEA) Journal Volume 4 Nomor 1, Juli-Desember 2020 e-ISSN :2597-3819 p-ISSN:2597-9248 DOI : https://doi.org/10.31539/leea.v4i1.1416 99 EXPLORING DIGITAL MULTIMODAL TEXT IN EFL CLASSROOM: TRANSFORMED PRACTICE IN MULTILITERACIES PEDAGOGY Yuslimu Ilmi 1 Universitas Negeri Surabaya Pratiwi Retnaningdyah 2 Universitas Negeri Surabaya Ahmad Munir 3 Universitas Negeri Surabaya yuslimui@gmail.com 1 Submit, 17-07-2020 Accepted, 22-12-2020 Publish, 28-12-2020 ABSTRACT This study investigates EFL students‟ composition of digital multimodal text. There were forty-four students recruited from tenth grade of a private senior high school in Sidoarjo. All of them were divided into seven groups and given authority to choose their own topic for the digital multimodal project; advertisement video. Based on the theory of multimodal analysis, this study examines both students‟ processes and products. Qualitative case study was chosen as the design of the study and document analysis was chosen as the data collection technique. Through qualitative case study, the finding of this research shows that all the students‟ groups used multimodal modes in creating their advertisement videos. Additionally, this study revealed that the qualities of the students‟ project depend on two important things: the ability of the group members and the collaboration of the members in doing the project. Keywords: Multimodal Text, Multiliteracies Pedagogy, Transformed Practice, EFL Classroom, Senior High School. INTRODUCTION In the twenty-first century, education is faced with the sophisticated technology and advance communication because people are now living in a global society with the increasingly different local contexts. This new environment changes the concept of text from reading and writing to be multimodal (Mills, 2008). Consequently, education needs to improve the quality of leaners that have high level of creativity to understand multimodal text. It is because multimodalities are always integrated with the advanced technology where mailto:yuslimui@gmail.com 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 100 information comes from many sources in different forms (Boling, 2008; Cope & Kalantzis, 2009). As a part of education, English language teaching must be aware if the global context. The curriculum should be related to many different subjectivities (Kellner, 2001). English teachers should be equipped with the appropriate skill, strategies and insight to cope with the different views of literacy and to support their students successfully (Boche, 2014). They not only need to include the students with all their diversities in situated contexts, but also need to engage them in multiliteracies (Davis, 2008). It has been considered that multiliteracies pedagogy is a meaningful way that can effectively engage students in teaching and learning practice by offering four benefits. First, it strengthens the relationship between teacher and students. Second, it increases the inclusivity for multiplicity. Third, it develops performances of literacy practices, and last it creates positive classroom community (Doucette-vanthuyne, 2016). The concept of multiliteracies pedagogy is suitable with the twenty-first century education. It has four major components which can be implemented in teaching practices. Those components are situated practice, overt instruction, critical framing, and transformed practice (New London Group, 2000). Every component has its important role in teaching and learning practice. They do not stand in linear hierarchy, but each of them can happen repeatedly, randomly, or simultaneously in complex ways (Mills, 2006). Transformed practice is the climax of multiliteracies pedagogy. In this section, students are expected to apply the knowledge they gained (from situated practice, overt instruction, and critical framing) appropriately and creatively (Mills, 2006). Applying appropriately involve some activities such as writing, drawing, acting out in the „common way‟ and solving problem. While applying creatively can be implemented through using knowledge innovatively, taking an intellectual risk, applying knowledge in different setting, offering a new problem, translating knowledge into different mix of „modes‟ of meaning (Pullen & Cole, 2010). The implements for multimodal compositions have been developed together with emerging technologies. In the twenty-first century, students have discovered various ways of using media to communicate, collaborate, and create in the most of digital space (Alvermann, 2015). Therefore, as one of the component in the process of “applying creatively”, research on multimodalities need to be conducted. Much research has been done on multiliteracies pedagogy and a substantial numbers of studies have investigated the multimodal texts. (Shin, et.al, 2020) examined the digital multimodal text of writing composition of L2 learners with the development of their metalanguage. Such examination have focused on 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 101 learners‟ uses on multimodal text, types of multimodal writing task for undergraduate students, and some ways to engages teachers in exploring multimodal text (Anstey & Bull, 2014; Jocius, 2013; Lim & Polio, 2020) . a study on ESL students was done by Ganapathy & Seetharam (2016) to examine the effectiveness of multimodal approach in ESL classroom. Shepard-carey (2020) has started research on an emergent bilingual student in demonstrating and responding to reading comprehension text using various semantic texts. While a research among college Freshmen students has been worked by Jacobs (2012) to examine the multimodal text and academic literacies from an expository video created by students. To address the research gap, the current study examines tenth-grade EFL learners‟ composition, both processes and products, of digital multimodal text. In particular, it highlights the use of more than one mode in the digital multimodal text created by the EFL learners in form or advertisement video. LITERATURE REVIEW Multliteracies Pedagogy The pedagogy of multiliteracies has four major dimensions. Those four major dimensions are called as situated practice, overt instruction, critical framing, and transformed practice (New London Group, 2000). Situated practice means the involvement of meaningful practices that able to relate to students‟ experience and background. Through situated practice, teacher needs to construct the students‟ life world experience and putting the meaning-making process in the real world context (New London Group, 2000). Overt instruction is defined as an active interaction between teacher and students that helps students to understand what they learn (Henderson, 2012). Critical framing is similar with analyzing the social and cultural meaning of text by assessing particular design of meaning and transformed practice is transferring the previous design to create a new design of text in different context and cultural sites (Kalantzis & Cope, 1997). Picture 1. Components of Multiliteracies Pedagogy 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 102 Multimodality Multiliteracies pedagogy concerned with the use of multimodal layers of learners‟ world in the classroom to engage students with the tools and technology that they are already familiar with. Multimodal meaning is the other modes of meaning working together; linguistic, visual, spatial, gestural, and musical (New London Group, 2000). It involves the process of integration and moving the emphasis backward and forwards between the several modes. 1. Linguistic mode includes written or spoken words. This mode includes the choice of words, the delivery of written or spoken words, the organization of words into sentences and sentences into paragraphs, and the development and coherence if words and ideas. The examples of linguistics mode are: narrative text type, report writing, script writing of a role play or radio talk script (Pullen & Cole, 2010). 2. Visual mode refers to the images and characters that people see. Visual includes the screen format, view, still or moving image, sculpture, and page- layout. visual mode also discuss about the props, vector, size, shape of images, color and white space, borders, costumes, perspectives and maps concept (Pullen & Cole, 2010). 3. The audio or musical mode focuses on sound including music, sound effect, ambient noises, silence, and tone of voice, volume of sound or accent (Pullen & Cole, 2010). 4. Gestural mode refers to the way movements are interpreted. It focuses on behavior (group/ individual), posture, facial expression, hand gesture, body language, items, artifacts, or kinesics in a role play (Pullen & Cole, 2010). 5. The spatial mode refers to the arrangement of elements in space. It involves the organization of items and the physical closeness between people and objects. For instance, the different ways in which chair and desks are arranged in classroom, the position of performer in a roleplay, movement, proxemics, layout of collage, layout of PowerPoint (Pullen & Cole, 2010). RESEARCH METHOD This study is based on qualitative case study. The subjects of the study were recruited from an EFL classroom consists of forty-four students‟ at the 10 th grade. All the students were divided into seven groups that each of them produced an advertisement video as the implementation the transformed practice to access multiliteracies pedagogy. The study was conducted in one of private senior high school in Sidoarjo. This school was chosen because it is one of private school that has supporting items and facilities for the implementation of multiliteracies pedagogy. The data were collected through observation and document analysis. 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 103 The data were analyzed and processed based on these steps: familiarizing and organizing, coding and reducing, then interpreting and representing FINDING Table 1. Example of Video Transcription Made by First Group Scene# (time stamp) Linguistic mode (written) Linguistic mode (oral) Audio mode (sound and music) Visual mode (still and moving images) Spatial mode (arrangeme nt of elements) Gestural mode (Gaze or behaviour) 1: in a school park (00.01- 00.23) Voice- dubbing A girl is sitting in the school park. A boy comes and insults her. Two boys come and do the same thing There is a table surrounded by three chairs made from cements. Next to the teable ans seats there is a tree. The male students are pointing and looking at the girl sarcastically Female student is sad (00.24- 00.32) We have a solution on the drinking Hilo. This is Hilo Voice- dubbing Three girls are standing back to a fountain and having conversati on The fountain is located behind the girls Showing a bottle of milk Showing a sachet of Hilo chocolate banana. (00.33- 00.41) Instrument al music plays throughout ; it has a pleasant tone and seems to be both strings and piano. There is a display of a sachet Hilo chocolate banana with a bottle of milk added with decorative glasses lying next to it Hilo chocolate banana is leaned in front of a bottle of milk. Then a glasses is put next to it. 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 104 The first group advertised milk. The video included five modes. Those five modes are oral linguistic mode, audio mode, visual mode, spatial mode, and gestural mode. The oral linguistic mode was shown through the statement „we have a solution on the drinking. Hilo.. this is Hilo’. And the audio mode was in form of voice-dubbing and instrumental music. For the visual mode, the video displayed a girl who was insulted by the boys, and then she was helped by her friends. Her friends entertained her and give her suggestion to drink Hilo. The spatial mode supported the visual mode. It showed a table surrounded by long chairs all are made from cement. The table and chairs are located near a three. Some gestural mode also found when the boys insulted and looked at the girl sarcastically, the girl felt sad after insulted by the boys. But then she felt happy again after drinking Hilo. The second group promoted bread in their advertisement video. There were four modes inserted to the video. Those are linguistic modes, audio mode, visual mode, and gestural mode. The linguistic mode found was a writing „2000 years later‟ and the dubbing sound saying the same words. For the audio mode, there was a song of Jessy Jay-Price Tag played at the end of the display. The visual mode showed a display of SMPTE colour bar at the beginning of the video, and then it shows some teenagers in the middle of yard eating Sari Roti bread. There were some gestures found from the visual mode such as: a boy touching stomach as if he feels hungry and two girls showing Sari Roti bread in front of the camera while saying “Sari Roti, bread Sari Roti.” The third group had decided to choose Eskulin cologne gel to be advertised. The modes used were linguistic mode, audio mode, visual mode spatial mode and gestural mode. There was a sound of a mobile legend game played while the visual showed two boys playing a game. Then voice-dubbing was used to complete the visual of some girl who are having a conversation in a class. A special effect was added. It was the displayed of the conversation when the boy was chatting with the girl via WhatsApp. The spatial mode displayed through some chairs and tables inside the classroom. The boys were sitting not far one another. For the other scene, four students were sitting like having a group project. Four tables are gathered in in the middle and four chairs were surrounding them. An example of gesture in the video was when a girl smells her body and one of her friend showing an expression of bad smell. At the end of the video there was found oral and written linguistic mode for the slogan of Eskulin cologne gel. The fourth group had produced the simplest video. It talked about candy and included linguistic, audio, visual, spatial and gestural modes. The linguistic mode was written as “Cold Mintz Exist”. The visual showed a girl who was studying and feeling hot. The girl fluttered her face to give a gesture mode. It was 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 105 supported with the instrumental music as the audio mode. The spatial mode found when the display showed three candies laid on a paper. The fifth group promoted a drink. It was fruit Tea. The modes included were only linguistic, visual, and gestural. The visual showed two students were reading books in front of the class. And then, one student came and gave them drinks. One student said “Many sensations, suitable for hot today”. That was included as the oral linguistic mode. And all of them showed the bottle of fruit tea to the camera as the gestural mode. The sixth group chose Sprite to be advertised. The video involved four modes: linguistic, audio, visual, and gestural. At the beginning, the video displayed a bottle of sprite with a bubble inside it. Supported with a linguistic mode “Hi guys let‟s think clearly” both written and oral. An audio mode is also inserted in form of instrumental music. For the next scene there was a boy standing in a basketball court. He puts a bottle of Sprite in his pocket, he drinks the Sprite. The gesture of the boy after drinking the Sprite was opening mouth widely as if he felt so fresh. All those activities were accompanied by voice- dubbing from the beginning to the end. The video then closed with the linguistic mode “Sprite is actually right, no more” both spoken and written. Last but not least, the seventh group created an advertisement video of Cornetto. It consisted of four modes: linguistic (oral and written), audio, visual, and gestural. Visually, there are two girls walking out from class. One of the girls fell down. Two boys walked by. One of them laughed at her, but the other one helped her and give her ice cream. The visual was supported with a back sound from chorus part of Make you Mine song by Public. Some written linguistic mode was also added such writing “Thank you Cornetto, I am happy”. At the end of the video, the girl showed a gestural mode that is a happy expression while she is holding cornetto. DISCUSSION From the findings described before, it can be concluded that all groups have used multimodal modes in the video. According to the New London Group (2000) when two or more meaning modes are working together, it is called multimodal meaning. From the results of the videos made by each group, most of groups involved four to five modes of meaning in the videos. Visual mode was mostly existed in all student products. Visual mode can be found in screen format, view, moving or still image, page-lay-out and sculpture. It sometimes discuss about color, shape, size, vector, boarders, etc. (Pullen & Cole, 2010). Related to visuals, of course gestural mode was also found in all advertisement videos made by the groups. From facial expressions, gestures and body languages, all get through the visual mode displayed on the screen. Kress (2003) and Westby (2010) said that gestural mode can be discovered 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 106 through; body language, face expression, gait, clothing and fashion, dance, hairstyle, eye movement and gaze, hands and arms movements. In addition, audio mode complemented almost all videos, except the Majapahit group. Audio mode was found in form of musical instruments, songs, or voice-dubbing. This mode is also called as musical modes which may discuss about dubbing, instruments, or probably a part of popular songs (Pullen & Cole, 2010). The linguistic mode involved was both in form of written and spoken. This mode was also included in almost the video. This mode includes the choice of words, the delivery of written or spoken words, the organization of words into sentences and sentences into paragraphs, and the development and coherence if words and ideas (Pullen & Cole, 2010). A little bit different from the other modes, spatial mode was minimally used in the videos. The spatial mode refers to the arrangement of elements in space. This meaning mode involves the organization of items and the physical closeness between people and objects (Pullen & Cole, 2010). There were two groups that did not include it in the video. The minimal use of modes in the advertisement videos were caused by inadequate quality of group members or lack of good cooperation between members. The different level of students‟ abilities certainly affects the quality of the video. Moreover, creating video requires some other skills, such as video editing, inserting sound effects, images or even slipping a chat process between two people. Therefore, good collaboration and contribution from all group members are needed. CONCLUSION In conclusion, all products from each group involve multimodal modes (linguistic mode, audio mode, visual mode, spatial mode and gestural mode). Although not all modes can be found in each video, but most of the products made have involved at least three modes of meaning (linguistic, visual and gestural), the rest have even used four to five modes at once. The differences in video quality, in terms of multimodality, are influenced by the ability of group members and also their ability to work in groups. These two things are mutually influential because groups that consist of proficient members do not certainly produce good quality products without good coordination and collaboration among the groups‟ member. While groups that have good cooperation and collaboration may make mistakes or deficiencies in providing materials without the support of good skills. 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 107 REFERENCES Alvermann, D. (2015). Why Bother Theorizing Adolescents ‟ Online Literacies for Classroom Practice and Research ? Journal of Adolescent & Adult Literacy, 52(1), 8–19. https://doi.org/10.1598/JA Anstey, M., & Bull, G. (2014). Helping Teachers to Explore Mulltimodal Texts. Curriculum & Leadership Journal, 8(16), 5–8. Retrieved from http://www.curriculum.edu.au/leader/helping_teachers_to_explore_multi modal_texts,31522.html?issueID=12141[11/07/2014 11:28:14 p.m. Boche, B. (2014). Multiliteracies in the Classroom : Emerging Conceptions of First- Year Teachers. Journal of Language & Literacy Education, 10(1), 114–135. Retrieved from http://jolle.coe.uga.edu Boling, E. C. (2008). Learning from Teachers ‟ Conceptions of Technology Integration : What do Blogs , Instant Messages , and 3D Chat Rooms have to do with it? Research in the Teaching of English, 43(1), 74–100. Cope, B., & Kalantzis, M. (2009). “Multiliteracies”: New Literacies , New Learning. Pedagogies: An International Journal, 4(3), 164–195. Retrieved from http://newlearningonline.com/files/2009/03/m- litspaper13apr08.pdf Davis, D. (2008). First We See: The National Review of Visual Education. Australian Government. Doucette-vanthuyne, A. (2016). An Investigation of Multiliteracies Pedagogies in Language Teacher Education : A Mixed Methods Study. Western Ontario. Ganapathy, M., & Seetharam, S. A. (2016). The Effects of Using Multimodal Approaches in Meaning- Making of 21st Century Literacy Texts Among ESL Students in a Private School in Malaysia. Advance in Language and Literary Studies, 7(2), 143–155. https://doi.org/10.7575/aiac.alls.v.7n.2p.143 Henderson, R. (2012). Teaching Literacies. Sedney: Oxford University Press. Jacobs, G. E. (2012). Developing Multimodal Academic Literacies among College Freshmen. Journal of Medi, 4(3), 244–255. Retrieved from www.jmle.org%0AThe Jocius, R. (2013). Exploring Adolescents ‟ Multimodal Responses to The Kite Runner : Understanding How Students Use Digital Media for Academic Purposes. Journal of Media Literacy Education, 5(1), 310–325. Retrieved from www.jmle.org Kalantzis, M., & Cope, B. (1997). Multiliteracies: Rethinking What We Mean by Literacy and What We Teach as Literacy the Context of Global Cultural Diversity and New Communications Technologies. Haymarket: Centre for Workplace Communication and Culture. Kellner, D. (2001). New Technologies / New Literacies : Reconstructing Education for the New Millennium. International Journal of Technology and Design Education, 11, 21–36. https://doi.org/10.1023/A:1011270402858 Kress, G. (2003). Literacy in the New Media Age. USA: Routledge. 2020. Linguistics, English Education and Art (LEEA) Journal 4 (1):99-108 108 Lim, J., & Polio, C. (2020). Multimodal assignments in higher education : Implications for multimodal writing tasks for L2 writers. Journal of Second Language Writing, 47(February), 1–8. https://doi.org/10.1016/j.jslw.2020.100713 Mills, Kathy A. (2008). Transformed Practice in a Pedagogy of Multiliteracies. Pedagogies: An International Journal, 3(1996), 109–128. https://doi.org/10.1080/15544800801929419 Mills, Kathy Ann. (2006). Multiliteracies : A Critical Ethnography. Queensland University of Technology. New London Group. (2000). Multiliteracies. (B. Cope & M. Kalantzis, Eds.). Canada: Routledge. Pullen, D. L., & Cole, D. R. (2010). Multiliteracies and Technology Enhanced Education : Social Practice and the Global Classroom. New York: Information Science Reference. Shepard-carey, L. (2020). Making Sense of Comprehension Practices and Pedagogies in Multimodal Ways : A Second-grade Emergent Bilingual ‟ s Sensemaking During Small-group Reading. Linguistics and Education, 55(March), 100777. https://doi.org/10.1016/j.linged.2019.100777 Shin, D., Cimasko, T., & Yi, Y. (2020). Development of metalanguage for multimodal composing : A case study of an L2 writer ‟ s design of multimedia texts. Journal of Second Language Writing, 47(February), 1– 14. Westby, C. (2010). Multiliteracies: The Changing World of Communication. Top Lang Disorders, 30(October), 64–71. https://doi.org/10.1097/TLD.0b013e3181d0a0ab