7 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) IZABELA KRAŚNICKA1 DOI: 10.15290/CR.2020.31.4.01 Jagiellonian University ORCID: 0000-0001-6746-780X Meaning in multisemiotic messages – functions of gestures accompanying speech as elements of utterance structure Abstract. The topic of the article will be the discussion of the issue of how gestures accompanying speech may complement a gap in a statement. The considerations will cover particular ways of filling such a gap, both semantically and taking into account the place of the gesture in the syntactic structure of the sen- tence. The subject of interest will be the analysis of spontaneous statements of interlocutors, which will allow for the isolation of word-gesture wholes; it will likewise describe the relationship between the two modes ahead of recounting the functions of gestures in the structure of the entire utterance. The ana- lysis is based on commentaries from one of the journalistic programs hosting politicians in Poland. Keywords: gestures accompanying speech, multisemiotic, embodiment of meaning, metaphoric ges- tures. 1. Introduction Observing people engaged in a discussion, involved in a dialog, including, or perhaps even, in particular, politicians talking in front of cameras in a TV studio, leaves no doubt that their gesticulation is an integral part of the utterances made. Gestures made while speaking are so strongly related to the uttered words that an attempt itself to separate these two modes of semiotic expression, even only for descriptive purposes, tends to be extraordinarily difficult. While the formal difference between an auditory and a visual mode is obvious, the meaning of an entire statement is of a multisemiotic nature and it should be understood as such. David McNeill’s concept of Growth Point takes as its starting point the assumption that a dialectic relationship between two semiotic modes, “dual semiosis” (McNeill 2016: 21), is the core of expression (McNeill 2016: 11-15, 21-22). 1 Address for correspondence: Wydział Polonistyki UJ, Katedra Teorii Komunikacji, ul. Gołębia 20, 31-007 Kraków, Poland. E-mail: izabela.krasnicka-wilk@uj.edu.pl 8 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) The meaning created in a dynamic mode of interaction is therefore the result of a total combination of both semiotic modes, while the one transmitted by gestures is not redun- dant in relation to the semantics of linguistics expressions (although it is sometimes so defined in situations where the linguistic and extra-linguistic content is expressed in the same way, cf. Müller et al. 2013: 66; Karpiński et al. 2008: 93), but is of a differ- ent character (e.g. Goldin-Meadow & McNeill 1999). The issue to be discussed here will hence be the occurrence of gestures accompanying speech, which, as part of the whole “usage event” (Cienki & Iriskhanova 2018: 26)2 also fills in the gap in a speech act at the syntactic level. The ways in which the verbal and gestural components of an utterance make up this multimodal whole will be the subject of this paper, as well as a reflection on the concept of “grammar of gesture” developed by a group of German research- ers (Müller et al. 2013: 707-732; Müller 2014a: 138-140), which as it seems, has much in common with the discussed issue. The combination of two semiotic modes raises a question about functions of gestures in a situation when pauses appear while speaking3 – when the speech act is paused for a moment to accommodate the speaker’s search to find a proper word or use a gesture relevant in the given context, in order to communicate the intended semantic potential. Whether, and if so, how, this “moment” in sometimes dynamic discussions and emotional statements is filled with gestures, whether it is possible to indicate specif- ic words that would fill this gap – these are the main questions that I will be looking for answers to in the analyzed material. 2. What is the “grammar of gesture”? German gesture researchers, especially Cornelia Müller, Jana Bressem, Silva H. Ladewig, Ellen Fricke, have adopted the assumptions of Adam Kendon, a pioneer of form-based gesture research (Müller 2014a, Kendon 2004), developed and presented issues related to a formal and functional side of gesture behaviors in their works. For example, Müller discusses the concept of four modes of gestural representation4 (but notes that in prin- ciple they are reduced to two5), which are expressed in gesture. While in the case of an artist’s depiction of a landscape, what matters is the medium chosen to depict it (pencil sketch, oil painting, photography), since visual thinking takes place “»through« 2 “Set of verbal and non-verbal (gestural, in our case) behaviors that interlocutors find relevant for their communication and choose to focus on” (Cienki & Iriskhanova 2018: 26). 3 Interesting research on the function of pauses in colloquial speech as well as a comprehensive review of literature on pauses can be found in Majewska-Tworek’s paper (2014). 4 These are: molding, drawing, acting and representing (see e.g. Müller 2014: 1691 for details). 5 Acting and representing: “in the acting mode, the hands mime themselves, while in the representing mode they mime other entities” (Müller 2014: 1696). 9 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) and »with« the ‘modes of representation’” (Müller 2014: 1688), in the case of gestures the speaker can draw iconic images, sculpt them with their hands or use their body to replace a described object (e.g. the hands as an imaginary book) and thus present the content in different ways, in different modes. This approach based on the form of a gesture is also reflected in the concept of “recurrent gestures”6, which in turn form groups of conventionalized gestures, belonging to individual families because of the common semantic axis which they have been grouped around (Fricke et al. 2014: 1630- 1640). For example, the palm up gesture family is united by the concept of “offering, showing, or receiving some object” (Müller 2014a: 137). The issue of gesture families is important for the theme of grammar of gestures (Müller et al. 2013: 718) – recur- rent gestures form families on the basis of common formal and semantic features; they are subject to the process of conventionalization (like the away gestures family discussed by the researchers) and in this sense they are elements of the “grammar of gestures”7. Researchers, therefore, understand this term as, firstly, a collection of formal and structural features of gestures and secondly, as a relation of substitution between linguistic and non-linguistic components of statements. “Notably, the term »grammar of gesture« refers to the basic form properties of gestures and their structures. It does not imply, however, that co-verbal gestures have anything like a grammatical structure. The formulation »Towards a grammar of gestures« underlines two aspects: first, co-verbal gestures show properties of form and meaning which are prerequisites of language and which – in case the oral mode of expression is not available – may evolve into a more or less full-fledged linguistic system such as a sign language or an alternate sign language […]. Second, when used in conjunction with speech, co-verbal gestures may take over grammatical functions, such as that of verbs, nouns, or attri- butes pointing towards a multimodal nature of grammar […]” (Müller et al. 2013: 711). Therefore, the term “grammar” refers to various types of formal regularities that char- acterize the gestures accompanying speech, but also touches upon an important issue – the function of gestures in the syntactic structure of a sentence, another issue that was also investigated by Ladewig (2014). Her research showed that gestures can replace nouns and verbs in utterances (Ladewig 2014: 1665-1668). Such approaches move in the direction of grammar as a multimodal phenomenon (Müller et al. 2013: 65), which treats gestures as a part of language, which is also the main premise of McNeill’s work (2016) – verbal and gestural actions of a human being form an inseparable whole that conveys meanings. 6 “In recurrent gestures, gestural form features are not random, by definition, but recur across speakers and contexts whilst sharing stable meanings” (Bressem & Müller 2014: 1577). 7 See also the conception of the “grammar of color” in T. van Leeuwen’s and G. Kress’ perspective (2013: 229-257), in which the authors consider the term “grammar” in relation to color as a semiotic system. 10 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) 3. Material and method The Ladewig study (Ladewig 2014) showed that gestures can replace concrete words, but taking into account the observations of Antas (2013), the question arises as to what the gestures illustrate. Taking an approach based on the form and concept briefly discussed above, the answer could go in the direction of concrete words that are demonstrated by gestures, but Antas notes: “Contrary to popular judgments about what gestures illustrate, they never show the words spoken, but the meaning of the concepts behind them” (Antas 2013: 92; see also: Załazińska 2012)8. “Apart from the fact that icons provide additional information about the described activity or objects used during its execution and not those contained in a verbal course, they usually precede and initiate the content to which they are related (thus, somehow announcing it)” (Antas 2013: 64). If iconic gestures “precede and initiate the content” they refer to, the question is posed as to how they fit into the semantic and syntactic structure of an utterance. Six episodes (Table 1) of one of the opinionated news programs, “Kawa na ławę” [lay it on the line], were selected for observation (access: https://www.tvn24.pl/kawa- na-lawe,59,m). The choice of the program was made based on the following factors: it involves several people talking, it enables spontaneous utterances even though they are recorded, and allows for the possibility of emotional involvement of speak- ers, sometimes to an extreme degree. This combination, in turn, provides conditions for effortless gesturing that accommodate observation of the embodied conceptualiza- tion of concepts and how they are expressed in a multimodal message. Table 1. Dates and duration of the episodes Date of episode Duration 17.02.19 57:00 24.02.19 56:55 03.03.19 59:50 10.03.19 1:00:45 17.03.19 58:29 24.03.19 57:36 The Elan program was used for the observations, which enables, among other things, a significant slowdown in playback and thus the accurate identification of the place 8 It is worth noting that we are talking about gestures accompanying speech, four types of which are dis- tinguished by McNeill (1992): iconic, metaphoric, beats, deictic — and not about emblematic gestures, which are of a completely different nature (cf. Ekman & Friesen 1969), where iconic and metaphoric ges- tures can generally be regarded as iconic, as they differ only in the referent (see Cienki 2008: 8). Below, however, I will use the original McNeill’s distinction of iconic/metaphoric gestures. 11 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) where verbal expression is interrupted by a pause not always heard well during the stan- dard pace of conversations. In the paper, the basis for determining the degree of dependence of gestures on speech is the so-called Gesture Continuum (McNeill 2013: 483), including Gesticulation – Language-slotted – Pantomime – Emblems – Sign Languages. The further we move on the scale to the right, the lower the dependence of gestures on speech, and at its last point we are dealing with independent systems of languages expressed only through gestures. The gestures that have been noted in the political discussions under analysis here are rather at the beginning of the above continuum, since their relation to speech is very close. However, since the boundaries between the different levels of the continuum are not sharp, in many cases it is not possible to unambigu- ously place the gestures I have observed. As McNeill (2016: 5) notes, the closest to them are language-slotted gestures which in combination with speech, form some “gesture- speech syntactic hybrids”. In the dynamic polylogue I observed, mainly of a persuasive nature, there was a certain difficulty to separate sentences with a gap and accompanying gestures from the gesture spectacle, which the audience witnessed9. On the one hand, it included many state- ments that sometimes gave the impression of being “designed” according to the ideolog- ical expectations of parties where there is no place for “stops” or hesitation, and on the other hand – there were emotional reactions to the statements of the previous speaker, open and dynamic quarrels between the two speakers that interrupted the statements of interlocutors. Therefore, this multilateral dialog was also characterized by the fact that speakers rather filled in the gaps in speech, so the pauses were usually filled with retardations – the cases of total replacement of words with gestures were few, although the material enabled me to excerpt and describe the examples. An almost complete lack of iconic gestures, and the excess of beats and deictic gestures, as well as gestures struc- turing discussion and interaction, confirmed the assumption that the concentration of speakers on persuasive interactions (instead of narration/description, as it happened in the case of materials studied by e.g. Cienki and Iriskhanova (2018) would impact the way they employed gestures. The material observations, which are the basic method of work, allowed for the iden- tification of such examples of gestures that fill the gap in an utterance in a differentiat- ed manner, which is implemented in several ways. The research attention was therefore focused on separating the types of gestures made, and not on the description of indi- vidual cases. These separate gestures are what will be discussed below. 9 The observations were also more difficult to analyze because of changes in the field of view of the camera and the bars covering the lower part of the screen – although only to a certain extent, because the gestures were still visible. 12 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) 3. Discussion of examples Example 1 Gesture accompanying an utterance: …osobiście odniosłem wrażenie, bezpieczeństwo gdzieś było na… na i to…no o to chodzi... [I personally got the impression that safety was somewhere on... on10... and that... well, it’s about that ...] (17.02.2019) Fig. 1. Removal from the focus area During his speech, the speaker made a gesture as if he were pushing an object away from himself, outside the area of his conscious attention. Interestingly, he made the gesture at the moment when the preposition na (on) was repeated, but he did not finish speaking – the sentence structure lacks a term that would determine where the safety he was talking about was located. Therefore, a gap emerges; however, filled in with redundancies, which is accompanied by a gesture without its verbal equiva- lent. But it has its specific place in the utterance structure, so one can point to the part of a sentence that is missing in a statement, but complemented by non-linguistic behav- ior. The whole is legible only when a recipient can see the speaker – because then it is possible to complement the missing part of the sentence with the content expressed with the gesture – safety was somewhere (on the side) – because this is the place located to the left of the central gesture space11 that the speaker indicated. On the other hand, the gesture clearly refers to the notion of insignificance – this is a kind of hand waving that I have written about previously (Kraśnicka-Wilk 2018: 75-87), and which connotes such meanings as lack of attention, disregard – I move it aside = ‘it is insignificant’ (the gesture of waving one’s hand will also be discussed below – it has several variant forms that change the meaning of the whole statement). It can be assumed, therefore, that the speaker does not verbalize thoughts, leaving an incomplete sentence without an adverbial12, because the whole concept has already been included in the gesture which is closely linked to the auditory mode and demonstrates the conceptual meta- phors important is central, unimportant is peripheral mentioned by Kövecses (2011: 309). It can be considered a prototypical example of a gesture-speech hybrid, 10 Phrases accompanied by a gesture are underlined. 11 Gestures space after Karpiński et al. 2008: 94. 12 In Polish: okolicznik. 13 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) i.e. McNeill’s language-slotted gesture, because it co-creates the whole semantically and syntactically with the verbal statement. The presence of both modes of semiotic expression is necessary for the sentence to be understandable, so one can talk about the “grammar” of the complete verbal-gestural whole, the “expression package”, whose semantics is the result of complementing the verbal part of the statement with a gesture. Example 2 Gesture accompanying an utterance: ...w jakiś sposób tutaj yyymm tą pedofilię zzzwww udelikatniamy no że tak powiem, jest jest większa możliwość no tak tak uważam … [somehow here, uhh, mmm, we soften this pedophilia of shshshsh, so to speak, there is a greater possibility, so, yes yes, this is what I think...] (10.03.2019). Fig. 2. A sequence of gestures searching for the word: I – w jakiś sposób tutaj yyymm [somehow here, uhh, mmm]; II – tą pedofilię [this pedophilia]; III – udelikatniamy [we soften]; IV – jest większa możliwość [there is a greater possibility] The speaker has a clear problem with finding an appropriate word; his utterance is incomplete, torn, unclear, and he makes several significantly different gestures (Fig. 2) – he starts with a balancing gesture (I), he imitates the movement of pushing an object on a slope (II), and afterward he makes a sharp cut with his whole hand (III) (as in the case of negation; cf. Antas 2013, Antas & Gembalczyk 2018: 50), and at the end of the utterance there is again the balancing gesture (IV). There is still an impres- sion that the word soften is not the word the speaker was looking for, because the chosen gesture does not confirm this conceptualization. It is rather about a certain concept expressed with gestures at the beginning and the end of the utterance: balance. Fig. 3. Balance gesture This movement resembles an examination of the land, seeking for proper support, something solid; the speaker’s hands press on something from above, alternately 14 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) seeking for something. It is accompanied by breaks in an expression filled with retarda- tions, which “support the extraction of words from the mental lexicon of the speaker” (Majewska-Tworek 2014: 192). It is difficult to mark gestures accompanying words in the recording of a statement; rather the opposite is true – the gesture performance is accom- panied by words, sought by the speaker looking for the right one. In the verbal structure of the statement, there is no object (as a part of the sentence) – the listener does not know what the greater possibility refers to, and even the first part of the statement seems a bit illogical: …w jakiś sposób tą pedofilię udelikatniamy [...somehow we soften this pedophilia] – given that a recipient who could not see the speaker would be left alone with this state- ment. Thus, taking into account the grammatical structure of the sentence, one can observe a complement to the word – a gesture – jest większa możliwość… [there is a greater possibility...] (+gesture) because it appeared in the place of a lexicalized pause, but the semantics of the whole remains unclear until the end. Although the broader context of the statement may lead the recipient to guess its meaning as a whole, the gesture does not specify the meaning in such a clear way as in the case of Example 1 – it is impos- sible to point to a single word that would complement a word gap. The gesture rather indicates in which direction the interpretation of the statement should go – it is about a certain fluidity, a balance between two values, but also – lowering the value (a gesture imitating sliding down) or even negation (a sharp cutting gesture); it is about expressing the concept, not a particular word. The word udelikatniamy ‘we soften’ is not, as it seems, the choice that the speaker meant to convey. In this example, the gestures are combined with the speech in an extremely sophisticated way – three different gestures of the speaker are to convey one notion, in front of the audience there is an explicit search for the most appropriate word – delivered through a nonlinguistic mode. Therefore, we can say that Example 2 is strong proof that gestures replace concepts, rather than individual words in the course of a statement in which the speaker is involved, but they are also its essential “syntactic” part, appearing at the moment of a verbal break. Example 3 Gesture accompanying an utterance: …czy będziemy im pro…proponować to, co dzisiaj Wielka Brytania przeżywa, czyli ten y… absurdalny w swojej istocie Brexit.... […are we going to pro... propose to them what Great Britain is experiencing today, namely this mm... Brexit, absurd in its essence] (10.03.2019). Fig. 4. Scheme of “exit from...” 15 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) The first gesture appears with the following words: czyli ten y… [namely this mm…]; it is a definite movement with both hands to the left, limiting a certain imaginary sphere. What has been conceptualized here is a scheme of movement appearing in a place where the utterance was interrupted with a short but clear pause. Carrying on with the statement, the speaker made another gesture with both hands in front of him, gently drawing a semi-circle when he said: absurdalny w swojej istocie Brexit [Brexit, absurd in its essence]. While the iconic gesture of a semi-circle, a sphere, is a pattern repeated in various verbal environments (cf. Antas 2013: 148-153), and conceptualizing a “closed set” or a “whole” – in this case the closed set is absurdalny Brexit [an absurd Brexit] (a metaphoric gesture), yet the first gesture has a different function here. It is also meta- phorical, but it accompanies not words, but a short pause, conceptualizing a thought that is not verbally expressed. The politician talks about a certain proposal for future generations, accompanying it with gestures illustrating an exit, a shift, and afterward he draws the semi-circle. If we omitted the fragment with a gesture showing the pattern of movement (exiting from..., exiting outside...), it would sound like this: czy będziemy im proponować to, co dzisiaj Wielka Brytania przeżywa, czyli ten (y…)13 absurdalny w swojej istocie Brexit [are we going to propose to them what Great Britain is experiencing today, namely this (mm…) Brexit, absurd in its essence], so it would be a syntactically complete whole, but characterized by a certain illogicality – because the sentence would suggest that Brexit will be proposed to Poles who had been mentioned a moment before. The speaker most probably wanted, however, to outline a dangerous vision of a decisive and sudden (precise and fast movement) exiting from, exiting outside as the most unfa- vorable decision dla naszych wnuków i prawnuków [for our children and grandchildren] (previous fragment of the statement), and replacing the gesture with a word (are we going to propose to them ? an exit) would not have the equivalent force of expression – the word exit does not connote an image as negative in this case as the gesture itself, a sudden shift out of the area of attention. The second gesture depicted a closed, finished whole (a sphere, a circle) and was accompanied with the words absurdalny w swojej istocie Brexit [Brexit, absurd in its essence]. The difference between these two types of gestures is described by McNeill: “Language-slotted gestures look like gesticulations but differ in the manner of integrating with speech. They enter a grammatical slot, semiotically merge with speech, and acquire syntagmatic values from it; gesticulation in contrast to gesture is co-produced obligatorily with speech but semiotically opposed to it” (McNeill 2013: 483). Although in this example it is generally difficult to speak of a “gram- matical slot”, only a momentary pause in the statement is a place that is complemented by a gesture expressing the concept of exit. 13 Intentional crossing out, as a sign of omitting a fragment. 16 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) Example 4 Gestures accompanying an utterance: …i zarówno jeśli chodzi o spółkę, czy państwo yyy rządzi tymi yyy… jednostkami, wspólnotami… [...and both in terms of a company or state, mmm ... these mmm... individuals, communities, are governed by...] (03.03.19) Fig. 5. Scheme of “being over” Fig. 6. Scheme of a whole In the statement, apart from the initial gestural enumeration, which will not be the subject of description here, two gestures appeared, and both during the articulated pauses ( yyy). The first one started during the pause when the speaker sought for a proper word to express his thought. This was a broad circle with an open hand with fingers spread out, pointing down at a certain area in front of the speaker and finally empha- sizing the word tymi ‘these’ with a short downward motion, as in Figure 5. It seems that the speaker’s hesitation expressed in a pause was supported with the gesture – the concept of governing was conceptualized in gestures as being over14, but the speaker found the missing word, although a little later, because the main phase of the gesture, the stroke, falls at the moment of the pause. The movement of the hand pointed to a way of understanding the concept of governance, but the gesture itself is not a part of the syntactic structure of the sentence, because there is a verbal complement to it that is appropriate in the given context: rządzi ‘governs’. The second gesture also occurred during the pause, but it also ended during the pause, because the speaker returned to the initial position of his hands15 when he spoke 14 Etymologically speaking, governing means to align, order, organize, watch something, take care of something, manage, administer (after: Wielki słownik języka polskiego, https://wsjp.pl/do_druku. php?id_hasla=23823&id_znaczenia=0), so not necessarily the “superiority” which is a metonymic conceptualization in the gesture described above. 15 Constituent elements of non-verbal behavior called a gesture after Kendon 2004. 17 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) the words: jednostkami, wspólnotami [individuals, communities]. The notion of a semi- circle, expressed in a non-linguistic manner, refers to the word wspólnoty ‘communities’ – more than to the word jednostki ‘individuals’ spoken earlier. Imaging by gestures might have guided the speaker to another word, which he used afterward, as a self-correction: …jednostkami, wspólnotami [...individuals, communities]. Therefore, it can be postulated that a sentence structure is clear and correct without a gesture, even though the utter- ance itself lacks a little bit of fluency, which is suggested by short pauses, where notions hidden under the uttered words are represented by gestures. In semantic terms, however, the sentence is complete only when the multisemiotic character of this state- ment is taken into account – the conceptualizations of concepts: governance as being over and community as a circle, a “closed set” is “received” by the recipient in a visual mode. Example 5 The gesture accompanying an utterance: …więc yy już tam yy, mniejsza o… o… semantykę y... [so mm… in any case mm…, it’s not about about semantics mm...] (10.03.19) Fig. 7. Throwing away gesture This example is particularly interesting because of the fact that the gestural phrase- ologism machnąć na coś ręką16 ‘forget about it’ is non-verbally depicted here (cf. Jarząbek 2016: 250; in detail on the gestural phraseology Krawczyk 1983; Kozak 2007; Kraśnicka- Wilk 2018, 2018a). The gesture appears at the moment of a short pause of the speaker and a broken utterance yyy już tam yyy [so mm… in any case mm…], and afterwards it is repeated during the next articulated pause o… o… [about… about…]17. As other mate- rial observations have shown (Kraśnicka-Wilk 2018), speakers usually make a gesture when speaking a gestural phraseologism; for example, a speaker states: machnijmy na to ręką [let’s forget about it] – and they actually make the gesture of a waving hand/ throwing away, therefore embodying its meaning. This notion, which is the subject 16 This expression can be regarded as a gestural phraseologism only in the Polish language, because in the English-language idiom – forget about it – there’s no specific “description” of a gesture, which is a characteristic feature of such phraseologisms. 17 In the original Polish-language utterance, only the vowel “o” was repeated. 18 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) of a separate research, is presented in a slightly different light in the discussed example, because the speaker only says the words mniejsza o [it’s not about], making the throwing away gesture twice in front of him, thus verbally and bodily, almost at the same time representing the notion of insignificance (important in the context of the whole utter- ance about a small mistake by the politician, which he, however, admitted to during the discussion). He does not utter the phraseologism machnijmy na to reką [let’s forget about it], but in his multisemiotic message, he stresses that what was said earlier should be considered invalid and therefore rejected. This is an example of the Away Gesture family: “An imaginary topic of the talk, sitting in the palm of the hand, is dismissed by throwing it away” (Müller et al. 2013: 719). The verbal statement lacks some fluency, there is still an impression that the words are rushing through its course; but, supple- mented by a gesture, it seems to have illocutionary force, appropriate in this context. The gesture complements it both syntactically and semantically – a wave of the hands complements the phraseologism expressed in the sentence, only in a shorter, variant form: mniejsza o… [it’s not about] instead of machnijmy na to ręką [let’s forget about it]. Conclusion The studied material allowed for placing the observed gestures between gesticulation and language-slotted gestures, taking into account the fluidity of borders between differ- ent categories. Gesticulation – … In the structure of utterance there are shorter or longer pauses, often filled with e.g. retardations accompanied by gestures - the gesture precedes the concept, which is afterward verbalized – Ex. 4 “community”, “govern”, Ex. 5 “it’s not about” - ... There is a gap in the sentence structure that can be accompanied by a pause/s – what is depicted in the gesture is not a substitute for a specific word/expression that says, “the word disappears but the thought remains” (Antas 2013: 121) – Ex. 2 “soften” - … The sentence structure can be supplemented by a specific, easily identifiable word that is only expressed by gestures – Ex. 1 “security” – ... Language-slotted - ... Although the studies were not quantitative, it could be observed that gestures appeared relatively often in retardation pauses filled with the most frequently repeated vowels. During pauses, the speakers made gestures illustrating conceptualizations of concepts such as: governance (raised hands illustrated a plane), community (hands outlined a circu- lar shape), longer perspective (the speaker gestured to extend the distance marked with 19 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) one hand: an example not mentioned above), insignificance (waving his hands, pushing the object out of the area of attention). In the case of this kind of relationship between gestural and linguistic imaging, the gesture was ahead of the concept which it referred to, and afterward, it was verbalized (govern, communities, it’s not about...). The second relation between the word and the gesture during pauses in an utter- ance was that the gap in the utterance was filled only with gestures – the speaker could not find the right word to replace what was depicted with gestures. An example is the balance gesture, repeated twice at the beginning and end of the statement, even though the sentence has been “supplemented” with words, expressed in the grammatically correct form (apart from hesitations and breaks in the statement). However, a repeti- tion of the gesture at the end of the statement would indicate that it was this conceptu- alization, expressed only with gestures, bodily, that was relevant for the speaker in the given context. Antas concludes: “the subject uses the name as the closest to the term. And when he feels that the name does not fully reflect the meaning that he imposes on the concept, or even »the word disappears«, but the thought remains – he uses a gesture” (Antas 2013: 121). In this case, it would be difficult to find the right word to complement the structure of the sentence, since the balance gesture, the search for balance remained only in the form of thought, while the word was used here as an “indirect supplement” – spoken, but, as it seems, significantly different from gestural conceptualization. The third type of relationship would be substitution, where the meaning expressed in a gesture can be supplemented with an easily identifiable word, so the meaning is contained in “gesture-speech syntactic hybrids” (McNeill 2016: 5). In this case, the gestures complement the statement not only on the semantic level, but also indi- cate what kind of gap they fill on the syntactic level – in the example discussed here, the gesture performs the function of an object (...safety was somewhere [+gesture – a hand stretched out from the central gestural sphere to the left]: on the side]) which is not verbalized. The material presented in this paper has shown that there are several kinds of “gaps” in statements – from the most common ones, pauses filled with retardations, during which the speaker hesitates in order to choose the most appropriate word, and encoun- ters “a temporary difficulty with verbalization of thoughts (»how to call it«, »how to express it«) and [may] have difficulty in extracting from memory any more or less accurate word” (Majewska-Tworek 2014: 192-193), to those during which the speaker creates the whole gestural representation, in the end failing to access the right word. The mere recognition of certain gestures such as entering a language is therefore completely conventional; since, as the observations have shown, this “entering” takes place on different levels and differs in degree, which in turn depends on the purpose and function of the statement on which the speaker is focused. Further research 20 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) may, therefore, move in the direction of seeking regularity in the occurrence of the above-mentioned word-gesture relationships to answer the question of whether there is a limited repertoire of such regularities. An interesting issue concerning the examined material is also the gestural concep- tualization of the concept, which precedes its verbalization. The question arises as to how the semantics of multimodal expression can be studied when a gesture is not accompanied by a linguistic pause and the verbalization of meaning occurs only later – and if the structure of the gesture itself can be described quite accurately by dividing it into phases (cf. Kendon 2004); it seems that the reference to the linguistic layer, which does not coincide with the gesture temporally, remains rather on an intuitive level, though sometimes obvious. However, this issue requires a separate, broader discussion and further research. The term “grammar of gesture” in relation to the material analyzed above will refer to the whole verbal-gesture unit in which the gesture takes a place equal to its verbal components – it is justified to talk about the “grammar of a multisemiotic message”18 rather than the grammar of gesture in the case of the approach I proposed. It is not so much the form of the gesture itself that is important, but its place in the struc- ture of verbal expression, which is emphasized by the above-mentioned researchers, who speak of the multimodal character of grammar. As Cienki and Iriskhanova point out, gestures form a partially dependent semiotic system that “co-exists with language in a mutually beneficial symbiosis” (2018: 33). A necessary dependence in this “symbiosis” is not only the fact that gestures “support” the information transmitted through language, but that language in an interactive manner utilizes the semiotic and semantic potential of gestures. References Antas, J. 2013. Semantyczność ciała. Gesty jako znaki myślenia. Łódź: Primum Verbum. Antas, J. & Gembalczyk, S. 2018. Człowiek negujący – multimodalne studium cieles- nych wyrazów negacji towarzyszących mowie. In: A. Załazińska & J. Winiarska (eds.), Multimodalność komunikacji, 47-56. Kraków: Księgarnia Akademicka. Bressem, J. & Müller C. 2014. A repertoire of German recurrent gestures with prag- matic functions. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 2. (38.2), 1575-1591. Berlin/Boston: Walter de Gruyter. Cienki, A. 2008. Why study metaphor and gesture? In: A. Cienki & C. Müller (eds.), Metaphor and Gesture. Amsterdam: John Benjamins. 18 I formulated this conclusion together with Prof. Jolanta Antas during the discussion on the “grammar of gesture”. 21 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) Fricke, E., Bressem, J. & Müller, C. 2014. Gesture families and gestural fields. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 2 (38.2), 1630- 1640. Berlin/Boston: Walter de Gruyter. Ekman, P. & Freisen, W. V. 1969. The repertoire of nonverbal behavior. Categories, ori- gins, usage, and coding. Semiotica 1(1): 49-98. Goldin-Meadow, S. & McNeill, D. (1999). The role of gesture and mimetic representa- tion in making language the province of speech. In: M. C. Corballis & S. Lea (eds.), Evolution of the Hominid Mind, 155-172. Oxford: Oxford University Press. Iriskhanova, O. K. & Cienki, A. 2018. The semiotics of gestures in Cognitive Linguistics: Contribution and challenges. Voprosy Kognitivnoy Lingvistiki 4: 25-36. Jarząbek, K. 2016. Słownik mowy ciała Polaków. Katowice: Wydawnictwo Uniwersytetu Śląskiego. Karpiński, M., Jarmołowicz-Nowikow, E., Malisz, Z., Szczyszek, M. & Juszczyk, K. 2008. Rejestracja, transkrypcja i tagowanie mowy oraz gestów w narracji dzieci i dorosłych. Investigationes Linguisticae 16(December): 83-98. Kendon, A. 2004. Gestures: Visible Action as Utterance. Cambridge: Cambridge University Press. Kozak, K. 2007. Polska frazeologia gestu: od informacji somatycznej do idiomu. Etnolingwistyka 19: 253-265. Kövecses, Z. 2011. Język, umysł, kultura. Praktyczne wprowadzenie. Kraków: Universitas. Kraśnicka-Wilk, I. 2018. Między słowem a gestem – frazeologia gestyczna w ujęciu kognitywnym. In: M. Gębka-Wolak, I. Kaproń-Charzyńska & J. Kamper-Werejko (eds.), Języki słowiańskie w kontekstach kultur dawnych i współczesnych, 75-87. Toruń: Wydawnictwo Naukowe Uniwersytetu Mikołaja Kopernika. Kraśnicka-Wilk, I. 2018a. Gesty zachowane w języku. Relacje między językowymi i nie- językowymi sposobami wyrażania znaczeń na przykładzie analizy semantyki fra- zeologizmu gestycznego podać komuś rękę. In: J. Winiarska & A. Załazińska (eds.), Multimodalność komunikacji, 57-71. Kraków: Księgarnia Akademicka. Krawczyk, A. 1983. Frazeologizmy mimiczne i gestyczne (na materiale gwarowym). Socjolingwistyka 5. Ladewig, S. H. 2014. Creating multimodal utterances: The linear integration of ges- tures into speech. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 2 (38.2), 1662-1677. Berlin/Boston: Walter de Gruyter. Leeuwen van Theo, K. G. 2013. Kolor jako system semiotyczny: wstęp do gramatyki ko- loru. In: A. Duszak & G. Kowalski (eds.), Systemowo-funkcjonalna analiza dyskursu, 229- -257. Kraków: Universitas. 22 ......................................................................................... CROSSROADS. A Journal of English Studies 31 (2020) (CC BY-NC-SA 4.0) Majewska-Tworek, A. 2014. Niepłynność wypowiedzi w oficjalnej odmianie polszczyzny. Propozycja typologii. Wrocław: Quaestio. McNeill, D. 1992. Hand and Mind. What Gestures Reveal about Thought. Chicago: The University of Chicago Press. McNeill, D. 2013. The co-evolution of gesture and speech, and downstream consequenc- es. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 1 (38.1), 480-512, Berlin/Boston: Walter de Gruyter. McNeill, D. 2016. Why We Gesture? The Surprising Role of Hand Movements in Communication. New York: Cambridge University Press. Müller, C. 2014. Gestural modes of representation as techniques of depiction. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 2 (38.2), 1687- 1702. Berlin/Boston: Walter de Gruyter. Müller, C. 2014a. Gesture as deliberate expressive movement. In: M. Seyfeddinipur & M. Gullberg (eds.), From Gesture in Conversation to Visible Action as Utterance, 127-151. Amsterdam: John Benjamins. Müller, C., Bressem, J. & Ladewig, S. H. 2013. Towards a grammar of gestures: A form-based view. In: C. Müller, A. Cienki, E. Fricke et al. (eds.), Body, Language, Communication. An International Handbook on Multimodality in Human Interaction, vol. 1 (38.1), 707-733, Berlin/Boston: Walter de Gruyter. Poprawa, M. 2009. Telewizyjne debaty polityków jako przykład dyskursu publicznego. Kraków: Universitas. Załazińska, A. 2012. Metoda analizy środków niewerbalnych pojawiających się pod- czas wypowiedzi (na podstawie programu „Kuba Wojewódzki”). In: J. Wasilewski & A. Nita (eds.), Instrukcja obsługi tekstów. Metody retoryki, 253-269. Sopot: Gdańskie Wydawnictwo Psychologiczne. * * * Izabela Kraśnicka is an Assistant Professor at the Communication Theory Department of the Faculty of Polish Studies at the Jagiellonian University. The author’s research interests focus primarily on gestures accompanying speech, their role in creating and transmitting meanings, as well as gestural phraseology as a category illustrat- ing relations that combine two semiotic modes – speech and gesture – into a semantic whole. She is the author of the book Mówiące ciało [the talking body], in which the issue of the cognitive view of the Polish language phraseology is discussed.