Tikka, Heidi. 2021. “The Body, the Threshold, the Cut: The Aesthetics and Ethics of Measuring in Interactive Media Art.” Catalyst: Feminism, Theory, Technoscience 7 (1): 1–7. http://www.catalystjournal.org | ISSN: 2380-3312 © Heidi Tikka, 2021 | Licensed to the Catalyst Project under a Creative Commons Attribution Non- Commercial No Derivatives license The Body, the Threshold, the Cut: The Aesthetics and Ethics of Measuring in Interactive Media Art Heidi Tikka Aalto University, School of Arts, Design and Architecture heidi.tikka@aalto.fi Abstract New media artists working on interactive installations often rely on different monitoring techniques, such as variable sensors in the design and the production of responsive environments and objects. In this short commentary, I will inquire into my installation Mother, Child (2011/2000) to address a new media art practice as productive alignment of agencies at the interface. The term body-sensor co- performance is used to foreground both the performative nature and the fundamental integrity of the technology and the body in interactive art. I will suggest that the setting of threshold values for different measuring operations can be understood as the boundary-making process, through which the installation feeds off the embodied liveliness of its audiences for its responsive actions. Drawing on Karen Barad’s work, these thresholds can be thematized as agential cuts. A number of specific examples in using sensors for interactivity are then addressed in order to inquire into the ways in which the questions of ethics and aesthetics entangle in creative and collaborative labors for Mother, Child. Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 2 Figure 1. Mother, Child (2011/2000) interactive installation at Wäinö Aaltonen Museum of Art, Turku, 2014. Agencies at the Interface Interactive art is a domain of experimental art practice that investigates and imagines new kinds of relations between humans and machines. These explorations are often staged as responsive environments or other entities that react to the actions of the visitors in museum and gallery spaces. In interactive installations these digitally produced others give an appearance of aliveness, as they seem to respond in meaningful ways to their audiences. How are these essentially inanimate others animated in and through human-machine interactions? I will suggest, by inquiring into the use of monitoring techniques in interactive art, that we, in a sense, endow these others with our embodied aliveness. The performativity of the human-machine relation is the key for addressing the aesthetic and ethical questions of measuring in interactive art. My own media art practice spans several decades of experimental work on different media and sensor technologies. My installations are often dramaturgically orchestrated as encounters between the embodied human visitors and the liminal others, configured as assemblages of images, material reminiscences of human bodies, collaborative labors, media devices, sensors, and code. I conceive of these installations as relational environments in which visitors enact the dramaturgical event of an encounter with these heterogenous Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 3 installation elements. As different kinds of sensors play a significant role in these enactments, the encounters could be called body-sensor co-performances to foreground both the performative nature and the fundamental integrity of the technology and the body in interactive art. This art form then is about enacted, unstable, and ephemeral events, rather than fixed, movable objects. Here I want to pursue the line of questions and observations that touch on the agency of the sensors in my installations. How are the operations of measuring, monitoring, and tracking understood in media art practice? My understanding of the artistic work and the use of measuring technologies in it is informed by Donna Haraway’s (1997, 8) account of figurations as material-semiotic arrangements. The development of a new installation always involves an orchestration of the relational interactions for particular technical elements, materials, and embodied processes that resonate across these relations, and by doing so, convey meaning. The dramaturgically active entities in my installations can be understood as figurations, consisting of an amalgam of material, technical, and narrative capacities. The question how the agency of a particular kind of sensor links with these other capacities is then of great importance in artistic design. The site of design and development in media art practice is the interface. Following Lucy Suchman’s (2007) work on human-machine reconfigurations, I conceive of the interface as the site of distributed agencies, heterogeneity, and multiplicity of enactments “objectified as persons and machines” (268). Thus, in my work, the entire installation and the way it affords the unfolding of an interactive dramaturgical event constitutes a human-machine interface. It is necessary to point out that these interfaces are not high-tech, but rather are the outcome of independent art practice, which relies on low-cost, off-the-shelf electronics and other materials. The creative challenge then is to figure out inventive uses for the simplest of devices, such as sensors measuring movement, temperature, pressure, or electric conductance. But it is perhaps because of the simplicity of these technical parts that I have come to inquire into their performativity. What happens in the process of measurement at the interface, and how does each sensor enact an interactive event in its particular way? Why are sensors so important in interactive installations? In the dramaturgies of interactive art, the machine as other, whether configured and represented as an environment, an animal, a plant, or a child, as in my installation Mother, Child (2000/2011), has to respond to the actions of a visitor in meaningful ways. The cues to these responses, which from the point of view of the machine can be Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 4 understood as state changes, are taken from the actions of visitors. Mother, Child installation consists of a responsive video projection in which a visitor, once seated, and holding a white fabric as a projection surface, may enact a relation to something, which appears to be more than a moving image of a child but less than a sentient being. The child reacts to the intrusions of the audience by crying, and to the actions of the visitor, who holds the projection fabric, by laughing, nursing, and finally falling asleep. What we experience as the liveliness of the child, his moods and the rhythm of his bodily motility, is in a sense, borrowed from the audience. Sensors are important because they translate human motility into code, so that in the case of Mother, Child, for instance, the body-sensor co- performance can be understood to animate the actions of the child and when doing so create the appearance of aliveness. Setting Thresholds The machine, which in the case of Mother, Child is a combination of a custom- made hardware switch, a video tracking system, and an interactive moving image playback system, sees these actions as if from an extremely narrow keyhole (Suchman 2007, 11). The state changes of the system are controlled by two monitoring systems: a switch that generates the data of the visitor’s presence, and the video tracking software that generates data of the motility of both the audience and the visitor. These two data flows provide the necessary information for the interactive dramaturgy as soon as the threshold values, the key points that instruct the system of state changes, are determined. An example of how such a state change would appear to a visitor of the Mother, Child installation is that if the “child is crying,” the software plays a moving image sequence of a “nursing child” after the visitor has been swaying the projection fabric to “calm the child.” The threshold values then constitute a direct link between the measuring of the data and the narrative of the installation. There are two observations that I want to address in regard to these threshold values in data flows. First, the setting of a threshold value is never innocent, as a threshold operation is always a boundary-making process, and as such implies the question of ethics. The making of a threshold includes certain values as the condition for an unfolding event, and excludes others, under which that event will never occur. And since these thresholds are determined into the data flows generated in the body-sensor co-performances of the audience interactions, they impose these exclusive operations to the participating bodies and their particular properties. For instance, in the first version of Mother, Child we used a pressure- sensitive pad for determining whether someone was seated, which was the Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 5 condition for the “waking up of the child.” The pad functioned as a simple on/off switch. But because of the way the seat was constructed, the pad was not sensitive enough to register the weight of a child. On the other hand, it was not exclusive enough because a bag full of heavy books thrown onto the seat would have turned the switch on, betraying the narrative logic of the installation. In the next version, a capacitive sensor was used instead. It registers a change in the electric conductance of the circuit in the presence of a fluid with a sufficient volume, such as the human body. This made it possible to determine the threshold values to include the bodies of both children and grown-ups, and to exclude the bodies of inanimate objects. Because the electric circuit was implemented directly into the seat, it turned the entire seat into a measuring apparatus. This in turn exposed the measuring operation to all the material changes in the seat and its environment, such as changes in the humidity of the timber used in it. As a consequence, when the piece was installed in a museum, the setting of the threshold values for the seat involved a lot of testing and tweaking. This experience brings up the second observation. The setting of the threshold values does not only involve metrics that can be abstracted from the apparatus. The experimental nature of my installations makes this explicit: the pains of adjusting the proper threshold values suggest the inseparability of the measuring apparatus from its environment, which conditions its operations. Sometimes the metrics are reduced to structural decisions as in the first example, in which the boundary-making process was embedded directly into the construction of the switch. These retrospective observations serve as a reminder that the ethics of boundary making are at play also in practices that are often perceived as the autonomous zone of artistic creativity and experimentation. The setting of the threshold values into a data flow is then conceivable as boundary work that involves labors and negotiations and always produces the differentiation between the included and the excluded. In the materialist reading of my art practice, a reading that draws on Karen Barad’s (2007) agential realism, a threshold value in a particular sensor would be understood as an agential cut. For Barad, a physical apparatus of measuring—such as a sensor—provides conditions for a meaning of a set of variables, while it excludes others. A physical apparatus then embodies a particular concept, while the concept acquires its meaning through this embeddedness. The apparatus specific concept marks the cut, an agential cut, between the object of measurement and the agencies of observation. The activity of measuring must therefore be understood as the Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 6 apparatus specific enactment of the phenomenon being measured. Objects of measuring do not have independent reality (Barad 2007, 118–20). For instance, in Mother, Child “presence” only denotes to the threshold value for proximity and for a volume of a liquid. Barad conceptualizes this material-conceptual relationality as intra-activity, under which the becoming of the world must be understood. As she words it, “what is ‘disclosed’ [by technoscientific practices] is the effect of the intra-active engagements of our participation with/in and as part of the world’s differential becoming” (Barad 2007, 361). She calls for the ethics of accountability, which I understand as the call for practices that are able to intra-actively articulate phenomena that can be perceived as objective, specifically because of these articulated intra-actions. It matters how apparatuses are configured and how these configurations and their conditions are made accountable, for “it matters which cuts are enacted: different cuts enact different materialized becomings” (Barad 2007, 361). Engagement with Barad’s work has also had material consequences to how Mother, Child is instantiated since its reconstruction in 2011. In the new version, the performativity of the installation is foregrounded, as particular attention is paid to the visibility and the aesthetics of the technical parts, which facilitate human-machine interaction. In these structural decisions the ethics and aesthetics of installation design entangle. Enfolding Temporalities Yet Mother, Child is also and genuinely a cinematic installation in which a visitor and the audience enact a relational event where different modalities of bodily movements converge with audiovisual experience. The aim was not to create a life-like simulation of a child. Rather, I wanted to explore the possibility of an experience in which the cloth, weightlessness, fluctuating image projection, self- experience of one’s body in movement, and the facial, bodiless moving image of a child would resonate together in the production of an affective overflow, feeding simultaneously off the excessive audiovisual and motile sensations and the absence of the other body. In the pursuit of something beyond the cinematic, the setting of the threshold values for the video tracking system became the video tracking system became a poetic venture, rather than an operational one. Perhaps, in this case, the question of the ethics and aesthetics of measuring is a more complex one. The monitoring of the audience activity was conducted by a video camera, placed above the installation, and software that registered the continuity of movement of what it identified as moving objects in the real-time video feed. In the interactive narrative, the child’s moods swing between the impulses of nursing Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 7 and crying, depending on how he experiences his surroundings. The reality of the video tracking data feed and the interactive narrative are linked through the reading of the data, which registers bodily motility in the installation space. The threshold values for this data were essentially used as cues for real-time moving image editing and playback from the existing prerecorded audiovisual database. The material was recorded during one nursing session with a stationary camera and edited for seamless combinatory interactivity logics. The shared labors of the artist and the software developer focused on finding a rhythm for the moving image narrative that “felt” right. I spent days in the studio, breathing and counting breaths to get the sense of right rhythmic flows, which were then translated into threshold values by the software developer. In an uncanny way, these adjustments also seemed to affect the temperament of the child. How to understand this “feeling right”? What kind of boundary-making process was taking place here? My tentative suggestion is that in this creative artistic and collaborative labor, the setting of the thresholds for the mood changes of the child became an embodied process, another kind body-sensor co- performance, implying the acknowledgement of distributed and relational agencies of production at the interface. This process can be seen as yet another entanglement of the ethics and aesthetics of human and nonhuman collaborations in media art practice. In other words, I do not see my embodied labor as a way of imposing my artistic vision on the dramaturgy of the installation. Rather, looking for the rhythms that would “feel” right was a process of tuning in with the sensors and their potentiality. As a creative process, it can be conceived as a body-sensor co-performance, a process of radical opening for the new and the unanticipated, and embodied affective labor for those agential cuts that organize the unfolding of the interactive narrative. In this creative work the rhythm figures as the folding in of different temporalities: my breathing, calling for the bodily memories of the past, the other moving bodies actualizing in the temporalities of the sensors and their data flows, the cinematic time of the moving image and the processing time of the computer. 7 References Barad, Karen. 2007. Meeting the Universe Halfway. Durham, NC: Duke University Press. Haraway, Donna J. 1997. Modest_Witness@Second_Millennium.FemaleMan©_Meets_OncoMouse™. New York: Commentary | Catalyst: Feminism, Theory, Technoscience 7 (1) Heidi Tikka, 2021 8 Routledge. Suchman, Lucy. 2007. Human-Machine Reconfigurations: Plans and Situated Actions. 2nd ed. New York: Cambridge University Press. Author Bio Heidi Tikka works as an independent artist-researcher, curator and writer. She is currently completing her doctoral thesis at the Department of Media in Aalto University, School of Arts, Design and Architecture. The documentation of her work, including the installation Mother, Child (2011/2000), can be found at http://heiditikka.com. http://heiditikka.com/