Firenze University Press www.fupress.com/bsfm-jems Citation: V. Rachiele (2020) Multimedia and Interaction Design in the ‘Stone with a Sto- ry’ Project. Jems 9: pp. 203-212. doi: http://dx.doi.org/10.13128/ JEMS-2279-7149-11197 Copyright: © 2020 V. Rachiele. This is an open access, peer-reviewed article published by Firenze University Press (https://oajournals.fupress.net/ index.php/bsfm-jems) and distrib- uted under the terms of the Cre- ative Commons Attribution - Non Commercial - No derivatives 4.0 International License, which per- mits use, distribution and repro- duction in any medium, provided the original work is properly cit- ed as specified by the author or licensor, that is not used for com- mercial purposes and no modifi- cations or adaptations are made. Data Availability Statement: All relevant data are within the paper and its Supporting Infor- mation files. Competing Interests: The Author(s) declare(s) no conflict of interest. ISSN 2279-7149 (online) http://www.fupress.com/bsfm-jems 2020 Firenze University Press JEMS - Journal of Early Modern Studies Multimedia and Interaction Design in the ‘Stone with a Story’ Project Valentina Rachiele ISIA Urbino () Abstract The article reports on the evaluation of the multimedia and interaction design for the exhibition ‘La pietra racconta. Un palazzo da leggere’ (‘Stone with a Story. Reading the Palace’), which opened in the spring of 2017 at the Palazzo Ducale in Urbino. I discuss the forms that the communication model took, using different technologies in different locations, offering visitors the opportu- nity to perceive the artifacts in a way that would not be possible in a traditional exhibition. The use of advanced technologies along with interaction-design research can truly transform and improve the museum experience, providing better learning and understanding, but only if there is collaboration among professional figures. Keywords: Graffiti, Interaction Design, Museum, Physical Interfaces, User Experience 1. Introduction For years, Italian museums and museums worldwide have been increasing the presence of technology and interactive devices in their institutions so as to allow visitors a more enjoyable and informative experience. With audio guides, interactive kiosks, videos and digital apps, museum goers no longer ‘look’ at an exhibition, but rather become proactive and even physically involved (Pekarik 2002). Consequently, as each exhibit takes visitors beyond just the visual and engages them in a sensorial experience where aspects of the art can be manipulated and combined to give a deeper understanding and meaning, the role of the visitor as passive observer becomes one of creator and even curator. The aim of this article is to investigate how these forms of interaction and participation can create a synergy between historical research and curatorship of exhibitions through an examination of the real case study of ‘La pietra racconta. Un palazzo da leggere’ (‘Stone with a Story. Reading the Palace’) which opened in the spring of 2017 at the Palazzo Ducale in Urbino. valentina rachiele204 2. ‘Stone with a Story. Reading the Palace’: The Exhibition The initial idea that became the basis for the exhibition can be traced back to the 1990s when Raffaella Sarti (University of Urbino ‘Carlo Bo’) began to do research into the graffiti1 on the walls of the Palazzo Ducale in Urbino. Names, dates, drawings, games, reflections on the meaning of life, signs and symbols – a veritable narrative chorus of stories had been carved and scratched into the stone or scribbled on the walls with sharp instruments, pencils and charcoal over the course of approximately five and a half centuries (Sarti et al. 2017, 28). Sarti conducted a systematic review of the graffiti which was recorded, transcribed and interpreted, thereby reconstructing a genesis and a context. In 2015 Manuele Marraccini, a Master student in Photography of Cultural Heritage at ISIA in Urbino, put forward a thesis proposal which would include the use of advanced digital photography to expand the previous research done by Sarti. The following year ISIA was invited by the Galleria Nazionale delle Marche to install and curate Marraccini’s thesis exhibition which would involve not only Marraccini himself, but also Raffaella Sarti, students and faculty members at ISIA, and a collaboration with Matteo Dellepiane, the director of the Department of 3D Technology for Cultural Heritage at the Visual Computing Lab, ISTI-CNR in Pisa. This project is therefore the result of the successful collaborative effort of an inter-disciplinary group of historians, researchers, graphic designers, interaction designers, computer programmers and videographers. The true protagonist of the exhibit was the graffiti itself which, due to its very nature, was spread throughout the rooms and external walls of the Palazzo Ducale of Urbino as well as displayed in specifically designated exhibition spaces through photographic reproduction. Using this unique installation and curatorial organization, visitors were able to find information and details about the actual graffiti on the walls, door frames and columns through traditional information panels, charts and videos and at the same time they could experience the photography of cultural heritage through high-resolution photographic reproductions that revealed graffiti, or aspects thereof, which were otherwise not visible to the naked eye. The specific technologies of photography of cultural heritage, which include RTI (Reflectance Transformation Imaging),2 3D photography, image tracing using vector paths and 3D printing (Sarti et al. 2017, 198-214) were also used to decipher illegible graffiti and to correct earlier transcriptions and interpretations. These innovative research and analysis technologies were part of a two-part communication model commonly found inside museums (Mandarano 2011, 214-215): the first being ‘in loco’ along the exhibition itinerary and in the rooms inside the Palazzo Ducale, the other being ‘remote’, i.e. allowing visitors access to information from wherever they are, not necessarily in Urbino. The multimedia designed for the exhibition were therefore schematically sub-divided into the two aforementioned major categories, ‘in loco’ or ‘remote’. In this specific case, the first category can be further divided into two sub-categories, ‘in loco’ being both the areas located within the specific exhibition itinerary (temporary exhibition spaces, the so-called ‘Banchetti’ area) and those areas immediately outside the itinerary yet still within the perimeter of the Palazzo Ducale. 1 For convenience I will use the generic term ‘graffiti’, even if I must note that the corpus covered here consists of many miscellaneous artifacts – nevertheless all written, carved and scratched on the walls of Palazzo Ducale. This article has been translated from Italian by Christine Di Staola. 2 Reflectance Transformation Imaging (RTI) on , accessed 10 January 2020. multimedia and interaction design 205 3. ‘In loco’: Part One of the Multimedia Communication Model For this part of the multimedia communication model, touch screens were located along the exhibition itinerary and in two other points in the Palazzo Ducale of Urbino. Visitors using the interactive multimedia started with a stylized floor plan of the exhibition rooms which were marked with the location of each of the graffiti (figure 1). Figure 1 – Stylized floor plan of the exhibition rooms () Licensed by Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported By tapping on each marker, visitors had immediate access to detailed charts for the graffiti and were able to view the multimedia reproductions (RTI, 3D, high-definition photography with vector tracing). The interactive kiosks, which served both as a physical anchor and a map inside the exhibition, were aimed at making the location and the characteristics of all graffiti in the exhibition as clear as possible. Interactive screens placed along the exhibition itinerary also allowed groups to participate in the interaction together, which in turn allowed for greater inclusion in the process of active and collaborative learning (Hornecker and Stifter 2006, 135). Visitors could also interact directly with the graffiti on the map, observing it in its original state and then analyzing it through digital manipulation; something that clearly would not have been possible with just an exhibition of the original writings and drawings (figure 2). valentina rachiele206 Figure 2 – A visitor interacting with the graffiti on the map () Licensed by Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported The various levels of interaction (viewing the map with the markers, selecting a marker and visualizing the information charts, or digitally manipulating the 3D or RTI objects) allowed visitors to personally engage with the exhibition and actively participate in the learning process (Prince 2004; Birchfield et al. 2008, 965). The length of time and depth of engagement had been specifically calculated so as not to create long queues at the kiosks or to block visitor traffic. Interactive kiosks with a website created ad hoc for the exhibit were designed to be located at the ticket desk on the ground floor of the Palace and at the entrance to the first floor (not part of the exhibition itinerary) so that any interactive content that might require lengthier viewer engagement would take place in areas away from the displays (Mandarano 2011, 218-219). As previously mentioned, the graffiti was not only located in specific exhibition rooms but was also to be found in all the rooms and on some of the outer walls of the Palazzo Ducale. In this case, the informational organization included the use of QR codes in areas where there were one or more examples of graffiti (such as door frames, walls and other architectural supports). By framing the QR code with a mobile device, visitors could view a webpage that showed charts detailing the specific graffiti. The QR codes and their information charts were visible and available for consultation in the entire Palazzo Ducale and, as they are part of the website which in itself acts as a sort of permanent exhibition, have remained so even after the end of the exhibition (figure 3). multimedia and interaction design 207 Figure 3 – QR codes in the Palazzo Ducale () Licensed by Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported 4. ‘In Loco’ and ‘Remote’: an Extension of Part One of the Multimedia Communication Model Created as a repository for an entire corpus of digital media including high-resolution photography, vector tracing, 3D photography and RTI, the website (which can be viewed or navigated anywhere there is an internet connection) is without a doubt the main multimedia element of the exhibition. Much in the way that the rooms along the itinerary act like display cases for objects being exhibited, the website contains all the graffiti from the Palazzo Ducale included in the exhibition and displays it in precisely the same manner. What is more, the website allows virtual visitors to ‘handle’ the works on display – rotate them along three spatial axes, modify the light sources in the RTI, zoom in and look at details literally down to the last millimetre – by way of digital manipulation of the photographs, something that would not be traditionally possible with the real graffiti.3 In this instance, what is most interesting is the continuous exchange occurring between real and digital, between real and virtual, between real and enhanced. Initially, for example, the idea was to give visitors torches so that they would be able to see the less visible parts of the wall and therefore be able to better appreciate the graffiti. The idea, later discarded, was brought back through the digital potential of RTI which provided visitors with a sort of ‘virtual torch’. By illuminating the graffiti from various angles, 3 La pietra racconta. Un palazzo da leggere, , accessed 10 January 2020. valentina rachiele208 visitors have an enhanced understanding of the execution, drawing process and meaning of each specifi c example of graffi ti. Th e real-digital dichotomy, which creates a sort of ‘hybrid’ experience (Bannon et al. 2005, 64) within the exhibition itinerary, is also refl ected in the structure of the website. According to user analysis, there is no ideal web user for this kind of exhibition. Due to the nature of the works on display – graffi ti and high-resolution photography – the user pool is extensive and includes interest groups that range from researchers, historians and arts enthusiasts to members of the Urbino community, teenagers and children. What emerged as the most logical question to consider was the real physical museum experience: how does one actually ‘visit’ an exhibition? Museumgoers can be placed into two general categories: those who want to be accompanied or guided throughout the exhibition following information that has been taxonomically sub-divided, and those who prefer to wander freely and discover things by chance or luck instead. Knowing this, the information architecture on the website was structured so that visitors could choose how they wanted to navigate the site, either to ‘consult’ it or ‘explore’ it (fi gure 4). Th e former is aimed at the fi rst category of visitors, while the latter is aimed at the second category. ‘Consult’ divides the graffi ti by theme, matching each with detailed descriptions and allowing viewers to visualize all the graffi ti belonging to a specifi c category – with no particular attention paid to their layout in the space (fi gure 5). ‘Explore’, on the other hand, allows visitors to literally ‘walk around’ the inside of the rooms on a virtual tour and to locate graffi ti by using the markers. By reproducing the physical experience of walking through the rooms of a museum, the digital is able to mimic reality (fi gure 6). Figure 4 – The webpage that offers the possibility to ‘consult’ or ‘explore’ the website () multimedia and interaction design 209 Figure 5 – An example of the taxonomy of graffiti () 5. ‘Remote’: Part Two of the Multimedia Communication Model Th e best example of the real-digital dichotomy mentioned above is the creation of a virtual tour through the rooms in the Palazzo Ducale. It is perhaps more precise to call it a simulation of the Palazzo Ducale, given that virtual tours replicate the exact geography and geometry of the actual rooms through photography and software, as opposed to creating just a virtual version. Moving from room to room on the virtual tour, visitors see markers that indicate the location of specifi c graffi ti and can see it in detail and also view informational charts linked through the site. Even virtual visitors, who are not physically in Urbino, can wander through the rooms in the Palazzo Ducale and see the graffi ti as if they were right before them – while they are seated comfortably on their living-room couch (Forte and Franzoni 1998, 199-200) (fi gure 6). valentina rachiele210 Figure 6 – An example of the virtual tour () 6. Conclusion As mentioned at the beginning of the article, many museums are currently exhibiting their collections through digital and multimedia communication models. It is now more important than ever to consider carefully the forms that these models take, at the level of quality in the experiences that they off er visitors, and at their value in terms of providing greater learning and deeper understanding of exhibitions. Th is case study demonstrates that these types of communication can become eff ective and can truly ‘heighten’ the museum experience, but only if professional fi gures – researchers, historians, scientists, museum directors, graphic designers and IT specialists – are involved in their creation and their preparation. Technologies such as advanced digital photography and disciplines such as interaction design have much in common with cultural heritage: they are all forms of communication essentially based on the impact, eff ect and experience of the visual (Mandarano 2011, 239). In this specifi c case, it was only when the graffi ti of the Palazzo Ducale were made truly visible through the use of digital technology that researchers were able to fully comprehend and catalogue the work. In this sense, technology’s gift to cultural heritage was rendering the invisible more visible, by giving museum visitors and researchers the ‘glasses’ they needed to be able to see reality more clearly. multimedia and interaction design 211 Works Cited Bannon L.J., Steve Benford, John Bowers and Christian Heath (2005), ‘Hybrid Design Creates Innovative Museum Experience’, Communications of the ACM 48, 3, 62-65, doi: 10.1145/1047671.1047706. Birchfield David, Brandon Mechtley, Sarah Hatton and Harvey Thornburg (2008), ‘Mixed-reality Learning in the Art Museum Context’, in A. Del Bimbo, A. El Saddik, A. Jaimes, K.S. Candan, C. Grivodz, S. Wuong, eds, MM ‘08. Proceedings of the 16th ACM International Conference on Multimedia (Vancouver, 26-31 October), New York, Association for Computing Machinery, 965- 968, doi: 10.1145/1459359.1459534. Forte Maurizio and Margherita Franzoni (1998), ‘Il museo virtuale: comunicazione e metafore’, Sistemi intelligenti 10, 2, 193-240, doi: 10.1422/3476. Hornecker Eva and Matthias Stifter (2006), ‘Learning from Interactive Museum Installations about Interaction Design for Public Settings’, in T. Robertson, ed., Proceedings of the 2006 Australasian Computer-Human Interaction Conference (Sydney, 20-24 November), Sydney, OZCHI, 135-142, doi: 10.1145/1228175.1228201. Mandarano Nicolette (2011), ‘Musei connessi: le nuove tecnologie nei musei romani’, in M. Ilie and C.M. Travaglini, eds, Rapporto sul patrimonio culturale a Roma, Roma, Centro per lo studio di Roma (CROMA), 213-240. Marraccini Manuele (2015), Un Palazzo da Leggere, MA Dissertation in Cultural Heritage Photography, ISIA (Istituto Superiore per le Industrie Artistiche), Urbino. Pekarik Andrew and Smithsonian Institution, Office of Policy and Analysis (2002), Developing Interactive Exhibitions at the Smithsonian, Washington, Smithsonian Institution, Office of Policy and Analysis. Prince Michael (2004), ‘Does Active Learning Work? A Review of the Research’, Journal of Engineering Education 93, 3, 223-231, doi: 10.1002/j.2168-9830.2004.tb00809.x. Reflectance Transformation Imaging (RTI), , accessed 10 January 2020. Sarti Raffaella, Manuele Marraccini, Angelo Rubino and Matteo Dellepiane (2017), La pietra racconta. Un palazzo da leggere, Urbania, Arti Grafiche Stibu, , accessed 10 January 2020.