Microsoft Word - 31442_GG3.docx Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 The Flexibility of Dense 3D Data Capture: Rapid Documentation of Monumental Fountains in Rome MARTINA ATTENNI, MARIKA GRIFFO, CARLO INGLESE, and ALFONSO IPPOLITO, Sapienza University of Rome, Italy ERIC LO and DOMINIQUE RISSOLO, University of California, San Diego, USA The knowledge and study of built heritage is now deeply connected to methodologies associated with the measure of surface details by recording point cloud coordinates. These methodologies enable researchers to gather a wider range of information, which is increasingly connected to technological advances. Such approaches influence the management of data, and these data are often redundant due to the ways in which they are captured. Massive or raw data capture does not include preliminary selection based on metric, geometric, and material features of the object. A multi-scalar approach, in which the criteria for data capture depends on the goals of the survey, is needed to optimize the relationship between information and the scale of the models to be built. This case study involving a selection of fountains in Rome aims to apply these principles to urban contexts defined by a strong spatial connection between architectural and sculptural elements. Survey can express this distinctiveness through complex, dynamic, and effective digital models. Keywords: 3D Data Capture, Integrated Survey, Fountains, Rome. SDH Reference: Martina Attenni et al. 2021. The Flexibility of Massive Data Capture: Rapid 3D Documentation and the Fountains of Rome. SDH 5, 1, 62-74 https://doi.org/10.14434/sdh.v5i1.31442 1. INTRODUCTION The interpretation of architecture uses a deconstruction and subsequent organized reconstruction of both quantitative and qualitative information. This is possible because of the on-site “experience” of the object as well as the researcher’s related knowledge and domain expertise. This modality is related mainly to a heuristic approach that enables one to extract the meaningful characteristics of the object from the “whole indistinct.” The work for this article was carried out under project C26V17TSAF, which was funded by Sapienza University of Rome. Authors’ addresses: Martina Attenni, Marika Griffo, Carlo Inglese, Alfonso Ippolito, Department of History, Representation and Restoration of Architecture, Sapienza University of Rome, Piazza Borghese 9, 000186, Rome, Italy; email: (martina.attenni, marika.griffo, carlo.inglese, alfonso.ippolito)@uniroma1.it; Eric Lo, Dominique Rissolo, Qualcomm Institute, University of California, San Diego, 9500 Gilman Drive, La Jolla, California, 92093-0436, USA; email: (eklo, drissolo)@ucsd.edu © 2021 by the authors; licensee Studies in Digital Heritage, IU, Bloomington (IN), USA. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution License (CC BY-NC) Rapid Documentation of Monumental Fountains in Rome 1:63 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 In the field of architectural heritage, the concept of “whole indistinct” refers to the amount of information derived from the tools and techniques of massive data capture performed by optical imaging or 3D scanning, which is anything but a selective approach. Data related to superficial qualities of the object are acquired uncritically and objectively without semantic distinction or hierarchy among the millions of points captured. Even though this stage of the process seems to be completely automated due to its connection to tools and devices, one must make several choices related to how such devices should be used. This confirms the importance of the survey project, in the traditional sense, even in an era in which tools and technologies are increasingly sophisticated and automated [Bianchini et al. 2017b]. On the other hand, massive data capture implies placing trust in processing operations to define concepts, theories, and thematic and semantic connections in addition to geometrical and physical features. The construction of a complete digital copy of the object results in a manifestation – temporal and physical – that is unique to virtual space. The model is therefore disconnected from the limits of our perception of the physical world and assumes an absolute value. This modalità di lettura (way of reading) is not a substitute for experiencing the object, in a physical sense, but is nonetheless empowering and rich in possibilities. The experimentation presented here intends to evaluate the relationship between the data capture stage of quantitative data production and the qualitative data extraction by emphasizing to what extent the first affects the second [Bianchini et al. 2017a]. The present study has been conducted on a selection of fountains (and their urban context) in Rome and is considered meaningful from a morphological, symbolic and – concerning the fountains – a decorative point of view. The research project applied structure-from-motion (SfM) / Image Matching (IM) photogrammetric processes with the long-term objective of obtaining multi-scalar numerical models. The goal is to further develop analysis on a global level – studying the urban context of architectural and sculptural elements – and on a detailed level, by focusing on the deconstruction of the sculptural artifact into its own elementary parts [Centofanti et al. 2012]. This process emphasizes the management of an increasing amount of data (attributable to the use of technology). The sheer quantity of data results in significant geometrical overlap (with respect to numeric models) and produces a redundancy of data (prompting some level of intervention). For this reason, a pre-dimensioning of the data-capture stage – according to the objectives of the study and the scale of the final model – can reduce or limit disorder in the virtual world. 2. METHODOLOGY These concepts guided the study of the fountains in Piazza Mattei, Piazza Santa Maria in Trastevere, and the twin fountains of Piazza Farnese (Fig. 1). The fountains are essentially analogous in function, with each one exhibiting a unique morphology and a range of sculptural or decorative details. From a formal point of view, the axis of each fountain is connected to the geometrical layout of the squares in which they are located. Their urban background or context is composed of historical buildings with high architectural value. From a dimensional point of view, the perpendicular axes of the water basins vary from 11 m for the Piazza Farnese fountains to 5 m for the Fountain of the Turtles in Piazza Mattei (with their heights being 6 m and 2 m, respectively). Finally, from a typological point of view, there are pronounced differences in the historical sequence of events with respect to the architectonic construction and the related urban space. 1:64 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Figure 1. Fountains in Rome; an overview of the objects of study in the urban context. Rapid Documentation of Monumental Fountains in Rome 1:65 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 The sixteenth-century Fountain of the Turtles, which was constructed in 1581 as part of a Giacomo Della Porta project, is the barycenter of the square. Its sculptural layout is rich and elaborated and was realized by Taddeo Landini. In combination with the limited size of the square, it becomes, from a perceptual point of view, the focal point of an open-air “room.” The Piazza Santa Maria in Trastevere offers the wider perception of an empty urban space. It is set up on a quadrangular plan with the fountain in an eccentric or off-center position. The location of a fountain in this area is documented as early as Roman times. There were several restoration interventions that changed its position and configuration, and the main interventions are celebrated in engravings located at the base of the fountain. In Piazza Farnese, the twin fountains are the focal points of a rectangular structure. The basins were originally located in Piazza Venezia and then moved to Piazza Farnese as a decoration. They were turned into fountains in 1626 by Gerolamo Rainaldi. The architect decided to not only change their function, but also their configuration by adding basins at the base and at the top of each fountain [Delli 1985]. In each of the presented case studies, the architectural and sculptural elements are evocative of their era. The spatial and symbolic connection with urban façades gives them an urban role that is characteristic of the identity of Rome as well (Fig. 2). The process approaches the study of typologically comparable elements by identifying a procedure that can be adopted in all contexts in which basic documentation is required and the methods of data acquisition and processing are explicit and reliable. The efficacy of the documentation of the forms and sculptural elements of the fountains is tied to the potential to survey (with a high level of detail) the characteristic elements of the typology. The data acquisition, therefore, concerned all parts of the objects that were visually and physically accessible, with the aim of producing numerical models that could be easily explored and via which further contextual analyses could be conducted. Figure 2. Urban interpretation: single image rectification and photo stitching of façades of Piazza Santa Maria in Trastevere. This analysis has been performed to contextualize the fountain of the Piazza. 1:66 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 A primary consideration is the scale of the selected object. In the case of the fountains, the SfM process is more controllable and, to a certain extent, independent from other data capture techniques (making integration with other modalities not always essential). The opposite is true with respect to architectural and urban scales, where it is considered necessary to integrate different data capture methodologies (including survey) to achieve metric and geometric accuracies and to accommodate heterogeneous data gathered from different sources. Finally, on the diametrically opposite side, is the detailed relief of objects with dimensions on the centimeter scale. In this case the acquisition strategy cannot ignore the use of pre-arranged instrumentation and photographic settings, the effectiveness of which strongly determine or condition the obtained result (Fig. 3). Figure 3. The scale of survey, integrated data capture. Rapid Documentation of Monumental Fountains in Rome 1:67 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Even though this distinction highlights differences in the documentation approach used for heterogeneous elements, the process presented here demonstrates the added benefit of using a level of computational power that does not impose practical limits on data capture. For the analysis of the artifact from an architectural and compositional point of view, the reading of the detail and of the chromatic features of surfaces plays an important role. The nature of the documentation project – defined by a strategy that fits the objectives of the research – determines the methods and tools for data capture and data processing. In this case, the process is not linked to limits imposed on the number of images (and the level of resolution) that can be acquired. This makes it possible to obtain three-dimensional numerical models, with high point density, that better enables the user to “experience” the object. These models play a fundamental role in the documentation of the object as well as the dissemination of digital data (Fig. 4). The numerical model, meant as a product of a scientifically reproducible process, is well suited to sharing and implementation among researchers in all fields connected to the investigation of the object. The organization of a digital database open to users with different levels of specific knowledge about the object would allow continued systematic documentation would provide analytical tools for scientific research [Apollonio et al. 2018]. Figure 4. Architectural interpretation: the portico of church of Santa Maria in Trastevere. From left to right: numeric model derived from laser scanner 3D data capture, 2D ortho image extracted from structure-from- motion numeric model, and 2D geometric representation of the main construction lines from architectural reading of the portico. 1:68 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Figure 5. Fountain of the Turtles, Piazza Mattei. Rapid Documentation of Monumental Fountains in Rome 1:69 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Figure 6. Southeast fountain, Piazza Farnese. 1:70 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Figure 7. Northwest Fountain, Piazza Farnese. Rapid Documentation of Monumental Fountains in Rome 1:71 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 3. WORKFLOW A consistent image acquisition strategy for SfM photogrammetry was applied to each fountain. The cameras and lens used were a Sony a7RIII 42.4 MP full-frame mirrorless camera with a 16-35 mm lens (shot at 16 mm) and a Sony QX1 20.1 MP lens-style camera (APS-C type CMOS sensor) with a 20 mm lens. The a7RIII was hand-held and afforded the field technician between 0-2.0 m of vertical range. The compact QX1 was mounted on a 1.24 m telescopic prism pole that extended to 2.15 m. This allowed the QX1 to reach approximately 4 m above the piazza floor. The QX1 camera was controlled via a smartphone using the Sony PlayMemories mobile app. Attempts were made to achieve comprehensive coverage of each fountain (with approximately 60- 80% overlap between photographs) using a systematic convergent approach while following a circular to rising “dome-shaped” path (switching from hand-held to pole-mounted camera when appropriate). Once the “dome” of images was accomplished at a consistent distance from the fountain, the technician would then shoot their way towards the fountain to increase the degree of coverage and level of detail necessary for accurate geometric reconstruction and higher point density. Photo scales (1-m rods with 10 cm increments) were used, and each imaging run began and ended with a photo of a color calibration chart with 12 reference colors and 6-step grayscale (DGK Color Tools). A manageable number of images were acquired for each fountain. The numbers are as follows: Piazza Mattei: 436; Santa Maria in Trastevere: 399; Piazza Farnese east: 348; Piazza Farnese west: 249. The quantity of images (or poses) proved more than adequate to realize the desired result, though an increased level of detail could be achieved with additional imaging (Figs. 5-7). Numerous challenges were encountered in each piazza. Chief among them was pedestrian traffic as well as parked and moving cars, trucks, motorbikes, and bicycles. These resulted in line-of-sight occlusions and dynamic scenes (the latter being more manageable in pre-processing and processing phases). Railings and fences also created line-of-sight issues as well as practical access challenges. Both water and wet surfaces can also prove problematic, though in the case of the fountains, did not inhibit 3D reconstruction. Given the height of the fountains, a significant limitation turned out to be the height of the pole (for the QX1 camera). A longer pole (perhaps 3-5 m in length) would have afforded the camera additional perspectives and would have resulted in greater coverage overall. The noise along the upper edges of the fountain (in the unmeshed models) is partly the result of this limited range. Sunny, cloudless skies persisted during the acquisition period. Ideally, the fountains would have been imaged in overcast conditions (providing diffused and even lighting). Bright sunlight can result in erroneous feature-matching (as a result of hard shadow boundaries), inaccurate RGB values, and exposure challenges. Without proper metering, for example, the fountains would tend to be underexposed due to the sun-lit building façades behind them. It should be noted that we can take advantage of the 15 stops of dynamic range offered by the camera’s sensor to mitigate the potential loss of detail within the underexposed regions of each composition. Following the field acquisition, the images are pre-processed for image quality, and then processed through Agisoft Photoscan 1.4.2 to create the 3D digital surrogate. Pre-processing consists of a 1:72 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 number of steps to ensure the quality of the images to be used in the reconstruction. First, any superfluous images containing notes, beginning/end of acquisition markers are removed, along with any imagery of obviously poor quality, such as those with poor focus or exposure settings. If color accuracy is a requirement, and particularly if multiple cameras are used, a color calibration target can be used to correct the white balance to make the cameras more consistent. While it is ideal to acquire well-exposed images, field conditions do not always permit ideal lighting, and exposure correction and tone mapping can be applied to the imagery in the form of decreasing contrast, reducing highlights, and brightening shadows or darker areas. If necessary, noise reduction can also be applied before exporting the images for use in the 3D reconstruction. For each area, all images from relevant cameras were combined in the reconstruction project. The first stage of processing generates a sparse point cloud and an estimate of the location and orientation from where each image was taken. After a visual inspection of the point cloud and estimated camera positions, an appropriate bounding box is defined to maximize resolution on the area of interest. Next, we generate a dense point cloud, which is our primary 3D representation of the site. Finally, we define the scale of the model using multiple measurements from scale bars in the imagery. This scaled dense point cloud is then exported into Viscore (a proprietary point cloud renderer and point-based visual analytics software), for analysis (https://chei.ucsd.edu/viscore/). If the sparse point cloud reconstruction does not align well, and only a minority of the cameras are misaligned, we reset their alignment and try to align again, or simply remove the cameras if they are redundant. When a significant number of the cameras are misaligned, it is likely that there is either an issue with image quality or there may be inadequate overlap to robustly cover the area. If possible, these issues should be reviewed, and new imagery acquired, but when this is not possible, there are strategies for dealing with marginally reconstructible imagery. First, reducing the resolution on the imagery can allow blurry or poorly focused imagery to align. This is facilitated by changing the sparse reconstruction quality in the photogrammetry software. Another option is to reconstruct chunks which do align individually, and manually align those chunks visually. These "remediated" reconstructions should be assumed not to have the same geometric fidelity as those that reconstruct well (though they can still yield a visual result). In cases where sites of interest, such as in Piazza Farnese, are adjoining and may have overlapping coverage, the two sites (fountains) can be combined into one project in order to provide context and a common coordinate system. To preserve detail on each area of interest, we duplicate the project and use a bounding box for each individual area of interest, though an additional copy can be processed with a bounding box including the entire area for context. With respect to image acquisition in Piazza Farnese, the accurate geometric reconstruction of each of the two individual fountains was given priority over the spatial accuracy of the two fountains with respect to each other, the piazza, and the surrounding buildings. The 3D models turned out to be extraordinarily accurate, with only a 2-3 cm difference between the hand-measured physical widths of the lower basins and the corresponding measurements taken off the scaled models. In general, the fountain in Piazza di Santa Maria in Trastevere presented the fewest number of image acquisition challenges and yielded a good overall result. Rapid Documentation of Monumental Fountains in Rome 1:73 Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 To represent the building façades associated with each piazza, 2D orthophotos were extracted from the 3D point clouds in Autodesk ReCap, then imported into AutoCAD to produce 2D architectural drawings (e.g., Fig. 4). An additional interpretive approach involves the use of rectified photomosaics of building façades to contextualize the fountains and the elements of the piazzas (e.g., Fig. 2). The original data assets (images and point clouds) as well as all derivative products (3D and 2D) can be archived and interactively viewed within Viscore as a multimodal digital twin. 4. CONCLUSION The numerical model produced with photogrammetric imaging constitutes an effective form of basic documentation. The level of detail achieved enables one to identify and characterize materials and structural elements, evidence of degradation or weathering, and the proportional relationships of the parts to the whole. Indeed, the possibility of obtaining a very high level of detail allows for the evaluation of the consistency of an object. Shifting different scales or “ways of reading” within the same model is a way to validate the data capture and processing methodologies concerning typologically similar objects. Moreover, it permits one to evaluate and study the design and construction rules that connect the smaller scale of the fountains to the urban scale of the piazzas and the architectural façades. The numerical models are placed in continuity with the current logic of dissemination and use, constituting a fundamental element in the knowledge of architectural and urban heritage, while imposing a new definition for the relationship between the scale of the model and its informative content. In this context, further phases of the research will be related to the integration of data, metadata, and 3D models into a 3D database to enable further typological studies, to archive existing documentation, and to share contents among the research community. 5. ACKNOWLEDGEMENTS Research was supported by Sapienza University of Rome. The authors would like to thank the students, staff, and faculty of the Department of History, Representation and Restoration of Architecture, especially Carlo Bianchini. At the University of California, San Diego, the authors thank Michelle Hermas, Falko Kuster, Crystal Liu, Christopher McFarland, Dominique Meyer, Vid Petrovic, and Ramesh Rao. 6. REFERENCES Fabrizio Ivan Apollonio, Vilma Basilissi, Marco Callieri, Matteo Dellepiane, Marco Gaiani, Federico Ponchio, Francesca Rizzo, Angelo Raffaele Rubino, Roberto Scopigno, and Giorgio Sobra. 2018. A 3D-centered information system for the documentation of a complex restoration intervention. J. of Cult. Herit. 29 (2018), 89-99. Carlo Bianchini, Michael Hess, Carlo Inglese, and Alfonso Ippolito. 2017a. Quality or Quantity: The role of representation in Archaeological Architecture. In Marco Ceccarelli, Michela Cigola, and Giuseppe Recinto, eds. New Activities for Cultural Heritage. Proceedings of the International 1:74 Attenni et al. Studies in Digital Heritage, Vol. 5, No. 1, Publication date: June 2021 Conference HeritageBot 2017. Cassino (Italy) 21-22 September 2017. Springer International Publishing AG, 150-157. DOI 10.1007/978-3-319-67026-3 Carlo Bianchini, Carlo Inglese, and Alfonso Ippolito. 2017b. I teatri del Mediterraneo come esperienza di rilevamento integrato. The Theatres of the Mediterranean as integrated survey experience. Rome: Sapienza Università Editrice. Mario Centofanti, Stefano Brusaporci, and Vittorio Lucchese. 2012. Architectural Heritage and 3D Models. In P. Giamberardio, D. Iacoviello, R. Natal, and R.S Tavares, eds. Computational Modeling of Objects Presented in Images: Fundamentals, Methods and Applications. Switzerland: Springer International Publishing, 31-49. Sergio Delli. 1985. Le fontane di Roma. Rome: Schwarz & Meyer. Received September 2020; revised April 2021; accepted May 2021.