early warning identity threat and mitigation system eceasst_netsys2021_camera_ready_latex-2.pdf demo: traffic splitting for tor —a defense against fingerprinting attacks electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) demo: traffic splitting for tor — a defense against fingerprinting attacks sebastian reuter, jens hiller, jan pennekamp, andriy panchenko, and klaus wehrle 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst demo: traffic splitting for tor — a defense against fingerprinting attacks sebastian reuter1, jens hiller1, jan pennekamp1, andriy panchenko2, and klaus wehrle1 1communication and distributed systems, rwth aachen university, germany 2it security, brandenburg university of technology, germany {lastname}@comsys.rwth-aachen.de, {firstname.lastname}@b-tu.de abstract: website fingerprinting (wfp) attacks on the anonymity network tor have become ever more effective. furthermore, research discovered that proposed defenses are insufficient or cause high overhead. in previous work, we presented a new wfp defense for tor that incorporates multipath transmissions to repel malicious tor nodes from conducting wfp attacks. in this demo, we showcase the operation of our traffic splitting defense by visually illustrating the underlying tor multipath transmission using led-equipped raspberry pis. keywords: onion routing; website fingerprinting; multipath traffic; privacy 1 introduction tor is one of the most popular anonymization networks [cmh+20]. it is commonly used to bypass censorship, to maintain the freedom of speech, or to repel trackers on the internet and in the iot domain [hpd+19, phr+19]. to provide anonymity to its users, tor leverages the onion routing principle [cmh+20]: all traffic is dismembered into fixed-size cells and routed from the client (onion proxy, op) to the server through a virtual circuit consisting of multiple relaying onion routers (ors), known as entry, middle, and exit ors (depending on their position in the circuit). during transmission, tor applies a multi-layered encryption scheme to all cells, which obfuscates their content and the communication endpoints. more specifically, tor achieves anonymity as each or knows only its predecessor and successor in the circuit and, consequently, no or knows the identity of both the client and the server at the same time. hence, tor hides the client’s identity (ip address) from all involved entities except nodes on the path between the op and the entry node, which, however, are unaware of the accessed server [cmh+20]. website fingerprinting (wfp) wfp attacks enable attackers to subvert tor’s anonymity guarantees [cmh+20, plz+16]. to this end, they exploit the fact that tor’s multi-layered encryption does not properly obfuscate revealing timing and volume patterns within the transmitted data stream. a local passive adversary located at the first hop of a tor circuit (i.e., a malicious entry or or its isp) can exploit this knowledge to unveil the website retrieved over a tor circuit. specifically, the attacker employs machine learning to identify traffic patterns for the loading of websites over tor circuits. as its privileged position also provides the attacker with the ip address of the circuit’s tor user, it can thus link the tor user to the visited website. to counter such 1 / 4 volume 080 (2021) demo: traffic splitting for tor wfp attacks, several defenses have been proposed [cnj14, wg17, jip+16], which, however, usually introduce undesirable bandwidth or latency overheads [cmh+20]. our contributions in previous work [cmh+20, cmp+19, phr+19], we presented a novel defense against wfp attacks based on a multipath approach that distributes tor cells over multiple entry ors, and showed that a well-tailored distribution strategy achieves a superior defense against malicious entry nodes while incurring only limited overhead. in this demo, we visualize the traffic distribution of our defense to provide tor users with an intuition on the operation of the defense and thus foster its acceptance. 2 related work the discovery of wfp attacks started an arms race between corresponding defenses and improved versions of the attack. we briefly discuss previous defenses and refer to [cmh+20] for a comprehensive overview. walkie-talkie [wg17] combines half-duplex data transmission with traffic shaping to distort distinguishable traffic patterns. cs-buflo [cnj14] and wtfpad [jip+16] rely on transmitting padding data to warp the traffic patterns. yet, research has found that existing defenses either introduce huge overhead or are not sufficiently effective, even though wtf-pad is currently considered for inclusion into tor [cmh+20]. related to traffic splitting, research considered implementing multipath approaches in tor to improve its performance, e.g., conflux [abeg13], but not to counter wfp attacks [phr+19]. 3 design of the traffic splitting defense our wfp defense splits traffic between the tor client and middle or to distribute it over multiple entry ors (cf. figure 1a). thereby, a traffic splitting strategy distorts traffic patterns with respect to malicious entry nodes [cmh+20]. with a suitable strategy, this traffic splitting is highly effective in repelling wfp attacks (< 14% accuracy compared to more than 98% without defense) while introducing hardly noticeable overhead [cmh+20]. multipath circuit setup: to set up an m-fold multipath circuit, our design re-uses tor’s existing circuit mechanisms [cmh+20]: first, the client builds a common three-hop initial circuit and then establishes m−1 further two-hop sub-circuits via different entry ors to the middle or. to enable the middle or to join the created sub-circuits, the client sends a cookie value (20-byte nonce) to the middle or using the initial circuit and waits for an acknowledgement (set_cookie, cookie_set). following, the client sends the same cookie via each remaining sub-circuit to the middle or (join). for each sub-circuit, the middle or uses the received cookie to join the sub-circuit with the initial circuit and sends an acknowledgement (joined). multipath traffic transmission: a distribution strategy determines the order in which cells are sent over the sub-circuits such that the middle or can send cells to and merge cells received from the client in the correct order [cmh+20]. to set up this strategy, the client regularly sends splitting instructions to the middle or (instruction, info). the proper selection of this strategy is essential to prevent any fingerprintable traffic patterns and thus influences the effectiveness of the defense [cmh+20]. netsys 2021 2 / 4 eceasst entry 1 client server entry 2 entry 3 exitmiddle (1) 3-hop initial circuit (3) s et c ook ie (2) 2-hop sub-circuit(s) (4) join (5) instructions (6) split & merge (6) split & merge initial circuit additional sub-circuits modified nodes bi-directional traffic cells (a) schematic illustration. (b) demo setup. figure 1: overview of our tor traffic splitting wfp defense. implementation: we implemented our defense and added demo functionality in tor 0.3.5.11, the latest version with long-term support. our source code is publicly available [cod21]. 4 illustrating the traffic splitting operation (demo setup) to illustrate the operation of our wfp defense, we use seven raspberry pis (rpi) (cf. figure 1b). these rpis are interconnected among each other using ethernet (local setup) as well as the public internet (use of real tor network). in the local setup, we configure all rpis to form a separate local tor network and build multipath circuits using one rpi as client, three rpis as entry ors, and the remaining three rpis as middle or, exit or, and server, respectively. during the demo, the client repeatedly builds new multipath circuits to the exit or and requests a website from the server or transfers large amounts of data using the bandwidth measurement tool iperf. thereby, the circuit setup (building of sub-circuits as well as the joining operation) is visualized with live log output from the respective rpis. more importantly, to visualize the following multipath traffic, each rpi is equipped with leds which represent transmissions on each subcircuit with a different color. furthermore, to also differentiate the transmission direction, we use pairs of leds (same color), one led for each direction. thus, the client and middle rpis each have three pairs of leds (one for each sub-circuit), whereas the other rpis are equipped with one pair of leds. during the data transmission, each rpi uses the led corresponding to the transmission direction (and circuit, if applicable) to signal the transmission of tor traffic cells, thereby evidently illustrating the transmission behavior of our multipath defense. for improved visibility in the face of high networking speeds, the rpis are configured to signal only every 100th traffic cell per sub-circuit and transmission direction in case of the iperf (bulk) data transmission. in the case of the website request, due to the comparably small amount of data, we maintain the observability by artificially restricting the bandwidth of the contacted server rpi. the demo can alter the number of used paths and show different splitting strategies, i.e., from easily conceivable strategies like round-robin up to the best defense strategy determined in [cmh+20]. beyond this local setup, we also showcase the application of our traffic splitting approach within the real tor network. to this end, we replace the local ors (rpis) with nodes de3 / 4 volume 080 (2021) demo: traffic splitting for tor ployed in the live tor consensus and only use an rpi to act as the traffic-splitting enabled client. specifically, we use one of our own splitting-enabled public tor nodes as remote middle or for multipath circuits, which allows us to observe the splitting and merging activities via the live log output. this setup allows us to showcase the use of our defense while accessing virtually any internet resource reachable via the real tor network. 5 conclusion in this demo, we visualize and observe the traffic distribution of our wfp defense using multiple led-equipped rpis within both a local autonomous setup and the live public tor network. thereby, we provide tor users with an intuition on the operation of the defense, thus fostering its acceptance and raise further awareness towards this active area of research. bibliography [abeg13] m. alsabah, k. bauer, t. elahi, i. goldberg. the path less travelled: overcoming tor’s bottlenecks with traffic splitting. in pets. 2013. [cmh+20] w. de la cadena, a. mitseva, j. hiller, j. pennekamp, s. reuter, j. filter, t. engel, k. wehrle, a. panchenko. traffic sliver: fighting website fingerprinting attacks with traffic splitting. in acm ccs. 2020. [cmp+19] w. de la cadena, a. mitseva, j. pennekamp, j. hiller, f. lanze, t. engel, k. wehrle, a. panchenko. poster: traffic splitting to counter website fingerprinting. in acm ccs. 2019. [cnj14] x. cai, r. nithyanand, r. johnson. cs-buflo: a congestion sensitive website fingerprinting defense. in acm wpes. 2014. [cod21] https://github.com/trafficsliver/trafficsliver-net-demo, 2021. [hpd+19] j. hiller, j. pennekamp, m. dahlmanns, m. henze, a. panchenko, k. wehrle. tailoring onion routing to the internet of things: security and privacy in untrusted environments. in ieee icnp. 2019. [jip+16] m. juarez, m. imani, m. perry, c. diaz, m. wright. toward an efficient website fingerprinting defense. in esorics. 2016. [phr+19] j. pennekamp, j. hiller, s. reuter, w. de la cadena, a. mitseva, m. henze, t. engel, k. wehrle, a. panchenko. multipathing traffic to reduce entry node exposure in onion routing. in ieee icnp. 2019. [plz+16] a. panchenko, f. lanze, a. zinnen, m. henze, j. pennekamp, k. wehrle, t. engel. website fingerprinting at internet scale. in ndss. 2016. [wg17] t. wang, i. goldberg. walkie-talkie: an efficient defense against passive website fingerprinting attacks. in usenix security. 2017. netsys 2021 4 / 4 https://github.com/trafficsliver/trafficsliver-net-demo introduction related work design of the traffic splitting defense illustrating the traffic splitting operation (demo setup) conclusion property inference-based federated learning groups for collaborative network anomaly detection electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) property inference-based federated learning groups for collaborative network anomaly detection jens wettlaufer 5 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst property inference-based federated learning groups for collaborative network anomaly detection jens wettlaufer universität hamburg jens.wettlaufer@uni-hamburg.de abstract: while the use of anomaly detection in network security has a long research history, it is rarely used in practice. besides privacy concerns when applied in cross-network settings, and a more difficult attack interpretation, the major drawback consists of the high number of false alarms. one reason is the heterogeneity of sources the model is trained on. in this paper, we propose a network anomaly detection extension that counteracts the heterogeneity of participants by dividing them into learning groups during central or federated training. the learning groups finally contain similar behaving clients, e.g., light bulbs, or pcs of the same department. similar behavior is extracted by hierarchically clustering the predictions of all individual client models similar to a passive property inference attack. our preliminary results based on infiltration attacks of the ids2017 dataset show that the method increases the accuracy and f1 score up to 4.4% and 2.5%, respectively. keywords: anomaly detection, unsupervised learning, property inference, network security, hierarchical clustering, collaborative learning 1 introduction anomaly detection intends to identify deviations or non-conforming patterns from expected behavior. these patterns are called anomalies or outliers. it is broadly researched in tumor identification on mri images, credit card fraud, identity theft, spam detection, and intrusion detection [cbk09]. the common problem of a high false positive rate does not concern for example credit card fraud that much, because the users can clarify the situation easily. however, in network intrusion detection of large networks, even a false positive rate of 1% in a million network flows a day is impractical to manually assess [sp10]. additionally, the interpretation of an anomaly often requires expert knowledge when the anomaly detection task is not clearly formulated [sp10]. furthermore, heterogeneous networks with different types of devices, end users, and thus behavior patterns make it hard to learn normal behavior, and interpret anomalies afterwards. essentially, while anomaly models intend to capture a normal profile, such that subtle deviations are detected, the heterogeneity of entities counteracts this. in addition, real-world challenges arise for the usage of centralized anomaly detection. first, it is challenging for large networks to collect all data for training and prediction. second, startups or small businesses may have too few data samples to train the model effectively. this could be circumvented by collaborating with others, which introduces privacy concerns. in consequence, network anomaly detection is rarely used in practice, even though it is capable of detecting unknown attacks in contrast to signature or policy-based approaches. to overcome these problems, we suggest to extend existing 1 / 5 volume 080 (2021) property inference-based learning groups anomaly detection procedures by dividing heterogeneous clients during the training into groups with similar behaviors, e.g., service usage or user behavior, in an unsupervised fashion. this enables given anomaly models to learn normal behavior more precisely in order to detect even subtle deviations, such as a slight increase of ssh traffic in the it department. further, anomalies can be interpreted and solved easier due to a decreased number of participating clients for a particular anomaly. therefore, our main contributions include: identification of similar behavior. we provide an unsupervised method based on the principle of property inference to identify similar behaving clients, further called learning groups. the method is not restricted to, but evaluated on network anomaly detection in this work. central and collaborative learning. we show that this method can be applied to central and collaborative learning by providing a federated learning extension that does not interfere with its actual protocol nor its privacy and performance guarantees. preliminary results. our preliminary results achieve an increased accuracy by 2.3-4.4% and f1 score by 1.2-2.5% on average between different models. the rest of the paper is structured as follows. section 2 identifies differences to existing solutions. section 3 provides the main research questions, explains the approach, and discusses preliminary results. finally, section 4 summarizes the paper and states next steps. 2 related work network anomaly detection models for different attacks are widely researched. for example, fernandes et al. [frc+19] survey numerous papers emphasizing that the kind of data, i.e., tcpdump, snmp, and ip, determines the detection capabilities. our approach is not determined by a specific method or data type. some works intend to profile ip hosts based on previously analyzed traffic or host parameters [yr08, mdbc10, jgl15]. our method does not necessarily target a specific network anomaly problem, but rather improves existing models. additionally, we intend to integrate the division of learning groups into the actual training without the need for a prior assessment of hosts, which might be infeasible in cross-network scenarios in regard to privacy. aiello et al. [akm+05] analyze host-level communities of interest, i.e., entities that share a common goal or environment, based on host connections to other hosts in regard to popularity and frequency. while this approach is not characteristic dependent, we argue that host-level goals are not necessarily interchangeable with similar behavior of hosts, e.g., regarding time shifts of actions. nguyen et al. [nmm+19] present a federated anomaly detection in a twofold approach. calculated fingerprints are clustered to identify iot device types, which are then used as static properties to apply their anomaly model within a group of devices of one device type only. in contrast, our approach aims at integrating the grouping step into the actual anomaly model training. additionally, our work goes beyond device type similarities and allows for the creation of non-obvious learning groups. 3 property inference-based federated learning groups this work hypothesizes that if groups of similar behaving clients, e.g., smart light bulbs, web servers, or pcs of the hr department, separately train together, the anomaly detection results in netsys 2021 2 / 5 eceasst a more accurate prediction. thus, we aim at answering the following research questions: rq1 how can we automatically infer client behavior properties during training? the goal is to identify similar behaving clients independent of the anomaly detection task, model, and training procedure. rq2 which (non-obvious) properties exist that improve the anomaly detection after training in property-based learning groups? [nmm+19] achieve promising results with device type specific learning groups. however, we also aim at recognizing non-obvious groups of devices that improve the prediction. rq3 how can we apply learning groups in a privacy-preserving and collaborative scenario? to further apply this method in large networks with higher performance and privacy requirements, we need to show that the method also works for collaborative learning. the following paragraphs first sketch an answer to rq3 and focus further on rq1, while addressing rq2 partially in the discussion of our preliminary results. approach. federated learning (fl) is a distributed round-base learning technique between clients with private data and a coordinating server. we modify the fl procedure as follows. first, the server distributes a tuple (mglobal,sn), i.e., a randomly initialized global model and n random samples of network traffic, to the participating clients. then, each client ci trains the received mglobal with its own data, makes a prediction pcin of the random samples sn, and extracts its model updates u ciglobal . it then sends the tuple (u ci global, p ci n ) back to the server. the server now applies the learning group metric to all received predictions pcin and determines the learning groups. finally, the clients continue fl training together within their assigned learning groups. learning group metric. client behavior is expressed through client data. client data is used to train ml models, which then become a representation of the data. thus, client behavior can be inferred based on the trained model. therefore, a metric to identify similarities between models is required. related work in the field of property and membership inference attacks on ml has shown that it is feasible to infer properties of the training data that are solely represented in the model predictions [ssss17]. consequently, we measure the similarity by comparing the prediction outputs of each trained client model for a given input. to identify client behaviors that are closest to each other, we choose hierarchical clustering, in particular agglomerative clustering. while we used three default distance metrics for our first results, further distance metrics [pah+19] will be analyzed subsequently. dataset and implementation. the preliminary results are based on the ids2017 dataset [shg18] containing simulated corporate communication of ten clients and two servers. we choose a subset of the attacks containing infiltration attacks. then, we apply the recommended task related flow-based features [shg18] to our simple anomaly detection models, i.e., isolation forest and elliptic envelope. we provide four baselines including all clients training together and separate, as well as training in obvious learning groups, i.e., clients/servers and operating systems. the values in table 1 are the average of five rounds. further, we test three variants of establishing learning groups by varying the percentile of the training data before clustering, while keeping three as a fixed number of clusters. additionally, we apply three different clustering metrics (cm) to the client-wise predictions of the samples sn with n = 1000: average (average distance), 3 / 5 volume 080 (2021) property inference-based learning groups complete (maximum distance), and ward (minimizes the variance of the clusters being merged). preliminary results and discussion. table 1 compares all performed tests, where min and max correspond to the result of a single learning group, while mean averages all of them. cells highlighted in red provide the information that they consist of one learning group that scores worse than the all together prediction. the goal, however, is to achieve a set of learning groups that all score equal or greater than the all together learning group. thus, none of the baselines address the most effective client properties for learning group separation. our experiments have several limitations, including the lack of comparison in regard to the fixed size of clusters, the use of simple ml models with discrete instead of probabilistic predictions, and a restricted set of features. learning groups isolation forest accuracy isolation forest f1 elliptic envelope accuracy elliptic envelope f1 min mean max min mean max min mean max min mean max baselines all together 0.863 0.927 0.914 0.955 all separate 0.835 0.886 0.946 0.901 0.939 0.972 0.891 0.926 0.973 0.942 0.961 0.986 machine 0.883 0.899 0.930 0.938 0.947 0.964 0.903 0.911 0.925 0.949 0.953 0.961 machine + os 0.850 0.893 0.940 0.919 0.943 0.969 0.907 0.923 0.955 0.951 0.960 0.977 10% training 3 clusters cm: average 0.870 0.899 0.944 0.931 0.947 0.971 0.915 0.921 0.934 0.956 0.959 0.966 cm: complete 0.844 0.883 0.944 0.915 0.938 0.971 0.915 0.921 0.934 0.956 0.959 0.966 cm: ward 0.902 0.908 0.926 0.949 0.952 0.961 0.929 0.935 0.949 0.963 0.966 0.974 20% training 3 clusters cm: average 0.863 0.885 0.926 0.926 0.939 0.962 0.926 0.936 0.949 0.961 0.967 0.974 cm: complete 0.893 0.907 0.946 0.943 0.951 0.972 0.926 0.936 0.949 0.961 0.967 0.974 cm: ward 0.882 0.901 0.940 0.937 0.948 0.969 0.926 0.936 0.949 0.961 0.967 0.974 30% training 3 clusters cm: average 0.867 0.897 0.940 0.929 0.945 0.969 0.928 0.937 0.949 0.963 0.967 0.974 cm: complete 0.870 0.901 0.950 0.930 0.947 0.975 0.924 0.933 0.949 0.961 0.965 0.974 cm: ward 0.850 0.892 0.950 0.919 0.942 0.974 0.924 0.933 0.949 0.961 0.965 0.974 table 1: evaluation of predicting normal behavior on the infiltration attacks dataset. bold values represent the maximum in a column. highlighted values are less than the all together baseline. 4 conclusion and future work in this paper, we present an unsupervised division of participants into learning groups during the semi-supervised training of an anomaly detection model. clients within learning groups have trained the initial model in a similar way, meaning that they have similar data and thus similar behavior. we showed that it is also feasible to introduce this method in a collaborative learning setup without revealing private data. by applying it to network anomaly detection of infiltration attacks, it achieves an increased accuracy and f1 score by up to 4.4% and 2.5%, respectively, in contrast to the common training procedure. teh next steps include advancing from a fixed cluster size to a threshold, a detailed analysis of distance metrics for hierarchical clustering, and a deeper look into the data to answer rq2. additionally, further models, e.g., neural networks and datasets of different domains, e.g., iot, industrial networks, or image recognition, will be evaluated to show the independence of this approach. subsequent work will reduce the attack surface on the property inference step, e.g., by introducing secure multi-party computation, and generalize the approach on other ml tasks. netsys 2021 4 / 5 eceasst bibliography [akm+05] w. aiello, c. kalmanek, p. mcdaniel, s. sen, o. spatscheck, j. van der merwe. analysis of communities of interest in data networks. in passive and active network measurement. 2005. [cbk09] v. chandola, a. banerjee, v. kumar. anomaly detection: a survey. acm computing surveys, 2009. [frc+19] g. fernandes, j. j. p. c. rodrigues, l. f. carvalho, j. f. al-muhtadi, m. l. proença. a comprehensive survey on network anomaly detection. telecommunication systems, 2019. [jgl15] a. jakalan, j. gong, s. liu. profiling ip hosts based on traffic behavior. in ieee international conference on communication software and networks. 2015. [mdbc10] g. münz, h. dai, l. braun, g. carle. tcp traffic classification using markov models. in traffic monitoring and analysis. 2010. [nmm+19] t. d. nguyen, s. marchal, m. miettinen, h. fereidooni, n. asokan, a.-r. sadeghi. dïot: a federated self-learning anomaly detection system for iot. 2019. [pah+19] v. b. s. prasath, h. a. a. alfeilat, a. b. a. hassanat, o. lasassmeh, a. s. tarawneh, m. b. alhasanat, h. s. e. salman. distance and similarity measures effect on the performance of k-nearest neighbor classifier – a review. big data, 2019. [shg18] i. sharafaldin, a. habibi lashkari, a. a. ghorbani. toward generating a new intrusion detection dataset and intrusion traffic characterization:. in proceedings of the 4th international conference on information systems security and privacy. 2018. [sp10] r. sommer, v. paxson. outside the closed world: on using machine learning for network intrusion detection. in ieee symposium on security and privacy. 2010. [ssss17] r. shokri, m. stronati, c. song, v. shmatikov. membership inference attacks against machine learning models. 2017. [yr08] t.-f. yen, m. k. reiter. traffic aggregation for malware detection. in detection of intrusions and malware, and vulnerability assessment. 2008. 5 / 5 volume 080 (2021) introduction related work property inference-based federated learning groups conclusion and future work smart urban data space for citizen science electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) smart urban data space for citizen science heiko bornholdt, david jost, philipp kisters, michel rottleuthner, sehrish shafeeq, winfried lamersdorf, thomas c. schmidt, and mathias fischer 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst smart urban data space for citizen science heiko bornholdt, david jost, philipp kisters, michel rottleuthner, sehrish shafeeq, winfried lamersdorf, thomas c. schmidt, and mathias fischer*{ bornholdt,jost,kisters,shafeeq,lamersdorf,mfischer } @informatik.uni-hamburg.de universität hamburg, hamburg, germany{ michel.rottleuthner,t.schmidt } @haw-hamburg.de hamburg university of applied sciences, hamburg, germany abstract: in smart cities, the number of citizen-operated sensor devices continues to increase. this development multiplies the amount of collectible data about our urban environment. currently, many of those sensors are connected to proprietary cloud services with questionable effects on user privacy. simply avoiding interconnecting those devices misses out on opportunities to collectively use data as a shared source of information. the city of hamburg sponsored project smart networks for urban citizen participation (sane) aims at enabling citizens to connect existing sensors to create a city-wide data space while keeping control of devices and gathered data. interconnecting these sensors enables all citizens to generate higher value information and thus improve their urban environmental awareness. in this paper, we describe the demo of our sane architecture. the demo presents how citizens connect to the city-wide data space, how data can be collected, processed, and shared in a privately manner. keywords: smart city, internet of things, urban sensing, citizen science, distributed data processing, data governance 1 introduction as the number of privately owned sensors rises, so does the amount of data collected [bse21]. there are various kinds of sensors like home weather stations, smart home devices, smart gadgets, and smartphones. this collected data is currently not fully exploited; most data is only used in closed data spaces or only available via local access. data sent to a central server of the sensor provider is only shared with those who are part of provider’s network. by relying on a central entity, a single point of failure and trust is given. participants have no other option than to trust the central entity. in addition, these data spaces are usually limited to certain measurement types so that only specific queries can be made, making it challenging to link different information. sane [bjk+19] focuses on closing this gap by providing an open and fully decentralized city-wide data space. everyone can participate by collecting data, providing storage space or adding services that generate higher value information based on existing data. by opening this data space to the public, everyone can benefit from collected data: citizens without own sensor ∗ ahoi.digital 1 / 4 volume 080 (2021) smart urban data space for citizen science hardware interested in their surroundings, businesses needing specific information without the capacity to deploy their own sensor network, and city officials. while collected data is the foundation of the sane-network, own services can be developed and provided to process these data streams into higher-quality knowledge. with a hands-on demo, we would like to present on how simple it is to join and participate to the sane-network by adding new sensors. after the initial bootstrapping process of new sensors, we will show how participants can publish, process, and exchange collected data within the network. therefore, an intuitive user interface is shown to display collected data, communication between participants, and generated information. the following section 2 gives a short overview of the sane architecture. section 3 describes the presented demo. section 4 gives a short conclusion and provides an outlook for future work. 2 sane sane [bjk+19] establishes a city-wide and open data space1. citizens can participate by connecting their sensors and publishing collected data, creating data processing services, and providing storage space to cache existing data on their hardware. the sane framework provides interfaces to answer complex queries and explore existing data within the city. the sane architecture is partitioned into two main areas: field and network. the field consists of sensors contributing new data to the sane data space. for physical sensors, we provide a sane-compatible reference implementation using the open-source operating system riot [bgh+18]. additionally, sane integrates external providers, e.g., via their apis as virtual sensors, allowing them to connect existing data sources to the data space. the network consists of interconnected nodes of varying capabilities operated by citizens. nodes have multiple purposes, they connect to locally deployed sensors, act as gateways for collected data, and provide various services to the system and defining the functionalities of the sane middleware. these include data provisioning, data quality assessment and data processing, access control, and user interfaces. nodes may run any number of services, depending on their resources and user preferences. citizens can access the sane platform’s data via different interfaces, either tailored for visual user experience or direct machine communication. they allow for simple data exploration as well as complex queries connecting multiple data sources and measurement types. to achieve high accessibility, this will be as simple as browsing the www. 3 demonstration scenario our demonstration illustrates the sane approach from a citizen perspective by showing the most relevant aspects of our system in multiple combined example scenarios. the scenarios cover a practical interactive sensing application, intuitive and secure sensor device bootstrapping, integration of heterogeneous public data sources, and citizen-controlled data processing. to showcase the sensing aspect, we deploy ultrasonic and radar-based presence detection sensors in the rooms of the conference location. these sensors register when people move nearby 1 https://github.com/sane-city netsys 2021 2 / 4 https://github.com/sane-city eceasst observed room web-based interface reading sensor data to gather activity in room interactive data query creation with data visualization overlay network collecting and processing data, and providing services bootstrapping user-owned sensors local sensors local sane node remote sane node integrating external data repositories external sensors figure 1: demonstration setup showing locally deployed sensors, sensor bootstrapping, integration of external sensor data, and the interactive web interface to explore the sane data space. to indicate which rooms are occupied and how activity in different rooms changes over time. the sensors wirelessly connect to a sane node based on a raspberry pi. this node provides collected data to the network, consisting of other devices, like servers and laptops. the measured activity can be viewed through a graphical interface, querying the data from the sane middleware. it shows a heatmap of the activity in the rooms on a map of the area. in a hands-on experience with our sensors, shown in figure 1, we demonstrate how users are meant to securely add new devices that will later feed measurements into their digital data space. in this bootstrapping process, a sensor device provisioned with our sane-compatible riot os reference firmware is configured with initial parameters. the user may enter information like a custom name, location, etc., into a web form served by the node. upon finishing this, a prompt with an animated colored screen appears which is used as an out-of-band communication channel to transfer bootstrap meta-data and cryptographic key material from the computer to the constrained sensor device. the latter uses a low cost rgb light sensing peripheral to receive the emitted color signals and then decodes the original data. holding it up against the screen allows it to capture the data and configure itself accordingly to automatically and securely connect to the correct node. afterward, the web interface lists the newly added device together with its metadata and associated data sources. the data integration aspect is demonstrated by including data from publicly accessible external sources, like luftdaten.info and the klimabotschafter e.v.. these show sane’s versatility for data source abstraction, allowing us to benefit from already existing crowdsensing initiatives and thereby expanding the data space available to sane users. our concept of virtual sensors highlights how this external data integrates with our data processing layer so that users can transparently interact with it. for the user-facing component, we showcase a scenario that encompasses the interactive request and discovery of data via sane. the individual sane nodes autonomously discover each other and form an overlay network to orchestrate a distributed system. within this node pool, 3 / 4 volume 080 (2021) smart urban data space for citizen science each participant explicitly expresses trust towards other parties and shares data according to individual preferences. data shared by others is usable as a resource for individual purposes by each trusted participant. a web interface is presented to the user for creating complex queries and data processing steps matching for certain conditions. the user may then utilize these elements to interact with available data and ask the system for related information. we visualize the queries and communication messages sent between nodes in the network to show how sane triggers operations when interacting with it. 4 conclusion and future work with sane, citizens can easily take an active role in their smart city by participating in data collection and use. our approach provides an alternative to closed, commercial devices and data hogging cloud services by giving users full control of the devices that collect their data and the systems that process and share it. data collected from a user-operated sensor is consequently owned by the same user unless it is explicitly shared with others. sane uses virtual sensors to integrate individually shared and public data sources into a data space that can be combined with private data to serve as the foundation for custom data-driven applications and services. citizen participation is an important factor for the success of different smart city initiatives [bj16]. full control over devices, infrastructure, and data processing steps allows users to obtain first-hand experience and improves visibility of aspects that are commonly hidden from users. this additional level of active involvement is considered to eventually raise awareness and acceptance, which is needed for the broad adoption of a participatory distributed smart city infrastructure. for future work, we aim to focus on efficiently address existing information within our distributed network. here we are going to look at distributed complex event processing, fitting the event-based nature perfectly. furthermore, we are looking into improving data quality by evaluating incoming data and quantifying the trustworthiness of values to increase the confidence in information provided by sane’s open data space. references [bgh+18] e. baccelli, c. gündogan, o. hahm, p. kietzmann, m. lenders, h. petersen, k. schleiser, t. c. schmidt, m. wählisch. riot: an open source operating system for low-end embedded devices in the iot. ieee internet of things journal 5(6):4428–4440, december 2018. [bj16] l. berntzen, m. r. johannessen. the role of citizen participation in municipal smart city projects: lessons learned from norway. in smarter as the new urban agenda. pp. 299–314. springer, 2016. [bjk+19] h. bornholdt, d. jost, p. kisters, m. rottleuthner, d. bade, w. lamersdorf, t. c. schmidt, m. fischer. sane: smart networks for urban citizen participation. in proceedings of ieee 26th international conference on telecommunications. 2019. [bse21] a. bárdossy, j. seidel, a. el hachem. the use of personal weather station observations to improve precipitation estimation and interpolation. hydrology and earth system sciences 25(2):583–601, 2021. netsys 2021 4 / 4 introduction sane demonstration scenario conclusion and future work data serialization formats for the internet of things electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) data serialization formats for the internet of things daniel friesel and olaf spinczyk 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst data serialization formats for the internet of things daniel friesel1 and olaf spinczyk2 1 daniel.friesel@uos.de 2 olaf@uos.de institut für informatik universität osnabrück, germany abstract: iot devices rely on data exchange with gateways and cloud servers. however, the performance of today’s serialization formats and libraries on embedded systems with energy and memory constraints is not well-documented and hard to predict. we evaluate (de)serialization and transmission cost of mqtt.eclipse.org payloads on 8to 32-bit microcontrollers and find that protocol buffers (as implemented by nanopb) and the xdr format, dating back to 1987, are most efficient. keywords: iot, energy, data serialization 1 introduction by definition, an iot device does not come alone: it is connected to the internet of things and thrives by exchanging data with other iot devices or cloud servers. on the application layer, this requires a data exchange format suitable for resource-constrained embedded systems. several standardized data formats and accompanying implementations are available for this task, and preferable to custom implementations due to lower time investment and improved interoperability. however, the cost of data (de)serialization and transmission with these libraries is largely undocumented. previous studies are often bound to specific use cases and evaluated on powerful android smartphones or even x86 computers, not iot devices. we aim to fill this gap by giving a quick overview of the transmission and (de)serialization cost of currently available libraries on 8to 32-bit embedded microcontrollers. 2 evaluation setup we evaluate implementations of four data formats: arduinojson 6.18 (json), mpack 1.0 (messagepack), nanopb 0.4.5 (protocol buffers v3), and xdr (external data representation). see https://ess.cs.uos.de/git/software/netsys21-artifacts for source code and compiler options. we also take a quick look at six data formats without suitable embedded implementations: ubjson, bson, cbor, cap’n’proto, avro, and thrift. we leave out xml and exi, which have been shown to perform no better than json and protocol buffers [gt11, zww+18]. on the hardware side, we examine 8-bit atmega328p, 16-bit msp430fr5994, 32-bit esp8266, and 32-bit stm32f446re microcontrollers. as msp430fr5994 fram access is limited to 8 mhz, we set its clock speed to 8 mhz to avoid fram wait states. we use json payloads obtained from public mqtt.eclipse.org messages as well as data from two smartphone-centric studies for our measurements [mae12, sm12]. message objects have 1 / 4 volume 080 (2021) mailto:daniel.friesel@uos.de mailto:olaf@uos.de https://ess.cs.uos.de/git/software/netsys21-artifacts data serialization formats for the internet of things 0 20 40 60 80 100 120 140 160 180 200 220 xdr? thrift? avro? capnproto? protocol buffers? messagepack cbor bson ubjson json 46 72 36 50 40 85 85 105 96 111 data size [b] d at a fo rm at figure 1: serialized data size of encoded benchmark objects. star marker (?) indicates schemaenabled data formats; schema size is not included. bar elements represent 25th, 50th, and 75th percentile. mean values are denoted by the diamond symbol and also printed on the left. one to 13 key-value pairs, including lists and sub-objects. the smartphone study datasets are our largest and most text-heavy samples. in some cases, we made minor adjustments to ensure message compatibility with all evaluated data formats. given the payloads, data formats, and implementations, our evaluation program generates and executes (de)serialization code on the target mcus and measures clock cycles, serialized data size, text segment size, and memory usage (i.e., data + bss + stack). we use c++ and python3 libraries to measure serialized data size for data formats without embedded implementations. 3 observations fig. 1 shows observed serialized data sizes for each format. we see that avro, xdr, and protocol buffers provide the most efficient encoding and are thus cheapest to transmit, and json is least compact. this is in line with findings reported in earlier studies [sm12, gt11]. as fig. 2 shows, xdr (de)serialization is also by far the fastest operation, followed by mpack, nanopb, and arduinojson. on esp8266, arduinojson performs even better than nanopb. mpack appears to be a good choice for serialization-only applications. the nanopb outlier is caused by a benchmark object using lists with nested objects. in real-world use, a message is typically received and then deserialized, or serialized and then transmitted. depending on the relationship between per-cycle mcu energy consumption and per-byte radio transmission cost, fast (de)serialization may be more or less important than compact message objects. combined with different requirements for the data format in question, which may limit the set of available formats and implementations, this leads to a simple conclusion: there is no single best data format. nevertheless, we can make some observations. when combining an ultra-low-power mcu with a slow, high-power radio, the transmission cost per byte is most relevant. for instance, given an msp430fr5994 mcu and a ti cc1200 radio, datasheets indicate that the computation cost netsys 2021 2 / 4 eceasst 0 0.2 0.4 0.6 0.8 1 1.2 1.4 ·104 xdr nanopb mpack arduinojson 316 404 3,937 3,126 3,082 1,138 5,926 5,331 cycles l ib ra ry stm32f446re esp8266 msp430 atmega 2,931 1,849 2,100 6,125 4,587 5,511 616 463 58,207 21,566 3,692 11,049 18,246 19,424 3,305 2,581 33,646 16,310 2,192 10,149 15,172 18,330 1,476 1,332 figure 2: clock cycles for serialization (blue, top) and deserialization (red, bottom) on stm32f446re (boxplots, left) and other architectures (table entries, right). 0 0.2 0.4 0.6 0.8 ·104 xdr nanopb mpack arduinojson 215 704 466 7,016 240 5,002 215 6,842 bytes l ib ra ry stm32f446re esp8266 msp430 atmega 4,960 11,899 7,756 1,037 7,280 290 18,089 179 9,937 381 2,292 148 7,060 266 6,871 266 13,770 1,169 2,238 355 figure 3: relative text segment (blue, top) and data+bss+stack (red, bottom) usage for (de)serialization on stm32f446re (boxplots, left) and other architectures (table entries, right). of about 0.5 nj per clock cycle is four orders of magnitude lower than the transmission cost of 5 to 10 µj per byte. from an energy perspective, spending an additional 9,000 cpu cycles to save a single byte of data is already worth it. it follows from fig. 1 and 2 that the difference in (de)serialization speed is negligible in this case and nanopb is the most energy-efficient choice. with faster radios, the situation is less extreme. for instance, an esp8266 datasheet also gives about 0.5 nj per clock cycle, but just 5 nj per byte for a 65 mbit/s wi-fi connection. here, xdr is slightly more energy-efficient. however, unless data is transmitted non-stop, the differences between data formats are small compared to an esp8266’s overall energy requirements. finally, memory requirements are also an important aspect. in fig. 3, we see that xdr is extremely light-weight, and the other three implementations vary significantly between architectures. notably, nanopb uses more than half of the atmega’s ram, likely because it is not optimized for 8-bit architectures. we do not report esp8266 memory usage, as we were unable to determine its stack growth. 3 / 4 volume 080 (2021) data serialization formats for the internet of things 4 conclusion we find that nanopb and xdr are most energy-efficient. on low-power mcus with radios in the sub-1 mbit/s range, nanopb is slightly better; for devices with fast radios, xdr wins. assuming it can be implemented efficiently, avro is also an interesting candidate for iot usage. when it comes to rom and ram requirements, xdr has by far the lowest footprint. however, its messages lack schema and type information, and it has limited code generator and library support in modern programming languages. protocol buffers, on the other hand, provide type information and are better supported. taking this into account, we consider nanopb (and protocol buffers in general) to be a good choice for energy-efficient data serialization on today’s relatively powerful iot devices. when devices are required to interact with many different nodes and quickly evolving message formats, and have a sufficient amount of space and energy to spare, we also recommend the schema-less json and messagepack formats due to their ease of use. however, on extremely resource-constrained devices such as avr microcontrollers, which do not have much rom and ram to spare, the decades-old xdr format is still more efficient than any other serialization library we are aware of. bibliography [gt11] b. gil, p. trezentos. impacts of data interchange formats on energy consumption and performance in smartphones. in proceedings of the 2011 workshop on open source and design of communication. osdoc ’11, pp. 1–6. acm, new york, ny, usa, 2011. doi:10.1145/2016716.2016718 [mae12] k. maeda. performance evaluation of object serialization libraries in xml, json and binary formats. in 2012 second international conference on digital information and communication technology and it’s applications (dictap). pp. 177–182. may 2012. doi:10.1109/dictap.2012.6215346 [sm12] a. sumaray, s. k. makki. a comparison of data serialization formats for optimal efficiency on a mobile platform. in proceedings of the 6th international conference on ubiquitous information management and communication. icuimc ’12, pp. 48:1–48:6. acm, new york, ny, usa, 2012. doi:10.1145/2184751.2184810 [zww+18] c. zhang, x. wen, l. wang, z. lu, l. ma. performance evaluation of candidate protocol stack for service-based interfaces in 5g core network. in 2018 ieee international conference on communications workshops (icc workshops). pp. 1– 6. may 2018. doi:10.1109/iccw.2018.8403675 netsys 2021 4 / 4 http://dx.doi.org/10.1145/2016716.2016718 http://dx.doi.org/10.1109/dictap.2012.6215346 http://dx.doi.org/10.1145/2184751.2184810 http://dx.doi.org/10.1109/iccw.2018.8403675 introduction evaluation setup observations conclusion early work: path selection in a path-aware network architecture electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) early work: path selection in a path-aware network architecture thorben krüger and david hausheer 5 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst early work: path selection in a path-aware network architecture thorben krüger1 and david hausheer2 1 thorben.krueger@ovgu.de 2 david.hausheer@ovgu.de networks and distributed systems lab otto-von-guericke university magdeburg, germany abstract: modern path-aware networking (pan) architectures based on packetcarried forwarding state (pcfs) promise to support practical multipath communication with increased availability and redundancy, even for single-homed hosts. how exactly hosts can chose paths in a meaningful way while maintaining good network utilization as well as user satisfaction is the subject of this research. we demonstrate how overall network utilization is affected by the introduction of multipath communication and propose the term “cost of multipath” to reason about this observation. we also identify a collection of concrete practical techniques from networking research and engineering that can be adapted to allow hosts to make more informed path decisions. these tie into the practical implementation of an experimental pathselection engine with a high-level api that we aim to utilize to quantitatively investigate host-based approaches for traffic optimization in the existing path-aware scion network architecture keywords: path-awareness, scion, traffic engineering, cost of multipath, multipath networking 1 introduction in contrast to the current bgp-based internet, modern path-aware networking (pan) architectures guarantee verifiable packet forwarding across the network over precise paths specified by the sending host. in a pan, a host selects one or more desired paths to the destination from a set of alternatives, rather than merely delegating forwarding decisions to routers along the way. as a direct consequence, traffic distribution patterns across such a network are the product of the collective path-choices made on the hosts. moreover, even a single-homed host can easily distribute its communication with a given destination over an almost arbitrary number of alternative paths. we argue that these new capabilities enjoyed by the end hosts in the next-generation internet require more sophisticated mechanisms for intelligent path choices, beyond the usual set of simple heuristics that merely compare path lengths without any consideration for, e.g., network utilization patterns, congestion or other factors. the remainder of this paper is structured as follows: in section 2, we give some necessary context for our early work and identify the techniques we aim to use in our implementation work. in section 3, we discuss the impact that multipath communication can have on network utilization. we share a glimpse into our currently ongoing work before concluding with a summary in section 4. 1 / 5 volume 080 (2021) mailto:thorben.krueger@ovgu.de mailto:david.hausheer@ovgu.de traffic engineering in path-aware networks 2 background and related work in pan architectures, the ability to dictate guaranteed paths across the network resides exclusively with the end hosts [psrc17]. in the scion architecture, dedicated network-level services provide the necessary inter-domain topology information for a host to select among several possible paths to any destination. currently, there are no mechanisms to prohibit the sub-optimal use of paths that are, e.g., overly long or congested. a related problem is known from research about peer-to-peer (p2p) networks: the resources shared in p2p systems often exist in multiple replicas [sb09], i.e., they are available from different peers. however, peer selection for resource retrieval in p2p overlay networks is often arbitrary and very rarely takes underlying network topology into account. this results in inefficient traffic patterns on the network, where resources that would also be available from nearby peers are instead retrieved from far away and usually with lower performance, while needlessly increasing network transit traffic and costs. as a remedy, [afs07] proposes an “oracle” service for p2p overlays that could be deployed by isps to help their customers identify and contact peers that are nearby. in our work, we aim to generalize this approach to network paths instead of peers, where a network operator can offer comparable functionality to hosts that wish to adjust their multipath communication to current network conditions for mutual benefit. in the absence of such centralized mechanisms, another way for a host to obtain some early path quality estimates is an adaption of the “connection racing” approach detailed in [wy12]. connection handshakes to a destination are simultaneously initiated across a set of alternative paths. the different round-trip-times until handshake completion can then be used as estimates for “path latency” properties. efforts are underway at the ietf to develop and agree upon a suitable vocabulary to reason about path property concepts [ek20]. our theoretical work will follow the current ietf consensus in the relevant terminology wherever possible. our practical implementation of a suitable path-selection mechanism will also mirror the corresponding ietf proposal [twe+21]. lack of coordination between independent selfish agents in game theory and distributed mechanism design is often referred to as the “price of anarchy” [ck05]. we aim to develop our own notion of the “price of multipath” in deference to this metaphor. 3 early and ongoing work intuitively, communication between two nodes across a network should happen over the shortest paths possible. an abstract example is depicted in figure 1. needlessly using longer paths, i.e., taking “detours” consumes more resources, resulting in worse network utilization, which is demonstrated in figure 2. where multipath communication combines the use of optimal shortest paths with the additional use of longer detours, network utilization is still worse than the optimum but improved when compared to the previous case. see figure 3. for the purposes of quick failover and redundancy, the capacity to engage in communication over multiple paths is assumed to be desirable, even in cases where global network utilization suffers. however, we need a qualitative understanding of this trade-off to further reason about traffic enginnering under path-awareness. netsys 2021 2 / 5 eceasst figure 1: singlepath direct communication: 3 gbit/s total bandwidth figure 2: singlepath indirect communication: 1.5 gbit/s total bandwidth figure 3: multipath communication using both direct and indirect paths: 2 gbit/s total bandwidth 3.1 research questions how can a price of multipath be defined? we aim to define a term for the price of multipath (pom), deriving the utilization penalties for a given network topology in terms of the node degree distribution and other parameters. it this pom constant for a given network? we will need to work confirm or disprove the suspicion that the pom is generally a fixed property of a network, in the way that figure 4 suggests. what is fair path selection? can we come up with sane defaults for the path selection mechanism that avoids the pathological case depicted in figure 4a)? how large should the fraction of bandwidth over a redundant, longer path be? can there be special allowances and acceptable unfair behavior to ensure qos? what effects can path-selection have for network utilization? using a suitably sophisticated path-selection mechanism, could a host explore multiple alternative paths to a destination simultaneously, autonomously avoiding congested links and ultimately 3 / 5 volume 080 (2021) traffic engineering in path-aware networks a) aggressive multipath, degenerated example b) considerate multipath, fairer example figure 4: total network utilization is a constant 4 gbit/s in both cases helping to improve global network utilization? on the other hand, does an overly naı̈ve approach to path choice result in an under-utilization of viable alternative routes while compounding traffic congestion issues in common choke points? 3.2 methodology to also approach these questions from a practical side, we are currently in the process of designing and implementing a next-generation networking api that translates high-level user demands into actual path choices on the scriptable back-end, leveraging information about path performance and network conditions to automatically optimize for throughput, latency, etc. 4 summary upcoming pan architectures promise a world with easy and ubiquitous multipath. this requires the reconsideration of a lot of assumptions and thereby poses some interesting challenges. our notion of the cost of multipath reflects the observation that network utilization appears to be slightly penalized when compared to traditional shortest-path approaches. once the theoretical underpinnings are more firmly in place, we seek to demonstrate the viability of our ideas through a practical implementation of a path-selection engine that incorporates these mechanisms behind a high-level next-generation networking api that echoes current proposals under development at the ietf. netsys 2021 4 / 5 eceasst bibliography [afs07] v. aggarwal, a. feldmann, c. scheideler. can isps and p2p users cooperate for improved performance? acm sigcomm computer communication review 37(3):29–40, 2007. [ck05] g. christodoulou, e. koutsoupias. the price of anarchy of finite congestion games. in proceedings of the thirty-seventh annual acm symposium on theory of computing. pp. 67–73. 2005. [ek20] t. enghardt, c. krähenbühl. a vocabulary of path properties. internet-draft draftirtf-panrg-path-properties-00, internet engineering task force, mar 2020. work in progress. https://datatracker.ietf.org/doc/html/draft-irtf-panrg-path-properties-00 [psrc17] a. perrig, p. szalachowski, r. m. reischuk, l. chuat. scion: a secure internet architecture. springer, 2017. [sb09] j. seedorf, e. burger. application-layer traffic optimization (alto) problem statement. rfc 5693, oct 2009. doi:10.17487/rfc5693 https://rfc-editor.org/rfc/rfc5693.txt [twe+21] b. trammell, m. welzl, t. enghardt, g. fairhurst, m. kühlewind, c. perkins, p. s. tiesel, c. a. wood, t. pauly, k. rose. an abstract application layer interface to transport services. internet-draft draft-ietf-taps-interface-12, internet engineering task force, apr 2021. work in progress. https://datatracker.ietf.org/doc/html/draft-ietf-taps-interface-12 [wy12] d. wing, a. yourtchenko. happy eyeballs: success with dual-stack hosts. rfc 6555, apr 2012. doi:10.17487/rfc6555 https://rfc-editor.org/rfc/rfc6555.txt 5 / 5 volume 080 (2021) https://datatracker.ietf.org/doc/html/draft-irtf-panrg-path-properties-00 http://dx.doi.org/10.17487/rfc5693 https://rfc-editor.org/rfc/rfc5693.txt https://datatracker.ietf.org/doc/html/draft-ietf-taps-interface-12 http://dx.doi.org/10.17487/rfc6555 https://rfc-editor.org/rfc/rfc6555.txt introduction background and related work early and ongoing work research questions methodology summary reform: a tool for rapid requirements formalization electronic communications of the easst volume 079 (2020) interactive workshop on the industrial application of verification and testing etaps 2020 workshop (interavt 2020) reform: a tool for rapid requirements formalization georgios giantamidis, georgios papanikolaou, marcelo miranda, gonzalo salinas-hernando, juan valverde-alcalá, suresh veluru, stylianos basagiannis 8 pages guest editors: stylianos basagiannis, goetz botterweck, anila mjeda eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst reform: a tool for rapid requirements formalization georgios giantamidis1,2, georgios papanikolaou1, marcelo miranda3, gonzalo salinas-hernando1, juan valverde-alcalá1, suresh veluru4, stylianos basagiannis1 1united technologies research centre ireland, ireland 2aalto university, finland 3university of minho, portugal 4servisbot ltd. arclabs research centre, ireland abstract: formal methods practices can sometimes be challenging to adopt in industrial environments. on the other hand, the need for formalization and verification in the design of complex systems is now more evident than ever. to the end of easing integration of formal methods in industrial model based system engineering workflows, utrc ireland has developed a tool aiming to render requirements formalization as effortless as possible to the industrial engineer. the developed approach is an end-to-end solution, starting with natural language requirements as input and going all the way down to auto-generated monitors in matlab / simulink. we employ natural language processing and machine learning techniques for (semi-)automatic pattern extraction from requirements, which drastically reduces the required formalization workload for both legacy and new requirements. for monitor generation, we provide our own approach which outperforms existing state-of-the-art tools by orders of magnitude in some cases. keywords: requirements formalization, formal verification, natural language processing, clustering, monitor generation, matlab, simulink 1 introduction traditional requirements management procedure followed in industrial environments is characterized by a set of important problems that can significantly limit engineer productivity and in the worst case even have catastrophic results for the end product. one such problem is handling requirements in unstructured, natural language format which prevents early potential requirement inconsistency detection as well as analysis and tool support opportunities in general. another issue is that typically test cases and requirement monitors are constructed manually, which is time consuming and error prone. while formalization of requirements could address these issues, it is often not performed as simply the vast volume of legacy requirements makes this prohibitively time consuming. to address these problems at their core, in utrc ireland we have developed a tool for rapid requirements formalization and subsequent analysis. our proposed approach, by employing 1 / 8 volume 079 (2020) reform: a tool for rapid requirements formalization natural language processing (nlp) and machine learning (ml) techniques for pattern identification, leads to huge acceleration in formalization for both legacy and new requirements. once formalization is complete, consistency checking can be performed, which prevents early design error propagation. formalized requirements also enable automatic monitor and test-case generation which rapidly accelerates the verification process. overall, the proposed methodology can be shown to drastically reduce certification costs as well, an important consideration for industrial adoption of any technology. in this report we outline the proposed solution, highlight important aspects that contribute to its effectiveness, present results in real world case-studies and conclude with discussion for future development goals. the presentation is structured as follows: in section 2 we outline the nlp part of the pipeline, which focuses on abstracting requirements to ease subsequent pattern identification. in section 3 we discuss grouping of abstract requirements into clusters for pattern identification and extraction. in section 4 we demonstrate how the extracted patterns speed-up the formalization process for both legacy and new requirements. in section 5 we talk about the additional capabilities of consistency checking and automatic monitor generation we provide for formalized requirements. in section 6 we present results of applying the proposed approach on real world industrial case studies. finally, in section 7 we conclude with some ideas for future extension of the tool. 2 nlp for domain entity identification in order for pattern discovery to yield high quality results, it is imperative that irrelevant details are abstracted away from the provided input (figure 1). to this end, we apply a range of built-in transformations to remove textual clutter that typically appears in a variety of domains, but we also provide the user with the option to insert their own abstraction rules in order to address the specific domain at hand. the provided built-in transformations are regular expression and context free grammar based, and serve to identify and abstract away information such as signal names, quantities accompanied by units (e.g. time duration), as well as entire mathematical expressions. the user defined abstraction rules are based on nlp analysis of the requirement text. specifically, we have developed our own heuristics for named entity recognition on top of off-the-shelf nlp libraries [hon15, msb+14] and suggest these to the user so they can (potentially after tweaking) enable their use in the requirement abstraction procedure. finally, the user is able to enrich the abstraction procedure by simply adding their own regular expression rules. 3 ml for pattern discovery once requirements are brought into an abstract representation, clustering for pattern discovery follows as the next step of the proposed solution (figure 2). the aim of this step is to discover commonalities among requirements and group them based on these, so that the workload for interavt 2020 2 / 8 eceasst figure 1: abstraction view subsequent formalization is reduced (e.g. the user only has to formalize a couple dozen patterns instead of thousands of requirements). clustering requires defining what the distance between two requirements means. a variety of approaches have been explored here based on just syntactic information (e.g. jaccard similarity of sentence n-grams), on just semantic information (e.g. similarity of dependency parsing results), as well as combinations of the two, along with additional heuristics. the clustering algorithm we employ is a so called hierarchical clustering algorithm. initially, each requirement is placed in its own group, and then an iterative phase takes place where, on each iteration, the groups that are closest to each other are merged together. this process continues until there is only one requirement group left that contains all requirements. as the algorithm executes, information about various requirement (sub)groups and their merges is kept track of and forms a binary tree data structure called a dendrogram. the final clustering is obtained by providing a distance threshold that slices the dendrogram at the corresponding height. an important property of the algorithm is that the resulting clusters are guaranteed to respect the given distance threshold, i.e. within each cluster, all pairwise requirement distances are less than that threshold. note that while clustering results might not be perfect, the user (apart from adjusting the distance threshold) is allowed to intervene by manually expanding and collapsing dendrogram nodes in order to fine tune the obtained partitioning. 4 formal pattern definition & requirements formalization after partitioning of the requirements into clusters, pattern definition and formalization takes place (figure 3). while this phase is manual, we argue that the preceding clustering significantly reduces the workload required by grouping together requirements that can potentially be 3 / 8 volume 079 (2020) reform: a tool for rapid requirements formalization figure 2: clustering view described by the same pattern. a pattern consists of two parts: a natural language part and a formal language part. this is done so that once the pattern is defined and a coupling between the two representations is made, the user can refer to the pattern by using the natural language part, while the tool can use the formal language part under the hood. a variety of formal languages is supported for defining the formal part of a pattern, namely the property specification language (psl), an ieee standard [psl], the spear domain specific language, developed by rockwell collins [fwh+17], as well as the structured assertion language for temporal logic (salt) [bls06]. all these languages share the characteristic that are extensions of linear temporal logic (ltl), a widespread formalism for property specification towards verification purposes. however, in contrast to ltl, the languages we support are high-level, in the sense that they provide syntax features that facilitate expression of complex properties with minimal amount of text; which is another way our proposed approach considerably reduces the formalization workload for the user. given a set of patterns, formalization of legacy requirements is trivial – all the user needs to do is associate a cluster with a pattern and all requirements in that cluster are formalized according to the formal part of the corresponding pattern. for new requirements, we provide two ways formalization can be done: the user can either select a pattern to serve as a template and fill in the missing information (e.g. signal names, mathematical expressions etc.) according to the new requirement, or use the requirements editor we provide with syntax checking and autocompletion features based on a grammar automatically derived by the set of defined patterns. it might, of course, happen that none of the existing patterns are suitable for the new requirement at hand, in which case a new pattern has to be defined before proceeding. however, as more requirements are formalized using the tool and the pattern library grows, this situation becomes less likely to occur. interavt 2020 4 / 8 eceasst figure 3: formalization view 5 consistency checking & monitor generation having obtained a set of formalized requirements we can perform, through the tool, consistency checking as well as monitor generation (figure 4). note that both these procedures are supported for all available specification languages by means of translating the high-level property representations into low-level ltl formulas, the fundamental formalism our supported languages are based on. for consistency checking we extended an existing algorithm [gmr17] by integrating with the z3 smt solver [mb08] in order to be able to handle mathematical expressions as well. for monitor generation we use our own approach [gbt20] based on active automata learning [ang87] which, in some cases, is able to outperform conventional translation strategies by orders of magnitude. the resulting monitors can be coupled with the system model (i) serving as assertions for run-time monitoring, (ii) for automatic test-case generation, as well as (iii) for formal verification of the design. currently we only support simulink as a monitor generation target, however, since we first generate a tool-agnostic intermediate representation, supporting additional targets is trivial. 6 industrial case studies the developed solution has been applied so far on two industrial case studies: (1) low-level requirements for the fpga specification of airbus a350 etrac (electrical thrust reverser actuation controller), and (2) high-level requirements for the brake control unit of mitsubishi regional jet. in the former case, the entire tool pipeline was used, from importing natural language 5 / 8 volume 079 (2020) reform: a tool for rapid requirements formalization figure 4: consistency checking & monitor generation view requirements all the way down to formal verification of the space vector modulation (svm) subsystem of the design using the automatically generated monitors. specifically, we were able to fit 40% of the 750 given requirements into 25 clusters, and managed to formalize the 100 requirements for the svm subsystem using just 6 patterns. automatically generated monitors from these 100 requirements were coupled with the system model and successful formal verification of the design took place by employing the simulink design verifier (sldv) matlab toolbox. in the latter case, only the parsing and clustering part of the tool were exercised, in order to demonstrate that our solution provides benefits (e.g. better documentation and traceability by enabling easy subsequent mapping to a more structured representation) even for high-level requirements that cannot be easily mapped to simulink model representations; in particular, we were able to fit 50% of the 700 given requirements into just 15 clusters. 7 conclusion & future work in this report we presented an effort carried out in utrc ireland to develop a tool for rapid requirements formalization. this is achieved by pattern identification from legacy requirements with nlp and ml methods, and subsequent use of the extracted patterns to drive formalization of both legacy and new requirements. a variety of formal languages is supported by the tool and, once requirements are formalized, consistency checking and automatic monitor generation can be performed as well. the approach has been tested on industrial case studies with several hundreds of requirements in each case and the results have been very promising so far. one limitation of the developed solution is that it currently only focuses on functional requirements (i.e. system behavior). therefore, a direction we plan to explore in the future is handling interavt 2020 6 / 8 eceasst non-functional requirements as well (e.g. timing and architectural constraints). another direction for future development is extending the tool with more specification languages and monitor generation targets in order to enable further interoperability with other tools and ease adoption from industrial users. bibliography [ang87] d. angluin. learning regular sets from queries and counterexamples. inf. comput. 75(2):87–106, nov. 1987. doi:10.1016/0890-5401(87)90052-6 [bls06] a. bauer, m. leucker, j. streit. salt—structured assertion language for temporal logic. in liu and he (eds.), formal methods and software engineering. pp. 757– 775. springer berlin heidelberg, berlin, heidelberg, 2006. [fwh+17] a. w. fifarek, l. g. wagner, j. a. hoffman, b. d. rodes, m. a. aiello, j. a. davis. spear v2.0: formalized past ltl specification and analysis of requirements. in barrett et al. (eds.), nasa formal methods. pp. 420–426. springer international publishing, cham, 2017. [gbt20] g. giantamidis, s. basagiannis, s. tripakis. efficient translation of safety ltl to dfa using symbolic automata learning and inductive inference. in computer safety, reliability, and security 39th international conference, safecomp 2020, lisbon, portugal, september 15-18, 2020, proceedings. springer nature switzerland ag, 2020. https://doi.org/10.1007/978-3-030-54549-9 8 [gmr17] n. gigante, a. montanari, m. reynolds. a one-pass tree-shaped tableau for ltl+past. in eiter and sands (eds.), lpar-21, 21st international conference on logic for programming, artificial intelligence and reasoning, maun, botswana, may 7-12, 2017. epic series in computing 46, pp. 456–473. easychair, 2017. http://www.easychair.org/publications/paper/340363 [hon15] m. honnibal. spacy: industrial-strength natural language processing. 2015. https://spacy.io/ [mb08] l. de moura, n. bjørner. z3: an efficient smt solver. in ramakrishnan and rehof (eds.), tools and algorithms for the construction and analysis of systems. pp. 337– 340. springer berlin heidelberg, berlin, heidelberg, 2008. [msb+14] c. d. manning, m. surdeanu, j. bauer, j. finkel, s. j. bethard, d. mcclosky. the stanford corenlp natural language processing toolkit. in association for computational linguistics (acl) system demonstrations. pp. 55–60. 2014. http://www.aclweb.org/anthology/p/p14/p14-5010 7 / 8 volume 079 (2020) http://dx.doi.org/10.1016/0890-5401(87)90052-6 https://doi.org/10.1007/978-3-030-54549-9_8 http://www.easychair.org/publications/paper/340363 https://spacy.io/ http://www.aclweb.org/anthology/p/p14/p14-5010 reform: a tool for rapid requirements formalization [psl] iec 62531:2012(e) (ieee std 1850-2010): standard for property specification language (psl), in iec 62531:2012(e) (ieee std 1850-2010), vol., no., pp.1-184, 28 june 2012. interavt 2020 8 / 8 introduction nlp for domain entity identification ml for pattern discovery formal pattern definition & requirements formalization consistency checking & monitor generation industrial case studies conclusion & future work microsoft word campus08_1.8_feature interaction in pervasive computing systems_cameraready.doc electronic communications of the easst volume 11 (2008) guest editors: romain rouvoy, mauro caporuscio, michael wagner managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 proceedings of the first international discotec workshop on context-aware adaptation mechanisms for pervasive and ubiquitous services (campus 2008) feature interaction in pervasive computing systems yu liu and rené meier 6 pages eceasst 2 / 7 volume 11 (2008) feature interaction in pervasive computing systems yu liu and rené meier lero @ tcd, department of computer science, trinity college dublin, ireland {yuliu, rene.meier}@cs.tcd.ie abstract: feature interaction describes a situation where the combination of two or more services that individually perform correctly results in unexpected and possibly adverse behaviour. such feature interaction issues have first been identified in telecommunication systems and are now beginning to be considered in other distributed software systems. we expect significant feature interaction research in pervasive computing where very many applications collaborate and adapt to changes to their environment or to user needs in order to provide tailored services to users. this paper presents a classification of feature interaction issues in pervasive computing systems. the classification captures, with a focus on automotive systems and systems for smart homes, feature interaction issues related to types of interaction, channels of interaction, and user needs. the classification aims to aid the understanding of feature interaction in pervasive computing systems, and to serve as a guideline for designers of pervasive applications. keywords: feature interaction, pervasive computing, adaptation 1 introduction the complexity of pervasive applications has grown dramatically, with the involvement of a diverse set of devices, the dynamic nature of the environment that these applications monitor and control, and the changing user needs. examples of pervasive applications can be found in areas such as automotive systems and systems for smart homes. today’s luxury cars are equipped with a multitude of software intended to assist car users to enhance safety, to strengthen security, and to improve comfort. a car essentially becomes a smart object, reacting dynamically to changing driving conditions as well as to the condition of the vehicle and the driver. smart homes include services for controlling lighting, heating, and ventilation as well as for delivering information and entertainment. these services become networked smart objects to adjust to the needs of residents. generally, pervasive computing systems exist in a dynamic environment that has a range of parameters that must be catered for. pervasive applications need to be context aware so that information about the context of a device can be captured and utilized. for example, in smart homes, relevant pattern of occupancy and alternative means of heating, such as sunlight, are important parameters a heating control application should capture and adapt to. the changes in the context ought to be dynamically accommodated and as such pervasive applications must be adaptable. collaboration and adaptation of pervasive applications is hindered by feature interaction issues, which were first identified in telecommunication systems [1] and are now considered feature interaction in pervasive computing systems proc. campus 2008 3 / 7 relevant to other distributed software systems. feature interaction is a situation where a number of services work properly in isolation, but exhibit undesired behaviour when combined. a wealth of literature [7] on feature interaction can be found in the telecommunication domain, but relatively little work has focused on addressing the distinct requirements of pervasive computing systems. a notable distinction, between telecommunication systems and pervasive computing systems, is that they have very different notions of context. telecommunication applications treat the signalling channel as their context whereby the set of status signals of a call is transmitted and shared. pervasive applications, on the other hand, consider the context in terms of the physical environment and of the needs of individual users. consequently, pervasive computing systems have to deal with a much more complex context, and therefore face greater possibilities of feature interaction. this paper discusses feature interaction in pervasive computing systems presented in the form of a classification. the classification captures types of interaction, channels of interaction, and user needs. types of interaction describe how pervasive applications interact with their context and with each other. channels of interaction are concerned with identifying the pathway whereby pervasive applications may interfere with each other. user needs are important inputs to pervasive computing systems and aim to describe the overall system behaviour. however, individual users might have conflicting goals that may not be satisfied at the same time or users have quality of service constraints that can not be met at runtime, thus leading to feature interaction. calder [2] proposes a classification of telecommunication features, and his classification serves as basis for our types of interaction. in the pervasive context, however, both physical environment and user needs play a crucial part. our classification extends his classification to deal with physical environment and user needs. the remainder of the paper is organized as follows: section 2 introduces the three major categories of our classification and uses them to discuss feature interaction in pervasive computing systems; section 3 describes related work; section 4 concludes this paper and outlines future work. 2 the classification feature interactionfeature interactionfeature interaction typestypestypes channelschannelschannels user needsuser needsuser needs feature interactionfeature interactionfeature interaction typestypestypes channelschannelschannels user needsuser needsuser needs figure1. root of the classification as shown in figure 1, feature interaction in pervasive computing systems can be described in three categories, namely, types of interaction, channels of interaction, and user needs. eceasst 4 / 7 volume 11 (2008) 2.1 types of interaction types of feature interaction describe how features, or services, are triggered. features can trigger each other, or respond to the same external trigger. in telecommunication systems, external triggers reside in a two-way signalling channel and features usually detect their own triggering conditions. however, in pervasive computing systems, external triggers of a feature are defined by a possibly large set of context properties, which can be changed by the environment, which in turn can be influenced by the feature or indeed by other features. shared trigger. shared trigger interaction takes place when more than one feature is activated by the same event trigger, and their responses conflict with each other. for example, consider the application that opens windows and an application that controls air conditioning [4] in a modern house. both applications are triggered once temperature reaches a certain threshold value chosen by the occupant of the house. opening the windows will compromise the effectiveness of air conditioning. this type of interaction can affect the environment and might result in a fluctuating room temperature. sequential trigger. sequential trigger interaction occurs when the responses of one feature cause another feature to be triggered [2]. sequential trigger takes two forms: 1) one feature sends a notification directly to another feature. 2) one feature affects environment variables, such as temperature and humidity, through actuators, for example, air conditioner and heater, and these changes to the environment variables then lead to other features being triggered. looping trigger. there are situations where individual features may run correctly but multiple features as a whole are stuck in a loop [2]. this is due to cyclic generation of sequential events, whereby the features involved get triggered repeatedly. looping trigger is considered to be a special case of sequential trigger. for example, in order to keep humidity of a room at a constant level, the humidifier application is activated once humidity drops below the constant level. meanwhile, the ventilation application is triggered by a high humidity level that is caused by humidifier. hence, this may cause the humidifier application and ventilation application to be switched on and off repeatedly. missed trigger. missed trigger interaction refers to a situation where the presence of one feature in the system prevents the second feature from operating. some features are designed to use the same device; however, the undesirable situation could be that one feature has full control over the device or disable it, thus preventing other features from correctly functioning. 2.2 channels of interaction channels of feature interaction are concerned with the pathway whereby pervasive applications may interfere with each other. pervasive computing systems can be modelled through three layers, namely, application layer, device layer and environment layer [8]. feature interaction might occur at each layer for different reasons. application layer. experiences from telecommunication domain [1] have shown that distributed support for applications, such as information sharing or transactions, might be problematic and as such may give rise to unanticipated interaction between features. two issues arising from distributed application support also find their way into pervasive computing systems. the first issue is concerned with the assumption of data availability of pervasive applications. based on its context, one application assumes that certain data is feature interaction in pervasive computing systems proc. campus 2008 5 / 7 available while the applications that hold the data keep it private. the second issue is transaction support of pervasive applications. apart from the issues in distributed support for applications, dynamic adaptation of pervasive applications can also lead to unanticipated interactions, as some of the applications might collaborate for the first time and lack knowledge of each other’s interface and behaviour. device layer. the device layer comprises sensors, actuators and control devices. in this layer, feature interaction issues boil down to the conflicts in accessing control devices. some control devices may only be used by one application at the same time, while other devices can be used by multiple applications simultaneously. lack of control over the access to the shared devices can result in a deadlock situation for the applications attempting to use such devices. another issue regarding application-device interaction is the absence of the notion of a session. feature interaction in telecommunication applications is dealt with in a specific period of time, namely, a session. however, there is no indication of the completion of a control request delivered to a device [4]. for the devices controlled by a pervasive computing system, some of them can carry out a control request almost instantaneous while other devices carry out a request over a certain length of time, for example, to request a vcr to record a tv program. hence, the start and endpoint to look for feature interaction are not well defined. environment layer. the environment layer is the source of an implicit coupling between different applications in a pervasive computing system. environment variables, such as temperature and humidity, can be changed by control devices. one feature might be triggered by the changes in an environment variable that other features have influence upon. kolberg [8] advises to explicitly specify the links between control devices and environment variables. in addition, two seemingly irrelevant environment variables can affect each other, thus creating an implicit relationship between trigger conditions of several otherwise independent features. metzger [5] proposes to build an environment simulator to establish relationships between environmental variables. 2.3 user needs an integral part of the context that pervasive applications deal with is the needs of user. user needs are arguably the important inputs to a pervasive computing system. user needs also constrain the overall behaviour of the system. user needs can be divided into behaviour constraints and quality of service (qos) constraints. behaviour constraints are used to specify the composite behaviour of features. qos constraints are generally concerned with timeliness, resource consumption and other qos metrics associated with features. behaviour constraints. users of a pervasive computing system typically define specific behaviour constraints. these constraints range from simple tasks such as controlling devices individually, e.g., switch on the heater at 9pm, to situations that involve very complex user activities, such as controlling the level of illumination of a room according to user profiles. behaviour constraints typically have been embedded in the implementation of applications. this is at the expense of customizability highly demanded by users. we believe that behaviour constraints are part of the context and should be separated from the implementation of the applications. policy based solutions [3] are a promising approach and have already been applied in smart homes, whereby users can explicitly specify the behaviour constraints they can accept in terms of policies as opposed to embodying these constraints inside implementation of features. some industrial strength platforms for smart homes, such as knx eceasst 6 / 7 volume 11 (2008) [9], make it easier to integrate devices from different vendors and suppliers. by employing this capability, users are writing their own policies using an elementary set of basic built-in features of a platform. policy based solutions consider feature interaction as two policies having inadvertent embedded conflicts. in adopting policy based solutions, there are two advantages: 1) users have greater flexibility to express system behaviour constraints in terms of policies. 2) conflicting behaviour constraints can be detected through policy conflicts. qos constraints. qos constraints such as timeliness, resource consumptions of individual features are essential considerations for dynamic adaptation. for instance, time-bounded adaptation in automotive systems is a pressing issue [6]. adaptation typically involves swapping in new features or swapping out existing features to suit the current context. the issue of feature interaction comes down to how to guarantee qos constraints of existing features when new features are dynamically activated. very little current work on feature interaction considers qos constraints of a system. this must be addressed, especially in mission-critical pervasive computing systems, such as automotive systems. 3. related work feature interaction issues in telecommunication systems have been extensively studied. cameron [1] identifies three dimensions for feature interaction: 1) the kind of features involved in the interaction (customer features, system features). 2) the number of users involved (single user, multiple users). 3) the number of network components involved in the interaction (single component, multiple components). calder [2] proposes a newer classification of telecommunication features, and his classification contains four types of interaction using trigger condition as the criteria. our classification extends calder’s classification to deal with a much more complex context which incorporates the physical environment and user needs. feature interaction issues have also been researched in other domains. metzger [5] discussed feature interaction issues in embedded control systems introducing the notion of physical environment and proposes to build an environment model to reveal implicit relationships of environment variables. kolberg [4] [8] addresses only four types of interaction in smart homes, and he proposes a simplified model of environment variables. shehata [9] recognises a paradigm shift from feature interaction to policy interaction in smart homes, so that users can explicitly specify behaviour constraints and interaction detection can be carried out at both design time and runtime. however, we believe that in addition to behaviour constraints, qos constraints must also be considered for (mission critical) pervasive computing systems. 4. conclusion and future work this paper proposed a classification of feature interaction issues in pervasive computing systems. this is achieved through extending existing work on feature interaction in telecommunication systems. the classification comprises three categories, namely, types of interaction, channels of interaction, and user needs. these categories have been discussed with an aim to foster understanding of feature interaction in pervasive computing in light of current feature interaction in pervasive computing systems proc. campus 2008 7 / 7 research in the area. we intended to further diversify our classification with a special focus on distinguishing between desired and unwanted interactions, on challenges driven by the physical environment and on detecting and resolving conflicts at the device layer. acknowledgement the work described in this paper is partly supported by lero the irish software engineering research centre. references [1] e.j.cameron, n.griffeth, y.j.lin, m.e.nilson, h.velthuijsen and w.k.schnure. a feature interaction benchmark for in and beyond. in feature interactions in telecommunication systems, ios press, pp.1-23, may 1994. [2] m.calder, m.kolberg, e.magill, d.marples, and s.reiff-marganiec. hybrid solutions to the feature interaction problem. in proc. 7th. feature interactions in telecommunications and software systems, ios press, june 2003. [3] p.dini, a.clemm, t.gray, f.j.lin, l.logrippo, and s.reiff-marganiec. policy-enabled mechanisms for feature interactions: reality, expectations, challenges. in computer networks: the international journal of computer and telecommunications networking, volume 45, issue 5, august 2004. [4] m.kolberg, e.magill, d.marples, and s.tsang. feature interactions in services for internet personally appliances. in proceedings of ieee international conference on communications 2002, new york, usa, vol. 4, pp. 2613-2618. [5] a.metzger. feature interactions in embedded control systems. in computer networks: the international journal of computer and telecommunications networking, volume 45, issue 5, august 2004. [6] s.fritsch, a.senart, d.c.schmidt, and s.clarke. time-bounded adaptation for automotive system software. in proceedings of the 30 th international conference on software engineering, 2008. [7] m.calder, m.kolberg, e.h.magill, s.reiff-marganiec. feature interaction: a critical review and considered forecast. in computer networks, volume 41, number 1, 15 january 2003, pp. 115-141(27). [8] m.kolberg, e.h.magill, and m.wilson. compatibility issues between services supporting networked appliances. in communications magazine, ieee. volume 41, issue 11, 2003. [9] m.shehata, a.eberlein, and a.o.fapojuwo. managing policy interactions in knx-based smart homes. in proceedings of the 31 st annual international computer software and application conferences, vol 2, 2007. on the resilience of opportunistic networks against dos attacks electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) on the resilience of opportunistic networks against dos attacks s. afzali, a. udugama, a. förster and m. fischer 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst on the resilience of opportunistic networks against dos attacks s. afzali1, a. udugama1, a. förster1 and m. fischer2 1 university of bremen, germany 2 university of hamburg, germany abstract: opportunistic networks (oppnets) enable contact-based networking and service provisioning when no infrastructure exists, e.g., in disaster areas. in such sensitive scenarios, maintaining their availability is important, but most existing work on oppnets mainly assume fully cooperative and thus not malicious nodes. in this paper, we study the impact of different flavors of low-intensity denial of service (dos) attacks on oppnets, which are hard to detect and to counter. our results indicate that low-rate dos and black hole attacks as a special case of dos, seem to have a huge impact on the packet delivery ratio and the delivery delay of an oppnet. keywords: opportunistic networks, attacker models, security, ops, omnet++ 1 introduction opportunistic networks (oppnets) are networks used to communicate in environments where end-to-end paths and networking infrastructure are non-existent. applications that use oppnets have to be delay tolerant and nodes exploit any available communication opportunity to exchange information with other nodes using direct communication. communication in the aftermath of disasters, for remote or unconnected villages, or for offloading saturated networks are some of the scenarios where oppnets can play an important role. an integral and critical part of an oppnets node is the forwarding mechanism employed to exchange data between nodes. there are a number of forwarding protocols developed to efficiently exchange data [ktuf19]. however, these protocols assume well behaving nodes only. however, there are many types of attacks that can inflict damage on a network and thus also on oppnets. oppnets might be seriously affected by denial of service (dos) and as special case of dos also by blackhole attacks [ada16]. the main contribution of this paper, is an evaluation of the impact of low-rate dos and black hole attacks on oppnets, so that efficient countermeasures can be developed in the future. our results indicate that both attack types can significantly decrease the packet delivery ratio of oppnets. the remainder of this paper is structured as follows: section 2 briefly summarizes related work. section 3 describes our methodology and section 4 our evaluation results. section 5 concludes the paper. 2 related work a survey of the previous work on evaluating attacks on oppnets shows that almost all focus on individual attack models and improvements to protocols rather than on a collective look on the severity of the problem[ada16]. there exist also some work on analysis of attack models for some specific protocols like maxprop [ccc10]. in this work, we look at the impact of low-rate dos and black hole attacks and quantify to which extent they can degrade the network operation. 1 / 4 volume 080 (2021) on the resilience of opportunistic networks against dos attacks in a low-rate dos attack, the attacker injects apparently valid packets into the network to deplete the network bandwidth and forwarding resources of other nodes for legitimate users. a special case of dos is a blackhole attack, in which malicious nodes do not cooperate with the rest of the network and thus only request and receive messages, without ever forwarding them. 3 methodology dos attacks in this paper we mainly focus on low-rate dos attacks and black hole attacks. both types of attacks are hard to detect and hard to counter in an oppnet setting. in a low-rate dos attack, a malicious node will send garbage packets to other nodes to consume resources and to impede the forwarding of legitimate content. in a black hole attack, a malicious node will receive packets, but will never forward them. hence, such a node is consuming transmission resources of other nodes without contributing the own resources for spreading content. simulation the evaluation of the effects of dos attacks on oppnets are performed using the omnet++ network simulator together with the oppnets framework ops [ufdk19]. in this work, ops has been extended with two attacker models described in section 2. unlike traditional networks, oppnets have a specific set of evaluation metrics that highlight their performance: the delivery ratio, the network wide ratio of packets successfully delivered over the total expected packets, and average delay, the network wide average of the time taken for a packet to reach a destination. these two metrics are independent of each other and typically used to benchmark oppnets. simulation setup the simulated oppnet consists of 50 nodes, including attackers and genuine nodes. each simulation run is set to 5 days. data (packets) by genuine nodes are injected into the network every 900 seconds, with a payload of 10 kb each. the forwarding of packets are limited by a maximum of 30 hops. nodes are configured to have infinite caches. the wireless connectivity is configured to be bluetooth le like with a range of 30 meters and a bandwidth of 100 kbps. mobility of nodes is modelled using the swim mobility model. the mobility area is 1,500 meters by 1,500 meters. epidemic routing [vb00] is used to disseminate data, probably the best one and used protocol for oppnets. this experimental setup is quite standard for oppnets and sufficient to show the severity of dos attacks [ktuf19]. the attack configuration includes the attacker model, the attack frequency, and the number of attackers explored. unless otherwise stated, attackers inject malicious data every 90 seconds. 4 results in this section we summarize our results on black hole and low-rate dos attacks on oppnets. black holes figure 1 plots the delivery ratio (left) and the delivery delay (right) in 95% confidence intervals in dependence on a varying number of black hole nodes and by keeping the overall number of nodes constant at 50 (red graph) and in dependence on a varying number of legitimate nodes without malicious nodes as baseline (blue graph). with an increasing number of black holes, the delivery of data decreases and the delivery delay increases. even at this rather low attack intensity it can be seen that the effect is significant. the results also indicate that a black hole node does not correspond to a decreased density of the network. while 25 black hole nodes in a setting with in total 50 nodes decrease the delivery ratio to under 50%, the delivery netsys 2021 2 / 4 eceasst 50/0 50/2 48/0 50/5 45/0 50/10 40/0 50/25 25/0 node combinations 0.00 0.25 0.50 0.75 1.00 d el iv er y ra tio blackhole attacks mix of total nodes to attackers 50/0 50/2 48/0 50/5 45/0 50/10 40/0 50/25 25/0 node combinations 0 20 40 d el iv er y de la y (h rs ) blackhole attacks mix of total nodes to attackers figure 1: delivery ratio (left) and delivery delay (right) for a varying number of malicious nodes carrying out black hole attacks. 50/2 refers to 50 total nodes and two of them are attackers. ratio stays at around 75% for a comparable setting with 25 legitimate nodes only and thus lower density. this is due to the fact that attackers still occupy networking resources, forcing legitimate nodes to waste their contact times on black hole nodes that do not forward their content. low-rate dos figure 2 shows the delivery ratio and delivery delay of oppnets during dos attacks for varying number of attacking nodes and attack frequencies and for a setting with malicious nodes (red graphs) and a setting without (red graphs). the graphs indicate that with an increasing number of attackers (top row) and increasing attack intensity (bottom row), the performance deteriorates quickly. already five attacking nodes are sufficient to decrease the delivery ratio from close to 100% to around 60%, while the average delivery delay nearly doubles at the same time. at the same time, delivery ratio and delay stay nearly constant from 50 to 45 legitimate nodes without any attackers. 5 conclusion in this work, we have evaluated the performance of oppnets in the presence of two relevant attacker models. we have explored dos attacks and blackhole attacks via simulations. our results are very interesting and show potential new directions for security-aware protocols in oppnets. blackhole and dos attacks severely impact the performance of the network, even at low-intensity attacks which are very hard to detect. in the future, we plan to explore systematically further oppnets relevant attack models.at the same time, we will focus on developing counter-measures for them. furthermore, we will explore also the impact of dos and blackhole attacks with data dissemination protocols other than epidemic. references [ada16] m. alajeely, r. doss, a. ahmad. security and trust in opportunistic networks – a survey. iete technical review 33(3):256–268, may 2016. 3 / 4 volume 080 (2021) on the resilience of opportunistic networks against dos attacks 50/0 50/2 48/0 50/5 45/0 50/10 40/0 50/25 25/0 node combinations 0.00 0.25 0.50 0.75 1.00 d el iv er y ra tio dos attacks mix of total nodes to attackers 50/0 50/2 48/0 50/5 45/0 50/10 40/0 50/25 25/0 node combinations 0 20 40 d el iv er y de la y (h rs ) dos attacks mix of total nodes to attackers 900 sec 450 sec 180 sec 90 sec 45 sec attack frequency (sec) 0.00 0.25 0.50 0.75 1.00 d el iv er y ra tio dos attacks x sec inter-packet interval 900 sec 450 sec 180 sec 90 sec 45 sec attack frequency (sec) 0 20 40 d el iv er y de la y (h rs ) dos attacks x sec inter-packet interval figure 2: effect of dos attacks by varying attackers (top row) and attack intervals (bottom row). in the top row 50/2 refers to 50 nodes from which two are attackers (95% confidence intervals). in the bottom row, the network uses a 50/10 node combination for each attack frequency. [ccc10] f. c. choo, m. c. chan, e. chang. robustness of dtn against routing attacks. in 2010 second international conference on communication systems and networks (comsnets 2010). pp. 1–10. 2010. [ktuf19] v. kuppusamy, u. m. thanthrige, a. udugama, a. förster. evaluating forwarding protocols in opportunistic networks: trends, advances, challenges and best practices. future internet 11(5):113, 2019. [ufdk19] a. udugama, a. förster, j. dede, v. kuppusamy. simulating opportunistic networks with omnet++. in virdis and kirsche (eds.), recent advances in network simulation: the omnet++ environment and its ecosystem. pp. 425–449. 2019. [vb00] a. vahdat, d. becker. epidemic routing for partially-connected ad hoc networks. technical report, 2000. netsys 2021 4 / 4 introduction related work methodology results conclusion bandwidth performance analysis of mobile voip solutions electronic communications of the easst volume 75 (2018) 43rd international conference on current trends in theory and practice of computer science student research forum, 2017 (sofsem srf 2017) bandwidth performance analysis of mobile voip solutions rafael dantas, chris exton and andrew le gear 12 pages guest editors: anila mjeda eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst bandwidth performance analysis of mobile voip solutions rafael dantas1, chris exton2 and andrew le gear3 1rafael.dantas@lero.ie lero the irish software research centre university of limerick, ireland 2chris.exton@ul.ie lero the irish software research centre university of limerick, ireland 3andrew.legear@horizon-globex.ie horizon globex ireland dac nexus innovation centre, ireland abstract: despite the efforts to improve current 4g technologies by developing the new 5g mobile network, much of the world’s population still relies on older 2g and 3g infrastructures. although old hardware can be replaced, the costs of such endeavour can be prohibitive for companies operating on developing nations. network traffic optimisations can be used to provide a better experience for the increasing population of mobile devices connected. this paper provides a base comparison of the bandwidth usage between horizon globex’s smart packet voip solution (spvs) and a group of popular applications like skype, viber and whatsapp. the experiment shows spvs consumes less than 50% the amount of data when compared against the second place, whatsapp. nevertheless, more research is required to measure the impacts of the optimisations with respect to quality. keywords: voip, mobile, bandwidth, consumption, network 1 introduction as the 5g era approaches fast and the number of connected smart devices continues to increase, we find existing network infrastructures under increasing strain [cis]. this change disproportionately affects developing nations, as old hardware will be required to support the increasing demand of growing populations [cis]. while old hardware can be upgraded, and new equipment added, it can quickly become an expensive task, especially considering that most users will expect access to the internet with 3g or faster connections. with the majority of the world’s mobile user population continuing to use 2g networks [cis], network traffic optimisations can still provide benefits for mobile voice over internet protocol (voip) applications where modern mobile internet infrastructure is not available. 1 / 12 volume 75 (2018) mailto:rafael.dantas@lero.ie mailto:chris.exton@ul.ie mailto:andrew.legear@horizon-globex.ie bandwidth performance analysis of mobile voip solutions horizon globex’s smart packet voip solution (spvs) [cd15] includes an android and ios mobile application, a series of node servers responsible for broker signalling between the phones’ mobile applications, voip servers for hosting calls between mobile applications and gateways to standard telephony protocols, if necessary. it also employs a series of network and codec optimisations to allow real-time communication that is also stable, reliable and bandwidth efficient. this paper presents the results of an experiment comparing the bandwidth usage between the spvs and nine popular mobile voip applications, which suggests that the former uses less bandwidth than the other applications tested. 2 related work performance and quality comparisons between applications are commonplace in mobile voip literature [dcr+95], [zwl10], [swb01], [kc11], [ll16], [mm01], [räm10]. often the objective of these tests is to evaluate the performance of existing technologies to select the best solution for the product under development. for example, [dcr+95] performed such an analysis for selecting the best codec for the inmarsat mini-m system. similarly, [zwl10] also performed comparisons for the s3c241o micro-controller. the objective may also be to test the performance of a solution against the market standards and alternative solutions [swb01], [kc11], [ll16], [mm01], [rt11]. when compared against other codecs like amr and amr-wb, the then newly developed opus voice codec provided usable voice quality in [rt11]. to decrease the degradation of quality on voip systems caused by packet loss, [swb01] proposed a new amr codec and tested it against the usual codecs used with the h.323 protocol. a performance evaluation of skype was made by [kc11] comparing it to industry standards. to improve bandwidth utilisation on mobile networks, [ll16] tested a tcp implementation with four alternatives and found significant improvements. even classic algorithms can be subject to such tests. [mm01] tried to find which change would optimise nagle’s algorithm [nag84] for network traffic control. lastly, quality tests may also be used to propose new ideas. [räm10] used mean opinion scoring (mos) [swh16] to test if subjects could distinguish between many narrowband and wideband codecs, and found a correlation between higher scores and higher bit rates. [wd15] evaluated the 3g network infrastructure in bangkok, by testing how the mos of skype and line were affected by it. 3 the smart packet voip solution the spvs is a complete mobile voip network solution including public switched telephone network (pstn) breakout capabilities. while spvs allows two users to communicate via traditional telephony infrastructure, it also supports one or both sides to be completely disconnected from regular pstn if that user has access to the internet. figure 1 shows the basic workings of a typical voip call using spvs: • first the application contacts a central node, which will try to find the other user inside the network. sofsem srf 2017 2 / 12 eceasst • the central node will then assign a voip server for handling the call, sending this information to both participants. • both ends will try to connect with the voip server. this is done to avoid any network address translation (nat) or proxy problems that would arise otherwise. • once connected, the voip server becomes responsible for relaying the call between the participants. • if one or both the participants are not able to connect to the voip server, a session initiation protocol (sip) server will call the unreachable user using the normal telephony infrastructure. 3.1 optimisations: header elimination most protocols carry some metadata alongside the payload. this header information is used by the receiver to understand and access the information contained inside, especially in cases where the configuration parameters can be changed for each individual packet. to improve bandwidth efficiency, the spvs’ protocol doesn’t carry metadata on every single packet. instead, the developers of the solution use a series of metadata packets to coordinate the call between the devices, allowing the packet to carry only the actual encoded speech data. in a similar way to protocols, codecs also carry a series of parameters that must be sent alongside the encoded data, which will be used by the decoding process. as an example, speex parameters can be quite complex; as detailed by [tma11]. if both sides of the communication agree on the configuration parameters of the payload the header section that carries this information becomes redundant. it is then possible to reduce header size, and ultimately remove it altogether, by standardising every parameter of the transmission. the spvs uses only one codec with a very specific set of parameters, making payload headers completely obsolete and allowing every packet to carry only the compressed speech signal. figure 1: typical workflow of a spvs call. 3 / 12 volume 75 (2018) bandwidth performance analysis of mobile voip solutions even if the payload does not carry a single bit of header information, every single network protocol will envelop all data produced by the layer above and append its own headers to it. since those protocols are usually outside the scope of the application and only accessible by the os layer or lower, the changes previously proposed in this section cannot be applied here. to reduce the overhead imposed by the network protocol stack, we have used a sampling size of 100ms instead of 20ms. this change allows for a better usage of the network infrastructure, similar to the optimisations provided by nagle’s algorithm [nag84] on tcp. since more data is being sent for each packet, the overall overhead size is drastically reduced as a larger part of the information is part of the actual payload. 3.2 optimisation: silence detection in a normal conversation, usually only one of the participants is speaking while the other is listening to what is being said. if an application is not aware of this special case, it will always record and send speech information both ways. using silence detection algorithms, it is possible to avoid sending useless data through the wire. the spvs has a special silence packet, which is sent by the application whenever it detects its user has gone silent. this packet also doubles as a keep-alive message, being sent periodically by the silent side to maintain the connection open. 3.3 optimisation: codec selection as a freely available open source voice codec, speex was selected to be integrated into horizon globex’s solution. it also provides good results when compared with other narrowband voice codecs like amr and ilbc [räm10] [tma11] [kc11]. speex has many compression levels that decrease speech fidelity in order to decrease the overall size of the stream. this value ranges between 0 and 10, the former being the on with the highest compression but lowest quality while the latter is the exact opposite. horizon’s developers decided to use compression level 3, since they have found that any higher setting does not provide improvement of the perceived speech quality through the phone speaker. 3.4 optimisations: transport protocol selection there are two main protocols that operate over the internet protocol, version 4 (ipv4) [pos81a]: 1. the user datagram protocol (udp) [pos80] is a very simple protocol, designed to provide a minimalistic mechanism for message exchange on an ip network. with simplicity as its main feature, a datagram has only 8 bytes of metadata. the error checking algorithm guarantees that every message received was not corrupted along the way but, since there is no error recovery built into the protocol, not all messages sent will arrive at their destination. 2. another protocol is the transmission control protocol (tcp) [pos81b]. it was built for reliable communication between two computers inside the same network or between two sofsem srf 2017 4 / 12 eceasst networks. to provide such reliability, the header is much bulkier than its simpler counterpart, containing at least 24 bytes of metadata. although this feature is very important for all sorts of applications, its retransmission routines are usually more harmful to real-time applications like voip than the lost data they were designed to retrieve. due to its simplicity and smaller footprint, udp was chosen as the best option as a transport layer protocol. 4 evaluation section 4.1 will analyse the network performance of the spvs, described in section 3, against the other solutions to test the assertions made in section 1. for this experiment, we have selected some of the most commonly used applications in the western market, namely “facebook messenger”, hangouts, line, skype, telegram, viber and whatsapp. we have also selected two popular chinese applications to represent that market, namely qq and wechat. table 1 presents the number of installed devices for all aforementioned applications. the numbers for all applications except wechat and qq were taken from google’s play store, in 01/03/2017. since wechat and qq’s mainly operate on the chinese market, their numbers were taken from huawei’s application store. lastly, hangouts’ numbers might not reflect its actual popularity since android devices usually come with it installed by default. 4.1 experimental design this experiment was designed to provide quantitative data for the analysis in section 4.3. we used two android mobile phone where the tested application was installed with two gnu/linux laptops acting as access points for both mobile phones and an earphone to reproduce a half minute-long audio file through both devices’ microphones. figure 2 is an illustration of this experiment. to properly intercept all data going through the internet, both phones were connected to it via application installations wechat 2,344,124,542 qq 2,004,169,512 messenger 1,000,000,000 hangouts 1,000,000,000 whatsapp 1,000,000,000 line 500,000,000 skype 500,000,000 viber 500,000,000 telegram 100,000,000 table 1: number of installations per application. 5 / 12 volume 75 (2018) bandwidth performance analysis of mobile voip solutions the wireless access points provided by both laptops, which were connected to the internet via two independent ethernet cables. using a personalised command-line interface (cli) described in section 4.2 to schedule 40 test executions for each application described in section 1. this cli was responsible for synchronising the actions of both computers during each test. all tests performed followed these steps: 1. all tests are queued on the console, grouped by application. 2. using a “next” command, the terminal loads the next test and waits for a user input. this input signals that both phones are connected in a call and ready for the audio playback. 3. once the “enter” key is pressed, “tcpdump” is invoked as a background process, to record all network activity necessary for data collection. 4. immediately after the previous step, the tool plays a half-minute-long audio file. this file contains phrases that alternate between the left and right channels to simulate a conversation with moments of activity and silence. 5. both sides wait until the audio reproduction is complete. then the terminal signals “tcpdump” to stop recording the network. 6. a counter is incremented to signals the completion of this test and the interface waits for a new “next” command or another user interaction. all data collected by tcpdump was filtered using wireshark, a network packet capture tool, which can read the format used by tcpdump and output the desired fields into the console, for further analysis. since all data going through the laptop was recorded into files, the most accessed ip address for that call was used to filter the data. figure 2: representation of the experiment. sofsem srf 2017 6 / 12 eceasst application wire throughput (kb/s) payload throughput (kb/s) packet rate (p/s) frame size (bytes) payload size (bytes) payload-to-header ratio horizon 8.24 5.90 7.97 130 93 2.52 whatsapp 23.59 17.44 22.23 133 99 2.83 qq 28.64 20.07 31.08 116 81 2.34 facebook 29.23 24.02 16.76 219 180 4.61 viber 40.66 28.22 45.02 113 79 2.27 wechat 44.85 35.83 33.13 170 136 3.98 telegram 48.22 40.40 27.71 218 183 5.17 line 53.82 42.21 42.33 159 125 3.64 hangouts 59.06 32.35 50.80 146 80 1.21 skype 84.54 61.69 83.98 126 92 2.70 table 2: comparison between various applications performances the following section details the tools used for the experiment. 4.2 tool support a personalised command-line interface was created to coordinate both laptops in all the tasks they were performing. this interface is able to connect the computers, interface with local network configuration by invoking “tcpdump” [jlm89] and “tc” [kuz01], and it’s also capable of rolling back a test if needed. “tc” [kuz01] is a tool created to configure traffic control inside the linux kernel. it is used for defining rules for incoming and outgoing packets on a device. this tool has many extensions, so called queuing disciplines or “qdiscs”, which are queues used to save packets while the network interface isn’t ready to handle them. all data was collected using “tcpdump” [jlm89], a tool for inspecting all traffic on a network and storing all data onto a file. the output format is widely supported by many tools. two bash scripts were written for the filtering of data after its collection. these filters use “tshark” [com12] to filter the pcap files created by “tcpdump”.1. the audio sample was reproduced using “mplayer” [tea05], a command-line interface player which was called by the script before changing the network properties. for data analysis, “tshark” [com12] and “wireshark” [c+07] were selected for filtering the results, removing unrelated network traffic and for selecting only the fields necessary for the analysis. in this experiment, every call happened after “tc” was used to create a new class. this class was configured to limit the bit rate to 100kbps, ensuring the application would use a narrowband codec and giving the previously mentioned terminal some band to run commands without interfering with the experiment. 4.3 quantitative analysis: bandwidth usage table 2 has a comparison between all applications analysed as part of our evaluation. • “wire throughput” and “payload throughput” are averages of the data rate sent to the network. the former’s numbers include the network overhead produced by all protocols in 1 these scripts, along with set up advice, are available from the authors on request. 7 / 12 volume 75 (2018) bandwidth performance analysis of mobile voip solutions the network stack while the latter’s numbers only include the data carried by the transport layer. smaller numbers represent smaller data consumption and cheaper calls. • “packet rate” is the average number of packets sent per second. applications with smaller values for this column might be less demanding on the network infrastructure while higher values may imply higher tolerance to individual packet losses. • “frame size” and “payload size” are the average sizes of the individual packets sent by each solution. the former includes the size of the network overhead while the latter does not. although higher numbers may imply higher network usage, this number only has true meaning when used together with packet rate. • “payload-to-header ratios” are simple ratios between the payload sizes and the average network overhead size, which is the frame size minus the payload size. smaller rations imply higher amount of data wasted on network headers. the solutions were presented in ascending order by wire throughput, this is the values that most closely relates to the cost of the calls made by each solution. when compared against the second closest solution in the first three categories, the spvs uses, in average, 65%, 66% and 52% respectively less resources. the results are striking using less than half the bandwidth than the second closest solution, whatsapp. the spvs has a network performance lead in this experiment. viber had the best results on both frame size and payload size. these numbers alongside a higher packet rate than the average of the other applications suggest they may be using a buffer size smaller than the other solutions. lastly, in the payload-to-header ratio column, telegram has the best payload-to-header ratio. they also have one of the biggest payload size, which naturally dwarfs the size of the header and inflates this ratio. 5 discussion the spvs shows the best results with respect to the amount of data sent through the network, using at worst half as much data as the second closest application but on average 80% less throughput than other applications. table 3 shows the average cost per second of a call using each solution tested here. to calculate the correct cost, we have assumed an approximate 0.10 usd/mb, the standard rate for intertelecom company gsm data, since a precise figure would vary between contract agreements between the solution’s developer and its internet service provider. although both qq and wechat performed well on their respective runs of this experiment, the sound received by the callee was of very poor quality. for all other applications, the voice message was adequate and understandable in its entirety. skype had the best payload-to-header ratio, but that was because the payload carried by each packet was also larger in comparison to the size of the header. since the only poor network condition tested was packet loss, most solutions probably could not trigger changes in protocol. sofsem srf 2017 8 / 12 eceasst application wire throughput (kb/s) cost (usd cents/s) horizon 8.24 0.08 whatsapp 23.59 0.23 qq 28.64 0.28 facebook 29.23 0.29 viber 40.66 0.40 wechat 44.85 0.44 telegram 48.22 0.47 line 53.82 0.53 hangouts 59.06 0.58 skype 84.54 0.83 table 3: average approximate cost of a call (assuming 10cent/s cost) it is also surprising that most of the tested solutions were using tcp as their transport protocol instead of udp, which would be the more sensible choice for real time applications like mobile voip, given the reasons discussed in section 3.4. we believe the authors of the solutions using tcp over udp believe that modern day internet connections are stable enough that the extra cost of developing for the latter protocol might not be justifiable. tcp already has concepts that are useful to voice calls like session and packet ordering that would need to be implemented by the application when using udp. 6 threats to validity network fluctuations could affect the results of this experiment, making one solution appear better or worse than it would have been otherwise. traffic shaping rules could also affect the numbers in a similar fashion. to minimise the effects of this threat, the experiments were run 40 times for the same application. a better separation between inbound and outbound traffic could also be more useful for analysis than the total amount of data exchanged between both sides of the call. finally, network performance isn’t the only important measure for a mobile voip solution the perceived quality of the audio must also be maintained. the experimenter listened to the transmitted audio for each data point and confirmed that quality audio was being maintained, but we feel that although these initial tests provide some bases for comparison, we intend to further investigate these results using a mos test in the next phase of our research. 7 future work the next step in this research is to develop a more comprehensive experiment for both assertions in section 1, which will record both how much data is being sent through the network and how much of it is getting to the other side of the call. this will require data from both cell phones to be recorded. 9 / 12 volume 75 (2018) bandwidth performance analysis of mobile voip solutions bandwidth savings are irrelevant if the overall quality of the audio sample has been degraded. therefore, another important step is to properly measure how much has this property been affected by the optimisations and how spvs’s audio quality compares against the other solutions using a mos assessment. finally, we intend to analyse ways to improve the current protocol. ideas for future refinements include: • a dictionary that will store common patterns produced by the codec to further compress the information being transmitted, • a permanent personal audio profile to improve codec compression when talking with the same person more than once. 8 conclusion although the spvs showed very promising results in this experiment, a more detailed experiment will provide us with better data regarding current mobile voip solutions, especially with respect to quality. 9 acknowledgements this work was supported with the financial support of the science foundation ireland grant 13/rc/2094 and co-funded under the european regional development fund through the southern & eastern regional operational programme to lero the irish software research centre (www.lero.ie). this work was possible thanks to brian collins, ceo of the horizon globex ireland dac, for permitting technical analysis of the product. (sow2016-034) bibliography [c+07] g. combs et al. wireshark. web page: http://www. wireshark. org/last modified, pp. 12–02, 2007. [cd15] b. collins, c. dziedzic. method and devices for routing in a satellite-based communication system. sept. 15 2015. us patent 9,137,729. [cis] c. v. n. i. cisco. global mobile data traffic forecast update, 2015–2020 white paper, 2016. [com12] g. combs. tshark-dump and analyze network traffic. wireshark, 2012. [dcr+95] s. dimolitsas, f. corcoran, c. ravishankar, a. wong, s. de campos neto, r. skaland. evaluation of voice codec performance for the inmarsat mini-m system. in digital satellite communications, 1995., tenth international conference on. pp. 101–105. 1995. sofsem srf 2017 10 / 12 eceasst [jlm89] v. jacobson, c. leres, s. mccanne. the tcpdump manual page. lawrence berkeley laboratory, berkeley, ca 143, 1989. [kc11] k. kim, y.-j. choi. performance comparison of various voip codecs in wireless environments. in proceedings of the 5th international conference on ubiquitous information management and communication. p. 89. 2011. [kuz01] kuznetsov, a. tc. 12 2001. [ll16] k. liu, j. y. lee. on improving tcp performance over mobile data networks. ieee transactions on mobile computing 15(10):2522–2536, 2016. [mm01] j. c. mogul, g. minshall. rethinking the tcp nagle algorithm. acm sigcomm computer communication review 31(1):6–20, 2001. [nag84] j. nagle. congestion control in ip/tcp internetworks. technical report, internet engineering task force (ietf), 1984. [pos80] j. postel. user datagram protocol. technical report, internet engineering task force (ietf), 1980. [pos81a] j. postel. internet protocol. technical report, internet engineering task force (ietf), 1981. [pos81b] j. postel. transmission control protocol. technical report, internet engineering task force (ietf), 1981. [räm10] a. rämö. voice quality evaluation of various codecs. in acoustics speech and signal processing (icassp), 2010 ieee international conference on. pp. 4662–4665. 2010. [rt11] a. rämö, h. toukomaa. voice quality characterization of ietf opus codec. in interspeech. pp. 2541–2544. 2011. [swb01] j. w. seo, s. j. woo, k. s. bae. study on the application of an amr speech codec to voip. in acoustics, speech, and signal processing, 2001. proceedings.(icassp’01). 2001 ieee international conference on. volume 3, pp. 1373–1376. 2001. [swh16] r. c. streijl, s. winkler, d. s. hands. mean opinion score (mos) revisited: methods and applications, limitations and alternatives. multimedia systems 22(2):213–227, 2016. [tea05] m. team. mplayer–the movie player. http:/www. mplayer. hq. hu, 2005. [tma11] e. touloupis, a. meliones, s. apostolacos. implementation and evaluation of a voice codec for zigbee. in computers and communications (iscc), 2011 ieee symposium on. pp. 341–347. 2011. 11 / 12 volume 75 (2018) bandwidth performance analysis of mobile voip solutions [wd15] p. wuttidittachotti, t. daengsi. quality evaluation of mobile networks using voip applications: a case study with skype and line based-on stationary tests in bangkok. international journal of computer network and information security (ijcnis) 7(12):28, 2015. [zwl10] j. zhou, t. wu, j. leng. research on voice codec algorithms of sip phone based on embedded system. in wireless communications, networking and information security (wcnis), 2010 ieee international conference on. pp. 183–187. 2010. sofsem srf 2017 12 / 12 introduction related work the smart packet voip solution optimisations: header elimination optimisation: silence detection optimisation: codec selection optimisations: transport protocol selection evaluation experimental design tool support quantitative analysis: bandwidth usage discussion threats to validity future work conclusion acknowledgements askthecode: interactive call graph exploration for error fixing and prevention electronic communications of the easst volume 77 (2019) interactive workshop on the industrial application of verification and testing, etaps 2019 workshop (interavt 2019) askthecode: interactive call graph exploration for error fixing and prevention robert husák, jan kofroň and filip zavoral 6 pages guest editors: anila mjeda, stylianos basagiannis, goetz botterweck eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst askthecode: interactive call graph exploration for error fixing and prevention robert husák1, jan kofroň2 and filip zavoral3 1 husak@ksi.mff.cuni.cz 2 jan.kofron@d3s.mff.cuni.cz 3 zavoral@ksi.mff.cuni.cz charles university faculty of mathematics and physics, prague, czech republic abstract: in order to prevent and fix errors in program code, developers need to understand its semantics to a significant extent. for this purpose, they use various approaches, such as manual call graph exploration or dynamic analysis with a debugger. however, these techniques tend to be cumbersome in a larger codebase, because they provide either underapproximate or overapproximate results and it is often hard to combine them. therefore, we present askthecode, a microsoft visual studio extension enabling to interactively explore a call graph, ensuring that only feasible execution traces are taken into consideration. askthecode is based on control flow analysis and backward symbolic execution. we show its potential to significantly improve developers’ experience on a complex code example. keywords: control flow analysis, symbolic execution, call graph, interactive, debugging 1 introduction when developers want to fix and prevent errors by understanding the semantics of the source code, they usually use manual call graph exploration and debugging. however, call graphs are often complicated and the calls can be constrained by various conditions. debugging, on the other hand, might be misleading and incapable of capturing all the possible situations. therefore, finding the exact context under which a bug can occur is tedious and error-prone [lm10]. we believe there is an opportunity to utilize a sound analysis technique for this task, for several reasons. first, the context is usually much more limited than in the case when we try to verify the program as a whole. second, the developers might be more willing to wait long enough for the analysis to complete, because it may save much more time to them than if they wanted to do it manually. last, even if the problem is too complicated to be handled properly in a reasonable time, developers can interact with the tool, applying appropriate abstractions and assumptions. therefore, we have created a tool named askthecode, which utilizes control flow analysis and backward symbolic execution to help developers investigate particular problems in the source code. 1 / 6 volume 77 (2019) mailto:husak@ksi.mff.cuni.cz mailto:jan.kofron@d3s.mff.cuni.cz mailto:zavoral@ksi.mff.cuni.cz askthecode: interactive call graph exploration for error fixing and prevention 2 design the primary purpose of askthecode is to reason about the context under which a certain situation in a program can occur. therefore, an efficient way to formulate what problem we need to address is writing an assertion a0 specifying the expected semantics and check in which situations it can be violated. notice that this approach is suitable not only for bug fixing, but also for their prevention, because developers can use the tool to confirm their possibly incorrect ideas about the program semantics. to verify an assertion, a natural approach is to inspect the code against the control flow and look for any input that eventually violates it. because askthecode is a microsoft visual studio extension aimed at c# code analysis, it can access the roslyn .net compiler [nh14]. we use it to construct control flow graphs and the call graph of the methods related to the problem being solved. next, we utilize backward symbolic execution [bcd+18, cfs09] to perform the assertion verification itself, employing the z3 smt solver [db08]. depending on the user configuration, an entry point is either a non-private method in the same class as where a0 occurs, or a public method in a public class within the project. to tackle path explosion and other symbolic execution problems, users can also adjust loop unwinding, recursion limit, and timeout. all the execution traces from entry points to violating a0 are continuously gathered during the run of backward symbolic execution. we provide users with a panel allowing them to interactively explore those traces as if they were a program under execution. they can step forward and backward throughout the particular methods and statements and see their intermediate values. furthermore, they can also inspect the relevant heap structure and its changes. there is also a high-level overview in the form of an annotated call graph, rendered using the microsoft automatic graph layout library [nnb+16]. the graph is continuously updated as backward symbolic execution runs on background. at the beginning, it consists only of the method m0 containing a0, but then it expands as the explored states extend to the callers and callees of m0. whenever an execution trace is discovered, all the methods it traverses are emphasized in red to draw the user’s attention. on the other hand, if there are not any states capable of extending to a method mu, we mark mu as unreachable by making its background green. the user can also explicitly ignore certain methods, letting the analysis focus on less complicated ones. 3 example to demonstrate how can askthecode help developers, let us inspect the code example in figure 1. it presents an excerpt from a library working with singly linked lists. they are represented by the node class containing an integer field val and a reference to the following node called next. due to a certain implementation issue, it is demanded that whenever a node is last in a list, its value must be zero. in lastnode, we check this property using an assertion. imagine that we have received a bug report stating that this property is sometimes violated, but we were not given any exact context under which it can happen. the most common approaches to solving these tasks are manual call graph exploration and debugging, both of which can be problematic in certain situations. manual call graph exploration interavt 2019 2 / 6 eceasst private bool lastnode(node n) { debug.assert(n.next != null || n.val == 0); return n.next == null; } public void checkeduse(node n) { if (n.val == 0 && lastnode(n)) { /*...*/ } } public node uncheckeduse(node n) { node gen = randomnode(); if (lastnode(gen)) { gen.next = n; } return gen; } private node randomnode() { int v = getrandomnumber(); if (v == 0) return new node(0, null); if (v == 1) return new node(10, null); return toocomplicatedoperation(); } figure 1: example c# code with an assertion in lastnode to be verified requires us to inspect the logic of each affected method to ensure the situation is handled in there. in our example, we need to inspect checkeduse, although it never allows the error to happen. debugging, on the other hand, reveals only the situations which can indeed cause the error being inspected. however, it might be often difficult to reproduce the error and we cannot be sure that we have discovered all the contexts under which it can occur. furthermore, the information provided by the debugger is limited. considering our example, after reproducing the error and pausing the program execution on the violated assertion, we can see uncheckeduse in the call stack. however, we cannot see what exactly happened in the call of randomnode, because we no longer know the indeterministic value v and hence the executed branch. the mentioned problems can be solved using askthecode, as we can see in figure 2. the call graph displayed in the top right panel shows that lastnode is called both by checkeduse and uncheckeduse. checkeduse is displayed with a green background and dashed arrow, as it is proven not to cause the error. on the other hand, uncheckeduse is emphasized in red together with its callees randomnode, getrandomnumber and the node constructor, as there was found an assertion violating program trace going through them. randomnode can also potentially call another method, toocomplicatedoperation, whose definition we have intentionally skipped. as we can tell from its name, it was too complicated for backward symbolic execution to handle automatically. therefore, as it is shown in the call graph, we at least know that it is worth to be inspected manually, unlike checkeduse. regarding the found trace, we can explore it in the bottom panel by inspecting its call tree and the particular statements. whenever we select a statement in the table, the appropriate piece of code is selected in the opened code editor in the top left part. furthermore, there is a replay panel on the right in which we can see the contents of the heap and of all the local variables at the given step of the trace. on its bottom, there are buttons to navigate through the trace in a debugger-like fashion, with the added capability of stepping backward in the history. from the trace, we can now easily discover that at least one of the problematic situations happens when v in randomnode is 1. 3 / 6 volume 77 (2019) askthecode: interactive call graph exploration for error fixing and prevention figure 2: askthecode in microsoft visual studio 2017 4 related work in the field of human interface design, there are numerous tools helping developers to understand the semantics of the code [sbm17, dr10]. probably the closest one to our approach is reacher [lm11], a tool to interactively and intuitively explore complex call graphs. however, these tools do not reason about reachable paths as soundly as symbolic execution. furthermore, they are not directly aimed at verifying assertions, hence no production of error traces. on the other hand, to discover erroneous program inputs, we can use symbolic execution, whose current state of the art is summarized in [bcd+18]. most of these techniques analyse a program by systematically exploring its state space from various entry points in a forward fashion, aiming for high code coverage. in order to reason about particular assertions, we use the backward variant of symbolic execution, whose most important representative is snugglebug [cfs09]. despite its advanced capabilities used to alleviate the path explosion problem, it can still occur in practice. in such situations, snugglebug and other related tools [da14, myfh11] cannot provide sufficient information due to their lack of interactivity. interavt 2019 4 / 6 eceasst 5 conclusion on a complex example which models situations known from practice, we demonstrate that askthecode can help developers understand causes of errors. utilizing interactive approach, certain limitation of used backward symbolic execution can be alleviated, e.g. by voluntarily omitting certain problematic places or controlling loop unwinding. in the future, we plan to extend the interactivity even more, and implement more advanced features such as state merging [kkbc12] or directed call graph construction [cfs09], ultimately making askthecode production-ready. acknowledgements: this work was supported by the project progress q48, the czech science foundation project 18-17403s and the grant svv-2017-260451. bibliography [bcd+18] r. baldoni, e. coppa, d. c. d’elia, c. demetrescu, i. finocchi. a survey of symbolic execution techniques. acm computing surveys (csur) 51(3):50, 2018. doi:10.1145/3182657 [cfs09] s. chandra, s. j. fink, m. sridharan. snugglebug: a powerful approach to weakest preconditions. sigplan not. 44(6):363–374, june 2009. doi:10.1145/1543135.1542517 [da14] p. dinges, g. agha. targeted test input generation using symbolic-concrete backward execution. in 29th ieee/acm international conference on automated software engineering (ase). acm, västerås, sweden, september 15-19 2014. doi:10.1145/2642937.2642951 [db08] l. de moura, n. bjørner. z3: an efficient smt solver. in proceedings of the theory and practice of software, 14th international conference on tools and algorithms for the construction and analysis of systems. tacas’08/etaps’08, pp. 337–340. springer-verlag, berlin, heidelberg, 2008. doi:10.1007/978-3-540-78800-3 24 [dr10] r. deline, k. rowan. code canvas: zooming towards better development environments. in proceedings of the 32nd acm/ieee international conference on software engineering volume 2. volume 2, pp. 207–210. 05 2010. doi:10.1145/1810295.1810331 [kkbc12] v. kuznetsov, j. kinder, s. bucur, g. candea. efficient state merging in symbolic execution. in proceedings of the 33rd acm sigplan conference on programming language design and implementation. pldi ’12, pp. 193–204. acm, new york, ny, usa, 2012. doi:10.1145/2254064.2254088 5 / 6 volume 77 (2019) http://dx.doi.org/10.1145/3182657 http://dx.doi.org/10.1145/1543135.1542517 http://dx.doi.org/10.1145/2642937.2642951 http://dx.doi.org/10.1007/978-3-540-78800-3_24 http://dx.doi.org/10.1145/1810295.1810331 http://dx.doi.org/10.1145/2254064.2254088 askthecode: interactive call graph exploration for error fixing and prevention [lm10] t. d. latoza, b. a. myers. developers ask reachability questions. in proceedings of the 32nd acm/ieee international conference on software engineering volume 1. icse ’10, pp. 185–194. acm, new york, ny, usa, 2010. doi:10.1145/1806799.1806829 [lm11] t. d. latoza, b. a. myers. visualizing call graphs. in 2011 ieee symposium on visual languages and human-centric computing (vl/hcc). pp. 117–124. sep. 2011. doi:10.1109/vlhcc.2011.6070388 [myfh11] k. ma, k. yit phang, j. s. foster, m. hicks. directed symbolic execution. in static analysis. pp. 95–111. springer berlin heidelberg, berlin, heidelberg, 2011. doi:10.1007/978-3-642-23702-7 11 [nh14] t. neward, j. hummel. rise of roslyn. msdn 29(11):70, 2014. https://msdn.microsoft.com/en-us/magazine/dn818501.aspx [nnb+16] l. nachmanson, a. nocaj, s. bereg, l. zhang, a. holroyd. node overlap removal by growing a tree. in graph drawing and network visualization. pp. 33– 43. springer international publishing, cham, 2016. doi:10.1007/978-3-319-50106-2 3 [sbm17] j. smith, c. brown, e. murphy-hill. flower: navigating program flow in the ide. in 2017 ieee symposium on visual languages and human-centric computing (vl/hcc). pp. 19–23. 10 2017. doi:10.1109/vlhcc.2017.8103445 interavt 2019 6 / 6 http://dx.doi.org/10.1145/1806799.1806829 http://dx.doi.org/10.1109/vlhcc.2011.6070388 http://dx.doi.org/10.1007/978-3-642-23702-7_11 https://msdn.microsoft.com/en-us/magazine/dn818501.aspx http://dx.doi.org/10.1007/978-3-319-50106-2_3 http://dx.doi.org/10.1109/vlhcc.2017.8103445 introduction design example related work conclusion composing control flow and formula rules for computing on gridswork done while the first author was working in aizu university as adjunct professor. electronic communications of the easst volume 10 (2008) proceedings of the seventh international workshop on graph transformation and visual modeling techniques (gt-vmt 2008) composing control flow and formula rules for computing on grids1 p. bottoni and n. mirenkov and y. watanobe and r. yoshioka 15 pages guest editors: claudia ermel, reiko heckel, juan de lara managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 1 work done while the first author was working in aizu university as adjunct professor. http://www.easst.org/eceasst/ eceasst composing control flow and formula rules for computing on grids2 p. bottoni1 and n. mirenkov2 and y. watanobe2 and r. yoshioka2 1 bottoni@di.uniroma1.it dep. of computer science, ”sapienza” univ. of rome, italy 2(nikmir,yutaka,rentaro)@u-aizu.ac.jp dep. of computer software, univ. of aizu, japan abstract: we define computation on grids as the composition, through pushout constructions, of control flows, carried across adjacency relations between grid cells, with formulas updating the value of some attribute. the approach is based on the identification of a subcategory of attributed typed graphs suitable to the definition of pushouts on grids, and is illustrated in the context of the cyberfilm visual language. keywords: grids, control flow rules, dpo 1 introduction graphs have been long proposed as a universal formalism for describing the structure of system configurations and to support computational specifications of the transformations they may undergo. moving from this common ground, the areas of graph transformations and graph algorithms have taken two divergent, possibly complementary paths. on the one hand, graph transformations propose a declarative approach to computation based on the iteration of local modifications to the graph structure, so as to define a language of admissible graph configurations, each depicting a possible state of the system being modelled. on the other hand, algorithms on graphs exploit procedural definitions of visits to the graph structure, usually to extract some global property of it. in many cases, graph transformations – typically performed by enriching the graph with additional features such as types [cmr96], attributes [mw93, hkt02, dbe+07] or control structures on rule application [kk99, swz99, bkpt00] – are capable of replicating many relevant features of the algorithmic approach. however, the general approach to graph transformations – based on the search for a subgraph isomorphism between the antecedent of a rule and the host graph under scrutiny – is not optimal for spatially organized structures, such as grids (in any number of dimensions), trees, or pyramids [ym02, wmym08], as the inherent non-determinism of the matching process fails to take advantage of the existence of privileged relations among elements, and of orders for their visit. we propose to reconcile the use of graph transformation as a general computational framework with the existence of some spatial structure on the host graph. to this end, we combine a suite of meta-models for diagrammatic languages – defining the possible spatial relations among identifiable elements [bg04], their transformation semantics [blg07], and the relations between the two [dgb07] – with a form of algebraic composition of rules in the framework of the double pushout approach to graph rewriting. the proposal is applied to the cyberfilm visual environment, which provides the user with iconic representations of computational flows on spatial structures. these representations are 2 work done while the first author was working in aizu university as adjunct professor. 1 / 15 volume 10 (2008) mailto:bottoni@di.uniroma1.it mailto:(nikmir,yutaka,rentaro)@u-aizu.ac.jp composing rules on grids arranged as sequences of frames highlighting the set of nodes which at each step contribute to the production of a new result [wmym08]. in a separate view, the formulas defining the computations can be defined, thus allowing their reuse according to different control flows. in particular, we focus on bidimensional grids on which several control flows can be defined, and use a categorical construction to provide a formal treatment of the composition of control flow and computational formulas. in the rest of the paper, after related work in section 2, we provide background on graph transformations and the adopted metamodels in section 3. section 4 introduces the categories on grids needed to define control flow rules in section 5. finally, section 6 shows how to compose formulas and control flows, before drawing conclusions in section 7. 2 related work spatial structures, such as those defined by grids or trees, have been the subject of many studies from the algorithmic point of view, in particular as regards the identification of paths with particular properties over them [ips82]. from the algebraic point of view, trees have been studied as representations of computational structures, such as terms [hp95] or abstract syntaxes [mos94], whereas images, rather than grids, have been studied in relation to the sets of languages definable on them [gr97]. the translation morphism discussed in this paper may be seen as an analogous of the ”positional overlapping” operation for images [bl07]. the technique for composing control flows and formulas differs from the notion of (local) application of rules to rules in [par94], based on finding a match from a rule component to a component of another rule, as well as from that of action pattern in [blg07], where a pattern is matched to the right-hand side of a rule to produce a rule whose effects conform to the pattern. a construction analogous to the one here is in [tb94], exploiting common subrules to identify possible agreements on a host graph and construct amalgamated versions of the rules. although we can also use this notion to find agreements between rule application, we are mainly interested here in the construction of new rules from rules defined on different graph types. finally, we point out a similarity with notions of modularity and viewpoints [gemt00], proposed as a way to modeling a system through the integration of partial models. however, we combine different aspects of the behaviour of the system into an integrated specification, rather than considering behavioral and structural aspects together. the approach to coordination proposed in [afgk02] is also based on pushouts (actually colimits), to allow separation of concerns when defining different aspects of a program behaviour. 3 background: metamodels and graph transformations according to the metamodel for diagrammatic languages presented in [bg04], and shown in figure 1, a diagram is composed of identifiable elements among which significant spatial relations exist. a whole diagram is itself an identifiable element, with global properties. an identifiable element is a recognizable unit in the language, associated with a graphical representation defined by a complex graphic element, composed in turn of one or more graphic elements, each possessing some attach zone, representing the geometrical support for spatial relations. the existence of proc. gt-vmt 2008 2 / 15 eceasst figure 1: the overall metamodel for diagrammatic languages. a relation is assessed via a predicate isattached() implemented by each zone. symmetries may exist between spatial relations. two relations σ and ρ are tied by a symmetry if there is a size-preserving diagram transformation changing all instances of ρ into instances of σ . specializations of these abstract types define language families. for example, in connectionbased languages, an entity acts as an endrole for connection elements, while the significant relation is touches, determined by the coincidence of a dot at the end of a connection with a point on the border of an entity. in this paper we are interested in languages based on adjacency, which indicates a class of relations between cells of regular shape tessellating the plan, and whose borders overlap for a finite segment. according to the type of tessellation, cells may entertain various adjacency relations, typically possessing symmetric companions, such as in left and right adjacency for regular arrangements of rectangles. based on this metamodel, we can represent diagrams as attributed typed graphs, where nodes are elements of classes in the metamodel and edges are instances of the associations. formally, a type graph is a construct t g = (nt , et , st ,t t ) with nt and et sets of node and edge types. st : et → nt and t t : et → nt define the source and target node types for each edge type. a typed graph on t g is a graph g = (n, e, s,t) with a graph morphism type : g → t g composed of typen : n → nt and typee : e → et , s.t. typen (s(e)) = st (typee (e)) and typen (t(e)) = t t (typee (e)). type graphs with node inheritance exploit a pair t gi = (t g, i), where i = (ni , ei , si ,t i ) is a node inheritance graph, with ni = nt , i.e. i has the same nodes as t g, but its edges are the inheritance relations. the inheritance clan of a node n is the set of all its children nodes (including n itself): clan(n) = {n′ ∈ ni|∃ path n′ →∗ n in i}⊆ ni . typed attributed graphs are typed graphs with additional data nodes and attribute edges (nodes of g are now called object nodes). a type graph t g has a set ∆ of data type nodes and a set a of attribute type edges, denoting the domains of the attribute nodes and the set of attributes associ3 / 15 volume 10 (2008) composing rules on grids ated with nodes, together with functions σa : nt →p(a), defining the attributes for a given type, and τa : a → ∆, defining the admissible domain for each attribute. these elements define a type graph with attributes t ga. a typed attributed graph on t ga is a construct (t g, g, n∆, ea, sa,ta), where t g and g are as before, n∆ is the set of data nodes, coinciding with the disjoint union of the domains of attributes, and ea is the set of attribute edges, from object nodes to data nodes. edges are typed on a and associate object nodes with the values of its attributes. sa : ea → n and ta : ea → n∆ define the valuation of attributes for a given node, coherently with σa and τa. we represent data nodes as typed items and distinguish them from object nodes through a dotted contour, following the convention proposed in example 8.5 of [eept06]. attributed typed graphs form the adhesive hlr category [ls04] agraphatg, so that transformations can be expressed through double pushout (dpo) derivations [eept06], in which rules are spans l ← k → r and k defines the part which is left unchanged by the rule application. we also exploit application conditions, as shown in section 4. 4 categories on grids rectangular grids are regular arrangements of cells according to symmetrical pairs of vertical and horizontal adjacency relations, with nrow rows and ncol columns, conforming to the family of adjacency-based diagrammatic languages depicted in figure 2. hence, nodes are instances of cell, with a position given by their row and col attributes and boolean values to distinguish border cells. adjacency relations can be of four types, with the obvious constraints on their pairing. moreover, there exists a set of additional constraints stating that a grid has to form a rectangle (i.e. its top and bottom borders must have the same number of elements, as must its left and right borders), and all border elements are adjacent to three other cells except the four corner elements, adjacent to two. mirror and rotation symmetries exists between pairs of adjacency relations. figure 2: the family of bidimensional grids. grids thus give rise to a subcategory agridt of agraphatg, whose morphisms are composed of a structural part, involving nodes of type cell and adjacency, and a data part involving proc. gt-vmt 2008 4 / 15 eceasst attributes. the structural part uses translations as morphisms. a translation exists from a grid g1 to a grid g2 if g2 is such that an isomorphism exists from g1 to a subset of its cells, preserving its connectivity and the relative directions. a translation is uniquely determined by the position of the image of the upper left corner (or any other cell) of the original grid in the context of the target grid. figure 3 shows the composition of two translations, where the highlighted rectangles show the new positions of the original grid. we assume that translations occur only rightwards and downwards. the pairs (r, c) labeling the morphisms indicate the offsets at which the nodes of the original grid are found in the new grid. the size of g2 is at least equal to that of g1. the identity morphism is the translation (0, 0) from a grid into itself, and morphism composition is the vectorial sum of the translations. we now study the subcategory tgridt, obtained by taking the structural part of agridt, i.e. maintaining the type information, but forgetting attributes. figure 3: translation morphism and composition. in tgridt, a pushout g1 p1→ p p2← g2 for a span g1 t1← g t2→ g2 between two grids can be constructed in the same way as the pushout in graph if and only if one of the following is true: (1) either t1 or t2 is an identity; (2a) t1 has a label of the form (r1, 0) and g1.ncol == g.ncol and (2b) t2 has a label of the form (0, c1) and g2.nrow == g.nrow; (3a) t1 has a label of the form (0, c2) and g1.nrow == g.nrow and (3b) t2 has a label of the form (r2, 0) and g2.ncol == g.ncol. then, p has size (max(g1.nrow, g2.nrow), max(g1.ncol, g2.ncol)); morphisms p1 : g1 → p and p2 : g2 → p are labeled by (max(r1, r2)−r1, max(c1, c2)−c1), and (max(r1, r2)−r2, max(c1, c2)−c2), respectively, so that parallel arrows have the same label (see figure 4). the pushout complement g x1→c x2→ g′, for the composition g t1→ g1 t2→ g′, where g is an object of 5 / 15 volume 10 (2008) composing rules on grids size (r, c), g1 of size (r1, c1) and g′ of size (r′, c′), uniquely exists only if t1 and t2 satisfy the constraints above, and has size ((r′−r1) + r, (c′−c1) + c), with x1 and x2 labeled as t1 and t2. figure 4: the pushout construction for grids we can now define structural dpo rules in tgridt in accordance to the construction above. in particular, in order to satisfy the the dangling condition, rules are non-deleting (i.e. l ← k is an identity). the gluing condition, if k → r is not an identity, requires border cells of l to be matched to border cells of the host grid g. the pushout complement object d is now always equal to g. hence, grids can be generated so that the constraints on their rectangular form are maintained through the pushout construction, without having to adopt regulatory mechanisms for rewriting, such as those needed for the so-called indian grammars: a set of horizontal rules is there first used to create the upper row, and then vertical rules are applied in parallel to populate the columns [sk74]. the pushout construction can now be lifted in order to consider also attributes. in particular, as typical of attributed graph rewriting, data morphisms are identities (no domain element can be created or deleted). hence, the effect of a rule can only be the addition of structural nodes and edges and the deletion and creation of attribute edges. application conditions can be used to describe the relations between values. all grids in agridt can now be generated by the iterated use of the two rules in figure 5 and figure 6, in which identifiers of adjacency nodes indicate their directions and an application condition defines the coordinates of the new cell. in both cases, we show the classical representation of dpo rules at the top of the figure, and use, at its bottom, a compact notation, already exploited in [dgb07]: the difference between k, l, and r is shown by highlighting the deleted and produced parts with different colours and marking them with tags {del} and {new}. the elements outside the tagged areas are those belonging to the k component. note that, differently from [eept06], we explicitly show k as presenting isolated data nodes, which are connected to different object nodes in l and r. proc. gt-vmt 2008 6 / 15 eceasst figure 5: the rule for letting a grid grow horizontally. 5 control flow rules figure 7 presents the metamodel triples describing the correspondences induced by assigning to the adjacency relation in the visual representation the semantic meaning of a carrier, along which either data or modifications in the activation state can travel from and to active elements. the left upper part of figure 7 constitutes the static semantics for the control flow variety on spatial structures, while the right upper part models the data variety, here simplified by considering a simple integer-valued attribute, called level. by applying the construction in [dgb07] one can incrementally define the flow structure through triple graph rules which introduce carriers in correspondence with the installation of adjacency relations in specified directions. hence, one can model the permeability of the cell wall to control or data flow. as an example, flows could travel rightwards and leftwards, but not downwards and upwards. a control flow (cf ) rule is a dpo rule in agraphatg with graphs conforming to the left upper part of figure 7, so that a contains the attribute state with values in some finite domain activationstate ∈ ∆. in general, as shown in the rule (in compact form) on the left of figure 83, the activation state may vary during transportation, e.g. a flow can decrease its intensity. the element reached by the flow could have possessed some other activation value and the one from which the flow originated may gain a new one, as defined by application conditions. the basic rule on the right of figure 8 deals with the case of cells entering an active state as the control flow reaches them traveling the grid rightwards from the origin to the destination, while the origin enters a quiescent state. 3 abbreviations are used for names of values and types. 7 / 15 volume 10 (2008) composing rules on grids figure 6: the rule for letting a grid grow vertically. the carrier identifier indicates the value of its direction attribute. similar rules are defined for other directions, exploiting rotational and mirror symmetries. cf -rules are composed to form more complex ones using a componentwise pushout construction in agraphatg, as shown in figure 9, where l ← k → r is the maximal intersection of l1 ← k1 → r1 and l2 ← k2 → r2, all the squares commute and those with curved arrows are pushouts. as an example, directional rules compose through a rule on a single cell passing from the active to the quiescent state. figure 10 illustrates the case of the two horizontal movements, while figure 11 that of one directional and one vertical movement. figure 7: the metamodel triples for control and data flows on grids. proc. gt-vmt 2008 8 / 15 eceasst figure 8: a generic rule for transmission of control flows and a basic rule. l �� 44k loo r // �� ** r �� 44l1 �� k1 l1oo r1 // �� r1 �� l2 44k2 l2oo r2 // **r2 44l′ k′ loo r // r′ figure 9: the construction for rule composition. 6 composing control flow and computation formulas we now introduce data-rules to specify the transformation of some attribute according to some formula. these are defined on the type graph in the right upper part of figure 7. data and cf rules are composed, again with a pushout construction, to produce rules which both apply the formula and propagate the flow, when an active element is reached by the control flow. the rule in figure 12 doubles the value of level. x and y are variables to indicate generic instances of an integer. the rules involved in their combination operate on four different types of graphs. the intersection is defined in a type graph where activeelement abstracts on controlactiveelement and dataactiveelement and has no attribute, while the pushout object complies with a type graph formed by taking the quotient of the disjoint union of the two type systems from figure 7 and identifying the activity types in a fullactiveelement type figure 10: the construction of the rule propagating control flow horizontally in both verses. 9 / 15 volume 10 (2008) composing rules on grids figure 11: the construction of a bidirectional rule. (abbreviated in factv). node morphisms go from less to more specific types. figure 12: a rule expressing a computational formula. in figure 13, the bidirectional rule of figure 11 is composed with the formula of figure 12, so that the latter is now evaluated only when the activation front leaves an element in both directions. using different mappings from the intersection to the cf -rule, the formula would be evaluated when the control flow reaches an element from a specific direction. the resulting rule does not specify the level values for the other elements. rules can be applied sequentially or, if they do not conflict on their result, combined to form amalgamated rules to achieve an effect of parallelism [tb94]. as an example, the rule of figure 13 agrees with itself on any node whose upper and left neighbours are both mapped, by two distinct matches, to the cell identified by 1. rules can be enriched with parameters and applied via rule expressions to realize complex computations [bkpt00]. 6.1 types of activation in cyberfilm the cyberfilm language [ym02, wmym08] provides a collection of predefined control flows, associated with program templates defining the loops realizing them, and with sequences of iconic schemes for an intuitive visualization of the main steps in the execution flow. cyberfilm allows the separated definition of computational formulae and control flow specifications. hence, the constructions above can be exploited to provide a compositional mechanism for it. in particular, in the cyberfilm framework, control flow is defined by the transformation of the proc. gt-vmt 2008 10 / 15 eceasst figure 13: the resulting rule specifying the condition of application. flashing state of a node different types of flashing are defined: for example, full flashing indicates that the node is able to perform reading and writing operations; contour flashing indicates that the node is referenced by other flashing nodes which can perform reading operations, but not change its value; half flashing indicates the activity state of an observer which can change the state of other nodes in a global fashion. other types of flashing are defined, but in this paper we restrict ourselves to the flow of the full and contour flashing, thus interpreting full flashing as an indication that the formula associated with the node can be evaluated to assign a new value to the node, and contour flashing, as the fact that the value of the node is available for formula evaluation by other nodes. at any time, a cell is in only one possible state. the control flows of the full and contour flashing can be independent or coordinated. independent flows can be specified as described in section 5, whereas coordinated flows require the identification of the conditions under which a cell is able to receive the contributions of other cells. figure 14 shows the composition of a coordinated cf -rule, for rightward transmission of both full and contour flows, with a formula rule where an element reads the value of its down neighbour (without changing it), to compute its new level. the upward adjacency relation is mapped, in the formula rule, to a data carrier (dc) element, as control and data may flow in different directions, i.e. cells can have different permeability to data and control flows. 11 / 15 volume 10 (2008) composing rules on grids figure 14: the rule resulting from the coordination of movements of full and contour flashing. in this case, the cf -rule is bound to consider both the full and contour flows simultaneously. the same effect could be achieved by considering the two flows independently. figure 15 shows the first step of the relative construction, in which the rightwards movement for the full flashing is combined with the same formula to produce a rule which does not affect the state of the controlactiveelement identified by 2. in figure 16, the obtained rule is composed with that for rightward movement of contour flashing. while the final effect is the same, the intermediate step could be combined with other movement rules. several such rules might be defined, for example to propagate the flow across several cells, so that only some elements are activated. 7 conclusions we have proposed an approach, based on componentwise pushout of dpo rules in the category of attributed typed graphs, to the specification of computations on grids. the approach allows proc. gt-vmt 2008 12 / 15 eceasst figure 15: coordinating movement of the full flashing flow with the formula in figure 14. the independent definition of two types of rules, one to specify control flow and the other to specify the actual computations. the construction is symmetrical in control and formula rules, so that it can be flexibly applied starting from either specification. symmetries between adjacency relations can also be exploited to generate different versions of flows and formulas. future work will explore other types of spatial structures, typically trees and pyramids, to define adequate cf -rules, also considering the distinction between formula evaluation on control flows reaching or leaving the involved cells, and develop ways of reasoning about the compatibility of independent cf -rules (e.g. one for reading and one for writing). bibliography [afgk02] l. f. andrade, j. l. fiadeiro, j. gouveia, g. koutsoukos. separating computation, coordination and configuration. j. of software maintenance 14(5):353–369, 2002. [bg04] p. bottoni, a. grau. a suite of metamodels as a basis for a classification of visual languages. in proc. vl/hcc 2004. pp. 83–90. 2004. [bkpt00] p. bottoni, m. koch, f. parisi presicce, g. taentzer. automatic consistency checking and visualization of ocl constraints. in proc. uml 2000. pp. 294–308. 2000. [bl07] p. bottoni, a. labella. pointed pictures. journal of visual languages and computing 18:523–536, 2007. 13 / 15 volume 10 (2008) composing rules on grids figure 16: coordinating movement of the contour flashing flow with the rule of figure 15. [blg07] p. bottoni, j. de lara, e. guerra. action patterns for incremental specification of execution semantics of visual languages. in proc. vl/hcc 2007. pp. 163–170. 2007. [cmr96] a. corradini, u. montanari, f. rossi. graph processes. fundamenta informaticae 26(34):241–265, 1996. [dbe+07] j. de lara, r. bardohl, h. ehrig, k. ehrig, u. prange, g. taentzer. attributed graph transformation with node type inheritance. tcs 376:139–163, 2007. [dgb07] j. de lara, e. guerra, p. bottoni. triple patterns: compact specifications for the generation of operational triple graph grammar rules. in proc. gt-vmt’07. pp. 81– 95. 2007. [eept06] h. ehrig, k. ehrig, u. prange, g. taentzer. fundamentals of algebraic graph transformation. springer, 2006. [gemt00] m. goedicke, b. enders, t. meyer, g. taentzer. towards integration of multiple perspectives by distributed graph transformation. in nagl et al. (eds.), proc. agtive 1999. pp. 369–377. 2000. [gr97] d. giammarresi, a. restivo. two-dimensional languages. in handbook of formal languages. volume iii, pp. 215–267. springer, 1997. proc. gt-vmt 2008 14 / 15 eceasst [hkt02] r. heckel, j. küster, g. taentzer. confluence of typed attributed graph transformation with constraints. in proc. icgt 2002. lncs 2505, pp. 161–176. 2002. [hp95] a. habel, d. plump. unification, rewriting, and narrowing on term graphs. electr. notes theor. comput. sci. 2, 1995. [ips82] a. itai, c. h. papadimitriou, j. l. szwarcfiter. hamilton paths in grid graphs. siam j. comput. 11(4):676–686, 1982. [kk99] h. kreowski, s. kuske. graph transformation units with interleaving semantics. formal aspects of computing 11:690–723, 1999. [ls04] s. lack, p. sobocinski. adhesive categories. in ehrig et al. (eds.), proc. fossacs 2004. pp. 273–288. springer, 2004. [mos94] p. mosses. recent trends in data type specification. chapter unified algebras and abstract syntax, pp. 280–294. springer, 1994. [mw93] m. k. m löwe, a. wagner. term graph rewriting: theory and practice. chapter an algebraic framework for the transformation of attributed graphs, pp. 185–199. john wiley and sons ltd, 1993. [par94] f. parisi presicce. transformations of graph grammars. in tagt. lncs 1073, pp. 428–442. 1994. [sk74] r. siromoney, k. krithivasan. parallel context-free grammars. information and control 24:155–162, 1974. [swz99] a. schürr, a. winter, a. zündorf. the progres-approach: language and environment. in ehrig et al. (eds.), handbook of graph grammars and computing by graph transformation, vol. 2. pp. 487–550. world scientific, 1999. [tb94] g. taentzer, m. beyer. amalgamated graph transformations and their use for specifying agg. in dagstuhl seminar on graph transformations in computer science. lncs 776, pp. 380–394. springer, 1994. [wmym08] y. watanobe, n. n. mirenkov, r. yoshioka, o. monakhov. filmification of methods: a visual language for graph algorithms. journal of visual languages and computing 19(1):123–150, 2008. [ym02] r. yoshioka, n. n. mirenkov. visual computing within environment of selfexplanatory components. soft computing 7(1):20–32, 2002. 15 / 15 volume 10 (2008) introduction related work background: metamodels and graph transformations categories on grids control flow rules composing control flow and computation formulas types of activation in cyberfilm conclusions verifying model transformations by structural correspondence electronic communications of the easst volume 10 (2008) proceedings of the seventh international workshop on graph transformation and visual modeling techniques (gt-vmt 2008) verifying model transformations by structural correspondence anantha narayanan and gabor karsai 14 pages guest editors: claudia ermel, reiko heckel, juan de lara managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst verifying model transformations by structural correspondence anantha narayanan1 and gabor karsai1 1institute for software integrated systems, vanderbilt university nashville, tn 37203 usa abstract: model transformations play a significant role in model based software development, and the correctness of the transformation is crucial to the success of the development effort. we have previously shown how we can use bisimulation to verify the preservation of certain behavioral properties across a transformation. however, transformations are often used to construct structurally different models, and we might wish to ensure that there is some structural correspondence to the original model. it may be possible to verify such transformations without having to explicitly specify the dynamic semantics of the source and target languages. in this paper, we present a technique to verify such transformations, by first specifying certain structural correspondence rules between the source and target languages, and extending the transformation so that these rules can be easily evaluated on the instance models. this will allow us to conclude if the output model has the expected structure. the verification is performed at the instance level, meaning that each execution of the transformation is verified. we will also look at some examples using this technique. keywords: verification, model transformations 1 introduction model transformations that translate a source model into an output model are often expressed in the form of rewriting rules, and can be classified according to a number of categories [mv05]. however, the correctness of a model transformation depends on several factors, such as whether the transformation terminates, whether the output model complies with the syntactical rules of the output language, and others. one question crucial to the correctness of a transformation is whether it achieved the intended result of mapping the semantics of the input model into that of the output model. for instance, a transformation from a statechart model to a non-hierarchical fsm model can be said to be correct if the output model truly reproduces the behavior of the original statechart model. models can also be seen as attributed and typed graph structures that conform to an abstract syntax. model transformations take an input graph and produce a modified output graph. in a majority of these cases, the transformation matches certain graph structures in the input graph and creates certain structures in the output graph. in such cases, correctness may be defined as whether the expected graph structures were produced in the output corresponding to the relevant structures in the input graph. if we could specify the requirements of such correspondences and trace the correspondences easily over instance models, a simple model checking process at the 1 / 14 volume 10 (2008) verifying model transformations end of a transformation can be used to verify if those instances were correctly transformed. in this paper, we explore a technique to specify structural correspondence rules, which can be used to decide if the transformation resulted in an output model with the expected structure. this will be specified along with the transformation, and evaluated for each execution of the transformation, to check whether the output model of that execution satisfies the correspondence rules. 2 background 2.1 great great [akl03] is a language framework for specifying and executing model transformations using graph transformation. it is a meta-model based transformation tool implemented within the framework of gme [lbm+01]. one of the key features of great is the ability to define cross-language elements by composing the source and target meta-models, and introducing new vertex and edge types that can be temporarily used during the transformation. such cross-metamodel associations are called cross-links. note that a similar idea is present in triple graph grammars [sch95]. this ability of great allows us to track relationships between elements of the source and target models during the course of the transformation, as the target model is being constructed from the source model. this feature plays a crucial role in our technique to provide assurances about the correctness of a model transformation. 2.2 instance based verification of model transformations verifying the correctness of model transformations in general is as difficult as verifying compilers for high-level languages. but for practical purposes, a transformation may be said to have ‘executed correctly’ if a certain instance of its execution produced an output model that preserved certain properties of interest. we call that instance ‘certified correct’. this idea is similar to the work of denney and fischer in [df05], where a program generator is extended to produce logical annotations necessary for formal verification of certain safety properties. an automated theorem prover uses these annotations to find proofs for the safety properties for the generated code. note that this does not prove the safety of the code generator, but only of a particular instance of generated code. in our previous effort [nk06], we have shown that it is both practical and prudent to verify the correctness of every execution of a model transformation, as opposed to finding a correctness proof for the transformation specification. this makes the verification tractable, and can also find errors introduced during the implementation of a transformation that may have been specified correctly. this technique was applied to the specific case of preservation of reachability related properties across a model transformation. reachability is a fairly well-understood property, and can be verified easily for a labeled transition system (lts), for instance by model checking [hol97]. if two labeled transition systems are bisimilar, then they will have the same reachability behavior. in our approach, we treated the source and target models as labeled transition systems, and verified the transformation by checking if the source and target instances were bisimilar. proc. gt-vmt 2008 2 / 14 eceasst figure 1: architecture for verifying reachability preservation in a transformation given an lts (s, λ, →), a relation r ⊆ s × s is a bisimulation [san04] if: (p, q) ∈ r and p α→ p′ implies that there exists a q′ ∈ s, such that q α→ q ′ and (p′, q′) ∈ r, and conversely, q α→ q ′ implies that there exists a p′ ∈ s, such that p α→ p ′ and (p′, q′) ∈ r. we used cross-links to relate source and target elements during the construction of the output model. these relations were then passed to a bisimilarity checker, which determined whether the source and target instances were bisimilar. if the instances were determined to be bisimilar, we could conclude that the execution of the transformation was correct. figure 1 shows an overview of the architecture for this approach. figure 2: meta-model for uml activity diagrams 2.3 uml to csp transformation the uml to csp transformation was presented as a case study at agtive ’07 [beh07], to compare the various graph transformation tools available today. we provide an overview of this 3 / 14 volume 10 (2008) verifying model transformations case study here from a great point of view, and we will use this as an example to explain our technique for verifying model transformations. the objective of this transformation is to take a uml activity diagram [omg06] and generate a communicating sequential process [hoa78] model with the equivalent behavior. the activity diagram consists of various types of activity nodes, which can be connected by directed activity edges. figure 2 shows the gme meta-model for uml activity diagrams. a csp process is defined by a process assignment, which assigns a process expression to a process id. process expressions can be a simple process, a prefix operator or a binaryoperator. figure 3 shows the gme meta-model for csp, highlighting the relevant parts for our example. figure 3: meta-model for csp the uml to csp mapping assigns a process for each activity edge. for each type of activity node, a process assignment is created, which assigns the process corresponding to the incoming activity edge to a process expression depending on the type of the activity node. figure 4 shows one such mapping, for the type action node. this assigns the incoming process to a prefix expression. the resulting csp expression can be written as a = action −→ b, which is shown in figure 4 as a model instance compliant with the csp meta-model. 3 structural correspondence as in the uml to csp case, model transformations can be used to generate a target model of a certain structure (csp) from a source model of a different structure (uml). specific structural configurations in the source model (such as an action node in the uml model) produce specific structural configurations in the target model (such as a prefix in the csp model). the rules to accomplish the structural transformations may be simple or complicated. however, it is fairly straightforward to compare and verify that the correct structural transformation was made, if we already know which parts of the source structure map to which parts of the target structure. in our technique to verify a transformation by structural correspondence, we will first define proc. gt-vmt 2008 4 / 14 eceasst (a) activity diagram (b) csp figure 4: csp process assignment for action node a set of structural correspondence rules specific to a certain transformation. we will then use cross-links to trace source elements with the corresponding target elements, and finally use these cross-links to check whether the structural correspondence rules hold. in essence, we expect that the correspondence conditions are independently specified for a model transformation, and an independent tool checks if these conditions are satisfied by the instance models, after the model transformation has been executed. in other words, the correspondence conditions depend purely on the source and target model structures and not on the rewriting rules necessary to effect the transformation. since the correspondence conditions are specified in terms of simple queries on the model around previously chosen context nodes, we expect that they will be easier to specify, and thus more reliable than the transformation itself. we also assume that the model transformation builds up a structure for bookkeeping the mapping between the source and target models. 3.1 structural correspondence rules for uml to csp transformation a structural correspondence rule is similar to a precondition-postcondition style axiom. we will construct them in such a way that they can be evaluated easily on model instances. we will use the uml to csp example to illustrate structural correspondence. consider the case for the action node, as shown in figure 4. the action node has one incoming edge and one outgoing edge. it is transformed into a process assignment in the csp. the csp structure for this instance consists of a process id and a prefix, with an event and a target process. this is the structural transformation for each occurrence of an action node in the activity diagram. we can say that for each action node in the activity diagram, there is a corresponding process assignment in the csp, with a prefix expression. when our transformation creates this corresponding process assignment, we can use a cross-link to track this correspondence. the structural correspondence is still not complete, as we have to ensure that the process id and the prefix expression are created correctly. we use a kind of path expression to specify the correctness of corresponding parts of the two structures, and the correspondence is expressed in the form sourceelement = out putelement. let us denote the action node by an, and the process assignment by pa. then the necessary correspondence rules can be written using path expressions as shown in table 1. 5 / 14 volume 10 (2008) verifying model transformations rule path expression the action node corresponds to a process assignment with a prefix pa.procex pr.type = pre f ix the incoming edge in the uml corresponds to the process id an.inedge.name = pa.procid.name the outgoing edge corresponds to the target process an.outedge.name = pa.procex pr.process.name the action of the action node corresponds to the event an.action = pa.procex pr.event table 1: structural correspondence rules for action node these rules together specify the complete structural correspondence for a section of the activity diagram and a section of its equivalent csp model. the different types of activity nodes result in different structures in the csp model, some of which are more complex than the fairly straightforward case for the action node. next, we look at the structural mapping for some of the other nodes. a fork node in the activity diagram is transformed into a process assignment with a concurrency expression. figure 5 shows a fork node with an incoming edge a and three outgoing edges b, c and d. this is represented by the csp expression a = b ‖ (c ‖ d), where ‖ represents concurrency (the actual ordering of b, c and d is immaterial). the structural representation of this expression as an instance of the csp meta-model is shown in figure 5. the fork node is transformed into a csp process assignment that consists of a process id corresponding to the incoming activity edge of the fork node, and a process expression of type concurrency. the concurrency expression consists of processes and other concurrency nodes, depending on the number of outgoing activity edges. if we denote the fork node by f n and the process assignment by pa, the structural corre(a) activity diagram (b) csp figure 5: csp process assignment for fork node proc. gt-vmt 2008 6 / 14 eceasst rule path expression the fork node corresponds to a process assignment with a concurrency pa.procex pr.type = concurrency the incoming edge in the uml corresponds to the process id f n.inedge.name = pa.procid.name for each outgoing edge, ∀o ∈ f n.outedge there is a corresponding process ∃p ∈ pa.procex pr..process : o.name = p.name in the process expression table 2: structural correspondence rules for fork node (a) activity diagram (b) csp figure 6: csp process assignment for decision node spondence rules can be described using path expressions as shown in table 2. we will use the double-dot ‘..’ to denote the descendant operator (similar to ‘//’ in xpath queries), to specify ancestor-descendant relationships. by evaluating these rules on the activity diagram and the csp models, we can determine whether the structural correspondence was satisfied for fork nodes. another type of node in the activity diagram is the decision node. the transformation mapping for the decision node is a slight variation of the fork node. figure 6 shows a decision node with an incoming edge a, and three outgoing edges b, c and d, with guards x, y and else respectively (in this case study, we will assume that the decision node always has exactly one ‘else’ edge). this is represented by the csp expression a = b 6< x 6> (c 6< y 6> d), where the operator c 6< y 6> d is the condition operator with the meaning that if y is true, then the process behaves like c, else like d. the decision node is transformed to the csp model shown in figure 6 as a model instance of the csp meta-model. the decision node is transformed into a process assignment that consists of a process id corresponding to the incoming activity edge of the decision node, and a process expression of type condition. the condition’s expression attribute is set to the guard of the corresponding activity edge, and a process corresponding to the activity edge is created as it’s lhs. for the final ‘else’ edge, a process is created in the last condition as it’s rhs. the 7 / 14 volume 10 (2008) verifying model transformations rule path expression the decision node corresponds to a process assignment with a condition pa.procex pr.type = condition the incoming edge in the uml corresponds to the process id dn.inedge.name = pa.procid.name for each outgoing edge, there is a corresponding condition in the ∀o ∈ dn.outedge ∧ o.guard 6= else process expression and a corresponding ∃c ∈ pa.procex pr..condition : process in the condition’s lhs c.ex pression = o.guard ∧ c.lhs.name = o.name for the outgoing ‘else’ edge, there is a condition in the ∀o ∈ dn.outedge ∧ o.guard = else process expression with a ∃c ∈ pa.procex pr..condition : corresponding process as it’s rhs c.rhs.name = o.name table 3: structural correspondence rules for decision node structural correspondence rules for this mapping are shown in table 3. 3.2 specifying structural correspondence rules in great specifying the structural correspondence for a transformation consists of two parts: 1. identifying the significant source and target elements of the transformation 2. specifying the structural correspondence rules for each element using path expressions the first step is accomplished in great by using cross-links between the source and target elements. a composite meta-model is added to the transformation, by associating selected source and target elements using a temporary ‘structural correspondence’ class. there will be one such class for each pair of elements. this class will have a string attribute, which is set to the path expressions necessary for structural correspondence for that pair. figure 7 shows a composite meta-model specifying the structural correspondence for fork nodes. the fork node class comes from the activity diagram meta-model, and the process assignment class comes from the csp meta-model. figure 7: composite meta-model to specify structural correspondence once the structural correspondence has been specified for all the relevant items, the transformation is enhanced to create the cross-link when creating the respective target elements. figure 8 shows the great rule in which the process assignment for a fork node is created, enhanced to proc. gt-vmt 2008 8 / 14 eceasst figure 8: great rule with cross-link for structural correspondence create the cross-link for structural correspondence. note that in incoming activity edge is represented as an association class named activityedgein. the transformation for the fork node is actually accomplished in a sequence of several rules executed recursively, as shown in figure 9. first all the fork nodes are collected, and a sequence of rules are executed for each node. these rules iterate through the out-edges of each fork node, creating the concurrency tree. though several rules are involved in the transformation for fork nodes, the cross-link needs to be added to one rule only. it must be noted that it is necessary to specify the structural correspondence rules only once in the composite meta-model. the cross-link must however be added to the transformation rules, and in most cases will be required only once for each pair of source and target element. 3.3 evaluating the structural correspondence rules once the structural correspondence rules have been specified, and the cross-links added to the transformation, the correspondence rules are evaluated on the instance models after each execution of the transformation. these rules can be evaluated by performing a simple depth first search on the instance models, and checking if the correspondence rules are satisfied at each relevant stage. this consists of two phases. the first phase is to generate the code that will traverse the instance models and evaluate the correspondence rules. since the meta-models of both the source and target languages are available with the transformations, and the path expressions are written in a standard form that can be parsed automatically, the model traverser code can be automatically generated from the structural correspondence specification. this needs to be done only once each 9 / 14 volume 10 (2008) verifying model transformations figure 9: sequence of great rules for fork node transformation time the structural correspondence specification changes. the second phase is to call the model traverser code at the end of each execution of the transformation, supplying to it the source and target model instances along with the cross-links. in the case of the uml to csp transformation, we traverse the input activity diagram model and evaluate the correspondence rules at each activity node. for each activity node, the crosslink is traversed to find the corresponding process assignment. if a correspondence rule has been defined for an activity node, and no corresponding process assignment is found, then this signals an error in the transformation. after locating the corresponding process assignment, the path expressions are evaluated. if any of the rules are not satisfied, the error is reported. if all the rules are satisfied for all the nodes, then we can conclude that the transformation has executed correctly. the instance model is traversed in a depth-first manner. the corresponding elements are located using the cross-links, which will take constant time. the path expressions are evaluated on the instances, which will take polynomial time in most cases. thus, the overall verification process does not incur a significant performance overhead in most cases. 3.4 remarks the structural correspondence based verification described here can provide important assurances about the correctness of transformations, while being practically applicable in most common transformations. the use of path expressions to specify correspondence rules makes it easy to specify correctness. the path expressions use a simple query language that can be easily evaluated on the instance models. our future research concentrates on the requirements of such a query language. most complex transformations may involve multiple rules executing recursively to transform a particular part of a model. however, it may be possible to specify the correspondence for that part of the model using a set of simple path expressions. such a specification would be simpler and easier to understand than the complex transformation rules. the structural correspondence is also specified orthogonal to the transformation specification. proc. gt-vmt 2008 10 / 14 eceasst thus, these rules may be written by someone other than the original transformation writer, or even supplied along with the requirements document. 4 related work [k0̈4], [khe03] present ideas on validating model transformations. in [khe03], the authors present a concept of rule-based model transformations with control conditions, and provide a set of criteria to ensure termination and confluence. in [k0̈4], küster focuses on the syntactic correctness of rule-based model transformations. this validates whether the source and target parts of the transformation rule are syntactically correct with respect to the abstract syntax of the source and target languages. these approaches are concerned with the functional behavior and syntactic correctness of the model transformation. [lt] also discusses validation of transformations on these lines, but also introduces ideas of syntactic consistency and behavior preservation. our technique addresses semantic correctness of model transformations, addressing errors introduced due to loss or mis-representation of information during a transformation. in [bh07], bisztray and heckel present a rule-level verification approach to verify the semantic properties of business process transformations. csp is used to capture the behavior of the processes before and after the transformation. the goal is to ensure that every application of a transformation rule has a known semantic effect. we use path expressions to capture the relation between structures before and after a transformation. these path expressions are generic (they do not make any assumptions about the underlying semantics of the models involved), and can be applied to a wide variety of transformations. ehrig et. al. [eee+07] study bidirectional transformations as a technique for preserving information across model transformations. they use triple graph grammars to define bi-directional model transformations, which can be inverted without specifying a new transformation. our approach offers a more relaxed framework, which will allow some loss of information (such as by abstraction), and concentrates on the crucial properties of interest. we also feel that our approach is better suited for transformations involving multiple models and attribute manipulations. in other related work, [vp03] presents a model level technique to verify transformations by model checking a selected semantic property on the source model, and transforming the property and validating it in the target domain. the validation requires human expertise. after transforming the property, the target model is model checked. in our approach, since the properties are specified using cross links that span over both the source and target languages, we do not need to transform them. [llmc06] discuss an approach to validate model transformations by applying ocl constraints to preserve and guarantee certain model properties. [ggl+06] is a language level verification approach which addresses the problem of verifying semantic equivalence between a model and a resulting programming language code. 4.1 mof qvt relations language the mof 2.0 query / view / transformation specification [omg05] addresses technology pertaining to manipulation of mof models. a relations language is prescribed for specifying relations that must hold between mof models, which can be used to effect model transformations. 11 / 14 volume 10 (2008) verifying model transformations relations may be specified over two or more domains, with a pair of when and where predicates. the when predicate specifies the conditions under which a relation must hold, and the where predicate specifies the condition that all the participating model elements must satisfy. additionally, relations can be marked as checkonly or enforced. if it is marked checkonly, the relation is only checked to see if there exists a valid match that satisfies the relationship. if it is marked enforced, the target model is modified to satisfy the relationship whenever the check fails. our approach can be likened to the checkonly mode of operation described above. however, in our case, the corresponding instances in the models are already matched using cross links, and the correspondence conditions are evaluated using their context. the cross links help us to avoid searching the instances for valid matches. specifying the correspondence conditions using context nodes simplifies the model checking necessary to evaluate the conditions, thus simplifying the verification process. since we verify the correspondence conditions for each instance generated by the transformation, these features play an important role. 4.2 triple graph grammars triple graph grammars [sch95] are used to describe model transformations as the evolution of graphs by applying graph rules. the evolving graph complies with a graph schema that consists of three parts. one graph schema represents the source meta model, and one represents the target meta-model. the third schema is used to track correspondences between the source and target meta models. transformations are specified declaratively using triple graph grammar rules, from which operational rules are derived to effect model transformations. the schema to track correspondences between the source and target graphs provides a framework to implement a feature similar to cross links in great. if the correspondence rules can be encoded into this schema, and the correspondence links persisted in the instance models, our verification approach can be implemented in this scenario. 5 conclusions and future work in this paper, we have shown how we can provide an assurance about the correctness of a transformation by using structural correspondence. the main errors that are addressed by this type of verification is the loss or misrepresentation of information during a model transformation. we continue to hold to the idea that it is often more practical and useful to verify transformations on an instance basis. the verification framework must be added to the transformation only once, and is invoked for each execution of the transformation. the verification process does not add a significant overhead to the transformation, but provides valuable results about the correctness of each execution. the path expressions must use a simple and powerful query language to formulate queries on the instance models. while existing languages such as ocl may be suitable for simple queries, we may need additional features, such as querying children to an arbitrary depth. our future research concentrates on the requirements of such a language. while the path expressions can be parsed automatically and evaluated on the instances, the cross-link for the relevant elements must be manually inserted into the appropriate transformation proc. gt-vmt 2008 12 / 14 eceasst rules. however, in most cases, it may be possible to infer where the cross-links must be placed. if the cross-links could be inserted into the rules automatically, the transformation can remain a black box. the main concern with this is that the cross-links are crucial to evaluating the correspondence rules correctly and also to keep the complexity down. we have seen simple string comparisons added to the path expressions in this paper. some transformations may require more complex attribute comparisons, or structure to attribute comparisons such as counting. we wish to explore such situations in further detail in future cases, to come up with a comprehensive language for specifying the path expressions. acknowledgement the research described in this paper has been supported by a grant from nsf/csr-ehs, titled “software composition for embedded systems using graph transformations”, award number cns-0509098. bibliography [akl03] a. agrawal, g. karsai, a. ledeczi. an end-to-end domain-driven software development framework. in oopsla ’03: 18th annual acm sigplan conference on oop, systems, languages, and applications. pp. 8–15. acm press, new york, ny, usa, 2003. doi:http://doi.acm.org/10.1145/949344.949347 [beh07] d. bisztray, k. ehrig, r. heckel. case study: uml to csp transformation. in applications of graph transformation with industrial relevance (agtive). 2007. [bh07] d. bisztray, r. heckel. rule-level verification of business process transformations using csp. in proceedings of the 6th international workshop on graph transformation and visual modeling techniques (gt-vmt’07). 2007. [lt] de lara, j. and taentzer, g. automated model transformation and its validation with atom3 and agg. in lecture notes in artificial intelligence, 2980. pp. 182– 198. springer. [df05] e. denney, b. fischer. certifiable program generation. in glück and lowry (eds.), gpce. lecture notes in computer science 3676, pp. 17–28. springer, 2005. [eee+07] h. ehrig, k. ehrig, c. ermel, f. hermann, g. taentzer. information preserving bidirectional model transformations. in fundamental approaches to software engineering. pp. 72–86. 2007. doi:http://dx.doi.org/10.1007/978-3-540-71289-3 7 [ggl+06] h. giese, s. glesner, j. leitner, w. schfer, r. wagner. towards verified model transformations. october 2006. [hoa78] c. a. r. hoare. communicating sequential processes. commun. acm 21(8):666– 677, 1978. 13 / 14 volume 10 (2008) http://dx.doi.org/http://doi.acm.org/10.1145/949344.949347 http://dx.doi.org/http://dx.doi.org/10.1007/978-3-540-71289-3_7 verifying model transformations [hol97] g. j. holzmann. the model checker spin. ieee trans. softw. eng. 23(5):279–295, 1997. doi:http://dx.doi.org/10.1109/32.588521 [k0̈4] j. m. küster. systematic validation of model transformations. in proceedings 3rd uml workshop in software model engineering (wisme 2004). october 2004. [khe03] j. m. küster, r. heckel, g. engels. defining and validating transformations of uml models. in hcc ’03: proceedings of the 2003 ieee symposium on human centric computing languages and environments. pp. 145–152. ieee computer society, washington, dc, usa, 2003. [lbm+01] a. ledeczi, a. bakay, m. maroti, p. volgyesi, g. nordstrom, j. sprinkle, g. karsai. composing domain-specific design environments. computer 34(11):44–51, 2001. doi:http://dx.doi.org/10.1109/2.963443 [llmc06] l. lengyel, t. levendovszky, g. mezei, h. charaf. model-based development with strictly controlled model transformation. in the 2nd international workshop on model-driven enterprise information systems, mdeis 2006. pp. 39–48. paphos, cyprus, may 2006. [mv05] t. mens, p. van gorp. a taxonomy of model transformation. in proc. int’l workshop on graph and model transformation. 2005. [nk06] a. narayanan, g. karsai. towards verifying model transformations. in bruni and varro (eds.), graph transformation and visual modeling techniques gt-vmt 2006. electronic notes in theoretical computer science, pp. 185–194. 2006. [omg05] omg. meta object facility (mof) 2.0 query/view/transformation specification. 2005. http://www.omg.org/cgi-bin/doc?ptc/2005-11-01. [omg06] omg. unifed modeling language, version 2.1.1. 2006. ”http://www.omg.org/technology/documents/formal/uml.htm” [san04] d. sangiorgi. bisimulation: from the origins to today. in lics ’04: proceedings of the 19th annual ieee symposium on logic in computer science (lics’04). pp. 298–302. ieee computer society, washington, dc, usa, 2004. doi:http://dx.doi.org/10.1109/lics.2004.13 [sch95] a. schürr. specification of graph translators with triple graph grammars. in wg ’94: proceedings of the 20th international workshop on graph-theoretic concepts in computer science. pp. 151–163. springer-verlag, london, uk, 1995. [vp03] d. varró, a. pataricza. automated formal verification of model transformations. in jürjens et al. (eds.), csduml 2003: critical systems development in uml; proceedings of the uml’03 workshop. technical report tum-i0323, pp. 63–78. technische universität münchen, september 2003. proc. gt-vmt 2008 14 / 14 http://dx.doi.org/http://dx.doi.org/10.1109/32.588521 http://dx.doi.org/http://dx.doi.org/10.1109/2.963443 http://www.omg.org/cgi-bin/doc?ptc/2005-11-01 "http://www.omg.org/technology/documents/formal/uml.htm" http://dx.doi.org/http://dx.doi.org/10.1109/lics.2004.13 introduction background great instance based verification of model transformations uml to csp transformation structural correspondence structural correspondence rules for uml to csp transformation specifying structural correspondence rules in great evaluating the structural correspondence rules remarks related work mof qvt relations language triple graph grammars conclusions and future work a framework for generating query language code from ocl invariants electronic communications of the easst volume 9 (2008) proceedings of the workshop ocl4all: modelling systems with ocl at models 2007 a framework for generating query language code from ocl invariants florian heidenreich, christian wende, and birgit demuth 10 pages guest editors: david h. akehurst, martin gogolla, steffen zschaler managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst a framework for generating query language code from ocl invariants florian heidenreich1, christian wende2, and birgit demuth3 1 florian.heidenreich@inf.tu-dresden.de 2 christian.wende@inf.tu-dresden.de 3 birgit.demuth@inf.tu-dresden.de http://st.inf.tu-dresden.de institut für softwareund multimediatechnik technische universität dresden, germany abstract: the semantical integrity of business data is of great importance for the implementation of business applications. model-driven software development (mdsd) allows for specifying the relevant domain concepts, their interrelations and their concise semantics using a plethora of modelling languages. since model transformations enable an automatic mapping of platform independent models (pims) to platform specific models (psms) and code, it is reasonable to utilise them to derive data schemas and integrity rules for business applications. most current approaches only focus on transforming structural descriptions of software systems while semantical specifications are neglected. however, to preserve also the semantical integrity rules we propose a query code generation framework that enables model-driven integrity engineering. this framework allows for mapping uml models to arbitrary data schemas and for mapping ocl invariants to sentences in corresponding declarative query languages, enforcing semantical data integrity on implementation level. this supersedes the manual translation of integrity constraints and, thus, decreases development costs while increasing software quality. keywords: ocl, code generation, query languages, model-driven integrity engineering 1 introduction the development of business applications involves a stepwise derivation of a software system starting from very abstract specification of the businesses domain concepts, their interrelations and their concise semantics. the idea of automating the transformation of more abstract representations to less abstract representations is at the heart of the mdsd [obj03a][bet04]. however most current approaches in mdsd only focus on transforming structural descriptions of software systems. since structural descriptions do not tackle all aspects of software systems, a plethora of other specification and modelling techniques exists. the ocl [obj03b] provides means to enrich models with detailed semantics in a formal way. unfortunately, ocl constraints are not preserved in current multi-staged transformation approaches and thereby are lost during pim to psm transformation. 1 / 10 volume 9 (2008) mailto:florian.heidenreich@inf.tu-dresden.de mailto:christian.wende@inf.tu-dresden.de mailto:birgit.demuth@inf.tu-dresden.de http://st.inf.tu-dresden.de generating query language code from ocl invariants the query code generation framework addresses this issue by providing a general framework for mapping ocl invariants to declarative query languages and thereby enables model-driven integrity engineering. we focus on query languages, because data in business systems is mostly managed by systems that are accessible through query languages (e.g. database systems). in section 2, we present the architecture of the query code generation framework that was developed within the dresden ocl2 toolkit [sof]. it consists of a model transformation framework which supports the generation of arbitrary data schemas from the uml [obj04] class model, and the ocl transformation framework realizing the mapping of ocl invariants to queries in the corresponding query languages. in section 3 we show the application of the query code generation framework by means of two examples, the mapping of uml/ocl to relational databases plus sql [ms93] and to xml schemas [fw04] plus xquery [bcf+07]. finally, section 4 and section 5 conclude this paper by discussing related work and summarizing the results of our work. 2 architecture of the query code generation framework our query code generation framework’s architecture is tripartite, where the first module is responsible for reading a uml/ocl model and building an abstract syntax model of it. this part is described in detail in [lo03, dhk05] and is not discussed in this paper. the second module performs the transformation of the uml model to the target data schema. we refer to this module as the model transformation framework. the third module maps ocl invariants to declarative query languages. we refer to this module as the ocl transformation framework. 2.1 model transformation framework models appear on several abstraction levels within the query code generation framework: uml models are used to describe domain concepts platform independently, cwm models, conforming to the common warehouse metamodel [obj01], describe data schemas, and socalled schema facade models provide a generic interface to these data schemas for the ocl transformation framework. to mediate between these levels of abstraction arbitrary model transformations are necessary. to address this issue we extended the existing ocl toolkit infrastructure with a java framework that includes a generic transformation engine to compose, configure and execute transformations in the context of a mdsd process. the models are stored in a central repository—the netbeans mdr—and can be accessed via java metadata interfaces (jmi) [sun02] which were automatically generated from the corresponding metamodels. thus, type-save model transformations can be implemented in java with no effort to learn a completely new transformation language. the frameworks’s architecture follows the strategy pattern [ghjv95]. a transformationengine provides input and configuration data for each transformation and steers its execution (see figure 1). the interface itransformation declares the common interface to be realised by all concrete transformation strategies. its abstract implementation—the class m2mtransformation—provides typical services that every transformation requires: loading source models from the repository, configuring transformation parameters, and preparing proc. ocl4all 2007 2 / 10 eceasst +invoke() +setconfigurationparameter() +getresult() +gettrace() «interface» itransformation +setconfigurationparameter() +getresult() +gettrace() m2mtransformation +invoketransformation() transformationengine 1 strategy * strategy context strategy concrete strategy +invoke() uml2ischemafacade +invoke() uml2dbschema figure 1: architecture of the transformation framework the target model. concrete transformations like the mapping of a uml class model to the ischemafacade and its ingredients (cf. subsection 2.2) or the object-relational mapping of uml-based domain models to a database schema can be implemented as subclasses (uml2ischemafacade, uml2dbschema) and use the provided transformation services. the design of the model transformation framework strongly addresses the specific needs of the dresden ocl toolkit with regard to repository access, transformation implementation and transformation configuration. however, in future work we will resolve these strict dependencies through the introduction of an adaptation layer which decouples the toolkit from repository and language specifics. this will provide new possibilities to integrate the upcoming transformation standard qvt [obj05] and existing transformation tools as atl [jk05]. 2.2 ocl transformation framework since we also want the semantic constraints to be preserved across the different abstraction levels, we provide the ocl transformation framework which transforms ocl invariants to equivalent sentences in declarative query languages. these expressions are used to ensure the integrity rules in the platform specific data schema. conceptually, we utilise a pattern-based approach to map ocl invariants to platform specific query languages. in our previous work [dh99] we have identified common patterns that occur when working with ocl invariants. examples of such patterns refer to the general structure of an ocl invariant, attribute access, or navigation across associations. for language-specific code generation we created templates for each of the identified patterns. although there are many different declarative query languages out in the wild, they all share 3 / 10 volume 9 (2008) generating query language code from ocl invariants <<interface>> ischemafacade <<interface>> ischemaelement guide figure 2: the ischemafacade is a generic interface to access target data schemas common concepts for querying data. they provide constructs for 1. projection, 2. cartesian product, and 3. restriction of data, that are similar to the concepts from relational algebra. it is true that in some languages (e.g. xcerpt [bs03]) these concepts are not that obvious than in other languages (e.g. sql [ms93]), where they exist as first-class constructs of the language. our code generator—the declarativecodegenerator—translates ocl invariants to equivalent expressions in the target query language. therefore it requires knowledge about both the target data schema and the target query language to realise e.g. class and attribute access or navigation expressions from the ocl invariants. the framework provides a generic interface for data-schema specific access to elements corresponding to the classes, attributes and associations in the source model—the ischemafacade that manages ischemaelements (see figure 2). to encapsulate the specifics of attribute access and navigation across associations in the target data model the code generator requires the developer to provide a language-specific realisation of the ischemafacade. during code generation the ischemafacade acts as lookup repository for elements that are referenced in the ocl invariants and provides information about how these elements can be accessed in the target data schema. since there is no generic format for the definition of data schemas, every ischemaelement offers a guide that gives hints to the code generator about the specific location of the element in the target data schema. the guide makes use of the common properties of query languages mentioned above. thereby it consists of a triple of attributes for projection, cartesian product, and restriction. this information is used to parameterise pattern templates—source code fragments containing holes—to build code fragments in the target query language. we follow a visitor-based approach [ghjv95] and use the template engine stringtemplate [par] for template expansion, which enforces a strict separation of generation logic and template definition by providing a template language for context-free templates [par04]. this interesting property of the template engine and the abstract notion of guides allows us to build a code generator that is independent of concrete query languages. 3 applications to show the applicability of our query code generation framework we present two examples where we apply the introduced concepts to uml-based domain models that are semantical enriched by ocl constraints. the uml models are mapped to platform-specific data schemas (using the model transformation framework) while the ocl invariants are transformed to equivaproc. ocl4all 2007 4 / 10 eceasst database schema database schema integrity views figure 3: mapping of uml/ocl-based domain models to relational databases lent representations in the corresponding query language (using the ocl transformation framework). 3.1 ocl2sql a mapping of uml/ocl-based domain models to databases (either relational or object-oriented) and additional integrity checks (formulated in sql or oql respectively) is motivated by the manifold of software applications employing databases as persistence mechanism. nowadays different techniques to ensure the integrity of application data are commonly used. checks to ensure data integrity are either realised directly in the user interface, or manually embedded in the application layer, or written by hand as a bunch of sql integrity checks at the persistence layer of the software system. all of these approaches share the drawback that it is not possible to automatically transfer integrity rules known and specified with ocl at design time into the system’s implementation. in the following we show how this issue can be tackled using the query code generation framework. we decided to realise integrity checking at the persistence layer to reduce its cohesion with the application layer. this alleviates the effort for client implementation and system maintenance—especially in the context of distributed systems sharing a common database. figure 3 depicts the steps taken by the query code generation framework to implement the mapping. the procedure is divided into two phases. first, schema propagation maps structural specifications of domain models to database relations. it consists of an object-relational mapping implemented as model transformation of uml class models to cwm (relational) [obj01] models. this transformation is highly configurable with regard to mapping strategies for inheritance structures, association mapping, or naming conventions in the database schema. however, the resulting models are still independent of a concrete database platform. to generate database-specific ddl (data definition language) code an additional code generation step is taken. since generation logic and code templates are strictly separated, arbitrary vendor-specific sql dialects can be supported with minimal effort. second, constraint propagation maps ocl constraints to integrity views which are used to ensure data integrity. it involves a model transformation which results in an ischemafacade used as interface for the ocl transformation framework to generate the sql integrity checks (cf. subsection 2.2). these integrity checks are realised with the view approach [dhl01] 5 / 10 volume 9 (2008) generating query language code from ocl invariants which generates sql views for all ocl invariants to determine data that violates semantical data integrity. database enforced constraints such as check constraints do not suit this issue, because they only refer on tuples of one table. indeed typical navigation expression in ocl invariants need to be mapped to a constraint including multiple tables respectively relational joins. the view approach supports this requirement and is well understood for relational databases and sql. database-specific trigger mechanisms can be used to integrate view-based integrity checks in the persistence layer, because they are part of the database schema. with the use of a template engine for code generation this works for several vendor-specific sql dialects too. i n v a r i a n t b o d y ( c o n s t r a i n t n a m e , c o n t e x t , c o n t e x t a l i a s , e x p r e s s i o n ) : : = << c r e a t e v i e w $ c o n s t r a i n t n a m e $ a s ( s e l e c t ∗ from $ c o n t e x t $ a s $ c o n t e x t a l i a s $ where n o t ( $ e x p r e s s i o n $ ) ) >> listing 1: sql template definition for ocl invariant l o g i c a l e x p r e s s i o n a n d ( e x p r e s s i o n 1 , e x p r e s s i o n 2 ) : : = << ( $ e x p r e s s i o n 1 $ and $ e x p r e s s i o n 2 $ ) >> listing 2: sql template definition for logical and listing 1 shows the template that is used to generate the body of an ocl invariant. the current implementation contains templates based on sql92 that range from very abstract patterns like ocl invariant bodies to very basic patterns like logical and expressions as shown in listing 2. it is notable, that all template code is of declarative nature. obviously, the illustrated approach provides a clear separation of the structural and the semantical mapping of domain models to database platforms. this reduces the complexity of the mappings, eases maintenance and results in a highly adaptive and configurable approach to ensure data integrity for business applications. 3.2 ocl2xquery since many applications are using xml as data format for storing their application data (either directly or indirectly by xml-based databases like the ibm db2 viper system [ibm]), it is also useful to support these systems with our query code generation framework. the second example for the application of our framework is the mapping of uml/ocl-based domain models to xml schema [fw04] and xquery [bcf+07] respectively. for transformation of uml models to xml schema we have used the strategies and patterns described in [car01]. we have developed a pattern catalogue for mapping ocl invariants to equivalent expressions in xquery (similar to the cataloque for sql presented in [dh99]) that is partly based on the work from gaafar and sakr [gs04] who describe the possibility to map xquery to ocl. since the ocl2xquery tool also heavily utilises our framework, the structure of the mapping process resembles the process described in subsection 3.1 (see figure 4). proc. ocl4all 2007 6 / 10 eceasst figure 4: mapping of uml/ocl-based domain models to xml these two examples show, that by simply exchanging model transformation strategies and pattern catalogues for code generation, a platform independent model can be translated to more platform specific models and code while preserving the platform independent ocl constraints. 4 related work in our work, we face up to the model-driven integrity engineering in data-intensive applications. knowing well that it is hard to draw a sharp dividing line between database and application rules [dat00], we consider database rules (cf. section 3). there is one project that is strongly related to our work. the andromda toolkit [and] transforms models of higher abstraction levels to models of lower abstraction levels (i.e. platform specific models or code) and also offers a means for transforming ocl constraints to other languages. at the moment it is limited to hql (hibernate query language) [hib] and ejb-ql (enterprise java bean query language) [sun] but due to its framework character, other target languages are also possible. however, it also differs in an important point from our work: in contrast to our metamodel-based approach it works on a string-based level, where ocl constraints are translated to target languages by a match parser that uses regular expressions. in [tg01] türker and gertz give an overview of the semantic integrity support in the sql standard sql:1999 [em99], and show that advanced concepts such as assertions and check constraints proposed in this standard are rarely supported in major commercial (object-)relational database management systems. the role of integrity constraints is often underestimated so that non-trivial integrity constraints are seldom considered in database design. one reason for this is the decreased performance when using an automatic constraint-enforcing mechanism. in our ocl2sql tool we therefore generate integrity views whereby the constraint evaluation can be performed in a batch-oriented manner. there are several ocl-to-sql case studies. in [pkv04] our (first) ocl-to-sql tool is used to generate sql code from spatial ocl, a domain-specific ocl version to model spatial constraints in environmental information systems. in [bc06] brambilla and cabot propose oclto-sql transformation and its tuning for web applications. vermeer and apers [va96] exploit object constraints for database interoperation. 7 / 10 volume 9 (2008) generating query language code from ocl invariants 5 summary in this paper we reported on our experiences on preserving ocl-based data-integrity rules for business applications in a multi-staged mdsd process. the automatic transformation of uml class models to other data schemas—while preserving the semantical integrity through the transformation of ocl invariants to sentences of corresponding query languages—reduces development costs and enhances the quality of the resulting system. we gave an overview on the architecture of the query code generation framework and its components to illustrate the abstraction mechanisms necessary to cope with the variety of implementation platform specifics. as illustrated in section 3, the presented approach is highly configurable and adaptable to a manifold of platforms and the corresponding query languages which advances the development of data-intensive business applications. acknowledgements: we would like to thank all people who have contributed over several years to the dresden ocl toolkit project. bibliography [abe+06] d. akehurst, b. bordbar, m. evans, w. howells, k. mcdonald-maier. sitra: simple transformations in java. in: acm/ieee 9th international conference on model driven engineering languages and systems, 2006. [and] andromda project team. andromda. http://www.andromda.org/ [bc06] m. brambilla, j. cabot. constraint tuning and management for web applications. in icwe 2006. pp. 345–352. acm 1-59593-352-2/06/0007, 2006. [bcf+07] s. boag, d. chamberlin, m. f. fernández, d. florescu, j. robie, j. siméon. xquery 1.0: an xml query language. w3c recommendation, jan. 2007. http://www.w3.org/tr/xquery [bet04] j. bettin. model-driven software development. in: mda journal, april 2004. [bs03] f. bry, s. schaffert. the xml query language xcerpt: design principles, examples, and semantics. in revised papers from the node 2002 web and database-related workshops on web, web-services, and database systems. pp. 295–310. springerverlag, london, uk, 2003. [car01] d. carlson. modeling xml applications with uml. addison-wesley, boston, münchen, 2001. [dat00] c. j. date. what not how. the business rules approach to application development. addison-wesley, 2000. proc. ocl4all 2007 8 / 10 http://www.andromda.org/ http://www.w3.org/tr/xquery eceasst [dh99] b. demuth, h. hussmann. using ocl constraints for relational database design. in france and rumpe (eds.), uml 1999 the unified modeling language. proc. 2nd international conference, fort collins, usa. pp. 598–613. springer lncs 1723, 1999. [dhk05] b. demuth, h. hussmann, a. konermann. generation of an ocl 2.0 parser. in proceedings of the models’05 workshop on tool support for ocl and related formalisms needs and trends. 2005. [dhl01] b. demuth, h. hussmann, s. loecher. ocl as a specification language for business rules in database applications. in gogolla and kobryn (eds.), uml 2001 the unified modeling language. proc. 4th international conference, toronto, canada. springer lncs 2185, 2001. [em99] a. eisenberg, j. melton. sql:1999, formerly known as sql3. acm sigmod record 28(1):131–138, 1999. [fw04] d. c. fallside, p. walmsley. xml schema. w3c recommendation, oct. 2004. http://www.w3.org/xml/schema [ghjv95] e. gamma, r. helm, r. johnson, j. vlissides. design patterns: elements of reusable object-oriented software. addison-wesley, reading mass., 1995. [gs04] a. gaafar, s. sakr. towards a framework for mapping between uml/ocl and xml/xquery. in baar et al. (eds.), uml 2004 the unified modeling language. proc. 7th international conference, lisbon, portugal. lecture notes in computer science 3273, pp. 241–259. springer, 2004. [hib] hibernate project team. hibernate query language. http://www.hibernate.org/hib docs/reference/en/html/queryhql.html [ibm] ibm. db2 viper. http://www-306.ibm.com/software/data/db2/xml/ [jk05] f. jouault, i. kurtev. transforming models with atl. in proceedings of the model transformations in practice workshop at models conference. 2005. [lo03] s. loecher, s. ocke. a metamodel-based ocl-compiler for uml and mof. in ocl 2.0 industry standard or scientific playground?, workshop proceedings, uml 2003 the unified modeling language. 6th international conference, san francisco, usa. entcs 154. 2003. [ms93] j. melton, a. r. simon. understanding the new sql: a complete guide. morgan kaufmann publishers, 1993. [obj01] object management group. common warehouse metamodel (cwm) specification. omg document, feb. 2001. http://www.omg.org/cgi-bin/doc?ad/2001-02-01 9 / 10 volume 9 (2008) http://www.w3.org/xml/schema http://www.hibernate.org/hib_docs/reference/en/html/queryhql.html http://www-306.ibm.com/software/data/db2/xml/ http://www.omg.org/cgi-bin/doc?ad/2001-02-01 generating query language code from ocl invariants [obj03a] object management group. mda guide version 1.0.1. omg document, june 2003. http://www.omg.org/cgi-bin/doc?omg/03-06-01 [obj03b] object management group. uml 2.0 ocl specification. omg document, oct. 2003. http://www.omg.org/cgi-bin/doc?ptc/03-10-14 [obj04] object management group. uml 2.0 infrastructure specification. omg document, oct. 2004. http://www.omg.org/cgi-bin/doc?ptc/04-10-14 [obj05] object management group. meta object facilities (mof) 2.0 query/view/transformation specification. omg document, nov. 2005. http://www.omg.org/cgi-bin/doc?ptc/2005-11-01 [par] t. j. parr. stringtemplate. http://www.stringtemplate.org [par04] t. j. parr. enforcing strict model-view separation in template engines. in www ’04: proceedings of the 13th international conference on world wide web. pp. 224–233. acm press, new york, ny, usa, 2004. [pkv04] f. pinet, m. kang, f. vigier. spatial constraint modelling with a gis extension of uml and ocl: application to agricultural information systems. in mis 2004, lncs 3511. pp. 160–178. springer, 2004. [sof] software technology group, technische universität dresden. dresden ocl toolkit. http://dresden-ocl.sf.net [sun] sun mircrosystems. enterprise java bean query language. http://java.sun.com/j2ee/1.4/docs/tutorial/doc/ejbql.html [sun02] sun microsystems incorporation. java metadata interface (jmi) specification. 2002. [tg01] c. türker, m. gertz. semantic integrity support in sql:1999 and commercial (object) relational database management systems. the vldb journal 10:241–269, 2001. [va96] m. w. vermeer, p. m. apers. the role of integrity constraints in database interoperation. in proceedings of the 22nd vldb conference. pp. 425–435. 1996. proc. ocl4all 2007 10 / 10 http://www.omg.org/cgi-bin/doc?omg/03-06-01 http://www.omg.org/cgi-bin/doc?ptc/03-10-14 http://www.omg.org/cgi-bin/doc?ptc/04-10-14 http://www.omg.org/cgi-bin/doc?ptc/2005-11-01 http://www.stringtemplate.org http://dresden-ocl.sf.net http://java.sun.com/j2ee/1.4/docs/tutorial/doc/ejbql.html introduction architecture of the query code generation framework model transformation framework ocl transformation framework applications ocl2sql ocl2xquery related work summary implementing dnssec soft delegation for microservices electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) implementing dnssec soft delegation for microservices andrés marı́n-lópez, patricia arias-cabarcos, thorsten strufe, gabriel barceló-soteras, daniel dı́az-sánchez, florina almenares-mendoza 5 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst implementing dnssec soft delegation for microservices andrés marı́n-lópez12∗, patricia arias-cabarcos1, thorsten strufe1†, gabriel barceló-soteras3, daniel dı́az-sánchez2, florina almenares-mendoza2 1 karlsruhe institute of technology, germany 2 universidad carlos iii de madrid, spain 3 parliament of the balearic islands, spain abstract: securing dns in edgeand fog computing, or other scenarios where microservices are offloaded, requires the provision of zone signing keys to the third parties who control the computing infrastructure. this fundamentally allows the infrastructure provider to create novel signatures at their discretion and even arbitrarily extend the certificate chain. based on our proposal on soft delegation for dnssec, which curtails this vulnerability, we report on our proof-of-concept: a c-implementation of chameleon hashes in openssl, a server side implementation of the mechanism in the ldns server, and an offline client that validates the signed records, in this paper. we also discuss different approaches for generating dnssec rrsig records, and the behavior of a resolver to verify the credentials and securely connect to an end point using tls with sni and dane. keywords: dnssec; dane; chameleon signatures; iot; microservices; fog computing 1 introduction interconnection of microservices requires name resolution and security support. dns and tls are by far the most used protocols for this purpose. local certification authorities (cas) and self-issued certificates are supported in tls, and suit in dynamic microservices scenarios. consider a group of off-shore windmills with dense local connectivity but expensive connection to the mainland, which is coordinated and managed by a local set of edge-servers, as one example. from the core of the network, several microservices are selected to be instantiated at the nodes in the windmills. once instantiated, the microservices will setup secure interconnections. most connections must be resolved inside the fog, including name resolution and certificate verification. our proposal in [dmaa19] was aimed at minimizing the intervention of the core of the network, especially the dns at the core (from now on the dnscore) with a soft delegation to the dns of the distant computing nodes, which we’ll call dnsfog. we proposed using chameleon hashes [kra00]. they allow an entity that possesses trapdoor information to create collisions – in ∗ funded by the comunidad de madrid project cynamon p2018/tcs-4566 and co-financed by european structural funds (esf and feder) † funded by the german research foundation (dfg, deutsche forschungsgemeinschaft) as part of germany’s excellence strategy – exc 2050/1 – project id 390696704 – cluster of excellence “centre for tactile internet with human-in-the-loop” (ceti) of technische universität dresden. 1 / 5 volume 080 (2021) dnssec soft delegation other words to compute signatures that verify alternative, arbitrary inputs. chameleon signatures are a lightweight solution compared to normal dnssec delegation. signed patterns impose the limits of the delegation to the dnsfog to only matching rrs. 2 dns security dns handles objects called resource records (rrs) of different types (a/aaaa, mx, name server ns, etc.) dns clients query for rrs matching a domain name (also known as owner field). dnssec adds integrity protection to dns, to prevent cache poisoning and spoofing. dnssec clients are provided with signatures to verify the integrity of the resource records in the response. validation requires the verification of the signatures, and dnssec introduces key chains for that matter. the chains originate at the root zone “.”. the zone records on each level are signed by a zone signing key (zsk). it’s integrity is verified by a using local key signing keys (ksk), which in turn are signed by the zsk of the parent zone in the ds records, basis for trust chain extension to a delegated zone. dnssec responses include the signatures (rrsig) of the rrs. as in dns, the answer may optionally include (1) the authoritative part including the name servers (ns) and corresponding rrsigs, and (2) the additional part, the prefetching mechanism of dns, with the rrs the server anticipates will be required by the client, for instance the a or aaaa records of the ns in the authoritative part, and corresponding rrsigs. the dns-based authentication of named entities (dane) specification [rfc6698] introduces the tlsa rr type. tlsa associates a certificate or a public key of an end-entity or a trusted issuing authority with the tls or dtls transport endpoint. dane tlsa records validated by dnssec can be used to augment or replace the use of trusted public cas. tls clients can use the service name indication (sni) when negotiating the connection with a tls server, and dane supports sni by tlsa records which can hold: a credential for the server (pkix-ee); a domain name trust anchor (ta) linking the authorized issuer ca for the domain (pkix-ta); a local ca entitled to issue trusted certificates for the domain (dane-ta); and even to indicate an ad-hoc key for the tls server (daneee). 3 dnssec soft delegation considering the usual dynamics in microservice deployments, dnssec would require the server to provide signatures for every local change. we hence propose soft delegation, allowing the dnscore to delegate signing of the microservices using chameleon signatures to the dnsfog. let us illustrate how the chameleon signatures work using discrete logarithms [am05]: 1. let dnsfog selects q a large prime, p = kq + 1, and g, a generator of z∗p. let sk f = x ∈ (1,q−1) be the trapdoor, with corresponding public key pk f = y = gxmod q. let (pks,sks) be the public and private keys of dnscore (the signer). 2. dnsfog sends the dnscore two random parameters a and b. the dnscore takes message m and computes e = sha256(m,b), cham-hash f (m,a,b) = a−(yegbmod p)mod q. 3. the dnscore signs it: sig = sigs(rrsig rdat a|a|b|cham-hash f (m,a,b)). netsys 2021 2 / 5 eceasst 4. the signature can be verified with (sig,m,a,b, p,g,q,y,pks). 5. when dnsfog wants to modify the message m′, it picks a new random l,computes the collision for the chameleon hash, the hash of m′, and the corresponding new b′: a′ = cham-hash f (m,a,b)+(g l mod p)mod q (1) e′ = sha256(m ′,a′) (2) b′ = k−e′xmod q (3) the new message signature can be verified with (sig,a,b,m′,a′,b′, p,g,q,y,pks), which means that the signature sig still holds for a modified m′. note that to verify the signature both the original parameters (a,b) and the new parameters (a’,b’) are required. that is how the dnscore limits the delegation to dnsfog by only allowing to generate modifications m′ from message m. m is obtained by hashing the serialization of a specially crafted rr, where the owner is modified to contain a placeholder (<t>). we call this rr a pattern. an a pattern looks like: <t>.example.com. 3600 in a rnd the pattern is serialized as: rrpattern=(owner|type|class|ttl|len(rdata)|rdata), rdata=<t>.example.com|rnd. and the message is computed as m= sha256(rrpattern). in our soft delegation approach, the dnscore will generate the signature sig=sign(rrsig_rdata|a|b|cham-hash(m,a,b)). the signature is thus bound to the (a,b) parameters and to the chameleon hash of the pattern, which includes the dnsfog public key y. the public parameters y,p,g,q are in a ds record with private oid at the dnscore. the record generated by the dnscore will be like: <t>.example.com 3600 in rrsig a 254 3 ttl d1 d2 32478 example.com. b64(sig,a,b) the algorithm value 254 (private_oid) indicates that the verification is different from standard. the (a,b) parameters are concatenated to the signature so that they can be recovered for verification. to create a new rr from the pattern, the dnsfog computes m′ as the sha256 of the serialization of the new rr, computes the parameters (a’,b’) to have a collision of the chameleon hash, and concatenates the (a’,b’) parameters to the signature. when the verifier gets the rrsig of the new rr, it will also query for the corresponding pattern rr and rrsig. it will compute the chameleon hash of the rrpattern using the (a,b) parameters from the rrsig of the pattern. next it will check the signature validity using the public key of the dnscore. then it checks that the chameleon hash of the new rr with (a’,b’) holds: cham-hash(m,a,b)=cham-hash(m’,a’,b’) finally, the verifier also has to check if the owner, type and ttl match the ones in the pattern. 4 implementation and evaluation the dnsfog has two alternatives to compute the collisions: approach 1: compute a single rrsig for the whole rrset of all the created microservices, or approach 2: multiple rrsigs: one per each of the created microservices. in approach 1, the signature is performed on the hash of the whole rr set. since no new rrsig 3 / 5 volume 080 (2021) dnssec soft delegation are added, there is no need to recalculate the nsec/nsec3 with each rr added. but this approach does not protect against domain enumeration, since the responses must include all the rrs to verify the signature of the rr set, disclosing thus all the existing a/tlsa records. besides responses size may exceed the limit of udp. in approach 2, each microservice will have its own rrsig, so equal number of nsec/nsec3 rrs have to be added. this approach provides domain enumeration protection but requires computing an extra collision for the nsec/nsec3 records, on the other hand responses are shorter, since only the required domain name a/tlsa are returned with the corresponding rrsig. we have opted in our implementation for approach 2. figure 1: verification of microservice queue.example.com figure 1 shows a microservice named queue.example.com instantiated by the orchestrator at ip 10.0.3.27 with a self-signed certificate entirely included in the tlsa record (daneee). the figure illustrates the whole process, asuming the client has already the public keys pks which is the zsk dnskey of the dnscore and the public parameters p,g,q,y which are in the ds record with private oid signed by the dnscore, i.e. the corresponding delegated public key of the dnsfog. 5 discussion and conclusions dnssec delegation is performed by signing a ds. normal delegation gives the dnsfog authority over a whole subdomain. an attacker may extract the delegated private key, if the dnsfog is in a remote site only visited for maintenance, as in the wind mill scenario. our proposal limits the delegation to a set of types (patterns), in the same way as we limit most verification traffic to the fog. in our proposal if the dnsfog is attacked, the attacker can only use the private key to alter rrs according to the pattern, and only locally at that fog. online signing requires the dnscore signing a key for each dnsfog and the key to be available at the dnsfog. this implies the above mentioned risk for remote unattended sites. in [dmaa19] we show that the collision computation time is negligible when compared with the signing time with rsa or ecdsa, while the time of computing the chameleon hash is 7 times slower than rsa signature, if advanced factorization is used. this increased load is pushed from the dnscore to the dnsfog, allowing the schema to scale. netsys 2021 4 / 5 eceasst the delegation increases the trust chain so that resolvers must also trust in the dnsfog. nevertheless, this is the same as in normal delegation scheme or in the pki trust model, where the trust is rooted at the top cas, but in the end you have to trust also the final entity presenting a certificate. at least for the time it takes a private security incident to be noticed and notified, and the final entity certificate to be revoked and the crl to be published or pushed to the ocsp server. trust is bounded by the patterns signed by the dnscore. they limit the delegation to the dnsfog, which has the trapdoor to add, delete or modify rrs according to the patterns. we call this a soft delegation since it is limited to the signed patterns: only rrs matching the delegated patterns can be modified by the dnsfog. the soft delegation alleviates the dnscore from the necessity of signing the whole zone with every new change in the microservices deployment in the fog. the dnscore only has to sign the delegated patterns using the chameleon hash. the dnsfog will compute the collision when a new rr is required using eqs. 1-3. we expect delegation to patterns of types a, aaaa and tlsa records. in an extended version of this article we will address more details of the implementation. our results from the server-side integration of the proposed soft delegation mechanism in the open source ldns dnssec server show that the chameleon hashes and the collisions are computed faster than in [dmaa19]. the proof of concept server side implementation is available at https://gitlab.pervasive.it.uc3m.es/100416942/ldns-tools-chamhash. besides we have developed an offline verifier which takes as input a json file containing all the required records to validate the signatures of dnssec records. we have verified that common cases of requests in microservice environments are protected through the use of the offline verifier together with some scripts and a python wrapper to invoke ldns. this can be found at https: //gitlab.pervasive.it.uc3m.es/100416942/core-fog-scripts. bibliography [am05] g. ateniese, b. de medeiros. on the key exposure problem in chameleon hashes. in blundo and cimato (eds.), security in communication networks. pp. 165–179. springer berlin heidelberg, berlin, heidelberg, 2005. [dmaa19] d. dı́az-sánchez, a. marı́n-lopez, f. almenárez mendoza, p. arias cabarcos. dns/dane collision-based distributed and dynamic authentication for microservices in iot. sensors 19(15):3292, 2019. [kra00] h. r. krawczyk. t: chameleon signatures. in proceedings of ndss. volume 2000, pp. 143–154. 2000. 5 / 5 volume 080 (2021) https://gitlab.pervasive.it.uc3m.es/100416942/ldns-tools-chamhash https://gitlab.pervasive.it.uc3m.es/100416942/core-fog-scripts https://gitlab.pervasive.it.uc3m.es/100416942/core-fog-scripts introduction dns security dnssec soft delegation implementation and evaluation discussion and conclusions uncertainty entangled; modelling safety assurance cases for autonomous systems electronic communications of the easst volume 079 (2020) interactive workshop on the industrial application of verification and testing etaps 2020 workshop (interavt 2020) uncertainty entangled; modelling safety assurance cases for autonomous systems anila mjeda and goetz botterweck 10 pages guest editors: stylianos basagiannis, goetz botterweck, anila mjeda eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst uncertainty entangled; modelling safety assurance cases for autonomous systems anila mjeda and goetz botterweck lero–the irish software research centre university of limerick limerick, ireland name.surname@lero.ie abstract: when designing and analysing autonomous systems and their environment it is necessary to consider uncertainty and multiple potential states (of the system and its environment). in this position paper, we explore the idea of notations and modelling tools that are based on ‘superpositions’ of states. more concretely, we present a treatment of uncertainty in autonomous systems inspired by quantum physics and propose an extension of the goal structuring notation (gsn), a common approach for the modelling of safety arguments, to model ’superposition’ and ’entangled’ nodes; and, incorporate guidelines of the emerging ul 4600 standard for autonomous systems. keywords: autonomous vehicles; uncertainty; safety assurance cases; gsn, ul 4600 1 introduction i am sitting with a philosopher in the garden; he says again and again ‘i know that that’s a tree’, pointing to a tree that is near us. someone else arrives and hears this, and i tell him: ‘this fellow isn’t insane. we are only doing philosophy. ludwig wittgenstein, on certainty in safety-critical systems, it is critical to give grounds for why the software can be trusted. current trends on autonomous vehicles (avs) heighten the challenge of assuring they are safe, especially in view that avs need to operate in unpredictable situations and to make real-time decisions which so far have been the remit of human drivers. part of the challenge stands with the fact that developing autonomous systems, entails designing for uncertainty [rjc12, fc19] which in turn exacerbates the need for uncertainty-aware software development methodologies [fc19]. one technique used by classic (not fully autonomous) safety-critical domains is the submission of a safety assurance case that provides a documented trail, which demonstrates that state of the art safety considerations have been followed and are traceable throughout the system. this, in essence, entails showing that the potential risks associated with using the safety-critical system 1 / 10 volume 079 (2020) uncertainty entangled; modelling safety assurance cases for autonomous systems in its intended domain have been identified and acceptably mitigated. it provides the grounds to convince the certification bodies (e.g. [rtc11, iso11, fa+10, iso19]) and other stakeholders that the system is acceptably safe [lev11, hhkm13, dp18]. using safety assurance cases in avs is still a new and brave frontier. that said, existing research of particular interest for the focus of this paper is the emerging standard1 for autonomous systems [und19] and research on treatments of uncertainty [fc19]. in this position paper, we take a multidisciplinary lens to the treatment of uncertainty. we draw ideas from quantum physics where uncertainty has been treated as a domain-native for decades to posit our ideas on modelling uncertainty for autonomous systems. we also build on previous work by [dp18] and [und19] and propose an extension to the goal structuring notation (gsn)2. the remainder of this paper is organised as follows: in section 2 we discuss safety assurance cases and the latest developments in this area. in section 3 we discuss the concept of uncertainty from a multidisciplinary point of view and present our position in the treatment of uncertainty and its implications for autonomous vehicles (including a running example in section 3.1). in section 4 we propose our ideas extending gsn to capture autonomous assurance cases and finally we draw conclusions in section 5. 2 assurance cases assurance cases are a technique used to argue that a system is safe for a given application in a defined environment. formally, the assurance case is a “reasoned, auditable artifact created for contention that its top-level claim (or a set of claims), is satisfied, including systematic argumentation and its underlying evidence and explicit assumption(s) that support the claim.” [iso19]. simply put, assurance cases are structured arguments backed by evidence on why a specific safety goal or claim is met. the typical anatomy of a safety case consists of (1) the safety goal (typically split into sub-goals) that needs be achieved; (2) the backing evidence for achieving this goal; and, (3) the structured argument, which establishes the systematic relationship between the evidence and the goals. both the arguments and the evidence can be somewhat informal. as such, the arguments can express some degree of confidence, while the backing evidence can take multiple forms such as results from model checking, testing, analysis, modeling, simulation; or, expert opinions. an in-depth analysis of assurance cases and exiting tool support for them can be found in [dp18]. while there is no particular notation required, the goal structuring notation (gsn) [scs18], is adopted widely. figure 1 provides an illustration of a safety argument in gsn notation constructed for the duckietown environment3. the nodes in gsn provide a simple description for the goals, strategies, assumptions, 1 the standard for safety for the evaluation of autonomous products, ul 4600 is being developed as a ul – underwriters laboratories standard by a standards technical panel (stp) composed from researchers and practitioners from industry and academia. it is projected to be ratified as a ul standard in the first half of 2020. 2 a graphical notation which is adopted widely to capture safety assurance cases 3 this simplified diagram is extracted from our work-in-progress research in safety arguments for autonomous vehicles where the duckietown environment (from an mit classroom started activity: www.duckietown.org) is used as a running example. interavt 2020 2 / 10 eceasst g1 vision-based line following behaviour is correct at all times g2 track illumination variability is compensated correctly s1 argue over reasons that cause lane following failures a1 assume that camera is working correctly a2 assume acceptable lighting conditions g3 illumination compensation req. uncertainties are mitigated g4 illumination compensation design uncertainties are mitigated g5 road markings are detected correctly g6 correct recall uncertainties are mitigated g7 false positive uncertainties are mitigated e2 simulation testing results g8 lane-relative estimation is correct at all times e1 illumination compensation req. spec. model s2 argue over uncertainty of prior belief in track illumination conditions s3 argue over uncertainty of correct recall s4 argue over uncertainty of false positives a3 assume a mean time failure of > 30 minutes is acceptable a4 assume a cross track error < = 1.5 cm is acceptable … safety requirements: the duckiebots follow lanes in a track built according to specification j1 prior belief uncertainty taxonomy (patterns to choose from) … … g1-n: goal s1-n: strategy a1-n: assumption j1-n: justification e1-n: evidence figure 1: a partial safety argument in gsn notation justifications and evidence and are meant as pointers to fully detailed artifacts. the aim is to trace all of the safety goals to credible evidence without gaps in coverage and where all assumptions are analysed vis-à-vis their validity. not surprisingly, assurance cases are especially useful in situations where the system is too complex to be ‘proven’ safe using formal verification approaches or more broadly when heavy-weight formal approaches are not feasible to be used. a promising development for the world of assurance cases is ul 4600 [und19] which aims to support a structured way to argue that autonomous vehicles are safe. its first version focuses in highly automated vehicles and looks at the challenges of av safety in the absence of human drivers and presence of non-determinism. it looks at all the different functionality provided by human drivers including fault recovery and expected misuse. the goal of ul 4600 is to provide guidance on adopting the required system-level engineering rigour and is meant to be used alongside the other standards that regulate the domain (e.g., iso 26262) [iso11]. that said, in contrast to, say, iso26262, instead of prescribing ‘how to do safety’, ul 4600 is goal-based and makes use of assurance cases to argue in a structured way that the system is acceptably safe [pk19]. typically the safety goal defines what acceptably safe means for the case in question and then it proceeds to argue how the av meets that definition. it does not prescribe any specific software engineering approaches but it requires the use of rigorous ones. for example, it does not prescribe how to do testing and verification, but it rather requires a sound argumentation that you have done sufficient testing and verification. its creators see it as a framework for systematically putting together all of the elements needed to create an acceptably safe av. it facilitates this task by providing reference lists, lists of safety case topics, prompts and epistemic defeaters (‘did you consider what happens in cases of black ice?’), reference lists of hazards and lists of techniques that can be used to identify hazards, good 3 / 10 volume 079 (2020) uncertainty entangled; modelling safety assurance cases for autonomous systems practices to follow and bad practices to avoid [pk19]. all the prompts provided by ul 4600 need to be considered but not necessarily adopted, and for guidance they are classified into: • mandatory – prompts that are required to be considered. • highly recommended – prompts that can be ignored or altered with non-trivial rationale. • examples – prompts that are meant as illustrative reminders and not required do be addressed exhaustively. • recommended –prompts that are entirely optional. 3 reasoning with uncertainty uncertainty has been studied in many disciplines and a significant amount of research has focused on analysing and interpreting it [hei27, ms58, laf, tk74, gps08, tk94, cw04, kni12]. in economics, typically uncertainty is treated as a stochastic or probabilistic situation [laf], where the frequency or outcome of an event is not known [gps08] including situations where it is not possible to specify definite numerical probabilities for a specific (desired) outcome [kni12]. treatments of uncertainty in economical studies often intersect with uncertainty in decision making and project management (e.g., [jaa01, pgw08]). in psychology, uncertainty is typically discussed against the backdrop of certainty and is conceptualized as a psychological state. wittgenstein posited that being ‘certain’ is not any ‘knowing’ but it rather is a matter of us feeling that ’we cannot imagine otherwise’4 [wav86]. this somewhat non-deterministic treatment of uncertainty resonates with quantum physics where what is commonly referred to as the ’uncertainty principle’ is one of the key concepts that brought forward a revolution on our understanding of the universe. uncertainty in quantum physics can be considered ’native’, and yet its meaning is subject to different schools of thought and interpretations. different conceptualisations of uncertainty include ontological interpretations (lack of knowledge by an observer) and, epistemic interpretations (e.g., measurement inaccuracy). in fact, even the terminology comprises a range of words such as indeterminacy, indefiniteness and, inaccuracy. at the heart of both the thought revolution; and, lack of consensus and generational debates brought forward by quantum theories is the issue of how (and whether) the measurement process affects the ontological state of an observed system (and in some interpretations the observer her/himself) and it has even entered common parlance as ‘the measurement problem’. for us, in dealing with uncertainty in autonomous systems, this struggle is mirrored in the somewhat ‘simpler’ struggle of developing a measure of uncertainty that we know how do deal 4 please note how wittgenstein’s thinking connects to nicolaus copernicus (1473–1543) and to the quantum physicist hugh everett iii (1930–1982) who gave us the multiverse quantum hypothesis (referred to as ‘the branching world’ below): “bryce dewitt, an american physicist who had edited the journal where everett’s thesis appeared, wrote a letter to him complaining that the real world obviously didn’t ‘branch’, since we never experience such things. everett replied with a reference to copernicus’s similarly daring idea that the earth moves around the sun, rather than vice-versa: ‘i can’t resist asking: do you feel the motion of the earth?’ dewitt had to admit that was a pretty good response.” – sean carroll, something deeply hidden interavt 2020 4 / 10 eceasst with operationally. borrowing from the quantum physics’ thought experiments we can posit that we can profitably reason about the behaviour of autonomous vehicles if we know a probability distribution of the likelihood that the av might be behaving in various ways. entanglement (in quantum physics), at its simplest, suggests that when two particles interact or share spatial proximity, their behaviour becomes intrinsically linked even after they are far apart. say two electrons were moving with equal opposite velocities and they bump onto each other. even though before their encounter they each had probabilities of travelling in certain paths which were completely unrelated to each other, after their encounter they will move in precisely opposite directions. we say that the electrons are entangled5. now let us draw our attention to the ’macro’ world of avs and conceptualise them as entities that have different probability distributions of the likelihood of behaving in various ways. we use the entangled uncertainties metaphor to posit that we should design for uncertainty by conceptualizing our design choices as superimposed ontological representations of reality. our overall design for uncertainty resembles a ‘solutions superposition-model’ 6. where, each solution taken (in other words the av’s behaviour at a specific time-step matches that particular solution), entangles the uncertainty in the av behaviour within at least the next time step (the length of the time step would depend on when the next interaction of the av will cause the uncertainties of its behaviour to become entangled with the environment (e.g., bumps into the next thing). hence the intuition is that if we have enough ’certain’ solutions designed in our ’solutions superposition-model’ we can draw upon the ’entanglement of uncertainties’ argument and collapse the problem-space into certain outcomes for the av behaviour. in more lay terms, this translates into arguing about narrowing down the all possible behaviours’ state-space into a subset of (designed for) acceptably safe behaviour by an av. while we characterise possible av behaviours as probability distributions, another point of frustration we need to clarify is the meaning we give to probability. one school of thought sees the concept of probability connected to the frequency of an event or outcome happening/occurring. that is perfectly satisfying for cases when we can wait to witness the same exact event reoccurring over and over for a large number of repetitions. how often do we get tails if a coin is tossed? but, how do we reason for cases when we are in reality dealing with epistemic probability, where what we already know is important to help us gauge the outcome. even more so in cases when waiting for a very large number of occurrences or experiments upon some events is not plausible. for example, what is the probability of the soccer team of ones’ own country will win the world championship? in these cases, we can reason in terms of attaching beliefs between zero and one hundred percent to the various possible outcomes where the total set of beliefs for the possible outcomes adds up to 100 percent7. next, let us consider some illustrative examples of uncertainties in an autonomous environment. 5 in the everettian view of quantum physics the entanglement (e.g., how do the two entangled electrons ‘know’ what the other is doing) is described via the universal wavefunction, in other words, positing that the quantum state of the whole universe in ’interlinked’ and can be ’captured’ in one wave function. 6 here the superposition-model plays on the quantum concept of superposition 7 in this regard, there are existing mathematical frameworks such as for example dempster-shafer theory which acknowledge the need for explicitly expressing and quantifying beliefs/confidence/lack of confidence into a certain outcome. 5 / 10 volume 079 (2020) uncertainty entangled; modelling safety assurance cases for autonomous systems 3.1 no duck left behind – uncertainty in duckietown duckietown [pta+17] is an environment that started at mit and is aimed at education and research in autonomous vehicles. duckieworld is comprised of avs (“duckiebots”) that autonomously drive their users (“duckies”) around their town (“duckietown”) that has roads, traffic lights, and obstacles. the duckiebots can follow lanes while abiding traffic lights and avoiding random obstacles and the duckie pedestrians. their only sensor is a monocular camera and all their on-board processing is done in a raspberry pi. one of the interesting aspects of the duckiebots for us is that they face a number of software and environment uncertainties (figure 1), that include: • prior beliefs on illumination conditions of the roads. • prior beliefs on road-markings colours (the duckiebots navigation depends on colour-coded markings). • errors in the estimation of fit quality that allows the system to detect road-marking detection failure. • false positives in road-markings’ detection. • errors in lane-relative estimation. in the following section, we use these uncertainties as our running example to illustrate our position in designing for uncertainty. 4 gsn entangled – designing for uncertainty we propose to extend gsn nodes with metadata (building on research by [dp18]) to: a) accommodate our vision of ‘solutions superposition-model’ to model autonomous systems; and, b) incorporate the extensive reference lists, different types of prompts and epistemic defeaters provided by the emerging ul 4600 standard as natural elements of our treatment of uncertainty (’entangled uncertainties’). specifically, to account for superposition-model nodes and our treatment of ‘entangled uncertainties’ we propose to extend the gsn node types with: • nodetype ::= classic | superposition | entangled note that when the node type is classic the node will look exactly as any other classic gsn node. to account for respectively the need to express beliefs in certain outcomes and ul 4600’s prompt classifications, we extend the parameter types of gsn nodes with: • belief ::= percentageindecimalnumber • prompttype ::= mandatory | highyrecommended | example | recommended | as discussed in the previous sections, these ideas hail from our treatment of uncertainty and more specifically from our thought experiments on ‘entangled uncertainties’. in figure 2 we illustrate these ideas via a partial assurance case that captures some of the software and environment uncertainties faced by the duckiebots (our running example of an autonomous environment). in here, the overall safety goal that ’duckiebots will not hit any types of duckies’, interavt 2020 6 / 10 eceasst g1 duckiebots will not hit any types of duckies [ 0.9 ] e1 results of analysis of coverage of different types of duckies a3 assume false positive uncertainties are mitigated g2 prompttype: mandatory duckiebots will stop or avoid collision with detected duckies g3_meta type: superposition identify and mitigate risks caused by difficult to detect duckies g4_meta type: entangledgoal belief: 0.9 all mitigations are identified and are sufficient g5_meta type: entangledgoal belief: 0.1 fault response and violation strategy include feedback to accident log a1 assume that camera is working correctly a2 assume acceptable lighting conditions e2 results of detect and avoid simulation testing g3_meta type: superposition prompttype: mandatory identify and mitigate risks caused by difficult to detect duckies [ 0.1 ]… … figure 2: partial assurance case illustrating uncertainties in extended gsn notation under the assumptions that duckies’ cameras are working correctly and they have to operate within acceptable lighting conditions8 is split into two sub-goals: 1) ’duckiebots will stop or avoid collision with detected duckies’; and, 2) ’identify and mitigate risks caused by difficult to detect duckies’. the later goal is modelled via a nodetype: superposition and both sub-goals have the parameter prompttype: mandatory in their metadata. the prompttype: mandatory parameter incorporates ul 4600 in gsn notation, while nodetype: superposition reflects our ideas on ‘entangled uncertainties’. we incorporate the concept of ‘entangled’ probability distributions by modelling the two ‘entangled’ nodes (’all mitigations are identified and are sufficient’ and, ’fault response and violation strategy include feedback to accident log’) that come out of the superposition node (’identify and mitigate risks caused by difficult to detect duckies’) by using the parameter type belief : percentageindecimalnumber. this (extended) gsn node’ metadata is expressed via a (real) decimal number (or probability), where all the entangled nodes’ beliefs that come out of a superposition type node add up to 1 (100%). this is particularly important since we believe that as the technology behind quantum computing progresses further we expect to benefit from the real power of notations and modelling tools that ‘think’ in terms of superpositions |ψ〉 of the two classical states |0〉 and |1〉 coded in quantum memory (for example in the same qubit): 8 naturally this would translate into accepted weather conditions’ in real-world scenarios. 7 / 10 volume 079 (2020) uncertainty entangled; modelling safety assurance cases for autonomous systems |ψ〉 := α |0〉+ β |1〉= ( α β ) ; where, |α|2 +|β|2 = 1. in here, the coefficients α and β are complex numbers9 and tie in nicely with our ‘entangled nodes’ with assigned probability beliefs, since in quantum theories |α|2 and |β|2 represent exactly the probability of a state being observed. finally, figure 2 illustrates examples of evidence (’results of analysis of coverage of different types of duckies’, and ’results of detect-and-avoid simulation testing’) that are used to back up the assurance case’s claims. 5 conclusions the current drive for autonomous systems will benefit from rigorous uncertainty-aware software engineering methodologies. we believe that software engineering could profitably build upon treatments of uncertainty from other disciplines that, in different capacities, have dealt for decades with uncertain environments, measurements, projections, occurrences, etc. the current safety assurance demands on autonomous systems could benefit from a paradigm shift towards ‘complex-state’ modelling. this, in conjunction with the gained momentum in quantum computing, could be fostered by notations and modelling tools that ‘think’ in terms of superpositions and make use of quantum memory (as in nodes conceptualised as quantum states |ψ〉 := α |0〉+ β |1〉= ( α β ) ; where, |α|2 +|β|2 = 1. and, |0〉 & |1〉 represent of the two classical states.) in this paper, we explore a conceptualisation of uncertainty in autonomous systems inspired by quantum physics which sees uncertainty as a native feature of a system. furthermore, we propose an extension of gsn (building on research by [dp18]) to: a) accommodate ‘superposition’ and ‘entangled’ nodes; and, b) incorporate the extensive reference lists, different types of prompts and epistemic defeaters provided by the emerging ul 4600 standard as natural elements of our treatment of uncertainty (‘entangled uncertainties’). in future work, we plan to focus on modelling and validating live feedback that drives the update of beliefs in probability distributions. acknowledgements this work was supported, in part, by science foundation ireland grant 13/rc/2094 and co-funded under the european regional development fund through the southern & eastern regional operational programme to lero the irish software research centre (www.lero.ie). bibliography [cw04] c. chapman, s. ward. why risk efficiency is a key aspect of best practice projects. international journal of project management 22(8):619–632, 2004. 9 in quantum mechanics they are the amplitudes of the wave function. interavt 2020 8 / 10 eceasst [dp18] e. denney, g. pai. tool support for assurance case development. automated software engineering 25(3):435–499, 2018. [fa+10] u. food, d. administration et al. guidance for industry and fda staff-total product life cycle: infusion pump–premarket notification. draft guidance (april 2010), 2010. [fc19] m. famelis, m. chechik. managing design-time uncertainty. software & systems modeling 18(2):1249–1284, 2019. [gps08] i. gilboa, a. w. postlewaite, d. schmeidler. probability and uncertainty in economic modeling. journal of economic perspectives 22(3):173–88, 2008. [hei27] w. heisenberg. uber den anschaulichen inhalt der quanten theoretischen kinematik und mechanik, zeit. für phys., 43, 172–198. available in english translation: goo. gl/fmq2j7, 1927. [hhkm13] r. hawkins, i. habli, t. kelly, j. mcdermid. assurance cases and prescriptive software safety certification: a comparative study. safety science 59:55–71, 2013. [iso11] iso. iso 26262: road vehicles-functional safety. international standard iso/fdis 26262, 2011. [iso19] iso/iec/ieee 15026: systems and software engineering — systems and software assurance . standard, iso/iec/ieee international organization for standardization, 2019. [jaa01] a. jaafari. management of risks, uncertainties and opportunities on projects: time for a fundamental shift. international journal of project management 19(2):89–101, 2001. [kni12] f. h. knight. risk, uncertainty and profit. courier corporation, 2012. [laf] j. laffont. j., 1989, the economics of uncertainty and information. [lev11] n. g. leveson. the use of safety cases in certification and regulation. 2011. [ms58] j. g. march, h. a. simon. organizations. wiley, 1958. [pgw08] o. perminova, m. gustafsson, k. wikström. defining uncertainty in projects–a new perspective. international journal of project management 26(1):73–79, 2008. [pk19] d. prince, p. koopman. ul 4600 technical overview. technical presentation at carnegie mellon university, oct 2019. https://users.ece.cmu.edu/∼koopman/talks/191010 ul4600 tech webinar.pdf [pta+17] l. paull, j. tani, h. ahn, j. alonso-mora, l. carlone, m. cap, y. f. chen, c. choi, j. dusek, y. fang et al. duckietown: an open, inexpensive and flexible platform for autonomy education and research. in 2017 ieee international conference on robotics and automation (icra). pp. 1497–1504. 2017. 9 / 10 volume 079 (2020) https://users.ece.cmu.edu/~koopman/talks/191010_ul4600_tech_webinar.pdf uncertainty entangled; modelling safety assurance cases for autonomous systems [rjc12] a. j. ramirez, a. c. jensen, b. h. cheng. a taxonomy of uncertainty for dynamically adaptive systems. in 2012 7th international symposium on software engineering for adaptive and self-managing systems (seams). pp. 99–108. 2012. [rtc11] rtca. do-178c, software considerations in airborne systems and equipment certification. rtca, 2011. [scs18] goal structuring notation community standard version 2. standard, 2018. [tk74] a. tversky, d. kahneman. heuristics and biases: judgement under uncertainty. science 185(4157):1124–30, 1974. [tk94] b. n. taylor, c. e. kuyatt. guidelines for evaluating and expressing the uncertainty of nist measurement results. 1994. [und19] underwriters laboratories. standard for safety for the evaluation of autonomous products, ul 4600 (draft). december 2019. https://edge-case-research.com/wp-content/uploads/2019/12/191213 ul4600 votingversion.pdf [wav86] l. wittgenstein, g. anscombe, g. von wright. on certainty/über gewissheit. harper collins, 1986. interavt 2020 10 / 10 https://edge-case-research.com/wp-content/uploads/2019/12/191213_ul4600_votingversion.pdf https://edge-case-research.com/wp-content/uploads/2019/12/191213_ul4600_votingversion.pdf introduction assurance cases reasoning with uncertainty no duck left behind – uncertainty in duckietown gsn entangled – designing for uncertainty conclusions towards an interest management scheme for peer-based virtual environments electronic communications of the easst volume 17 (2009) workshops der wissenschaftlichen konferenz kommunikation in verteilten systemen 2009 (wowkivs 2009) towards an interest management scheme for peer-based virtual environments florian heger, gregor schiele, richard süselbeck, christian becker 10 pages guest editors: m. wagner, d. hogrefe, k. geihs, k. david managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst towards an interest management scheme for peer-based virtual environments florian heger1, gregor schiele2, richard süselbeck3, christian becker4 1 florian.heger@uni-mannheim.de 2 gregor.schiele@uni-mannheim.de 3 richard.sueselbeck@uni-mannheim.de 4 christian.becker@uni-mannheim.de chair of information systems ii university of mannheim, germany abstract: a fundamental task in peer-to-peer-based massively multiuser virtual environments is providing all peers with a consistent view of the environment. to do so, state changes must be propagated to peers. to limit the resulting network traffic, existing approaches often restrict the distribution of a state change to peers for which it is relevant. the identification of such peers is known as interest management. typically interest management is done based on so-called areas of interest. an area of interest is a spatial area around a user’s avatar. if a change occurs inside this area, it is relevant and must be reported to the corresponding peer. we propose to extend this approach with so-called areas of effect. an area of effect specifies the spatial area of the virtual environment that is directly influenced by a given state change. a state change is propagated to all peers whose areas of interest intersect with the change’s area of effect. this allows us to model complex state changes with arbitrary and possibly dynamic influence areas. they may even affect multiple areas at once. in this paper we describe our approach and give an overview on the current state of it’s implementation. keywords: mmve, peer-to-peer, interest management, area of interest 1 introduction massively multiuser virtual environments (mmves) allow the interaction between thousands of users in a shared virtual environment. users are represented in the mmve by their avatars, virtual entities that are controlled by the users and act on their behalf. in order to maintain the illusion of a credible virtual environment, the users constantly have to be provided with updates about state changes. a lot of information has to be propagated, which results in a high volume of network traffic and induces the need for sophisticated filtering schemes. these schemes determine the information which is relevant for every user. network traffic is optimized by only propagating relevant information. the design and the implementation of such schemes are generally known as interest management [mor96]. because scalability of an mmve very much depends on optimization of network traffic, the implementation of an effective interest management scheme is a key factor for creating scalable mmves with millions of concurrent users. 1 / 10 volume 17 (2009) mailto:florian.heger@uni-mannheim.de mailto:gregor.schiele@uni-mannheim.de mailto:richard.sueselbeck@uni-mannheim.de mailto:christian.becker@uni-mannheim.de towards an interest management scheme for peer-based virtual environments this is even more challenging for peer-to-peer (p2p) based mmves, like [hl04] [klxh04] [cyb+07] [yv05], in which the management of the virtual world is distributed over all participating devices. such mmves have become increasingly popular in recent years, due to their large potential with respect to better scalability and lower operating costs. our goal is to provide an interest management scheme for p2p-based mmves that enables highly scalable and dynamic systems. we base our approach on the well-known concept of areas of interest (aoi) and extend it with a novel concept which we call areas of effect (aoe). an aoi specifies a spatial area around a user’s avatar in the virtual environment. this area describes the user’s interest in his environment. the relevancy of information is determined based on this area: a peer only receives updates about state changes inside the aoi. in addition to this, an aoe specifies the spatial area in the virtual environment that is affected by a given state change. thus, a change can affect only a specific point, e.g. in case of a slow movement of a virtual object, or a larger area in the mmve, e.g. a line in case of a very fast movement. this allows us to model complex state changes with a single update instead of a series of updates, reducing the amount of updates that must be distributed. in this paper we propose our initial approach for scalable interest management in p2p-based mmves based on aois and aoes. we present aoes in more detail, show their potential and possible shortcomings and discuss how this concept can be used in an interest management scheme. this work is part of the peers@play project [www]. the rest of this paper is organized as follows: in section 2 we describe our system model and discuss requirements for an interest management scheme. after that we discuss related work in section 3 before we present our approach in section 4. finally, we give an overview on the current implementation state of our scheme, draw conclusions and give a preview on our future work in section 5. 2 system model and requirements before discussing existing approaches for interest management in p2p-based mmves, we first describe our target system environment in more detail. after that we present a list of requirements which a suitable scheme must fulfill. our system model consists of a number of users that want to use an mmve, their devices, a communication network and the mmve software. users may be located at any place in the world. the number of users is a priori undetermined and can change dynamically. a device executes the mmve software and is connected to a common communication network, e.g., the internet. we call such a device a peer. a peer can join and leave the system at any time. each user is represented in the mmve by a special character, called avatar. to interact with the mmve, the user controls his avatar to perform different activities. activities can also be initiated by so-called non-player characters. if an activity is executed, the system creates events describing the resulting state changes and propagates them to other peers. to be usable in such a system, an interest management scheme must fulfill the following requirements. we adopt these requirements from [mor96] and adapted them to p2p-based systems. 1. restriction to relevance: an interest management scheme has to determine only the relevant information for each peer. relevant means all information which is needed to guarantee a correct game-play and to maintain the authenticity of the virtual world in the users’ proc. wowkivs 2009 2 / 10 eceasst perception. 2. consistency: an interest management scheme further has to provide all users with enough information to guarantee a consistent world state on each peer. because consistency very much depends on users’ perception, there is information which is more and information which is less important to guarantee consistency. 3. distribution: in p2p-based mmves there is no central entity. the filtering of information has to be done in a decentralized manner. hybrid forms, e.g. mmves using zones with coordinators, are possible. 4. density awareness: in mmves users tend to cluster at certain locations, e.g. inside cities. an interest management scheme has to take that into account and has to adapt to these clusters in order to propagate updates efficiently. 5. interest scope: the interests of users are not necessarily symmetric. for example one user may see another user but not vice-versa, because one user is hidden behind an obstacle and watches the other user. some capabilities are also directional, e.g. the range of view. two users are able to see each other except one is behind the other. an interest management scheme has to take the asymmetry as well as the direction of interest into account. 3 related work various proposals have been made on interest management in mmves. most proposals either use visibility or spatiality as a filter to determine the relevant updates. an overview can be found in [bkv06]. steed et al. [sa05] use visibility as a filter. they propose a network partitioning scheme called frontier sets. the environment is sub-divided into cells. frontier sets are computed and used to determine if users are visible to each other. updates are only propagated to the visible users. frontier sets have an advantage inside rooms and buildings. outside the scheme shows weaknesses because of the resource-intensive calculation of frontier sets. proposals using spatiality as a filter often include the concept of areas of interest (aoi), e.g. [klxh04] [hcc06] [bhs+08] [yv05]. the user is represented in the virtual environment by an avatar. the aoi is a space around the avatar describing the limitations of it’s sensing capabilities. updates are filtered based on the aoi. the dominant spatial structure of the aoi is circular, e.g. [hcc06] [bhs+08]. hu et al. [hcc06] propose aois which change size dynamically depending on the number of peers inside the aoi. the concept of aoi goes back to the proposal of a spatial model of group interaction in virtual environments by benford et al. [bf93]. in this model every object in a virtual environment is surrounded by an aura. objects carry their aura with them when they move through the environment. the environment monitors for aura collisions. when auras collide, interaction is possible. benford et al. also mention that an object typically has different auras (e.g. differentiated by size and shape) for different media. it may be possible that in a large space without obstacles one user can see another user but can not hear him, because the visual aura is larger than the audio aura. 3 / 10 volume 17 (2009) towards an interest management scheme for peer-based virtual environments the spatial model includes the concept of focus and nimbus, which is further described in [gb95] and [skh02]. auras are divided into focus and nimbus. focus represents a user’s perception, nimbus represents an object’s perceptivity. a user is aware of an object if its focus intersects with the object’s nimbus. filtering using spatiality is further done by dividing the environment into regions, e.g. [klxh04] [hcc06] [frp+08] [cyb+07] [yv05]. each region is managed by a central entity, e.g. by a server or a coordinator node. updates are propagated by the central entity. a user’s interest into a region is determined depending on the location of the user (e.g. [klxh04]) or the intersection between the user’s aoi and the region (e.g. [hcc06]). in these approaches, scalability very much depends on the right partition and the right choice for coordinator peers. an often used model for update propagation is the publish-subscribe model, e.g. [klxh04] [hcc06] [cyb+07] [yv05]. when a user is interested in a region, he subscribes to the central entity and from that point on receives updates about state changes inside that region. when the user loses his interest, he unsubscribes from the region. the publish-subscribe model has the disadvantage of creating additional network traffic for the handling of subscriptions and unsubscriptions when the users move. frey et al. [frp+08] break with the publish-subscribe model: bounding boxes around the avatars are computed and information is exchanged directly between avatars that may potentially collide with each other based on these boxes. they do not mention a distinct entity for state changes. the propagation of updates depends on the avatars. in [glz05] gauthierdickey et al. describe an algorithm for event ordering in peer-to-peer games. the peers are organized in an n-tree. they propose to model state changes explicitly as events. events are described as a tuple consisting of an action, the action’s location and a function representing the action’s scope of impact. the scope of impact is used to determine which event has to be propagated to which peer. their algorithm allows events to be propagated quickly between peers wich are located near to each other in the virtual environment. even they model state changes as events, they do not represent events as first level entities inside the mmve. we think that proposals using the concept of aoi can be enhanced concerning the aoi’s spatiality and the representation of state changes. the sensing capabilities of users are influenced by user and environmental properties, e.g. there are users which have a larger range of view and users which are hidden behind obstacles. therefore, the aoi’s spatial structure and size should be changed dynamically. in contrast to [hcc06], we think that the change should depend on user and environmental properties, not the number of surrounding users. in mmves there are state changes which have properties, e.g. state changes which influence a certain area or remain for a period of time inside the mmve. by representing state changes as entities in the interest management scheme, the changes’ properties can be taken into account for update propagation. according to [gb95] and [skh02], our approach includes areas complementing the aoi and interest management is done depending on the intersections. in contrast, the areas of our approach represent new first level entities describing the state changes instead of being tied to existing entities like users or objects. these first level entities are not only characterized by spatiality, they are also characterized by further properties like time and priority. in the following, we describe our approach. proc. wowkivs 2009 4 / 10 eceasst 4 an interest management scheme for peer-to-peer-based mmves our interest management scheme is based on the concept of aoi management. we modify the concept and extend it with the concept of so-called areas of effect (aoe). in our scheme, state changes are modelled as first level entities, so-called events. in case an activity is executed that induces a change in the virtual environment, update events have to be issued. events allow us to model even complex state changes which affect certain areas or last for a certain period of time by controlling their distribution. events are represented by aoes. an aoe is the event’s spatial extension in the virtual environment. it’s properties resemble the event’s characteristics. by representing events by aoes, we intend to optimize the amount of updates that has to be sent over the network. using explicit entities for complex state changes allows us to propagate information about the event in a single update and compute possible changes in the following without a need for further updates. for example the information about the emergence of rain clouds is sent only once in a single update to all affected peers. in the following, changes are computed locally on every peer according to environmental circumstances, e.g. the wind. in the following, we give details about aoi and aoe. then we describe how relevant updates are determined. finally we describe our system architecture and give an overview on the current implementation state. 4.1 the area of interest we mentioned before that the aoi’s spatial structure depends on the sensing capabilities of the users. these capabilities are influenced by user and environmental properties. these properties can change, e.g. the environment that surrounds a user changes when the user moves. therefore, we do not define a distinct spatial structure in our interest management scheme. we also include the possibility to change the structure and it’s size dynamically according to changes of these properties. our approach is based on the assumption that an appropriate spatial structure for the aoi is equivalent to finding a trade-off between complexity of the structure’s computation and accuracy of update propagation. possible structures range from a rectangular structure to an arbitrary formed geometric structure. while a rectangular structure means easy computation at the expense of an unaccurate propagation, a free formed structure means complex computation at the benefit of an accurate propagation. benford et al. [bf93] discussed that there should be more than one aoi regarding the different kinds of media, e.g. there should be different aois for viewing and hearing. following this, we do not limit our scheme to a single aoi per user. instead, for every user there is an indefinite number of aois. all aois surround the user. they are represented by indefinite spatial structures. if the user moves on, the aois also change location according to the user’s movement. 4.2 the area of effect the aoe is the representation of an event inside the mmve. it has a spatial structure, a lifetime and a priority. the spatial structure represents the event’s impact on the environment. because 5 / 10 volume 17 (2009) towards an interest management scheme for peer-based virtual environments figure 1: aoes of a teleport (top), an accumulation of clouds represented by a hexagon (bottom left) and a rectangle (bottom right). structure and size depend on the event’s characteristics as well as environmental properties and both factors may vary from event to event, the aoe has an indefinite spatial structure. the aoe also may exist for a certain lifetime and it’s structure may change during lifetime according to environmental properties. the information about priority is needed to determine the importance of events in case of overload, in order to detect the first events to drop. in the current version of our scheme we concentrate on simple geometrical forms, e.g. circles, rectangles and points. based on our requirements analysis we find that geometrical forms are a fit for most events in an mmve. they can be computed in an efficient way and, therefore, are a desirable solution. an example for an aoe given as a single point is a movement event. a line could be used for a very fast movement. an example for an event that affects a wide area in the environment is an accumulation of clouds which causes rain in a certain area. in our scheme the corresponding event may be represented by a circular, elliptic, rectangular or hexagonal structure. even though we concentrate on geometrical forms right now, we also think about using free formed structures in the future. free formed structures offer the possibility of a very accurate representation of an event. the main challenge concerning these structures will be to find an efficient way of computing them. in addition to this, the spatial structure of aoes can also be split into multiple parts. mmves often include events that affect multiple non-adjacent locations in the virtual environment, e.g. if a user is teleporting from one place to another (see figure 1 top). this event demands the propagation of updates to users at both locations. the aoe representing the teleport consists of two parts, both of them have the spatial structure of a point. because aoe’s have a lifetime, our scheme is able to represent events that affect the environment only for a single moment as well as over a period of time. the spatial effect of events proc. wowkivs 2009 6 / 10 eceasst figure 2: the determination of relevant updates based on aoes. can change during lifetime. for example the accumulation of clouds mentioned before may change it’s location according to the wind (see figure 1 bottom left). also the size of the accumulation may change according to changes in weather conditions (see figure 1 bottom right). the accumulation of clouds represented by the hexagon changes location according to the wind. another accumulation represented by a rectangle changes size according to changes in weather conditions. the priority of an aoe is used to decide whether the propagation of updates can be passed on in case of a high user density (requirement 4). for example, the rain caused by the accumulation of clouds mentioned before may not be propagated to users which are crowded together at a place. the rain is used to create a better atmosphere and is not an integral part of game-play and therefore has a low priority. at the moment, this is still work in progress and not included in out prototype, yet. we expect this concept to be a main topic of our future research. 4.3 determination of relevant updates to determine whether a given event is relevant for a certain user, we compute the intersection of the event’s aoe and the user’s aoi. a user is provided with updates about an event, if it’s aoi intersects with the aoe of the event. if there is an aoi of user a and an aoe of event e, then the information about e is propagated to a if a’s aoi intersects with e’s aoe. figure 2 shows an example. the aoi of user b intersects the aoe of event e. therefore, b has to be provided with updates about e. the aoi of user a does not intersect e’s aoe and no updates are propagated. if an aoe consists of multiple parts, updates are propagated to all users whose aoi intersects with at least one part of the aoe. if the aoi of a user intersects with multiple parts, the update is propagated only once. assuming that the aoi approximates the users’ interest in updates in an appropriate way, the determination of relevant updates in our scheme fulfills requirement 1 (restriction to relevance). requirement 5 (interest scope) is not fulfilled by our scheme at the 7 / 10 volume 17 (2009) towards an interest management scheme for peer-based virtual environments moment and will be part of our future work. 4.4 system architecture the main architectural question when realizing this approach is where the computation of intersections between aois and aoes is done. in a p2p system there are two main possibilities: either there is one central instance for interest management, i.e. a single peer, or interest management is done in a decentralized manner, e.g. determination of relevant updates is done on each user’s computer. using one central instance has the advantage of a good consistency control (requirement 2). there is also only one instance that has to be provided with updates about events. the main disadvantage of that architecture is the possibility of overloading the central instance. in that case, the central instance acts like a bottleneck for the whole architecture. decentralized interest management offers the advantage of avoiding the possible bottleneck, but makes it difficult to maintain consistency. also each user’s computer has to be provided with information about the positions of all other peers’ aoi, in order to compute the intersections of the areas in a decentralized manner. because both architectures offer advantages as well as disadvantages, we propose to use a hybrid architecture including aspects of both. we propose to tessellate the environment into zones. each zone is administrated by a coordinator peer. the coordinator peers are responsible for interest management. this architecture avoids the bottleneck and still offers the possibility to control consistency in a partly centralized manner. it also fulfills requirement 3 (distribution). the development of our hybrid architecture is still in progress and a closer description will be part of our future work. nevertheless, we give a short description of the propagation algorithm in the following. our algorithm is based on two assumptions: (1) for every zone there is a coordinator peer responsible for interest management. this peer is dynamically chosen from all peers that participate in the mmve. (2) every peer is able to determine the coordinator peers and their corresponding zones. in the peers@play project we use a distributed hash table (dht) for this lookup. updates about an event are propagated as follows: 1. in case an event is triggered on a peer, the information about the corresponding aoe is sent to the coordinator peer of the zone in which the peer is located. 2. the coordinator peer determines which zones overlap with the aoe. this is determined based on the maximum spatial expansion of the aoe. 3. the coordinator peer sends the event and its aoe to each coordinator peer which handles an overlapping zone. 4. each coordinator peer then computes the intersections between the aois of the users inside his zone and the aoe. 5. finally, the information about the event and its aoe is propagated by the coordinator peer to the affected peers. proc. wowkivs 2009 8 / 10 eceasst 4.5 implementation state we have implemented a basic version of our interest management scheme and integrated it into the peers@play framework. the computation of intersections between aois and aoes as well as the propagation of updates depending on the outcome of the intersection operations are already implemented and working stable for a single zone. the current version of the implementation includes variably sized circular aois and aoes. currently, we are working on implementing additional geometric forms and multi-part aoes. 5 conclusions and future work in this paper we presented an early view on our proposed interest management scheme which adopts the widespread concept of areas of interest and extends it with areas of effect. based on a requirements analysis we proposed to model state changes as explicit entities (so-called events). areas of effect are the spatial extension of events in the virtual environment. by modelling state changes as entities with distinct properties we are able to represent complex events whose properties change over time. we provided a first glimpse on our algorithm for update propagation and an overview on the current implementation state. the implementation and evaluation of our approach is currently on the way. a main topic of our future research will be to find a solution on how to handle users that cluster at certain places (requirement 2). event priorities are one possible approach that we will employ in this area. we are also going to develop an event model for describing the characteristics of complex state changes in order to ease the definition of events and their corresponding areas of effect. bibliography [bf93] s. benford, l. fahlen. a spatial model of interaction in large virtual environments. in proceedings of the 3rd european conference on computer-supported cooperative work. 1993. [bhs+08] j. botev, a. höhfeld, h. schloss, i. scholtes, m. esch. the hyperverse concepts for a federated and torrent-based ”3d web”. in proceedings of the 1st international workshop on massively multiuser virtual environments at the ieee virtual reality. 2008. [bkv06] j.-s. boulanger, j. kienzle, c. verbrugge. comparing interest management algorithms for massively multiplayer games. in proceedings of the 5th acm sigcomm workshop on network and system support for games. 2006. [cyb+07] l. chan, j. yong, j. bai, b. leong, r. tan. hydra: a massively-multiplayer peer-topeer architecture for the game developer. in proceedings of the 6th annual workshop on network and systems support for games (netgames). 2007. [frp+08] d. frey, j. royan, r. piegay, a.-m. kermarrec, e. anceaume, f. l. fessant. solipsis: a decentralized architecture for virtual environments. in proceedings of the 1st 9 / 10 volume 17 (2009) towards an interest management scheme for peer-based virtual environments international workshop on massively multiuser virtual environments at the ieee virtual reality. 2008. [gb95] c. greenhalgh, s. benford. massive: a collaborative virtual environment for teleconferencing. acm transactions on computer-human interaction (tochi) 2:239–261, 1995. [glz05] c. gauthierdickey, v. lo, d. zappala. using n-trees for scalable event ordering in peer-to-peer games. in proceedings of the 15th international workshop on network and operating systems support for digital audio and video (nossdav). 2005. [hcc06] s.-y. hu, j.-f. chen, t.-h. chen. von: a scalable peer-to-peer network for virtual environments. ieee network 20:22–31, 2006. [hl04] s.-y. hu, g.-m. liao. scalable peer-to-peer networked virtual environment. in proceedings of the 3rd acm sigcomm workshop on network and system support for games. 2004. [klxh04] b. knutsson, h. lu, w. xu, b. hopkins. peer-to-peer support for massively multiplayer games. in proceedings of the 23rd conference of the ieee communications society (infocom). 2004. [mor96] k. l. morse. interest management in large-scale distributed simulations. technical report, department of information & computer science, university of california, irvine, 1996. [sa05] a. steed, c. angus. supporting scalable peer to peer virtual environments using frontier sets. in proceedings of the ieee virtual reality. 2005. [skh02] j. smed, t. kaukoranta, h. hakonen. aspects of networking in multiplayer computer games. the electronic library 20:87–97, 2002. [www] the peers@play project: http://pap.vs.uni-due.de. [yv05] a. yu, s. t. vuong. mopar: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games. in proceedings of the 15th international workshop on network and operating systems support for digital audio and video (nossdav). 2005. proc. wowkivs 2009 10 / 10 introduction system model and requirements related work an interest management scheme for peer-to-peer-based mmves the area of interest the area of effect determination of relevant updates system architecture implementation state conclusions and future work refactoring of uml models using agg electronic communications of the easst volume 8 (2008) proceedings of the third international ercim symposium on software evolution (software evolution 2007) refactoring of uml models using agg alessandro folli1, tom mens 15 pages guest editors: tom mens, ellen van paesschen, kim mens, maja d’hondt managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 1 this paper reports on work that has been carried out by the first author in the context of a masters thesis supervised by the second author in the context of an erasmus exchange programme. http://www.easst.org/eceasst/ eceasst refactoring of uml models using agg alessandro folli2, tom mens service de génie logiciel, université de mons-hainaut, belgique abstract: model refactoring is an emerging research topic that is heavily inspired by refactoring of object-oriented programs. current-day uml modeling environments provide poor support for evolving uml models and applying refactoring techniques at model level. as uml models are intrinsically graph-based in nature we propose to use graph transformations to specify and apply model refactoring. more in particular, we use a specific graph transformation tool, agg, and provide recommendations of how agg may be improved to better support model refactoring. these recommendations are based on a small experiment that we have carried out with refactoring of uml class diagrams and state machines. keywords: uml, model refactoring, agg, graph transformation 1 introduction model-driven engineering (mde) is a software engineering approach that promises to accelerate development, to improve system quality, and also to enable reuse. its goal is to tackle the complexity of developing, maintaining and evolving complex software systems by raising the level of abstraction from source code to models. the mechanism of model transformation is at the heart of this approach, and represents the ability to transform and manipulate models [sk03]. model transformation definition, implementation and execution are critical aspects of this process. the challenge goes beyond having languages to represent model transformations. the transformations also need to be reused and to be integrated into software development methodologies and development environments that make full use of them. the term refactoring was originally introduced by opdyke in his seminal phd dissertation [opd92] in the context of object-oriented programming. martin fowler [fow99] defines this activity as “the process of changing a software system in such a way that it does not alter the external behaviour of the code, yet improves its internal structure”. current-day refactoring techniques focus primarily on the source code level and do not take into account the earlier stages of design. a need exists for refactoring tools that enable designers to better manipulate their model, not just their source code. furthermore, there is a need to synchronise and maintain consistency between models and their corresponding code; source code refactorings may need to be supplemented with model-level refactoring to ensure their consistency. this article will focus on the problem of model refactoring, which is a particular kind of model transformation. the unified modeling language (uml) [obj05] is the industry standard used to specify, visualize, and document models of software systems, including their structure and design. therefore, 2 this paper reports on work that has been carried out by the first author in the context of a masters thesis supervised by the second author in the context of an erasmus exchange programme. 1 / 15 volume 8 (2008) refactoring of uml models using agg the goal of this article is to explore the refactoring of uml models. we use graphs to represent uml models, and graph transformations to specify and apply model transformations. this choice is motivated by the fact that graphs are a natural representation of models that are intrinsically graph-based in nature (e.g., class diagrams, state machine diagrams, activity diagrams, sequence diagrams). graph transformation theory has been developed over the last three decades as a suite of techniques and tools for formal modelling and very high-level visual programming [roz97, eekr99, ekmr99]. it allows to represent complex transformations in a compact visual way. moreover, graph transformation theory provides a formal foundation for the analysis and the automatic and interactive application of model transformations. among others, it provides the ability to formally reason about such graph transformations, for example to analyse parallel and sequential dependencies between rules. in this article, we argue that the use of graph transformations for the purpose of model refactoring is both possible and useful. as a proof of concept, we implement a number of complex model refactorings in agg3 [tae04]. it is a rule-based visual programming environment supporting an algebraic approach to graph transformation [em93]. agg may be used as a general-purpose graph transformation engine in high-level java applications employing graph transformation methods. agg is also one of the rare tools that incorporates mechanisms such as critical pair analysis for formally analysing graph transformations, which can be very useful for analysing refactoring rules [mtr07, men06]. based on our experience, we provide recommendations on how the agg tool, and graph transformation tools in general, may be improved. 2 motivating example in the field of software engineering, the unified modeling language (uml) defined by the object management group (omg) [obj05] is the de facto industry standard specification language for modelling, specifying, visualising, constructing, and documenting software-intensive systems. uml provides a standardized graphical notation to create abstract models of a system, referred to as uml models. these models must be conform to the uml metamodel which describes the syntax and semantics. as an example, figure 1 shows the metamodel of uml state machine diagrams. model refactoring is a special kind of model transformation that aims to improve the structure of the model, while preserving (certain aspects of) its behaviour. like the process of source code refactoring, the process of model refactoring is a complex activity. a definition of refactoring has been introduced by opdyke in his phd dissertation [opd92]. he defines refactorings as program transformations containing particular preconditions that must be verified before the transformation can be applied. in [fol07] we have discussed eight primitive model refactorings for uml class diagrams and uml state machine diagrams. this clearly shows that it is possible to formalise the specification and execution of model refactoring using graph transformation rules. table 1 shows the list of model refactorings that we have discussed and implemented. each model refactoring was 3 http://tfs.cs.tu-berlin.de/agg/ proc. software evolution 2007 2 / 15 http://tfs.cs.tu-berlin.de/agg/ eceasst figure 1: uml metamodel of state machines formalised, explained and motivated using a concrete example. a detailed explanation of when it should be used and how it can be realised precedes a discussion of the list of mechanisms to accomplish the refactoring itself. in this article, we explore the introduce initial pseudostate model refactoring in more detail in order to explain and illustrate the main concepts.4 as suggested by the name, it adds an initial pseudostate to a composite state, or region. the introduce initial pseudostate refactoring is used to improve the structure of a state machine diagram. in general, it is a good convention not to cross boundaries of a composite state. figures 2 and 3 show a simple example of using this kind of refactoring. an initial pseudostate 4 for reasons of simplicity, the representation of uml state machines used for this article does not consider the actions attached to states, such as do, entry and exit actions. 3 / 15 volume 8 (2008) refactoring of uml models using agg uml class diagram uml state machine diagram pull up operation introduce initial pseudostate push down operation introduce region extract class remove region generate subclass flatten state transitions table 1: list of model refactorings has been added to the active composite state. the target of the transition that initially referred to the ready state has been redirected to its enclosing region. an automatic transition has been defined between the initial pseudostate and the ready state. the ready state has thus become the default initial state of the active region; a transition whose target is the active state will lead the state machine to the ready state. when the refactoring is used to introduce a final state as well, similar changes to the transitions involved in the composite state will need to take place. figure 2: uml state machine diagram before refactoring figure 4 shows the control flow of this model refactoring that is composed of more primitive refactoring actions. the notation of uml interaction overview diagrams has been used to formally depict the control flow and to specify in which order the action must be executed. the interaction occurrence frames that compose the diagram indicate activities or operations to be invoked. for the purpose of defining model refactoring, they have been associated to graph transformation rules. some custom notations have been added to enrich the diagram with all necessary information. in particular, input and output parameters for each atomic step have been specified. in order to apply the refactoring, it is necessary to provide two input parameters r and s. the parameter r specifies which composite state, or region, will be modified by the refactoring. the parameter s specifies which will be the default state of the region. before applying the refactoring it is necessary to verify that the composite state does not contain an initial pseudostate; this check proc. software evolution 2007 4 / 15 eceasst figure 3: uml state machine diagram after refactoring figure 4: introduce initial pseudostate model refactoring, specified as uml interaction overview diagram will be implemented as a precondition. if the precondition is respected, the refactoring proceeds by creating the initial pseudostate inside the composite state. subsequently, the refactoring changes the target of all transitions pointing to the default state. the new target state of those transitions will become the composite state that contains the region r. for technical reasons, a final cleanup phase is needed in order to remove auxiliary elements 5 / 15 volume 8 (2008) refactoring of uml models using agg that have been added during the transformation process. 3 formal representation as graph transformation uml models can be represented as a graph-based structure, and graphs must conform to the corresponding type graph, much in the same way as models must conform to their metamodel. a type graph corresponding to the uml metamodel is required to formally represent the uml models as graphs and to formally define the uml model refactoring. 5 for the purpose of this article, we have chosen to take into account a subset of the concepts defined by the uml metamodel. in particular, we focus on uml state machine diagrams only. figure 5 shows the type graph corresponding to the uml metamodel of figure 1. this type graph has been created using the agg graph transformation tool. figure 5: uml state machine diagram – type graph agg offers many concepts that are useful to define a type graph very similar to the corresponding uml metamodel. agg allows enrichment of the type graph with a generalisation relation between nodes, and each node type can have one or more direct ancestors (parents) from which it inherits the attributes and edges. moreover, it is also possible to define a node type as an abstract type, thereby prohibiting creation of instance nodes of this abstract type. the primitive refactoring actions shown in figure 4 can be implemented by means of graph transformation rules. transformation rules are expressed mainly by two object structures: the left-hand side (lhs) of the rule specifies a subgraph to search for, while the right-hand side (rhs) describes modifications generated by the transformation. the lhs and the rhs of a rule are related by a partial graph morphism. the applicability of a rule can be further restricted by 5 for a detailed account on the relation between refactoring and graph transformation, we refer to [men06]. proc. software evolution 2007 6 / 15 eceasst additional negative application conditions (nacs). the lhs or a nac may contain constants or variables as attribute values, but no java expressions, in contrast to an rhs. in order to link all these primitive refactoring actions together (as specified in figure 4), we need to ressort to some kind of controlled (or programmed) graph transformation mechanism. tools like fujaba offer this possibility by relying on the notion of so-called story diagrams [gz04]. agg, unfortunately, does not support controlled application of graph transformation rules, so we were forced to implement such a mechanism ourselves, as will be explained in section 4. the first primitive refactoring, named create initial pseudostate is shown in figure 6. it contains a nac to ensure that the region does not contain an initial pseudostate. the state s provided as input parameter (node number 3 in the figure) will become the default state of the region. the strname variable used in the rule is not an input parameter but is needed in order to define the name of the initial pseudostate. the state s provided as input parameter must be part of a composite state otherwise application of this kind of refactoring is no longer possible. if the precondition is respected, the transformation rule marks the default state with an auxiliary “refactoring” node in order to recognize it during the execution of the subsequent steps. figure 6: introduce initial pseudostate create initial pseudostate input parameters r : region ⇒ node1; s : state ⇒ node3 the second primitive refactoring step, named move incoming transition, is shown in figure 7. it takes into account the transitions which have the default state defined as target (the auxiliary “refactoring” node is used to identify the default state). the transformation rule replaces the target edge of a transition by one pointing to the composite state. for this transformation rule a nac has been added in order to ensure that only the transitions that are defined outside the region will be modified. the rule must be repeated as long as possible (i.e., until no further match can be found). the last step, named remove temporary reference is shown in figure 8. it removes the auxiliary “refactoring” node that has been attached to the default state during the execution of the first rule. 7 / 15 volume 8 (2008) refactoring of uml models using agg figure 7: introduce initial pseudostate move incoming transition figure 8: introduce initial pseudostate remove temporary reference 4 tool support in this section, we illustrate the feasibility of developing model refactoring tools using graph transformations. for this purpose, we have developed a prototype application in agg. the choice of agg was motivated by its good support for formal analysis techniques. based on our experience with implementing this prototype, we will discuss the current limitations of agg and graph transformation in general in section 5. the agg graph transformation engine is delivered together with an api (application programming interface) that allows to integrate the internal graph transformation engine into other environments. we used this api to develop our prototype application. this allowed us to specify the graph transformation rules of section 3, as well as the control flow specified in figure 4. figure 9 shows the graphical user interface of the model refactoring application that we developed in java by making use of the agg api. the application internally loads a file containing the model refactoring specifications and the necessary graph transformation rules. it then allows to open files containing the uml models to be refactored that respect the type graph. using the “refactoring” context menu, the user can apply the different model refactorings. when necessary the user will be prompted to enter the input parameter values and possibly to supply a match if the model refactoring can be applied to different parts of the uml model. the representation of the control flow explained in figure 4 has been a crucial point for the implementation of the prototype application. the control flow describes the order in which the individual graph transformation rules of each model refactoring have to be executed. at the moment, the agg tool does not provide a satisfactory solution for organizing and combining rules, and the supplied mechanisms were not sufficient for describing model refactorings. the proc. software evolution 2007 8 / 15 eceasst figure 9: model refactoring application prototype application avoids the underlying problem by using a custom control structure that represents the control flow of model refactorings. based on the uml interaction overview diagram syntax, we have represented the control flow as a graph structure, which is used to drive the application of graph transformation rules. figure 10 presents the type graph that we have defined in agg in order to represent this control flow. when the prototype application needs to apply a model refactoring, it first loads the corresponding graph representing the control flow. it searches the starting point and walks through the graph to determine which graph transformation rules have to be applied. it continues exploring the graph until it reaches a final point, and reports the result to the user. figure 11 shows the control flow we have implemented for the introduce initial pseudostate refactoring. it corresponds to the uml interaction overview diagram reported in figure 4. the prototype application has been enriched with an interpreter in order to evaluate the expression of decision points. that way, implementation of complex transformations is made possible. 5 discussion the main goal of this article was to present a practical study of the current limitations of graph transformation technology. seen in this light, we can use the insights gained during our experiments to discuss some of the limitations of agg, and of graph transformations in general. these suggestions may be used to advance the state-of-the-art in graph transformation tool support in the future. 9 / 15 volume 8 (2008) refactoring of uml models using agg figure 10: interaction overview diagram – type graph figure 11: control flow – introduce initial pseudostate 5.1 limitations of agg concerning the agg tool in particular, we encountered a number of limitations. agg does not allow to represent concepts like aggregation and composition used by the uml metamodel. therefore, the type graph needed to be simplified by using the more generic concept of association. moreover, agg does not have the notion of enumeration type. the property kind of the pseudostate element has been represented using a string value. the type graph also did not take into account ocl constraints imposed on the uml metamodel. these constraints typically proc. software evolution 2007 10 / 15 eceasst specify invariants that must hold for the system being modeled. ocl expressions need to be taken into account when specifying model refactorings in agg. this can be achieved by expressing them using so-called “graph constraints” in agg. trying to formalise ocl constraints as graph constraints, however, is a far from trivial task. agg does not provide a satisfactory control structure for organizing and combining rules, and the supplied mechanisms for composing rules were not sufficient to describe model refactorings. in order to reach our goal we needed to implement a notion of “controlled” graph transformation on top of agg. this is nothing new, in the sense that other graph transformation tools (e.g. progres, fujaba, great, viatra2) already support such mechanisms. agg also does not allow to specify optional patterns inside graph transformation rules. therefore, it is necessary to create similar graph transformation rules that take into account the different optional patterns. for example, the “guard” nodes may or may not be associated to a transition. in order to match transitions with an associated “guard” and transitions without “guard”, creation of two different graph transformation rules is necessary. again this limitation is not present in some of the other graph transformation tools around. another limitation of agg is that it does not support concrete domain-specific visual notation. this would be a nice feature, in order to be able to specify model refactorings using a notation that the designer is familiar with. as a solution to this problem, an eclipse plug-in called tiger is being developed for specifying domain-specific visual editors on top of agg [eetw06]. finally, in order to make model refactoring support acceptable to the community, it needs to be integrated in standard uml modeling environments. this could be realised, for example, by relying on the tiger emf transformation framework. it is a tool environment that allows to generate an eclipse editor plugin based on the eclipse modeling framework (emf) and the graphical editing framework (gef) using agg as underlying transformation engine [bek+06]. 5.2 limitations of graph transformation tools in general for the purpose of model refactoring, an important advantage of graph transformation is that rules may yield a concise visual representation of complex transformations. unfortunately, as identified by [dhj+06], the current state-of-the-art in graph transformation does not suffice to easily define model refactorings, so their expressive power needs to be increased. two mechanisms have been proposed by these authors: one for cloning, and one for expanding nodes by graphs. these are mechanisms that allow one to write more generic graph transformation rules. during our own experiments, we encountered the need for a reusability mechanism at the level of transformation rules. in our large set of transformation rules, there were many similarities among the rules, and we would like to have some mechanisms to capture these similarities, in order to write some of the rules in a more compact way, and also in order to be able to reuse (parts of) existing rules when specifying new ones. this reusability can take on many forms: the ability to share application conditions between different rules, the ability to call some rule from within another one (similar to subroutines and procedure calls), the ability to express multiple rules with the same lhs (resp. rhs) without needing to duplicate the lhs (resp. rhs) each time, to avoid redundancy, the ability to define a notion of specialisation between rules, and so on. in viatra2, for example, graph patterns can be decoupled and manipulated separately from the graph transformation rules themselves [bv06]. recently, an even more sophisticated 11 / 15 volume 8 (2008) refactoring of uml models using agg mechanism of recursive graph pattern matching has been introduced [vv07]. better modularisation mechanisms are also needed. whenever a large set of rules needs to be specified, we need to modularise them in some way. the layering or prioritisation mechanism offered by agg is too rudimentary for this purpose. other techniques for structuring transformations are therefore needed. one potentially useful approach could be the use of so-called transformation units [kk99]. other mechanism are available as well, and a detailed comparison of them has been made by [heet99]. integration of such mechanisms in graph transformation tools is starting to emerge. to give but one example, a new grouping operator has been introduced in the model transformation language great to enable the concise specification of complex transformations [bnn+07]. an important limitation of current graph transformation tools is that graphs and graph transformations are represented in a different way. ideally, graph transformations need to be treated as first class values. if graph transformations could be expressed as graphs, we would gain a lot of expressive power. for example, we could specify higher-order transformations, i.e., graph transformations that transform graph transformations. we could also use graph transformations to generate graph transformations from a graph specification (or vice versa). this could for example be very useful in the approach suggested in [ektw06] to generate an equivalent graph grammar specification for a given input metamodel. 5.3 alternative graph transformation tools many different tools for graph transformation exist. they all have their own specific reason of existence, and can be considered to be complementary in some sense. we already mentioned the progres [saz99] and fujaba tool6 [nz00] that offer built-in support for controlled graph transformation. viatra2 [bv06] and great [akn+06] are two other tools that perform sophisticated support for dealing with complex graph transformations. the moflon meta modeling framework7 [akrs06] is an extension of fujaba that additionally supports triple graph grammars [ks06], a technique that can be quite useful for synchronising different model views (such as class diagrams and state machines, for example). the moflon tool also offers a more standardised way to represent uml models, due to the similarities between the uml metamodel and the moflon concepts. atom3 is a domain-specific modeling tool based on graph transformation8. as such, it combines the virtues of visual concrete syntax with abstract graph transformations. we could continue our list by discussing other graph transformation tools such as motmot, groove, grgen, and so on. 6 conclusion [mt04] provided a detailed survey of research on software refactoring, and suggested model refactoring as one of the future challenges. in this article, we have shown how the formalism of graph transformation can be used as an underlying foundation for the specification of model 6 http://www.fujaba.de 7 http://www.moflon.org 8 http://atom3.cs.mcgill.ca proc. software evolution 2007 12 / 15 http://www.fujaba.de http://www.moflon.org http://atom3.cs.mcgill.ca eceasst refactoring. we have developed a prototype application in order to verify the feasibility of graph transformations for the purpose of model refactorings. the prototype application shows that it is possible to develop model refactoring tools this way. however, it is necessary to improve current graph transformation tool support in order to better support the specification of model refactorings. future work should formally explore the characteristics of model refactoring paying more attention on the preservation of the behaviour. model refactoring is a rather recent research issue and such definitions of behaviour preservation properties have not yet been completely given. there are some proposals about behaviour preservation but, in the context of the uml, such definitions do not exist because there is no consensus on a formal definition of behaviour. a uml model is composed of different diagrams that address different aspects of a software system. the application of model refactorings may generate inconsistencies between these uml diagrams. future work should explore the possibility to preserve the consistency among different kind of uml models after the application of model refactoring expressing inconsistency detections and their resolutions as graph transformation rules. mens, van der straeten and d’hondt [mvd06] propose to express inconsistency detection and resolutions as graph transformation rules, and to apply the theory of critical pair analysis to analyse potential dependencies between the detection and resolution of model inconsistencies. references [akn+06] a. agrawal, g. karsai, s. neema, f. shi, a. vizhanyo. the design of a language for model transformations. journal on software and system modeling 5(3):261–288, september 2006. [akrs06] c. amelunxen, a. königs, t. rötschke, a. schürr. moflon: a standardcompliant metamodeling framework with graph transformations. in rensink and warmer (eds.), model driven architecture foundations and applications: second european conference. lecture notes in computer science (lncs) 4066, pp. 361– 375. springer verlag, heidelberg, 2006. [bek+06] e. biermann, k. ehrig, c. köhler, g. kuhns, g. taentzer, e. weiss. graphical definition of in-place transformations in the eclipse modeling framework. in proc. int’l conf. model driven engineering languages and systems (models). lecture notes in computer science. springer-verlag, 2006. [bnn+07] d. balasubramanian, a. narayanan, s. neema, b. ness, f. shi, r. thibodeaux, g. karsai. applying a grouping operator in model transformations. in proc. applications of graph transformations with industrial relevance (agtive). pp. 406–421. wilhelmshöhe, kassel, germany, 2007. [bv06] a. balogh, d. varró. advanced model transformation language constructs in the viatra2 framework. in proc. 21st acm symposium on applied computing. pp. 1280– 1287. acm press, april 2006. 13 / 15 volume 8 (2008) refactoring of uml models using agg [dhj+06] f. drewes, b. hoffmann, d. janssens, m. minas, n. v. eetvelde. adaptive star grammars. in proc. int’l conf. graph transformation (icgt). lecture notes in computer science 4178, pp. 77–91. springer verlag, 2006. [eekr99] h. ehrig, g. engels, h.-j. kreowski, g. rozenberg (eds.). handbook of graph grammars and computing by graph transformation. volume 2: applications, languages and tools. world scientific, october 1999. [eetw06] c. ermel, k. ehrig, g. taentzer, e. weiss. object-oriented and rule-based design of visual languages using tiger. in proc. workshop on graph-based tools (grabats). electronic communications of the easst 1. 2006. [ekmr99] h. ehrig, h.-j. kreowski, u. montanari, g. rozenberg (eds.). handbook of graph grammars and computing by graph transformation. volume 3: concurrency, parallelism and distribution. world scientific, september 1999. [ektw06] k. ehrig, j. küster, g. taentzer, j. winkelmann. generating instance models from meta models. in 8th ifip international conference on formal methods for open object-based distributed systems (fmoods’06). lecture notes in computer science 4037, pp. 156–170. springer verlag, 2006. [em93] h. ehrig, michael löwe. parallel and distributed derivations in the single-pushout approach. theoretical computer science 109:123–143, 1993. [fol07] a. folli. uml model refactoring using graph transformation. master’s thesis, institut d’informatique, université de mons-hainaut, 2007. [fow99] m. fowler. refactoring: improving the design of existing code. addison-wesley, boston, ma, usa, 1999. [gz04] l. geiger, a. zündorf. statechart modeling with fujaba. electronic notes in theoretical computer science, 2004. [heet99] r. heckel, g. engels, h. ehrig, g. taentzer. classification and comparison of module concepts for graph transformation systems. pp. 669–689. world scientific publishing co., inc., river edge, nj, usa, 1999. [kk99] h.-j. kreowski, s. kuske. graph transformation units and modules. pp. 607–638. world scientific publishing co., inc., river edge, nj, usa, 1999. [ks06] a. königs, a. schürr. tool integration with triple graph grammars a survey. in heckel (ed.), proceedings of the segravis school on foundations of visual modelling techniques. electronic notes in theoretical computer science 148, pp. 113–150. elsevier science publ., amsterdam, 2006. [men06] t. mens. on the use of graph transformations for model refactoring. in ralf lämmel (ed.), generative and transformational techniques in software engineering. lecture notes in computer science 4143, pp. 219–257. springer, 2006. proc. software evolution 2007 14 / 15 eceasst [mt04] t. mens, t. tourwé. a survey of software refactoring. ieee trans. software engineering 30(2):126–162, february 2004. [mtr07] t. mens, g. taentzer, o. runge. analysing refactoring dependencies using graph transformation. software and systems modeling, pp. 269–285, september 2007. doi:10.1007/s10270-006-0044-6 [mvd06] t. mens, r. van der straeten, m. d’hondt. detecting and resolving model inconsistencies using transformation dependency analysis. in nierstrasz et al. (eds.), model driven engineering languages and systems. lecture notes in computer science 4199, pp. 200–214. springer-verlag, october 2006. [nz00] j. niere, a. zündorf. using fujaba for the development of production control systems. in nagl et al. (eds.), proc. applications of graph transformations with industrial relevance (agtive). lecture notes in computer science 1779, pp. 181–191. springerverlag, 2000. [obj05] object management group. unified modeling language: superstructure version 2.0. formal/2005-07-04, august 2005. [opd92] w. f. opdyke. refactoring: a program restructuring aid in designing objectoriented application frameworks. phd thesis, university of illinois at urbanachampaign, 1992. [roz97] g. rozenberg (ed.). handbook of graph grammars and computing by graph transformation. volume 1: foundations. world scientific, february 1997. [saz99] a. schürr, andreas winter, a. zündorf. the progres approach: language and environment. pp. 487–550. world scientific publishing co., inc., river edge, nj, usa, 1999. [sk03] s. sendall, w. kozaczynski. model transformation: the heart and soul of modeldriven software development. ieee software 20(5):42–45, 2003. special issue on model-driven software development. [tae04] g. taentzer. agg: a graph transformation environment for modeling and validation of software. in proc. applications of graph transformations with industrial relevance (agtive). lecture notes in computer science 3062, pp. 446–453. springerverlag, 2004. [vv07] g. varró, d. varró. recursive graph pattern matching. in proc. applications of graph transformations with industrial relevance (agtive). pp. 453–467. wilhelmshöhe, kassel, germany, 2007. 15 / 15 volume 8 (2008) http://dx.doi.org/10.1007/s10270-006-0044-6 introduction motivating example formal representation as graph transformation tool support discussion limitations of agg limitations of graph transformation tools in general alternative graph transformation tools conclusion selbstorganisierendes service level management basierend auf mechanismus-design electronic communications of the easst volume 17 (2009) workshops der wissenschaftlichen konferenz kommunikation in verteilten systemen 2009 (wowkivs 2009) selbstorganisierendes service level management basierend auf mechanismus-design bernhard jungk 12 pages guest editors: m. wagner, d. hogrefe, k. geihs, k. david managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst selbstorganisierendes service level management basierend auf mechanismus-design bernhard jungk1 1fachhochschule wiesbaden university of applied sciences, labor für verteilte systeme, kurt-schumacher-ring 18, d-65197 wiesbaden jungk@informatik.fh-wiesbaden.de, http://wwwvs.informatik.fh-wiesbaden.de abstract: immer komplexere it-infrastrukturen führen zu immer komplexeren itmanagementsystemen. selbstorganisierende systeme stellen einen möglichen ansatz zum umgang mit der zunehmenden komplexität dar. solche systeme konfigurieren sich selbst, optimieren automatisch die leistung des systems und reduzieren dadurch die sonst nötigen manuellen eingriffe. in diesem beitrag wird basierend auf mechanismus-design, eine teildisziplin der spieltheorie, ein selbstorganisierendes managementsystem für service-orientierte systeme beschrieben. das beschriebene, sogenannte slo-spiel, wird theoretisch sowie mit hilfe von simulationen untersucht und bewertet. die ergebnisse zeigen eine prinzipielle eignung des mechanismus für das management, wobei allerdings weitere verbesserungen für den realen einsatz untersucht werden müssen. keywords: service-oriented architecture, self-organisation, mechanism design 1 einführung die größe und komplexität von it-systemen, sowie die abhängigkeit vieler unternehmen von diesen nimmt immer weiter zu. deshalb wird es immer wichtiger ein durchgängiges it-management über systemund unternehmensgrenzen hinweg einzuführen. um dieses ziel zu erreichen, wurde das konzept des service level managements (slm) eingeführt. ein zentrales element des slm ist ein sogenanntes service level agreement (sla), welches einen vertrag zwischen anbieter eines services und dessen nutzer darstellt. dieser vertrag definiert für den angebotenen service bestimmte leistungskriterien in form von service level objectivs (slo), an deren einhaltung sich der service-anbieter bindet (vgl. [deb04]). um die dienstgütekriterien eines slas einhalten zu können, muss auf ereignisse in der itinfrastruktur durch geeignete konfigurationsänderungen für die einzelnen teilsysteme reagiert werden. manuelles management stößt hier durch die zunehmende größe, komplexität und erforderlichen kurzen reaktionszeiten an grenzen. sogenanntes selbstmanagement kann dieses management in teilbereichen automatisieren und löst damit das manuelle management ab (vgl. [sk05]). die zunehmende komplexität der systeme steigert auch den zur konfiguration des managementsystems nötigen aufwand, weshalb man aktuell die entwicklung von selbstorganisierenden 1 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design systemen vorantreibt (vgl. [mwj+07]). diese systeme verändern nicht nur konfigurationsparameter, sondern auch die struktur des managementsystems. bisherige managementsysteme sind im gegensatz dazu allerdings meist zentraler und statischer natur (z.b. [ybt05]) und besitzen deshalb eine reihe von nachteilen, z.b. probleme bei der skalierbarkeit oder geringe fehlertoleranz bei partitionierungen der kommunikationswege. ein möglicher ansatz für selbstorganisierende managementsysteme basiert auf dem algorithmischen mechanismus-design, einem teilgebiet der spieltheorie (vgl. [nrtv07]). modelliert werden sogenannte rationale agenten, die stets versuchen den eigenen nutzen zu maximieren. dieses verhalten ist beispielsweise zu beobachten, wenn das management über administrative domänengrenzen hinweg erfolgen soll. jede domäne versucht das management üblicherweise so zu gestalten, dass ihr eigener nutzen optimiert wird. weiterhin wird durch das algorithmische mechanismus-design die algorithmische komplexität der entworfenen mechanismen untersucht, damit ein entworfener mechanismus auch praktisch umsetzbar ist. eine parallel zur entwicklung immer komplexerer managementsysteme verlaufende entwicklung, ist die zunehmende verbreitung von service-orientierten systemen (vgl. [vas07]). ein kernbestandteil solcher systeme sind workflows, die bestimmte geschäftsprozesse eines unternehmens modellieren. ein workflow besteht aus verschiedenen aktivitäten, die in einem gerichteten graphen angeordnet sind. dieser beitrag beschreibt einen selbstorganisierenden mechanismus für das slm von solchen service-orientierten systemen auf basis des algorithmischen mechanismus-designs. dieser mechanismus soll immer dann aktiv werden, wenn bestimmte leistungskriterien eines services nicht eingehalten werden. prinzipiell eignet sich dieser ansatz für viele unterschiedliche leistungskriterien, allerdings betrachtet der beitrag die antwortzeit als einziges leistungskriterium, um die prinzipielle eignung zu zeigen. in abschnitt 2 wird das modell eines service-orientierten systems beschrieben, wie es im weiteren verlauf des beitrags genutzt wird. außerdem werden mögliche managementaktionen innerhalb dieses systems aufgezeigt. abschnitt 3 definiert das verwendete modell des mechanismus-designs, welches genutzt wird, um den mechanismus, das sogenannte slo-spiel, zu entwerfen (abschnitt 4). im weiteren verlauf werden theoretische ergebnisse vorgestellt und das slo-spiel mittels verschiedener simulationen bewertet (abschnitt 5). abschnitt 6 gibt eine zusammenfassung und einen ausblick über weitere ergänzende arbeiten, damit der vorgestellte mechanismus alle gängigen anforderungen erfüllt. 2 system-modell im folgenden werden ein service-orientiertes system und seine eigenschaften beschrieben. formal wird dieses modell in [jun08] definiert. ein solches system wird in zwei schichten beschrieben, einer logischen sicht, bestehend aus workflows und einer ausführungsschicht, bestehend aus services. die workflow-beschreibung ist an das in [tex08] genutzte workflow-modell angelehnt. jeder workflow besteht aus einer menge von aktivitäten, die in einem gerichteten azyklischen graph angeordnet sind. ein workflow-graph wird aus den im folgenden beschriebenen elementen gebildet. die zusammengesetzten elemente können dabei beliebig weiter geschachtelt werden. proc. wowkivs 2009 2 / 12 eceasst a 2a 1 a 3 a 4 a 5 a 6 a 7 loop (10) abbildung 1: ein beispielworkflow mit mehreren sequenzen, einer parallelausführung und einer schleife. die grundlegenden elemente sind atomare aktivitäten. jede atomare aktivität wird von genau einem service ausgeführt. zusammengesetzte aktivitäten dienen der logischen strukturierung eines workflows. möglich sind dabei sequenzen und parallelausführungen. zudem sind schleifen als abkürzende schreibweise möglich, wenn eine atomare oder zusammengesetzte aktivität endlich oft wiederholt ausgeführt wird. dazu gibt man mit einem schleifengewicht n an, wie oft eine schleife maximal durchlaufen wird. abbildung 1 zeigt einen beispielworkflow, der alle möglichen elemente nutzt. für ein effektives slm muss weiterhin definiert werden, welche eigenschaften des systems im rahmen des managements betrachtet werden sollen. relevant ist dabei der inhalt von slas und slos. zunächst wird das slm auf die betrachtung der antwortzeit eingeschränkt, um ein einfaches problem zu erhalten. ein sla-zielwert beschreibt die maximale gesamtantwortzeit eines workflows, ein slo-zielwert die maximale antwortzeit einer aktivität. um von der maximalen antwortzeit für den gesamten workflow zu einem zielwert für einzelne aktivitäten zu kommen, wird der sla-zielwert auf die einzelnen aktivitäten eines workflows aufgeteilt, wodurch die zielwerte für die einzelnen slos entstehen (vgl. [sk08], [mik08]). das ziel des managements ist, die antwortzeit für alle aktivitäten immer geringer als das jeweils zugewiesene slo zu halten. ist dies nicht der fall, so tritt eine slo-verletzung auf. als maßnahme müssen geeignete änderungen am system durchgeführt werden, so dass diese bedingung zu einem späteren zeitpunkt wieder erfüllt wird. geeignete managementaktionen müssen eines der folgenden beiden resultate bewirken: • lockerung des slos • verkürzung der antwortzeit die lockerung des slos erhöht dessen zielwert für die antwortzeit, so dass das slo wieder eingehalten wird, ohne dass die antwortzeit verbessert wird. da die summe aller slo-zielwerte in einem pfad durch den workflow nicht größer als der aus dem sla stammende zielwert sein darf, muss außerdem der slo-zielwert für mindestens eine andere aktivität gestrafft werden (vgl. [sch07]). der betrag um den ein slo-zielwert gelockert, bzw. gestrafft wird, wird im weiteren verlauf slo-anteil genannt. die zweite möglichkeit sind managementaktionen die zum ziel haben, die antwortzeit einer bestimmten aktivität zu verkürzen. dabei wird das slo nicht verändert. die generelle vorgehensweise ist dabei, dass dem service zur ausführung der aktivität mehr ressourcen bereitgestellt werden. da der zusammenhang zwischen tatsächlicher antwortzeit und ressourcenzuordnung sehr komplex ist (vgl. [mar08]), abstrahiert dieser beitrag davon, indem davon ausge3 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design gangen wird, dass man antwortzeitanteile zwischen aktivitäten verschieben kann, sofern diese vom gleichen service ausgeführt werden. d.h. antwortzeitanteile werden als ersatz für jegliche andere ressource betrachtet. eine zusätzliche möglichkeit, die im weiteren verlauf allerdings nicht betrachtet wird, ist die gezielte kombination der beiden managementaktionen. die lockerung von slos kann nur innerhalb eines workflows erfolgen, die verkürzung der antwortzeit nur zwischen zwei aktivitäten, die durch den gleichen service ausgeführt werden. eine kombination dieser möglichkeiten kann nun genutzt werden, um zwischen aktivitäten unterschiedlicher workflows slobzw. antwortzeitanteile zu verschieben, welche nicht durch den gleichen service ausgeführt werden. dazu kann eine aktivität als händler auftreten, welche beispielsweise das eigene slo lockert, um anschließend antwortzeit-anteile an eine aktivität zu vergeben, welche durch den gleichen service ausgeführt wird. führt keine mögliche aktion zur behebung einer slo-verletzung, muss diese an eine höhere managementebene eskaliert werden. 3 mechanismus-design nach der beschreibung des systems werden nun die grundlagen für den management-mechanismus beschrieben, welcher auf der basis des mechanismus-designs entworfen wurde. das dem mechanismus-design zugrunde liegende modell wird unter anderem in [nr01, nrtv07, ste08] detailliert beschrieben. in der spieltheorie sind folgende schreibweisen üblich (vgl. [nrtv07, ste08]): • ein vektor v−i = (v1,··· , vi−1, vi+1,··· , vn) enthält jedes element des ursprünglichen vektors v, ausgenommen des eintrags i. • ein vektor v kann auch als v = (vi, v−i) geschrieben werden. definition 1 (mechanismus-design problem) ein mechanismus-design problem wird durch ein gewünschtes ergebnis beschrieben, das von einer anzahl rationaler agenten erreicht werden soll. dabei gelten folgende annahmen: • ein mechanismus ist ein tupel m =de f (a, s, o, f , p1,··· , pn), mit einer endlichen menge agenten a ⊂ n, einer menge möglicher strategien s, einer menge möglicher ergebnisse o, der ergebnisfunktion f : sn → o und zahlungsfunktionen pi : sn → r für alle agenten i ∈ a. • jeder agent i besitzt private informationen, die im sogenannten typ ti ∈ ti wiedergespiegelt werden. die menge ti ist dabei für den agenten i eine teilmenge aller möglichen typen t. • jeder agent i hat mehrere mögliche strategien si ⊆ s. eine strategie j ist definiert als eine funktion s ji : t × s n−1 → si. anschaulich ist die strategie j des agenten i vom eigenen typ und von den strategien aller anderen agenten abhängig. proc. wowkivs 2009 4 / 12 eceasst • für jeden strategievektor s = (s1,··· , sn) existiert ein ergebnis o ∈ o, das durch die ergebnisfunktion f berechnet wird. • die ergebnisfunktion wird auch soziale entscheidungsfunktion genannt. eine soziale entscheidungsfunktion heißt effizient, wenn sie ein bestimmtes kriterium maximiert. • für jeden agenten hat ein ergebnis einen bestimmten wert, der durch eine wertfunktion vi : ti × o → r beschrieben wird. zusammen mit einer vom mechanismus zugeteilten bezahlung pi ergibt sich der tatsächliche nutzen für einen agenten i durch die lineare nutzenfunktion ui: ui(o, pi,ti) =de f vi(ti, o)− pi ein rationaler agent versucht, diese nutzenfunktion zu maximieren (vgl. [nrtv07]). ein beispiel für einen mechanismus ist die verallgemeinerte vickrey-auktion (vgl. [ac04]), welche zu einer wichtigen klasse von mechanismen, den sogenannten vickrey-clarke-grovesmechanismen (vcg) gehört (vgl. [nrtv07]). definition 2 die verallgemeinerte vickrey-auktion (gva) ist eine auktionsform und damit ein mechanismus m, durch die eine menge von diskreten objekten o versteigert wird. jeder bieter i gibt für jedes objekt o ∈ o ein gebot bi ab. weiterhin legt der verkäufer der objekte einen reservierungspreis pr fest, den jeder bieter für jedes objekt mindestens bieten muss. die ermittlung des ergebnisses erfolgt so, dass der soziale nutzen maximiert wird: f (v1,··· , vn) ∈ maxo∈o n ∑ i=1 vi(o) die ermittlung von preisen wird mit der sogenannten clarke-pivot-regel durchgeführt (vgl. [nrtv07]). der reservierungspreis kann dabei als gebot für alle objekte betrachtet werden. der zu zahlende preis pi lässt sich wie folgt berechnen: pi = maxc∈o ∑ j 6=i v j(c,t j)− ∑ j 6=i v j(o,t j) da die gva ein vcg-mechanismus mit clarke-pivot-regel ist, besitzt dieser mechanismus verschiedene nützliche eigenschaften, darunter individuelle rationalität, anreizkompatibilität und effizienz (vgl. [ac04, nrtv07]): 4 das slo-spiel bisher wurde beschrieben, aus welchen bestandteilen ein system besteht, welche ziele der slmprozess erreichen soll und welche managementaktionen möglich sind, um diese ziele zu erreichen. weiterhin wurde das abstrakte konzept eines mechanismus definiert und mit dem beispiel der gva veranschaulicht. 5 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design das slo-spiel kombiniert nun alle beschriebenen teile in einem management-mechanismus, der mittels auktionen slooder antwortzeitanteile zwischen verschiedenen aktivitäten umverteilt. im weiteren verlauf werden folgende varianten des slo-spiels untersucht. • das einfache slo-spiel lockert slos bzw. strafft diese innerhalb eines workflows. • das verallgemeinerte slo-spiel implementiert außerdem den transfer von antwortzeitanteilen innerhalb eines services zwischen verschiedenen aktivitäten. am slo-spiel nehmen alle aktivität des systems separat teil, d.h. die menge der agenten entspricht der menge aller aktivitäten im system. es wird also für jede aktivität unabhängig eine entscheidung getroffen, welche managementaktion durchgeführt werden soll. da der mechanismus mittels auktionen funktionieren soll, wird ein agent zu einem bieter, sofern er slooder antwortzeitanteile kaufen muss, d.h. wenn die antwortzeit den slo-zielwert übersteigt. ansonsten wird der agent zu einem verkäufer. jeder verkäufer führt anschließend auf basis der gva eine auktion durch. parallel dazu sucht jeder käufer nach aktiven auktionen, wählt eine davon aus, berechnet sein mögliches gebot und gibt das berechnete gebot in der ausgewählten auktion ab. anschließend berechnet jeder verkäufer das ergebnis der eigenen auktion, d.h. eine zuordnung von slound antwortzeitanteilen an die agenten, welche ein gebot abgegeben haben. dieses ergebnis wird den einzelnen bietern anschließend mitgeteilt. eine wichtige frage bleibt, wie findet ein käufer passende auktionen? aus der analyse der möglichen managementaktionen stellen sich folgende möglichkeiten dar (vgl. [jun08]): • slo-anteile können innerhalb von sequenzen umverteilt werden. • antwortzeitanteile können nur innerhalb eines services umverteilt werden. die umverteilung von antwortzeitanteilen geschieht lokal innerhalb eines services, deshalb kann auf alle informationen immer zugegriffen werden. somit ist das auffinden von auktionen innerhalb dieses services trivial über eine globale datenstruktur innerhalb des services implementierbar. für die umverteilung von slo-anteilen wird allerdings eine kompliziertere kommunikationsstruktur benötigt. es wird dazu ein gruppenkommunikationssystem genutzt (vgl. [ckv01]). jeweils eine gruppe wird pro zusammengesetzter aktivität gebildet, diese gruppe bildet sich selbständig aus allen enthaltenen aktivitäten über einen gemeinsamen gruppennamen, der aus der ursprünglichen workflowbeschreibung abgeleitet wird. in jeder dieser gruppen wird anschließend ein vertreter ausgewählt, der diese gruppe in der übergeordneten gruppe vertritt. um über dieses gruppenkommunikationssystem auktionen zu finden, sendet jeder käufer in seiner gruppe eine multicast-nachricht an alle gruppenmitglieder. alle verkäufer in dieser gruppe werden auf diese anfrage antworten. den vertretern innerhalb der zusammengesetzten aktivität kommt eine sonderrolle zu, sie sammeln alle performance-daten der in der zusammengesetzten aktivität enthaltenen aktivitäten. falls der vertreter an einer übergeordneten sequenz beteiligt ist, kann er auf basis der gesammelten performance-daten slo-anteile für die zusammengesetzte aktivität kaufen oder verkaufen. die jeweiligen anteile werden anschließend geeignet unter den beteiligten aktivitäten aufgeteilt (vgl. [jun08]). proc. wowkivs 2009 6 / 12 eceasst 5 bewertung um die qualität des slo-spiels aus theoretischer sicht beurteilen zu können, muss zunächst ein effizienzkriterium definiert werden, welches das slo-spiel erreichen soll. grundsätzlich lassen sich zwei effizenzkriterien unterscheiden. für das erste effizienzkriterium wird jedem workflow ein wert zugeordnet, der sich aus dem nutzen des workflows für den service-anbieter ableitet. für einen bestimmten zeitpunkt ist der gesamtwert des systems die summe der werte aller workflows, deren sla seit dem letzten zeitpunkt der wertermittlung eingehalten wurde. effizient wäre das ergebnis des slo-spiels, wenn es den gesamtwert des systems maximiert. da der vorliegende mechanismus dieses effizienzkriterium nicht erreichen kann (vgl. [jun08]), wird im folgenden ein zweites, schwächeres kriterium vorgestellt: definition 3 ein ergebnis o ∈ o ist dann in bezug auf einen workflow w effizient, wenn alle slos aller an w beteiligten aktivitäten a eingehalten werden. dies ist dann möglich, wenn alle pfade durch w eine gesamtausführungszeit kleiner dem zielwert des slas besitzen. bei der betrachtung des einfachen und des verallgemeinerten slo-spiels, stellt man die beiden folgenden eigenschaften fest. die vollständigen beweise finden sich in [jun08]. theorem 1 das einfache slo-spiel erreicht nach einer endlichen anzahl von auktionen immer ein nach definition 3 effizientes ergebnis, sofern alle pfade p durch den workflow w eine gesamtausführungszeit kleiner als die vorgegebene ausführungszeit des workflow-slas haben. theorem 2 das verallgemeinerte slo-spiel erreicht nicht garantiert die effizienz des einfachen slo-spiels. die beweisidee zu theorem 1 basiert auf einer geschickten wahl der wertfunktion für bieter, so dass es für die bieter immer möglich ist, in endlicher zeit die benötigten anteile zu kaufen, sofern irgendeine aktivität in der gleichen sequenz genügend anteile zur verfügung hat. der beweis von theorem 2 setzt voraus, dass zwei workflows mindestens einen service gemeinsam nutzen. dadurch kann es passieren, dass ein workflow dem anderen workflow benötigte antwortzeitanteile abkauft. hinterher können beide workflows schlechter dastehen, falls die abgekauften antwortzeitanteile für den kaufenden workflow nicht ausreichen und der verkaufende workflow nun ebenfalls nicht mehr genug antwortzeitanteile besitzt. um außerdem eine praktische bewertung zu ermöglichen, wurde eine simulation erstellt, welche das slo-spiel umsetzt. die prinzipielle architektur der simulation folgt abbildung 2. die kernkomponente ist der service-manager, welchem jeweils genau ein service und eine strategie zugeordnet wird. von außerhalb der simulation wird dem service-manager mitgeteilt welche aktivitäten eines workflows der service ausführt und jeder aktivität jeweils ein slo zugeordnet, welches der service-manager verwaltet. der service-manager überwacht zur laufzeit der simulation den ihm zugeteilten service und initiiert gegebenfalls managementaktionen, welche durch die strategie umgesetzt werden. die in der simulation eingesetzte strategie implementiert das beschriebene slo-spiel und nimmt somit einfluss auf den service oder die vom service-manager verwalteten slos. die 7 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design service-managerservice-manager m a n a g e m e n t n a c h r i c h t e n serviceservice performancesensor performancesensor s t r a t e g i es t r a t e g i e slosslos performanced a t e n performanced a t e n m a n a g e m e n t a k t i o n e n slo ä n d e r u n g e n m a n a g e m e n t a k t i o n e n abbildung 2: die architektur der simulation. strategie kommuniziert bei der umsetzung der managementaktionen mit den strategien anderer service-manager auf basis eines gruppenkommunikationssystems (vgl. [ckv01]), welches gemäß der beschreibung in abschnitt 4 implementiert ist. ein service innerhalb der simulation generiert aus einem vorgegebenen performance-modell performance-daten, im vorliegenden fall simulierte antwortzeiten. die performance-daten werden mittels eines performance-sensors an den service-manager übertragen, welcher diese daten nutzt, um managementaktionen über die strategie anzustoßen. es wurden insgesamt drei szenarien untersucht, die hier der reihe nach mit den jeweiligen ergebnissen vorgestellt werden. das erste szenario stellt sich folgendermaßen dar: • es wird der einschwingvorgang des systems mit 10 bzw. 100 knoten simuliert. dabei verletzten jeweils 50% der knoten initial das slo und 50% nicht. die gesamtantwortzeit ist dabei immer geringer als der sla-zielwert für den gesamten workflow. das ziel dieses szenarios ist zu untersuchen, ob das slo-spiel praktisch immer eine lösung findet, d.h. ob das theoretische effizienzkriterium tatsächlich eingehalten wird. gemessen werden kann dies durch die anzahl der bieter, da jede einzelne slo-verletzung da 0 1 2 3 4 5 0 2 4 6 8 10 12 a n za h l a kt iv e r b ie te r zeit [100 ms] (a) 10 0 10 20 30 40 50 0 20 40 60 80 100 120 a n za h l a kt iv e r b ie te r zeit [100 ms] (b) 100 abbildung 3: einschwingverhalten aus sicht der bieter mit 10 bzw. 100 agenten. proc. wowkivs 2009 8 / 12 eceasst 0 20 40 60 80 100 120 0 0.5 1 1.5 2 a n za h l n a ch ri ch te n zeit [1 s] (a) 10 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 0 2 4 6 8 10 12 a n za h l n a ch ri ch te n zeit [1 s] (b) 100 abbildung 4: nachrichten-overhead beim einschwingverhalten mit 10 bzw. 100 agenten. zu führt, dass genau ein bieter existiert. der umkehrschluss gilt auch, d.h. gibt es keine bieter, ist auch keine slo-verletzung vorhanden. deshalb wird hier die anzahl der bieter gegenüber dem zeitverlauf der simulation dargestellt. die ergebnisse der simulation in abbildung 3 zeigen, dass das slo-spiel tatsächlich immer eine lösung findet. anhand des zeitverlaufs lässt sich außerdem eine grenze der skalierbarkeit erkennen. in diesem szenario erkennt man, dass das slo-spiel mit der 10-fachen menge an agenten deutlich mehr zeit benötigt. dies lässt sich mit einer ebenfalls stark zunehmenden anzahl zu versendender nachrichten erklärten, denn jeder käufer muss an alle anderen agenten nachrichten verschicken (vgl. abbildung 4). das zweite szenario soll überprüfen, wie sich das slo-spiel bei hohen lasten verhält, d.h. wenn es keine garantierte lösung innerhalb eines workflows gibt: • das verhalten bei dynamischen änderungen und hoher last wird simuliert, d.h. es wird häufig das sla verletzt. es wird wieder mit 10 und 100 knoten gemessen. die änderung der antwortzeiten der services geschieht periodisch mit leicht zufälliger verzögerung. 0 1 2 3 4 5 0 500 1000 1500 2000 a n za h l a kt iv e r b ie te r zeit [100 ms] (a) 10 0 5 10 15 20 25 30 35 40 0 500 1000 1500 2000 a n za h l a kt iv e r b ie te r zeit [100 ms] (b) 100 abbildung 5: verhalten bei dynamischen änderungen und 10 bzw. 100 agenten. 9 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design 0 500 1000 1500 2000 2500 0 50 100 150 200 a n za h l n a ch ri ch te n zeit [1 s] (a) 10 0 5000 10000 15000 20000 25000 30000 35000 0 50 100 150 200 a n za h l n a ch ri ch te n zeit [1 s] (b) 100 abbildung 6: nachrichten-overhead bei dynamischen änderungen und 10 bzw. 100 agenten. dieses szenario soll aufschluss über das verhalten des systems im grenzfall geben, wenn das system sehr stark ausgelastet ist. wünschenswert ist in diesem fall, dass das managementsystem die performance auf keinen fall verschlechtert. da keine lösung für alle slo-verletzungen gefunden werden kann, ist die frage natürlich ob und wie das managementsystem das eigentliche system beeinflusst. eine wesentliche kenngröße dabei ist der erzeugte nachrichten-overhead, welcher möglichst gering ausfallen sollte, um das system nicht weiter zu belasten. in den auswertungen ist zu sehen (abbildung 5), dass sehr häufig keine lösung gefunden werden kann. dies zeigt sich deutlich an den plateaus in abbildung 5(b), jeweils nach einer kurzen verbesserungsphase zuvor. der nachrichten-overhead im falle der dynamischen änderungen bei sehr hoher last explodiert allerdings regelrecht (abbildung 6). die bei 100 agenten erzeugte sehr hohe spitzenlast würde in der realität die dienstgüte vermutlich weiter deutlich verschlechtern. erklären lässt sich dieses phänomen durch die art und weise, wie die käufer nach auktionen suchen. jeder bieter versendet dazu an alle anderen agenten eine nachricht. der nachrichtenaufwand wächst also quadratisch mit der anzahl der agenten. dieses verfahren lässt sich vermutlich abändern, so dass der nachrichtenaufwand wesentlich geringer ausfällt. die prinzipielle idee ist dabei, dass sich der nachrichtenfluss auch umkehren lässt, d.h. statt der suchanfrage der bieter, senden die verkäufer ein broadcast an alle agenten. da im schlechtesten fall keine agenten mehr verkäufer sind, tendiert der zusätzliche netzwerk-overhead gegen 0 (vgl. [jun08]). als drittes szenario soll das zweite ergebnis der effizienzbetrachtung praktisch untersucht werden, d.h. es sind zwei workflows vorhanden, deren aktivitäten sich services teilen: • es existiert ein workflow mit sehr hoher priorität, sowie einer mit sehr niedriger priorität. beide workflows teilen sich mehrere services. es gibt es nur ein effizientes ergebnis, wenn der workflow mit niedrigerer priorität alle slos und damit sein sla einhält. wie man in abbildung 7 erkennen kann, verletzen beide workflows das sla. das theoretische ergebnis wird also bestätigt. es zeigt sich, dass eine rein lokale optimierung des eigenen nutzen nicht ausreicht, um in jedem fall eine verbesserung zu erzielen. proc. wowkivs 2009 10 / 12 eceasst 0 1 2 3 4 5 0 5 10 15 20 25 30 a n za h l a kt iv e r b ie te r zeit [100 ms] (a) workflow 1 1 2 3 4 5 6 7 8 9 10 0 5 10 15 20 25 30 a n za h l a kt iv e r b ie te r zeit [100 ms] (b) workflow 2 abbildung 7: verhalten der bieter zweier workflows mit gemeinsam genutzten services im schlechtesten fall. diese problematik kann eventuell durch zusätzliche kommunikation zwischen den agenten verringert werden. ein effektives kommunikationsschema muss noch entwickelt werden. 6 zusammenfassung und ausblick im vorliegeden beitrag wurde ausgehend von der motivation für selbstorganisierende systeme das sogenannte slo-spiel, ein einfacher mechanismus auf der basis von mechanismus-design, entwickelt. dieser ist prinzipiell dazu geeignet einige arten von slo-verletzungen zu beheben. als grundlage für die entwicklung wurde ein service-orientiertes system, ein ziel für das management und die grundlagen des mechanismus-designs beschrieben. auf dieser basis wurde das slo-spiel als spieltheoretischer mechanismus entwickelt, welcher mit hilfe von verteilten auktionen anteile an logischen ressourcen umverteilen kann. das slo-spiel nutzt dafür eine verteilte verallgemeinerte vickrey-auktion, da diese einige gewünschte spieltheoretische eigenschaften aufweist. für das entwickelte slo-spiel wurde anschließend gezeigt, dass es ein effizientes ergebnis erreichen kann, sofern ein ergebnis existiert. allerdings erreicht das slo-spiel in verallgemeinerter form dieses ergebnis nicht, sondern verschlechtert die situation sogar. die durchgeführte simulation bestätigt die theoretischen ergebnisse und zeigt auch den weg für weitere verbesserungen auf. in stark ausgelasteten systemen erzeugt das bisherige spiel einen sehr hohen netzwerk-overhead auf grund der genutzten kommunikationsstruktur zur selbstorganisation. in weiteren arbeiten können diese schwächen sowohl theoretisch als auch praktisch durch änderungen an der bestehenden simulation weiter untersucht werden. literatur [ac04] l. m. ausubel, p. cramton. vickrey auctions with reserve pricing. papers of peter cramton 99wpvic, university of maryland, department of economics peter cram11 / 12 volume 17 (2009) selbstorganisierendes slm basierend auf mechanismus-design ton, 2004. [ckv01] g. v. chockler, i. keidar, r. vitenberg. group communication specifications: a comprehensive study. acm comput. surv. 33(4):427–469, dezember 2001. [deb04] m. debusmann. modellbasiertes service level management verteilter anwendungssysteme. phd thesis, universität kassel, dezember 2004. [jun08] b. jungk. bewertung von selbstorganisationsmechanismen für managementkomponenten auf basis von simulationen und leistungsmessungen. master’s thesis, fh wiesbaden, fb design informatik medien, september 2008. [mar08] d. marinescu. design and evaluation of self-management approaches for virtual machine-based environments. master’s thesis, fh wiesbaden, fb design informatik medien, februar 2008. [mik08] a. mikula. automatisierte überwachung von dienstgütekriterien in soa workflows. diplomarbeit, fh wiesbaden, fb design informatik medien, august 2008. [mwj+07] g. mühl, m. werner, m. a. jaeger, k. herrmann, h. parzyjegla. on the definitions of self-managing and self-organizing systems. in kivs 2007 workshop: selbstorganisierende, adaptive, kontextsensitive verteilte systeme. springer, 2007. [nr01] n. nisan, a. ronen. algorithmic mechanism design. games and economic behavior 35(1-2):166–196, april 2001. [nrtv07] n. nisan, t. roughgarden, e. tardos, v. v. vazirani. algorithmic game theory. cambridge university press, new york, ny, usa, 2007. [sch07] m. schmid. ein ansatz für das service level management in dynamischen architekturen. in kivs 2007 kommunikation in verteilten systemen industriebeiträge, kurzbeiträge und workshops. pp. 255–266. vde verlag, march 2007. [sk05] m. schmid, r. kröger. selbstmanagement-ansätze im ebusiness-umfeld. pik praxis der informationsverarbeitung und kommunikation 28 / 4:211–216, 2005. [sk08] m. schmid, r. kröger. decentralised qos-management in service oriented architectures. in meier and terzis (eds.), dais. lecture notes in computer science 5053, pp. 44–57. 2008. [ste08] j. steimle. algorithmic mechanism design. springer-verlag, 2008. [tex08] a. textor. monitoring unternehmenskritischer anwendungen unter verwendung modellbasierter performance constraints. bachelor’s thesis, fh wiesbaden, fb design informatik medien, september 2008. [vas07] y. vasiliev. soa and ws-bpel. packt publishing, august 2007. [ybt05] j. yu, r. buyya, c.-k. tham. qos-based scheduling of workflow applications on service grids. technical report, melbourne, australia, 2005. proc. wowkivs 2009 12 / 12 einführung system-modell mechanismus-design das slo-spiel bewertung zusammenfassung und ausblick massively multiuser virtual environments using object based sharing electronic communications of the easst volume 17 (2009) workshops der wissenschaftlichen konferenz kommunikation in verteilten systemen 2009 (wowkivs 2009) massively multiuser virtual environments using object based sharing michael sonnenfroh, kim-thomas möller, marc-florian müller, michael schöttner and peter schulthess 12 pages guest editors: m. wagner, d. hogrefe, k. geihs, k. david managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst massively multiuser virtual environments using object based sharing michael sonnenfroh1, kim-thomas möller2, marc-florian müller2, michael schöttner2 and peter schulthess5 1 michael.sonnenfroh@uni-ulm.de 2http://www.cs.uni-duesseldorf.de/ag/bs institut für informatik universität düsseldorf 5 peter.schulthess@uni-ulm.de http://www-vs.uni-ulm.de institut für verteilte systeme universität ulm abstract: massively multiuser virtual environments (mmves) are becoming increasingly popular with millions of users. commercial implementations typically rely on a traditional client/server architecture controlling the virtual world state of shared data at a central point. message passing mechanisms are used to communicate state changes to the clients. for scalability reasons our approach creates and deploys mmves in a peer-to-peer (p2p) fashion. we use standard java technology implementing only a few basic data-centric operations for the management of our distributed objects. higher consistency models can easily be implemented using these basic operations. currently, we have implemented transactional consistency offering convenient and consistent access to the shared scene graph. in this paper we describe our basic object model and the prototype implementation tgos (typed grid object sharing). furthermore, we discuss preliminary measurements with the virtual world wissenheim executed on top of tgos. keywords: mmve, distributed objects, multiple consistencies 1 introduction massively multiuser virtual environments including games and online getaways are attracting more and more people, e.g. second life exceeded 9 millions of users at the end of the year 2007. most of these systems are based upon classical client-server architectures regarding the communications topology and the programming model. providers often argue that client-server systems are superior in controlling user accounts and cheating. the installed servers are rather huge clusters which introduce latency and scalability constraints. as a consequence providers typically host different virtual worlds for each continent. in 2004 the wissenheim project was started to explore virtual presence applications using transactional memory for distribution. using transactional memory to distribute and share a 1 / 12 volume 17 (2009) mailto:michael.sonnenfroh@uni-ulm.de http://www.cs.uni-duesseldorf.de/ag/bs mailto:peter.schulthess@uni-ulm.de http://www-vs.uni-ulm.de massively multiuser virtual environments using object based sharing scene graph allowed us to deploy a simpler programming model in the field of virtual world development abstracting from the structure of the underlying network. evolving from the original wissenheim project we have developed a more flexible and portable object model and system, based on sharing objects. the typed grid object sharing system (tgos) allows wissenheim to use a data centric view to coordinate distributed access to virtual worlds. an important design aspect during the development was the requirement to cope with the challenges of wide area deployment such as high latency and potential node failures. the opportunity to use our approach for peer-to-peer deployment [13] was another aspect not only influencing the network layer but the programming model as well. visitors may test the prototype shown in figure 2 at the wissenheim website [1]. this paper is organised as follows. in section 2 we describe the tgos object model and in the following section its prototype implementation. in section 4 we present transactional consistency as a method to keep replicated objects consistent. how wissenheim has been adapted to run on top of tgos is described in section 5. subsequently, we present preliminary measurement results followed by related work and conclusions. 2 typed grid object sharing the object model of tgos is divided into two different parts called the object view and the global data store as shown in figure 1. the object view provides an object oriented and typesafe local-view of distributed objects. object views on different nodes can show different subsets of objects with different data versions. the global data store provides storage and distribution mechanisms for data blocks of arbitrary size. the data store is unaware of the internal structure of the data blocks it is managing. this allows different implementations for a global data store. each data block is tagged by a unique and persistent id provided by the global data store. in contrast to the object view where different object versions can be found the global data store provides a consistent view over all data blocks storing only one version per data block. node 1 node 2 node n global data store a 1 a.2 a 2 b 8 b.8 c.3 c 3 a 2 object view object view object view ... figure 1: object model schematics. figure 2: wissenheim worlds screenshot. proc. wowkivs 2009 2 / 12 eceasst 2.1 object view each object view is able to transform an object into a serialised form and merge serialised data into another object of the same type. each object of a view is connected to exactly one data block in the global store identified by the global id. although different nodes have different views and different data versions the type system remains unchanged on all views. we are saying that an object a residing in object view n is alike to object b residing in object view n+1 if both objects have the same id. two objects are called equal if they are alike and if they have the same version of data. 2.2 global data store the global data store (short: data store) can be described as a basic replication layer responsible for distributing arbitrary data blocks between nodes. it provides functions to read, write, invalidate, and lock data blocks it is managing. the granularity of a data store operation is at least one data block meaning that for example a write is updating one or multiple blocks. the data store also ensures that all write operation are performed atomically preventing, in case of a failure during block transfer, that corrupted data gets visible. all write accesses performed on the store are ordered by fifo consistency [12]. lock acquisitions are made on a first-come-first-served basis. beside the replication management the data store is also responsible for generating persistent ids for newly created data blocks. the data block size is determined by the size of the serialised object and the data format used by the object view and therefore not defined by the data store. the implementation of a global data store is not defined by our object model but for scalability reasons we recommend a peer-to-peer approach. 2.3 events the object views can receive asynchronous events from the global data store informing them about data changes of objects residing in their view. we have defined an update event and an invalidate event (shown in figure 3) which are triggered through a data store write or an invalidate respectively. when the update event is triggered the object view receives the newest version of the written data block for an object from the data store. in case of an invalidate event the object view will receive only a notification that a data block corresponding to an object in the view has changed. 2.4 basic operations the object view has a small set of basic operations which can be performed on objects to modify their corresponding data block in the global data store. with these basic operations different consistency models can easily be built inside the object model. definition 1 (push:o) the push:o operation will extract the data of the given object o and write it back to the global object store. views on other nodes will receive an update event. the push operation will block until the object data has been extracted and written to the data store. 3 / 12 volume 17 (2009) massively multiuser virtual environments using object based sharing node 1 node 2 node n global data store a 3 a.2 a 2 b 8 b.8 c.3 c 3 a 2 object view object view object view ... 1. write a.3 2. update event a.3 a.3 (a) update event. node 1 node 2 node n global data store a 3 a.2 a 2 b 8 b.8 c.3 c 3 a 2 object view object view object view ... 1. invalidate a.3 2. send event a a (b) invalidation event. figure 3: event mechanisms. definition 2 (push:o:b) the bounded push operation extracts the data of object o but defers writing to the global data store until the bounded object b is pushed. all object data is then written to the data store in one data store write. remark: bounded push operations are useful for creating an atomic push of multiple objects. definition 3 (inv:o, inv:o:b) corresponding to the push operations we have defined invalidate operations which have the same semantics beside that they will not generate an update event but an invalidate event at the other views instead. definition 4 (pull:o) to update an object in the object view we have the pull operation which will request the latest data block for o from the data store and merge it. the operation is performed synchronously meaning that it will block until object o has been updated. definition 5 (order:o) for asynchronous object updates we are providing the order operation which requests the newest data for object o. the data store will transfer the new data via an update event concurrently. definition 6 (sync:o) the sync operation will request an advisory lock for the data block corresponding to the object o. the operation will block until the request can be granted. the lock on o is removed by a push:o operation of the view which holds the lock. 2.5 naming service because the object views are accessing objects in a type-safe manner references to objects cannot be manufactured. to get an object currently not available inside a view there has to be a root object from which the desired object can be accessed. such a root object could be created automatically by each view at startup. but this would force systems without reference tracking to fully replicate every accessible object or at least to create proxies for every accessible object. therefore we have integrated a hierarchical naming service into our object model itself avoiding proxies or full replication. it is also useful for partitioning the accessible object graphs while still allowing access. for this purpose we have defined two basic naming service operations. proc. wowkivs 2009 4 / 12 eceasst definition 7 (put:o:k) this operation stores an object o into the name service with a given textual key k. definition 8 (o:get:k) a get operation retrieves the object stored under a key k. 3 tgos prototype all the features defined by our theoretical model have been implemented using sun java 1.5. figure 4 gives a short overview over the components implemented for the tgos model. jvm 1 jvm 2 jvm n replication layer a 1 a 2 b 8 c 3 a 2 ... replication stub object view replication stub object view replication stub object view figure 4: prototype schematics. jvm a 2 c 3 clo su re clo su re figure 5: closure building. 3.1 object extraction and merging as mentioned before the global data store is working on binary data and is totally unaware of the object model and type system used by the application. this makes it necessary to provide a way to serialise objects to a binary form and vica versa. java is already providing a native serialisation facility for transforming objects into a binary form. but the fact that java creates new objects during the deserialisation process rendered it unusable for our approach. to fulfil the requirements of our object model we needed a mechanism to extract and to merge the data of an object a with the data of an local object b keeping the local references to the objects untouched. therefore, we have created a new serialisation/merging facility using the java reflection api. our implementation is capable of extracting the binary data and merging this data into another java object of the same type in a similar way used by code versioning systems such as svn. 3.2 closure building the object view provides an automatic closure building mechanism by building the transitive closure over all references distinguishing between global and local objects. the solution provided with the implementation for the proposed closure building as shown in figure 5 is based upon inheritance mechanisms provided in most type safe and object-oriented languages. all objects applicable to the basic operations must be a subtype of the class sharedobject. so every reference from one shared object to another shared object is treated as a global reference, every other reference is treated as local reference. when an object gets pushed the local references of 5 / 12 volume 17 (2009) massively multiuser virtual environments using object based sharing a shared object are pushed using a copy by value semantic while references to shared objects use a copy by reference semantic. with this feature it is easily possible for a programmer to build object hulls with arbitrary size. using inheritance for marking objects with globally unique references is not an optimal solution and was chosen to simplify the implementation in the early stages of development. an alternative solution would be to use interfaces for marking these shared objects. 3.3 basic event handling currently, the object model defines two events which have been implemented in tgos. the update event is triggered when a node has pushed an object and the updated object data is arriving. the registered event method can now decide what to do with the updated object data: whether to integrate it fully, partial or not at all. it is also possible to store the updated information and integrate it at a later time which is used by the later described transactional consistency to keep the object views consistent at all times. when an invalidate event is triggered the globally unique object id of the invalidated object is transmitted. thus the application can pull a new version of the object or delete the object from the local object view. 3.4 scaleable global data store our implementation of the global data store is using a hierarchical approach as shown in figure 6(a). communication between nodes on the lower hierarchy level is implemented by a clientserver approach. the servers or superpeers are interconnected on a peer-to-peer basis to improve scalability. this approach was taken for simplicity and to allow rapid development in the early stages. due to our peer-to-peer object model it is possible to change the client-server model of a superpeer to a peer-to-peer based or mixed approach without altering the application. figure 6(b) shows a feasible setup planned for the further development of wissenheim worlds. super peer super peer super peer p2p (a) current super peer super peer p2p p2p (b) planned figure 6: scaleable global data store implementation. proc. wowkivs 2009 6 / 12 eceasst 4 consistency models by default the object model is providing no consistency other than the fifo consistency defined by the global data store. stronger consistency models can be implemented using the mechanisms provided in an object-oriented fashion within the object model. the basic operations of tgos simplify the adaption to different consistency requirements depending on the use case. stronger consistency models can be implemented as add-on libraries, too. 4.1 transactional consistency transactions are well-known in data-base applications but are hardly used by highly interactive applications. a first approach for using transactions in distributed interactive applications was the original wissenheim [4] running on top of a distributed operating system called plurix. the plurix os was designed to run on standard pc hardware and was used in a local network environment. however using transactions for highly volatile data in a system with high latencies like in a wide area network will lead to a sizeable overhead and will limit scalability. synchronised access to data which is often read but only rarely changed presents an ideal scenario for transactions. wissenheim is using transactions for synchronising read and write accesses to the scene graph structure, virtual items, and objects which are only accessible by one person at a time. these accesses are relatively rare and can therefore be synchronised via transactions without major performance drawbacks. because of the limited collision probability and the latency issue we are using an optimistic transactional approach [15]. our approach to optimistic transactions is using a backup mechanism to save objects before they are modified by a transaction to be able to rollback the object state in case of an abort. 4.2 transaction-based programming model to use transactions in the program flow we need a way to specify a transactional block. the start of such a block is defined by the begin operation. the commit operation is used to end a started transactional block and to validate all changes. finally, the abort operation is provided to voluntarily abort a transaction and thus undo all changes made within the started transaction block. rolling back a transaction requires to save backups of objects that are modified during a transaction. for this purpose we are providing the add2transaction o operation which will create a backup of object o. the rollback mechanism is using the same seralization mechanisms used by the object view. do { begin(); add2transaction(obj); obj.dostuff(); if (wrong) abort(); } while (!commit()); figure 7: basic transactional loop. class token { int tacount; int writesets[][] = new int[128][]; } figure 8: token object. a standard transaction is shown in figure 7. the do..while loop is used to restart the transaction in case the commit fails and we want to restart. to serialise our optimistic transactions 7 / 12 volume 17 (2009) massively multiuser virtual environments using object based sharing we are using a token-based mechanism. each transaction wishing to commit has to acquire the shared token. the token is implemented as ”normal” distributed object managed by tgos. figure 8 shows the content of the token object. the tanumber member is a steadily increasing number of transactions committed. access to the token object is synchronised via the sync operation provided by the object view. the writeset information is used to check if the committing transactions is colliding with any data altered by a previous transaction. in case of a conflict a rollback is performed by restoring the object backups. with these four methods it is possible to work transactionally on any object shared by our object model. to avoid making the backup operation for objects used inside the transactional block manually the aspectj framework [3, 7] can be used. creating aspectj pointcuts which observe the access to objects which are descendants of sharedobject can automate the backup creation. the use of aspectj is transparent for the implementation of the transactional consistency and can be used with any implementation. removing the do..while loop is a more difficult task because in a standard jvm we cannot use labels or access the program counter to jump back to the begin of the transactional block in case of an abort. extending the jvm however would allow us to implement the transactional block in a more transparent fashion alike the java synchronized blocks. but extending a jvm would exclude inexperienced users or users on strictly secured workstation from accessing wissenheim due to the inability to install or launch a modified jvm. 5 wissenheim worlds wissenheim is designed for edutainment combining interactive teaching content with entertaining games. currently, it is used to support lectures at the university of ulm by providing interactive exercises. wissenheim is running on three platforms: plurix (the original transactional operating system), linux (using the object sharing service provided by the xtreemos project [14]), standard java (using the tgos system described in this paper). wissenheim worlds [1] is an extension of the java version of wissenheim aiming at supporting a huge number of players and many scenarios. all virtual worlds are accessible via browser applets supporting windows vista/xp 64-bit and 32-bit, macos & macos x, linux 32-bit and linux 64-bit with working sun java vm. by using standard java applets we are able to launch wissenheim without the need to install any specific client software. 5.1 scene graph wissenheim is using a distributed scene graph [5] to share the scene information among the users. access to the scene graph structure is synchronised using transactional consistency. graphical data, position information and other volatile data is accessed using weak or scene-specifing consistency constraints to keep latency issues at a minimum. therefore, the virtual world is subdivided into disjunct scenes with their own scene graph and a separate transactional token. as a result each scene is independent from a consistency point of view and thus subdivides the network traffic on a scene basis. a user can be connected to one scene only but each node can access and modify the scene graph content and structure. proc. wowkivs 2009 8 / 12 eceasst 5.2 tgos integration wissenheim worlds is using the tgos service in a transparent way. the basic wissenheim application is unaware of the different services the replication layer provides for distribution. wissenheim is working solely on objects accessing scene and avatar data by either global references or naming service calls. the implementation of the global data store is automatically taking care of redirecting request, transparently handing over connections from one service to another. this gives the developer of a scene the opportunity to create his scene using an abstract network model. 5.3 transactions in wissenheim using a data centric approach for data management allows each client to directly access and modify the shared scene graph. thus may lead to race conditions especially if the structure of the graph is modified by two or more nodes concurrently. to synchronise concurrent accesses within wissenheim we are using the proposed transactional consistency for critical sections. thus we are using transactions on a fine-grained basis synchronising only critical program parts, e.g. when avatars are joining to a scene or are grabbing items. 6 measurements due to the fact that the implementation for the global data stores is interchangeable, latency characteristics can change as well. the remaining (constant) overhead is caused by the object view and serialiser implementation. for our measurements we have used some very common situations in wissenheim worlds to reproduce results as close as possible to real world situations. we have examined the serialisation overhead and the ratio of serialised data size to effective data size. for the calculations of the effective data size we are using a very strict definition counting only the size of members or array elements containing useful data. references or other structural members are ignored here. so the effective data is the most compact (uncompressed) form in which the information the object incorporates can be saved. our measurements were made on an intel pentium d 2.66ghz with 1gb ram running windows xp sp3 using sun java 1.6. class transformation { vect3d translation; vect3d rotation; vect3d scaling; matrix4d transmatrix; matrix4d basematrix; matrix4d invworld; } (a) class matrix4d { float [16] m; } (b) class vect3d { float x,y,z; } (c) figure 9: classes used for measurements. figure 9 shows the classes used for measurements, the transformation class shown in 9(a) is used by wissenheim worlds to position and orientate any virtual object and is therefore up9 / 12 volume 17 (2009) massively multiuser virtual environments using object based sharing dated whenever a movement occurs. as one can see the transformation class consists only of references which might need their class names saved during the serialisation process. our serialiser optimises this process by omitting the class names when the type of the member variable corresponds exactly to the type of the object it is referencing. the first measurement shown in figure 10 presents the average time to serialise and merge a transformation object and the size of the serialised data for best and worst case. best and worst case differ in the compaction the serialiser can perform on class names needed for de/serialising references. for comparison we are presenting the performance of the native java serialisation facility as well. in average the serialiser is at least as fast as the native java one, in most cases even faster. furthermore, we have tested the optimistic transactions over wide area and wlan connections with a round trip time of approximately 80ms. the average number of transactions per second was about 12 which correlates exactly to the round trip time. push time pull time overall 0 20 40 60 80 100 120 140 160 180 200 best case worst case java m ic ro s ec on ds (a) ef f ective data best case w orst case java 0 100 200 300 400 500 600 data size by te s (b) figure 10: serialisation measurements. 7 related work massively multiuser virtual environments such as wissenheim worlds refer to a wide variety of different topics. due to limited space we are comparing our ideas only with a limited assortment including mmves, scene graphs, and transactional memory. a large number of architectures and formats for scene graphs have been already proposed. popular ones such as vrml or java3d are unfortunately designed for single station use and lack the possibility to be used in a distributed manner. while distributed virtual environment systems such as avocado and dive or distributed scene graphs such as blue-c [11] or the distributed open inventor [9] present a comparable design by means of scene graph distribution, they lack support for transactional consistency. the fine-grained application-based control of the replication process and the ability to distribute scene graphs which structures and classes are completely application specific are another novelty of the tgos approach. the requirements proc. wowkivs 2009 10 / 12 eceasst and motivation for a peer-to-peer architecture has been described by schiele et al. [13] and a description of a consistency model suitable for mmves has been proposed by hähner et al.[8]. transactions are a key concept in database management systems providing significant benefit for concurrent data base access [6]. the tgos work on optimistic transaction has been strongly influenced by the concepts proposed by h.t.kung et al. [10]. related work on software transactional memory (stm) has be done by wende [15] and stms for large scale clusters are described by bocchino et al.[2]. although many good ideas have been adopted from these publications the tgos approach differs in the many properties, e.g. transactions used for an mmve, network environments, etc. 8 conclusions & future work the proposed tgos object model shows a new way of creating and sharing mmves by using a fine-grained data-centric approach. by allowing a peer-to-peer oriented programming model at the object level we can easily use different replication strategies customized for different distribution scenarios. the basic operations defined by the tgos model are the building blocks for different consistency models. the java-based tgos implementation includes transactional consistency providing a promising platform for virtual worlds. tgos has been tested hosting wissenheim for supporting lectures at university of ulm. we are encouraged by the feedback from beta users and by the preliminary measurement results presented in this paper. future work includes improved support of heterogeneity by allowing user-hosted scenes and the integration of more p2p like replication mechanisms. a framework for secure access to scenes and controlled avatar customization are the next steps. we plan to collect statistical data on user behaviour and load distribution when more avatars will show up in wissenheim worlds. bibliography [1] wissenheim worlds, www.wissenheim.de, 2008. [2] robert l. bocchino, vikram s. adve, and bradford l. chamberlain. software transactional memory for large scale clusters. in ppopp ’08: proceedings of the 13th acm sigplan symposium on principles and practice of parallel programming, pages 247–258, new york, ny, usa, 2008. acm. [3] tzilla elrad, robert e. filman, and atef bader. aspect-oriented programming: introduction. commun. acm, 44(10):29–32, 2001. [4] m. fakler, s. frenz, r. gockelmann, m. schoettner, and p. schulthess. an interactive 3d world built on a transactional operating system. electrical and computer engineering, 2005. canadian conference on, pages 235–238, may 2005. [5] markus fakler, s. frenz, m. schoettner, and p. schulthess. a demand-driven approach for a distributed virtual environment. electrical and computer engineering, 2006. ccece ’06. canadian conference on, pages 1538–1541, may 2006. 11 / 12 volume 17 (2009) massively multiuser virtual environments using object based sharing [6] j. gray and a. reuter. transaction processing: concepts and techniques. morgan kaufmann, 1993. [7] jeffrey palmand william g. griswold gregor kiczales erik hilsdale jim hugunin mik kersten. an overview of aspectj. in ecoop 2001 object-oriented programming, volume volume 2072/-1 / 2001, pages 327–354. springer berlin / heidelberg, 2001. [8] j. haehner, k. rothermel, and c. becker. update-linearizability: a consistency concept for the chronological ordering of events in manets. mobile ad-hoc and sensor systems, 2004 ieee international conference on, pages 1–10, oct. 2004. [9] gerd hesina, dieter schmalstieg, anton furhmann, and werner purgathofer. distributed open inventor: a practical approach to distributed 3d graphics. in vrst ’99: proceedings of the acm symposium on virtual reality software and technology, pages 74–81, new york, ny, usa, 1999. acm. [10] h. t. kung and john t. robinson. on optimistic methods for concurrency control. acm trans. database syst., 6(2):213–226, 1981. [11] martin naef, edouard lamboray, oliver staadt, and markus gross. the blue-c distributed scene graph. in in proceedings of the ipt/egve workshop 2003, pages 125–133. press, 2003. [12] j. sandberg r. lipton. pram: a scalable shared memory. technical report, princeton, 1988. [13] gregor schiele, richard suselbeck, arno wacker, jorg hahner, christian becker, and torben weis. requirements of peer-to-peer-based massively multiplayer online gaming. in ccgrid ’07: proceedings of the seventh ieee international symposium on cluster computing and the grid, pages 773–782, washington, dc, usa, 2007. ieee computer society. [14] marc-florian mueller kim-thomas moeller michael sonnenfroh michael schoettner. transactional data sharing in grids. in proceedings of the international conference on parallel and distributed computing and systems, orlando, usa, 2008. iasted computer society. [15] m. wende, m. schoettner, r. goeckelmann, t. bindhammer, and p. schulthess. optimistic synchronization and transactional consistency. cluster computing and the grid, 2002. 2nd ieee/acm international symposium on, pages 331–331, may 2002. proc. wowkivs 2009 12 / 12 introduction typed grid object sharing object view global data store events basic operations naming service tgos prototype object extraction and merging closure building basic event handling scaleable global data store consistency models transactional consistency transaction-based programming model wissenheim worlds scene graph tgos integration transactions in wissenheim measurements related work conclusions & future work electronic communications of the easst volume 17 (2009) guest editors: m. wagner, d. hogrefe, k. geihs, k. david managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 workshops der wissenschaftlichen konferenz kommunikation in verteilten systemen 2009 in kassel (wowkivs 2009) praxisfallbeispiel: modernisierung einer mainframe-anwendung durch eine verteilte soa carsten kleiner, arne koschel 11 pages eceasst 2 / 11 volume 17 (2009) praxisfallbeispiel: modernisierung einer mainframe-anwendung durch eine verteilte soa carsten kleiner, arne koschel fachhochschule hannover fakultät iv, abteilung informatik ricklinger stadtweg 120 30459 hannover {ckleiner,akoschel}@acm.org abstract: auch heute noch sind legacy-anwendungen auf basis von mainframe-dbms wie adabas und zugehörigen programmiersprachen wie natural häufig produktiv. sie sind jedoch oft nur schlecht mit neuen unternehmensanwendungen integriert. der vorliegende beitrag zeigt in einem fallbeispiel die modernisierung einer solchen anwendung unter einsatz von web services als basis ihrer integration in eine verteilte, service-orientierte architektur (soa). keywords: soa, legacy system, natural, web service, legacy integration, adabas 1 einführung 1.1 motivation legacy-anwendungen auf basis von mainframe-dbms wie adabas [date04] und zugehörigen programmiersprachen der 4. generation wie natural der software ag, sind heute noch häufig im produktiven einsatz, oft jedoch nur schlecht mit neuen unternehmensanwendungen integriert. die einbeziehung dieser existierenden it-werte ist somit eine wichtige aufgabe, die auch bereits seit etlichen jahren die it-abteilungen von unternehmen beschäftigt. zunächst wurde üblicherweise eine punktuelle anbindung der altanwendung an neuere software-komponenten durch die sogenannte enterprise application integration (eai) verwendet (vgl. [kel02], [chkt05]). service-orientierte architekturen (soa) sind heutzutage ein mittel der wahl für eine solche integration (vgl. [kbs05], [st07]). während damit die prinzipielle vorgehensweise für die integration festgelegt ist, gilt es in der praxis doch noch zahlreiche konkrete herausforderungen zu meistern, bis die legacy-integration lauffähig ist. dabei handelt es sich nicht nur um rein technologische sondern auch um konzeptionelle probleme. der vorliegende beitrag erhebt daher nicht den anspruch in diesem bereich grundlagenforschung darzustellen, sondern zeigt vielmehr anhand eines praxisfallbeispiels aus einer forschungsund lehrekooperation (studentisches projekt) mit der software ag, wie diese herausforderungen am beispiel gelöst wurden. 1.2 verwandte arbeiten situationen, in denen eine existierende mainframe-anwendung zu modernisieren ist, ergeben sich meist eher in der betrieblichen praxis in einem konkreten anwendungsfall. seltener modernisierung einer mainframe-anwendung mittels soa proc. wowkivs 2009 3 / 11 werden sie eher konzeptionell im akademischen umfeld betrachtet. nichtsdestotrotz gibt es einige wenige wissenschaftliche veröffentlichungen, die das vorgehen bei der integration einer existierenden mainframe-anwendung in eine soa untersuchen. während sich [cfft08] im wesentlichen auf interaktive funktionalitäten beschränkt und eine wrappervariante ähnlich der in diesem artikel verwendeten einsetzt, schlägt [eng08] einen bottomup-ansatz vor. dieser hat zwar den vorteil, dass er nicht nur interaktive komponenten betrachtet, ist aber aus sicht der prozessmodellierung eher ungünstig, da sich möglicherweise prozessoptimierungen nicht erschließen. das tutorial [smi07] beschreibt wege wie allgemein eine soa im unternehmen unter berücksichtigung von legacy-anwendungen eingeführt werden kann. es werden umfassende ansätze vorgestellt, die in der in diesem artikel beschriebenen fallstudie aufgrund des fehlenden umfelds sowie des mehr technischen integrationsfokus nicht eingesetzt wurden. in [lmso05] wird eine technik vorgestellt, wie eine bestehende legacy-anwendung analysiert werden kann, um festzustellen, ob und ggfs. welche der funktionalitäten sich für eine integration in eine soa eignen. diese technik musste im fallbeispiel nicht verwendet werden, da hier eine vollständige übernahme aller funktionalitäten der altanwendung von vornherein festgelegt war. in [eak06] wird schließlich eine kategorisierung von methoden zur integration von legacysystemen in eine soa vorgestellt. die verschiedenen realisierungsoptionen, die in kapitel 4.1 diskutiert werden, finden sich direkt auch in dieser klassifizierung wieder. aufgrund der einfachen beschaffenheit in der fallstudie konnten einige der dort vorgestellten varianten („direct data access“) direkt ausgeschlossen werden. andere komplexere verfahren (z.b. „reengineering“ ) sind nicht erforderlich, da das schließlich verwendete wrapping nicht auf basis der kompletten prozesse durchgeführt wird. 2 die existierende mainframe-anwendung – sag-tours sag-tours [sag07] ist eine natural-anwendung in einer klassischen 1-tier, terminalbasierten mainframe-architektur (siehe abb. 1). sie erlaubt es fiktive segelreisen zu buchen. technisch kommen terminal-emulationen zum einsatz, die via telnet-protokolle direkt mit einer unix-variation von natural kommunizieren. die natural-anwendung greift wiederum auf eine adabas-datenbank zu. im rahmen der projektentwicklung kam auch von adabas eine unix-variante zum einsatz. da adabas und natural aufgrund ihres alters nicht mehr als allgemein bekannt angesehen werden können, werden sie im folgenden exemplarisch etwas näher erläutert. abb. 1: 1-tier systemarchitektur der existierenden mainframe-altanwendung adabas adabas ist ursprünglich ein hoch-performantes mainframe-dbms, das intern auf sogenannten invertierten listen basiert. es wurde ca. 1971 erstmals produktiv installiert und seit dem eceasst 4 / 11 volume 17 (2009) kontinuierlich weiterentwickelt. konzeptuell basiert es auf einem nf²-ähnlichen datenbankmodell (non first normal form). dies bedeutet, dass adabas daten nicht nur als atomare werte in der ersten normalform zulässt, sondern zusätzlich sogenannte multiple felder und periodengruppen erlaubt. adabas wird in der regel in verbindung mit natural verwendet, bietet aber im prinzip auch schnittstellen für einige andere programmiersprachen. exemplarisch wird an einer anfrage erläutert, wie adabas intern arbeitet. es wird eine datenbankabfrage ausgeführt, die alle reisen mit starthafen „curacao“ ermittelt: find cruise with start harbor= ‘curacao’ abb. 2: adabas arbeitsweise intern – phase 1 in phase 1 werden die vom find-statement angeforderten datensätze anhand ihrer internen satznummer (isn) im temporärbereich der adabas-db gespeichert. abb. 3: adabas arbeitsweise intern – phase 2 in phase 2 wird eine schleife durchlaufen, welche die datensätze anhand der gespeicherten isn ausliest und dem programm zureicht. modernisierung einer mainframe-anwendung mittels soa proc. wowkivs 2009 5 / 11 natural natural ist eine mittlerweile über 30 jahre alte prozedurale programmiersprache der 4. generation, d.h. sie beinhaltet ein integriertes datenbank-zugriffskonzept. ursprünglich wurde natural für die mainframerechner von ibm und siemens entwickelt, steht aber mittlerweile auf sehr vielen plattformen zur verfügung. der sourcecode kann relativ problemlos zwischen einzelnen plattformen ausgetauscht werden. abhängigkeiten existieren jedoch teilweise zum verwendeten dbms-typ, z.b. einschränkungen bei verwendung von relationalen dbms vs. dem adabas-dbms. ein einfaches code-beispiel zeigt nachfolgend, wie mittels natural auf eine adabasdatenbank zugegriffen werden kann, die artikel verwaltet. im define data-abschnitt wird zunächst eine lokale datenstruktur zur aufnahme der artikeldaten als sicht (view) auf die artikel-tabelle deklariert. in der nachfolgenden read-anweisung wird sodann in einer impliziten schleife die artikel-tabelle ausgelesen. schön erkennbar ist der 4gl-konform verfügbare, inhärente tabellendatentyp. define data local 1 artikelansicht view of artikel 2 name (a30) 2 anzahl (n7) enddefine read artikelansicht by name display name 5x anzahl endread end bei ausführung erzeugt das programm eine liste aller artikel der tabelle (artikel) mit ihrem artikelnamen (name) und deren anzahl (anzahl) aus. name anzahl ----------------------------------- lego pneumatik kranwagen 10 lego schneemobil 50 3 geschäftsprozessmodell zunächst wurden aufgrund der gewählten kombination aus top-down und bottom-up-ansatz (vgl. auch abschnitt 4.1) die soll-geschäftsprozesse identifiziert. danach wurden basierend auf den vorhandenen funktionalitäten der altanwendung einzelne dienste aus dem bereich der business services 1 identifiziert. diese dienste entstanden dabei direkt aus den zugehörigen natural-programmen der altanwendung bzw. wurden in einzelnen fällen zusätzlich in natural 1 aufgrund der eher einfachen vorliegenden altanwendung in der fallstudie wurden nur dienste auf zwei ebenen, nämlich business process services sowie business services verwendet. basic services wurden nicht identifiziert. das vorgehen könnte aber auch analog auf basic services erweitert werden. eceasst 6 / 11 volume 17 (2009) implementiert, um eine einheitliche realisierung aller dienste der altanwendung zu erreichen. so gab es in der altanwendung ein natural-programm drinf-n0, das alle verfügbaren reisen zu einem startdatum und starthafen anzeigt; dieses wurde im dienst srvc_findcruises gekapselt und so verfügbar gemacht. schließlich wurden die auf diese weise gewonnenen dienste zu den zuvor definierten geschäftsprozessen kombiniert. so zeigt abb. 4 einen ausschnitt aus dem geschäftsprozessmodell, in dem der geschäftsprozess bp_addcontract zum hinzufügen einer reisebuchung aus diensten wie srvc_findcruises und srvc_findyachtwithskipper zusammengesetzt wird, die direkt den funktionalitäten der mainframe-anwendung entsprechen. abb. 4: auszug aus dem geschäftsprozessmodell der fallstudie bei der modellierung wurden die elementaren dienste der business-schicht so kombiniert, das sie teilweise in verschiedenen prozessen verwendet werden können. so besteht der prozess (abb. 4) des hinzufügens einer reisebuchung (bp_addcontract) aus den bestandteilen des suchens oder anlegens einer fahrtroute, des findens der dafür zur verfügung stehenden schiffe sowie des anlegens des eigentlichen buchungssatzes. wie in abb. 4 angedeutet, können einige dieser elementaren funktionen in anderen prozessen ebenfalls verwendet werden. modernisierung einer mainframe-anwendung mittels soa proc. wowkivs 2009 7 / 11 4 integrationsarchitektur 4.1 konzeption prinzipiell bieten sich bei der integration einer mainframe-anwendung mehrere vorgehensweisen an (vgl. auch kapitel 1.2). in unserer fallstudie wird letztendlich der weg des „business logic wrapping“ aus der familie der invasiven methoden (klassifizierung nach [eak06]) gewählt. der ansatz einer neuimplementierung (aus der familie der invasiven methoden nach [eak06]) der vorhandenen funktionalität ([syon08]) in einer programmiersprache, die sich besser für die verwendung in einer soa eignet (z.b. java oder .net), scheidet in vielen fällen schon aufgrund des erforderlichen hohen aufwands aus. im vorliegenden fall sollte daher ein ansatz verwendet werden, der einen weiterbetrieb der bestehenden anwendung vorsieht. außerdem sollte in dieser fallstudie unter beweis gestellt werden, dass eine integration unter beibehaltung der altanwendung tatsächlich nur einen überschaubaren aufwand erfordert. der kooperationspartner software ag bietet auch werkzeuge an, mit denen man mithilfe von screenscraping ganze geschäftsprozesse aus der altanwendung extrahieren und als dienste (in diesem fall meist web services) verfügbar machen kann. diese vorgehensweise wurde hier ebenfalls nicht eingesetzt, da sich auf diese weise nur sehr wenige, grob granulare dienste ergeben. somit erhält man nur dienste auf der ebene der geschäftsprozesse (vgl. [kbs05]) und keine dienste auf den darunterliegenden ebenen (business services, basic services). mit diesem vorgehen erreicht man zwar eine vermutlich noch schnellere integration der altanwendung; allerdings ist so keine erhöhte flexibilität der prozesse durch variable kombination der dienste der unteren ebenen zu erreichen. abb. 5: modellierung des geschäftsprozesses zum löschen einer reise die kapselung der funktionalitäten der altanwendung in business services findet sich auch in abb.5 wieder, in der exemplarisch der modellierte ablauf des geschäftsprozesses zum löschen einer reise dargestellt ist. man kann hier deutlich die elementaren aufgaben erkennen (business objekt suchen bzw. löschen), die früher in einem natural-programm eceasst 8 / 11 volume 17 (2009) implementiert worden sind. diese programme werden nunmehr in dienste verpackt, so dass der ablauf des geschäftsprozesses alleine durch adäquate kombination der business services erreicht werden kann. im unterschied zu realen szenarien, in denen diese kombination der dienste durch automatische generierung aus den modellierten diagrammen automatisiert vorgenommen werden würde, wurde dies im projekt manuell erledigt. dies hatte einerseits den grund, dass hier nur wenige prozesse statisch modelliert wurden, so dass der aufwand überschaubar war. andererseits sollte der overhead für die einarbeitung in ein weiteres umfassendes werkzeug eingespart werden. in der praxis ist eine verwendung solcher werkzeuge selbst bei mittelgroßen projekten jedoch zu empfehlen. 4.2 architektur in abb. 6 ist die entwickelte architektur für die integration der legacy-systeme dargestellt. die integration findet in der anwendungsschicht statt. die existierenden legacykomponenten adabas und natural bleiben unverändert und werden mittels eines rpc-servers genutzt. dieser rpc-server stellt also einen teil des in abschnitt 4.1 beschriebenen wrappers um die natural-funktionalitäten dar und wird von der für die integration erforderlichen komponente entirex-broker angesprochen. die interne kommunikation dieser beiden komponenten wird hierbei mittels xml mapping (xmm) durchgeführt. auch der entirex-broker gehört technisch gesehen noch zum wrapper, denn der entirexbroker wiederum kann von außen direkt über web services auf basis von soap angesprochen werden. daher bilden diese komponenten zusammen den web service wrapper. abb. 6: software-architektur zur integration der existierenden anwendung alternativ sprechen die web services die weitere komponente „integration server“ an. in diesem falle kontaktiert der integration server über rpc den zuvor erwähnten broker und gehört ebenfalls zur wrapper-schicht. im konkreten fallbeispiel wurden die web services der anwendungsschicht direkt aus jsp-seiten der server-präsentationsschicht angesprochen, um ein einfaches frontend zu erhalten. die konkret definierten geschäftsprozesse werden also hier direkt in der präsentationsschicht implementiert. in realistischen szenarien ist natürlich eher ein externes ansprechen der entsprechenden web services durch eine prozessmaschine zu modernisierung einer mainframe-anwendung mittels soa proc. wowkivs 2009 9 / 11 erwarten, in der die geschäftsprozesse in einer geeigneten sprache (z.b. bpel oder bpmn) definiert werden können. auf diese weise sind die legacy-komponenten vollständig als dienste gekapselt und können so in eine soa-gesamtlandschaft eingebettet werden. 4.3 implementierung anhand von ablauferklärungen und kleinen code-beispielen aus der integrierten (neuen) anwendung wird die umsetzung der integrationsarchitektur näher erläutert. „bottom up“ betrachtet, beginnt die entwicklung auf basis der bestehenden adabasdatenbank. hierzu bietet die sag eine integrierte entwicklungsumgebung, die den import von adabas-datenstrukturen (im folgenden beispiel: yacht-v) in entsprechende „define data“-blöcke für natural-programme (genauer: natural-sub-programme) leicht ermöglicht. für die subprogramme werden entsprechende ein-/ausgabe-parameter festgelegt, die sodann im eigenen code genutzt werden können. in nachfolgenden code-stück werden alle yachten mit dem yacht-branch „b“ gefunden. die daten werden zuerst in die ansicht yacht-view gelesen und danach werden die suchergebnisse in der struktur yacht-info gespeichert. die gefüllte struktur yacht-info wird vom sub-programm zurückgegeben. define data parameter using yacht-pd local using yacht-v 1 zaehler (n5.0) init <0> 1 gernze (n5.0) init <0> end-define find all yacht-view with yacht-view.yacht-branch = "b" . . . move yacht-view.yacht-name to yacht-info.yacht-name(zaehler) . . . die natural-sub-programme werden im nächsten schritt im entire-x-broker als mittels rpc aufrufbare programme registriert, für die eine schnittstellenbeschreibung zur verfügung steht. über werkzeuge der entwicklungsumgebung, kann aus dieser schnittstellenbeschreibung eine web services-beschreibung (wsdl) generiert werden. danach erfolgt eine normale client-entwicklung für web services. in unserem beispiel werden entsprechende axis-werkzeuge zur generierung von java-aufrufund holderklassen aus obiger wsdl eingesetzt. innerhalb von jsps bzw. servlets, werden diese javaklassen von der neuen web-anwendung genutzt. 5 fazit und ausblick die integration von legacy-anwendungen in dienstorientierte architekturen ist inzwischen weder unmöglich noch sehr aufwändig. am fallbeispiel von legacy-anwendungen auf basis von adabas und natural konnte in diesem projekt eine integration in die web service welt erreicht werden. es sind zwar gegenüber der bestehenden architektur (vgl. abb. 1) weitere komponenten in die architektur aufzunehmen (vgl. abb. 6). die tatsache, dass dieses aber im rahmen eines projekts mit bachelor-studierenden ohne größere probleme möglich war, zeigt, dass eine solche integration keine unverhältnismäßigen hohen kenntnisse bzw. aufwand mehr eceasst 10 / 11 volume 17 (2009) erfordert. außerdem wurde aufgezeigt, welche vorgehensweise bei der integration der altanwendung verfolgt wurde. diese methodik sollte auch bei anderen ähnlich strukturierten altsystemen anwendbar sein. das gleiche gilt für die technische realisierung der integration. ob eine solche integration in einem heterogeneren systemumfeld möglich wäre (z.b. ohne die vom selben hersteller angebotenen integrationskomponenten einzusetzen) bzw. welchen aufwand sie ansonsten erfordert hätte, könnte in einem folgeprojekt evaluiert werden. ebenso könnten alternativen zum kombinierten top-down-/bottom-up-vorgehen im detail angewendet werden. ein vergleich des dann erhaltenen zielsystems mit dem in kapitel 4 beschriebenen könnte aufschlüsse über die eignung geben. 6 danksagung wir möchten uns bei den studierenden des sag-tours projektteams für ihre produktive und motivierte mitarbeit in diesem projekt bedanken. 7 referenzen [atz08] francesca arcelli, christian tosi, and marco zanoni. can design pattern detection be useful for legacy system migration towards soa? in sdsoa’08: proceedings of the 2nd international workshop on systems development in soa environments, pages 63–68, new york, ny, usa, 2008. acm. [cfft08] gerardo canfora, anna fasolino, gianni frattolillo, p. tramontana. a wrapping approach for migrating legacy system interactive functionalities to service oriented architectures. journal of systems and software, 81(4):463–480, 2008. [chkt05] stefan conrad, wilhelm hasselbring, arne koschel, roland tritsch. enterprise application integration: grundlagen konzepte entwurfsmuster – praxisbeispiele. spektrum akademischer verlag, 2005 [date04] c.j. date, an introduction to database systems, 8th ed., pearson, 2004 [def+08] jürgen dunkel, andreas eberhart, stefan fischer, carsten kleiner, arne koschel: systemarchitekturen für verteilte anwendungen, hanser-verlag, 2008. [dja07] asit dan, robert johnson, and ali arsanjani. information as a service: modeling and realization. in sdsoa ’07: proceedings of the international workshop on systems development in soa environments, page 2,washington, dc, usa, 2007. ieee computer society. [eak06] abdelkarim erradi, sriram anand, and naveen n. kulkarni. evaluation of strategies for integrating legacy applications as services in a service oriented architecture. in ieee scc, pages 257–260. ieee computer society, 2006. [eng08] sandra englet. wiederverwendung von legacy systemen durch einen bottom up ansatz bei der entwicklung einer soa. in heinz-gerd hegering, axel lehmann, hans jürgen ohlbach, and christian scheideler, editors, gi jahrestagung (1), volume 133 of lni, pages 96–100. gi, 2008. [kbs05] dirk krafzig, karl banke, and dirk slama. enterprise soa: service oriented architecture best practices. prentice hall international, 2005. [kel02] wolfgang keller. enterprise application integration. erfahrungen aus der praxis. dpunkt-verlag, 2002. [kk09] carsten kleiner, arne koschel: legacy vs. cutting edge technology in capstone projects: what works better? erscheint in: proceedings of the 40th acm technical symposium on computer science education (sigcse09), 2009. modernisierung einer mainframe-anwendung mittels soa proc. wowkivs 2009 11 / 11 [lms06] grace lewis, edwin morris, and dennis smith. analyzing the reuse potential of migrating legacy components to a service-oriented architecture. in csmr ’06: proceedings of the conference on software maintenance and reengineering, pages 15–23,washington, dc, usa, 2006. ieee computer society. [lmso05] grace lewis, edwin morris, dennis smith, and liam o’brien. serviceoriented migration and reuse technique (smart). in step ’05: proceedings of the 13th ieee international workshop on software technology and engineering practice, pages 222–229, washington, dc, usa, 2005. ieee computer society. [ph07] mike papazoglou and willem heuvel. service oriented architectures: approaches, technologies and research issues. the vldb journal, 16(3):389–415, 2007. [pzl08] cesare pautasso, olaf zimmermann, and frank leymann. restful web services vs. big web services: making the right architectural decision. in www’08: proceeding of the 17th international conference onworldwide web, pages 805– 814, new york, ny, usa, 2008. acm. [smi07] dennis smith. migration of legacy assets to service-oriented architecture environments. in icse companion ’07: companion to the proceedings of the 29th international conference on software engineering, pages 174175,washington, dc, usa, 2007. ieee computer society. [soft07] software ag, sagtours: soa integration project, application document, darmstadt, 2007. [st07] gernot starke, stefan tilkov (hrsg.). soa-expertenwissen: methoden, konzepte und praxis serviceorientierter architekturen. dpunkt-verlag, 2007. [syon08] toshio suganuma, toshiaki yasue, tamiya onodera, and toshio nakatani. performance pitfalls in large-scale java applications translated from cobol. in oopsla companion ’08: companion to the 23rd acm sigplan conference on object oriented programming systems languages and applications, pages 685– 696, new york, ny, usa, 2008. acm. evaluation of multi-hop ad-hoc routing protocols in wireless seismic data acquisition electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) evaluation of multi-hop ad-hoc routing protocols in wireless seismic data acquisition aliyu makama, koojana kuladinithi, musab ahmed eltayeb ahmed, andreas timm-giel 5 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst evaluation of multi-hop ad-hoc routing protocols in wireless seismic data acquisition aliyu makama1, koojana kuladinithi2, musab ahmed eltayeb ahmed3, andreas timm-giel4 1aliyu.makama@tuhh.de, 2koojana.kuladinithi@tuhh.de, 3musab.ahmed@tuhh.de, 4timm-giel@tuhh.de, https://www.tuhh.de/et6 institute of communication networks hamburg university of technology, germany abstract: due to impediments associated with cable-based seismic survey, wireless seismic data acquisition (wsda) has recently gained much attention from contractors, exploration companies, and researchers to layout enabling wireless technology and architecture for wireless geophone networks (wgn) in seismic explorations. a potential approach is to employ multi-hop wireless ad-hoc communication. in this study, we propose a multi-hop wgn architecture consisting of several subnetworks to realize the expected network performance. we investigate the performance of proactive and reactive routing protocols to examine the optimal number of geophones that could be effectively supported within a subnetwork. the performance metrics used are packet delivery ratio (pdr) and average end-to-end delay. keywords: wireless geophone network (wgn), wireless gateway (gw), central control unit (ccu), wireless seismic data acquisition (wsda) 1 introduction seismic survey is a method of obtaining an image of the earth’s sub-surface structure in order to determine optimal places to drill for oil and gas. in land survey, an energy source generates variable frequency waves (referred to as sweep or shot) that propagates down the earth’s surface. these waves are reflected and refracted as they hit various subsurface layers, which are recorded by devices at the surface called geophones and then sent down to the central control unit (ccu) for further processing. the source is then moved to the next shot point and the process is repeated. depending on the coverage area, the survey can last for several weeks. a measured geophone response sampled at 0.5 ms and digitized with 24 bit resolution will result in a data rate of 48 kbit/s. wgns are often large-scale networks with tens of thousands of geophones deployed along receiver lines, covering several squares of kilometres of the survey area. for real-time wsda systems, defining a scalable network architecture and employing a suitable wireless technology is of essential importance. infrastructure based architectures for wgns have been proposed in many literatures, [rsa+19] etc., whereby group of geophones transmit their sensed data to a wireless gateway (gw), and from the gateway down to the ccu in single-hop. however, this increases the number of gws to be deployed in the survey. higher data rate, higher communication coverage, and regard for imposed cost are some of the key factors to consider when selecting wireless technology for wgns. 1 / 5 volume 080 (2021) mailto:aliyu.makama@tuhh.de mailto:koojana.kuladinithi@tuhh.de mailto:musab.ahmed@tuhh.de mailto:timm-giel@tuhh.de https://www.tuhh.de/et6 evaluation of multi-hop ad-hoc routing protocols in wireless seismic data acquisition network architecture: our proposal is to enable multi-hop ad-hoc communication in the wgn architecture proposed in [mkt21], to reduce the number of gws to be deployed, while keeping the wgns performance intact. therefore, a flat-based network architecture in which the survey area is divided into subnetworks is employed (figure 1). each subnetwork consists of certain number of geophones that relay their seismic data via neighbouring geophones to a gw node over multi-hop wireless network. the gw then forwards the aggregated data to the ccu. the size of the subnetwork determines the optimal number of hops supported satisfying the wgn requirements, with 100% pdr and the upper limit of the delay for real-time acquisition systems. to limit the effect of adjacent and co-channel interference, neighbouring gws are expected to use non-overlapping channels in such a way that no adjacent subnetwork uses the same channel. traffic demand in a subnetwork: the common orthogonal geometry where receiver and source lines are placed perpendicular to each other is employed [cgp00]. geophones are placed on four parallel receiver lines, separated by a distance of 200 m and an inter-geophone distance of 25 m as defined in a typical survey [rsa+19]. the traffic from each geophone is generated according to the geophone recording period acquisition technique in [mkt21], at a rate of 48 kbit/s which is transmitted to the gw node. depending on the number of hops, a forwarding geophone will transmit twice the rate or more, as shown in figure 1 (logical view). although the data generated per geophone is quite small, the aggregate data generated in a subnetwork could be significantly large. if the subnetwork size has been selected with up to 200 nodes maximum, around 9.6 mbit/s of the network capacity will be required with per node data rate of 48 kbit/s, assuming all nodes can reach the gw in a single hop. however, this value increases with more number of hops in the network. moreover, when various protocol headers are taken into consideration the available bit rate is less than the nominal data rate. wireless technologies: considering the large-scale geophone deployment and higher data rate requirements, wireless communication technologies/protocols used in conventional wireless sensor networks (wsn) such as, ieee 802.15.4 or lora are not suitable for wgns. longer range random access technologies like ieee 802.11af and ieee 802.11ah are more suitable as they can cover more geophones and can provide higher data rates. the main goal of this work is to evaluate a suitable routing protocol to enable multi-hop adhoc communication in a wgn subnetwork with acceptable network performance. ad-hoc in this work refers to infrastructure-less communication. many routing protocols for wireless adhoc networks have been discussed and classified based on the underlying network structure (i.e. flat, tree, mesh, and hierarchical based) in literature [pnv13]. the ad-hoc routing protocols are categorized as pro-active, reactive, and hybrid [pnv13]. in proactive protocols route to destinations are established and maintained by periodically distributing routing information in the network. in reactive protocols, route discovery is initiated only on demand from source nodes. two prominent proactive protocols i.e., optimised link state routing “olsr” (mostly used in mobile ad-hoc networks) and ipv6 routing protocol for low-power and lossy networks “rpl” (commonly used in wsns), and a widely used reactive protocol i.e., ad-hoc on-demand distance vector “aodv” are investigated in this study. ieee 802.11g, which has same random access behavior of csma/ca like in ieee 802.11af, ieee 802.11ad, is used as an example mac protocol to evaluate the multi-hop routing protocols. olsr is based on link-state algorithm, that uses the concept of multipoint relays (mpr) to netsys 2021 2 / 5 eceasst 200 m 25 m central control unit gateway geophone channel a channel b channel c source line subnetwork subnetwork subnetwork receiver line shot points wgn with multiple subnetworks (physical view) multi-hop connectivity in a subnetwork (logical view) figure 1: architecture for wireless geophone networks optimize the number of control messages flooded in the network. rpl is an ipv6 distance vector protocol that employs the concept of destination oriented directed acyclic graph (dodag) to establish network topology, with all links oriented towards the sink or dodag root. it uses an objective function to construct routes, which define how routing metrics and other constraints are taken into account when building the network topology[wtb+12]. aodv is a distance vector protocol based on the bellman-ford algorithm in which route discovery is initiated on demand by a transmitting node. using omnet++ simulation tool, we evaluate the performance of these protocols considering wgn’s traffic pattern, network size and structure to analyze the size of a subnetwork. 2 results analysis one subnetwork with various network size settings is configured in omnet++ simulator as shown in figure 1. each geophone accumulates the data during the interval of 250 ms and send the accumulated data periodically till the end of geophone recording period of 5 s, for a single shot. the payload size of a single packet is configured as 1500 b (= 48 kbit/s at 250 ms). ieee802.11g is used with ad-hoc mode of operation supporting both 11 mbit/s and 54 mbit/s data rates. the latter is used to investigate the routing protocol performance for the maximum achievable data rate in 802.11g, as 11 mbit/s might not be enough to cater for our application data rates over several hops. figure 2 shows wgn requirement of the pdr of 100% can be achieved only up to 20 nodes irrespective of the type of routing protocol used to enable multi-hop communication in a sub network, when using 11 mbit/s. the offered load approaches the maximum capacity the network can handle, thereby leading to severe packet loss at the mac layer due to retransmission failures or queue overflow. in addition to the offered load, routing overhead resulting from periodic exchange of hello message and topology control messages further occupies the network capacity. in general, higher number of routing overhead is involved in proactive compared to reactive routing. here, aodv shows a slight improvement in pdr beyond 20 nodes with 11 mbit/s. some cases, rpl shows better pdr, compared to olsr as it uses the trickle timer algorithm to dynamically adjust the rate of sending control packets. 3 / 5 volume 080 (2021) evaluation of multi-hop ad-hoc routing protocols in wireless seismic data acquisition figure 2: packet delivery ratio figure 3: average end-to-end delay the pdr of rpl and olsr outperform aodv when using 54 mbit/s. the pdr for all protocols closely follows till the network size (n) of 80 where the network capacity can bear the offered load and the routing overhead with mulit-hop communication. at n = 100, the pdr for aodv protocol significantly drops to almost 50%. this is caused by the specific traffic model of the wgn. geophones transmit data packets almost simultaneously during the geophone recording period of 5 s. this is a bottleneck for aodv as route discovery process is initiated whenever a node wants to transmit a packet, thereby overloading the network capacity with route request (rreq) packets leading to collisions. this results in failure in route discovery process causing more packets to be dropped. figure 3 shows the average packet end-to-end delay in the network. as expected, the delay decreases with higher data rate as a lower data rate implies longer packet transmission time. the delay increases with increase in network size as a packet might travel over two or three hops. in addition, as the probability of collision increases with an increase in network size, packets will take longer time in the queue due to the csma/ca back-off algorithm, especially in aodv protocol during the route establishment process. 3 conclusion the preliminary study sets to investigate the performance of proactive and reactive protocols in wgns, with focus on determining the optimal number of geophones that could be supported in a subnetwork. the initial results show that this clearly depends on the type of link layer technology (data rate, mac) and the underlying routing protocols strategy. rpl and olsr show more promising results as compared to aodv protocol. these protocols offer low delay during the route setup process as routes are immediately available when data transmission begins. however, high bandwidth requirement might be a major drawback in olsr as the control overhead is proportional to the number of nodes in the network. rpl looks more promising in terms of cost effectiveness as it employs trickle mechanism to optimize the dissemination of topology control information over the network. future work will be directed towards realizing the architecture presented in figure 1 with enhancements to mac scheme, e.g., dedicating the resources, and also to optimizing the routing strategy within the subnetworks. netsys 2021 4 / 5 eceasst acknowledgements: this work is supported by deutscher akademischer austauschdienst (daad) under the nigerian-german postgraduate training programme, in collaboration with the nigerian petroleum technology development fund (ptdf). bibliography [cgp00] a. cordsen, m. galbraith, j. peirce. planning land 3-d seismic surveys. society of exploration geophysicists, jan. 2000. [mkt21] a. makama, k. kuladinithi, a. timm-giel. wireless geophone networks for land seismic data acquisition: a survey, tutorial and performance evaluation. sensors 21(15), 2021. doi:10.3390/s21155171 [pnv13] n. a. pantazis, s. a. nikolidakis, d. d. vergados. energy-efficient routing protocols in wireless sensor networks: a survey. ieee communications surveys tutorials 15(2):551–591, 2013. doi:10.1109/surv.2012.062612.00084 [rsa+19] v. a. reddy, g. l. stuber, s. al-dharrab, w. mesbah, a. h. muqaibel. a wireless geophone network architecture using ieee 802.11af with power saving schemes. ieee transactions on wireless communications, pp. 5967–5982, 2019. doi:10.1109/twc.2019.2940944 [wtb+12] t. winter, p. thubert, a. brandt, j. hui, r. kelsey, p. levis, k. pister, r. struik, jp. vasseur, r. alexander. rpl: ipv6 routing protocol for low-power and lossy networks. technical report rfc6550, rfc editor, mar. 2012. 5 / 5 volume 080 (2021) http://dx.doi.org/10.3390/s21155171 http://dx.doi.org/10.1109/surv.2012.062612.00084 http://dx.doi.org/10.1109/twc.2019.2940944 introduction results analysis conclusion demo: using lora communications and epidemic routing in disaster rescue operations electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) demo: using lora communications and epidemic routing in disaster rescue operations yamani dalpathadu 1, showry thumma †, asanga udugama, anna förster 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 1 the authors performed the primary work and contributed equally. http://www.easst.org/eceasst/ eceasst demo: using lora communications and epidemic routing in disaster rescue operations yamani dalpathadu1 †, showry thumma1 †, asanga udugama2, anna förster3 1 sachitha@uni-bremen.de, 1 thumma@uni-bremen.de university of bremen, bremen, germany. 2 adu@comnets.uni-bremen.de 3 anna.foerster@uni-bremen.de university of bremen, bremen, germany. abstract: a casualty of disasters is the communication infrastructure. rescuers, in the aftermath of the disaster, require solutions to maintain communications in order to communicate critical information gathered by them. despite the numerous solutions proposed, a drawback is the communication range. in this work, we propose a communication system based on lora, a long-range, low-power communication technology. we use the commercially available, off-the-shelf lora based pycom lopy4 platform with opportunistic networking to demonstrate the viability of using lora for post-disaster recovery operations. keywords: epidemic forwarding, lora, pycom lopy4, disaster rescue operations 1 introduction the job of emergency first responders is important in saving as many lives as possible and assisting the affected in the aftermath of disasters. since disasters destroy communication infrastructure, satellite communications are widely used to provide broadband point-to-point connectivity for devices carried by rescuers. but, satellite communications are expensive due to costly equipment and subscriptions. since almost every mobile device these days is equipped with device-to-device (d2d) communication technologies (e.g., wifi, bluetooth), they are considered a viable solution during rescue operations [gw14, mtn18]. however, the short range of these communication methods makes them inefficient when rescuers are dispersed and operating in debris. lora is a low-power wide-area networking (lpwan) technology with a low data rate and a long range (usually 3 to 30 kilometers), making it an excellent alternative. some of the crucial data exchanged during rescue operations are short messages consisting of environmental information such as position coordinates or detected dangers (gas leaks, etc.).therefore, we propose a solution for rescuers based on a low-cost, commercially available microcontroller platform equipped with lora and multiple sensors. this platform, called pycom lopy41, carried by each rescuer, is configured to sense environmental information and † the authors performed the primary work and contributed equally. 1 https://pycom.io 1 / 4 volume 080 (2021) mailto:sachitha@uni-bremen.de mailto:thumma@uni-bremen.de mailto:adu@comnets.uni-bremen.de mailto:anna.foerster@uni-bremen.de demo: using lora communications and epidemic routing in disaster rescue operations use opportunistic networking (oppnets) with epidemic forwarding [vb00] and the loramac mode to disseminate data between rescuers and a disaster management center. in this work, we demonstrate the operation of our solution using a scenario mimicking communication during rescue operations. this work has been evaluated with multiple scenarios previously2 and the architecture and the performance evaluation have been accepted for publication [dtk+21]. 2 related work there are a number of work focusing on using lora in disaster communications. an iot framework based on lorawan and wifi technology for disaster management has been presented in [ddz+19]. the authors of [cfm+19] suggest a communication scheme for disaster communications. the authors of [hbk+20] describe a firmware that enables users to use lorawan technology via smartphones in post-disaster conditions. considering these works, our work is unique in that it is the first time loramac has been deployed in emergency communication applications. loramac refers to the mode of operation in lora for direct, device-to-device (d2d) communications with other devices in a wireless neighborhood. in the lorawan mode, the communications are performed over a deployed gateway. in contrast to [ddz+19], [hbk+20], and [sftd18], we employ loramac and an enhanced version of epidemic forwarding [vb00] to disseminate data. our approach is closely related to the work described in [sftd18], but the authors propose the use of the loramac interface as an attachment to a smartphone, thereby limiting their solution to the sensing capabilities offered by the smartphone. 3 system architecture the architecture of the system deployed in each pycom lopy4 device consists of a 3-layer protocol stack with an additional shim-layer for neighborhood management. the primary purpose of this protocol stack is to exchange sensed data between the devices of the rescuers. the figure 1a shows the micro-python programmed protocol stack of a device. following is a brief description of the protocol stack. • application layer is for applications that sense environmental information (e.g., position coordinates) and to use the received data of other rescuers. currently, we use the pytrack and pysense extensions of the pycom lopy4 to retrieve the last available localized gps coordinates, ambient light, pressure, and humidity. this traffic is transmitted periodically and the period of transmitting packets is configurable. • forwarding layer is the oppnets forwarding layer where an extended version of epidemic forwarding [vb00] is implemented. the extended version consists of anti-entropy sessions based on timeouts to overcome the effects of deadlocks that arise due to the packet losses during transmissions. 2 https://github.com/comnets-bremen/epidemic-on-pycom/tree/master/results netsys 2021 2 / 4 eceasst application epidemic forwarding sensed data collector and transmitter lora neighbour management forwarding link (a) the 3-layer protocol stack including the shim-layer x xx x x x path of mobile nodes x stationary node x mobile node building (b) the view of the 6-node scenario demonstrated figure 1: node architecture and demonstrated scenario • link layer is realized through lora where the loramac mode is configured at initialization to perform d2d communications. because lora only offers broadcast communications, this module manages unicast operations internally by considering packets that are destined to themselves. the addressing is realized through the unique mac-like address provided by the lora network interface. • neighbor management shim-layer is responsible for maintaining the neighbor list of a node using a beaconing mechanism. there are four different packet types used: hello message is used by neighbor management to beacon the presence and to know about other devices in its wireless vicinity, data message carries the sensed data, request and summary-vector messages are used by epidemic forwarding to perform the anti-entropy operation [vb00]. 4 demonstration the focus of our demonstration is to show the dissemination of data to the different devices of rescuers. the figure 1b shows a view of the placement of devices during the demonstration. the demonstration consists of six devices (nodes), three moving (mobile), and three stationary. we have chosen three initial nodes that are responsible for sensing data while all nodes spread the sensed data throughout the network. in this demonstration, we visually show the spread of data throughout the network and at the end, convey how successfully the epidemic routing protocol disseminated the data throughout the network, and how lora enabled this spread to occur. 5 summary and future work the work presented here for the demonstration focuses on disseminating sensed data of rescuers in the aftermath of a disaster when usual networking infrastructure-based communications 3 / 4 volume 080 (2021) demo: using lora communications and epidemic routing in disaster rescue operations are unavailable. the solution we propose uses lora and oppnets with epidemic forwarding for communications. the solution uses the commercially available off-the-shelf pycom lopy4 platform with micro-python to implement the functionality. the code and the performance evaluation is available at github3 and published at [dtk+21]. though we have an operational version, the performance results show us a number of challenges to address in the future. the primary challenge is the identification of optimal configuration parameters (e.g., the spreading factor). additionally, we plan to undertake an extensive performance analysis to include evaluations based on realistic post-disaster rescue operations that take into account factors such as node velocity, the distance between nodes, and weather conditions. bibliography [cfm+19] r. p. centelles, f. freitag, r. meseguer, l. navarro, s. f. ochoa, r. m. santos. a lora-based communication system for coordinated response in an earthquake aftermath. proceedings 31(1), 2019. [ddz+19] a.-m. dragulinescu, a. dragulinescu, c. zamfirescu, s. halunga, g. suciu. smart neighbourhood: lora-based environmental monitoring and emergency management collaborative iot platform. in 2019 22nd international symposium on wireless personal multimedia communications (wpmc). pp. 1–6. 2019. [dtk+21] y. dalpathadu, s. thumma, v. kuppusamy, a. udugama, a. förster. disseminating data using lora and epidemic forwarding in disaster rescue operations. in acm international conference on information technology for social good (goodit 2021). 09 2021. [gw14] b. p. gautam, k. wasaki. using a redundant wi-fi network as an emergency detour route to proactively reduce disaster risk in wakkanai, hokkaido. in 2014 international conference on information science, electronics and electrical engineering. volume 3, pp. 1830–1837. 2014. [hbk+20] j. höchst, l. baumgärtner, f. kuntke, a. penning, a. sterz, b. freisleben. lorabased device-to-device smartphone communication for crisis scenarios. 05 2020. [mtn18] m. murase, k. tanaka, k. naito. prototype implementation of human management system with ble beacon devices in natural disasters. 2018 15th ieee annual consumer communications and networking conference (ccnc), pp. 1–2, 2018. [sftd18] l. sciullo, f. fossemo, a. trotta, m. di felice. locate: a lora-based mobile emergency management system. in 2018 ieee global communications conference (globecom). pp. 1–7. 2018. [vb00] a. vahdat, d. becker. epidemic routing for partially-connected ad hoc networks. technical report, 06 2000. 3 https://github.com/comnets-bremen/epidemic-on-pycom.git netsys 2021 4 / 4 introduction related work system architecture demonstration summary and future work towards reuse on the meta-level electronic communications of the easst volume 74 (2017) 7th international symposium on leveraging applications of formal methods, verification and validation doctoral symposium, 2016 towards reuse on the meta-level dominic wirkner, steve boßelmann 11 pages guest editors: anna-lena lamprecht eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst towards reuse on the meta-level dominic wirkner1, steve boßelmann2 1 dominic.wirkner@udo.edu 2 steve.bosselmann@cs.tu-dortmund.de chair for programming systems, department of computer science, tu dortmund university, 44227 dortmund, germany abstract: modern software development peaks in complex product lines and utilizes features of programming languages to their full extend. on the other hand, model driven development shines by abstraction from implementation details to ease communication between programmers and domain experts. in this paper an extension to the language family of the cinco meta tooling suite is proposed, which allows a more flexible and efficient way to reference elements between different models. the definition of how representational elements reference their origins in another model is shifted away from code-level to make it accessible to domain experts. it also enables the possibility to reuse such specification without the need for code or model duplication. in addition, validation and needed updates between referenced elements and their representation is supported by a centralized transformation definition. this highly stretches reuse of implementation to overcome reoccurring problems in the context of inter-model references and acts as one part of a foundation for developing software product lines with the cinco framework. keywords: model driven development, software product lines, cinco meta tooling suite, graphical modeling 1 research motivation since the early days of programming developers tried to optimize development time and maintenance costs to ensure a desired level of quality. one key part in this challenge is minimizing code duplication. this is getting more and more difficult due to increase of complexity in modern software systems. nowadays this complexity is reflected by the emerging use of software product lines (spls). classical spls consist of a core part and a collection of features, which some of them are mutually exclusive and other ones are purely optional. the definition of rules for feature combination builds its own scientific field [bps04, lns13, vv11] and achieving a desired level of source code reuse requires an implementation language to have sufficient expressive power. typical continuous evolution of spls, which is supported by version control systems, faces the problems of software variants. this leads to a variability problem in space and time [ssa14]. finding a language that is capable of dealing with different variants and versions of software components, which means definition (design-time) and execution (runtime) of behavior, is another challenge to solve. general-purpose programming languages are capable to offer a solution for this problem, but they disadvantage application experts that typically lack programming knowledge. 1 / 11 volume 74 (2017) mailto:dominic.wirkner@udo.edu mailto:steve.bosselmann@cs.tu-dortmund.de towards reuse on the meta-level through history, a major motivation in the design of programming languages has been the strive for simplicity in terms of language features that enable definitions in a simple yet precise manner. to achieve simplification for a specific domain the introduction of new language features might even come along with the sacrifice of expressiveness. as an example, the replacement of goto statements by means of various loop constructs means a reduction in expressive power. on the other hand, this step made it possible to define what happens without the need to dictate how it is performed. such a shift from how to what has repeatedly been motivation for the development of domain-specific languages that pursue a declarative approach instead of using imperative instructions. within the last decade the paradigm of model driven development (mdd) has become quite popular because of its ability to narrow the gap between application experts and software developers [sm09]. furthermore, providing a one-thing approach (ota) i.e. a common and corresponding software is key for reaching such goal [ms09]. driven by the aspect of simplicity [nnl+13] and full code generation [kt08] the cinco meta tooling suite has been developed [nlks17], which supports an application expert by generating a graphical model editor from defined meta models. the analysis of the development process based on various applications of cinco [bnns16, bfk+16] highlighted possibilities for improvements in reusing parts of meta models to increase the benefits of ota even further. up to this point, general-purpose languages are capable of defining spls, but application experts that rely on domain knowledge instead of source code can hardly access them. on the other hand, modeling environments like those created with cinco support application experts, but lack the possibility to implement spls. there are some first concepts to enhance cinco by bootstrapping new features and reusing implemented effort [nnms16]. however, achieving a complete model driven spl development with cinco requires enhancements on the core language concepts. one major problem to overcome is a specification of meta model families and the linking between their languages without redefinition and duplication. the first question to answer is whether there are existing mechanisms, e.g. in programming languages which can be integrated and advanced on a model level. beyond that other aspects could be considered such like model transformations. the goal is to establish an easy reuse of meta models, or parts of them, to enable time-efficient and error-minimizing design of meta model families. in the following section 2 some related work is presented which served as inspiration to this research and ideas for possible solutions. section 3 gives a brief introduction to the cinco meta tooling suite and how model reuse is implemented today. this leads to reoccurring problems which are described in section 4. in section 5 we introduce a more potential solution for model reuse in cinco to overcome these problems. finally section 6 concludes presented results and gives some outlook to our future work. 2 related work a different but very related problem to reuse implementation effort is the famous expression problem [wad98], which addresses the question on ”how to extend a program?”. usually an efficient solution is requested and efficiency is understood in terms of source code complexity. isola ds 2016 2 / 11 eceasst the problem can be extended by limitation of recompiling or redeployment of old program parts. although the problem came up at the end of the last century, research for solutions never stopped [tor04, wo16, lh06]. furthermore, pure::variants1 has to be mentioned as a software, which already provides development of product lines in an integrated environment but only on a source code level. leaving the technical level of source code, some published results can be found in the sector of mdd. the concept of archimedian points [sn16] addresses the aspect of transforming a new variant of a meta model to an older version to enable reuse of meta model languages and corresponding code generators. the idea of splitting the meta model up into fragments and reusing them as extensions in other ones is proposed in [rvms12]. besides these general design concepts, there are at least two more practical approaches which define and implement spls in mdd. [dsw14] covers a more declarative way to distinct variable features by only describing their differences. in [gv09] the paradigm of aspect oriented programming is applied on the model-level to propose not only a declaration, but also a modelto-code transformation to implement software variants. development of product lines not only results in the question of extensibility as part of variant management. the paradigm of higher-order process modeling in the context of mdd, which is researched in [nsm13], can also provide mechanisms to improve the flexibility of meta model declaration. 3 meta tooling with cinco the cinco meta tooling suite [nlks17] offers a development environment for graphical modeling tools. based on textual specifications a complete eclipse-rcp-based integrated modeling environment (ime) is created by full code generation. these textual specifications consist of the meta-graph-language (mgl) to define the meta model and a description of appearances in the graphical editor by the meta-style-language (msl). in addition, the ime can be enriched with model validation, which is either generated from the meta model or manually implemented. to support development on tool level cinco offers integration of code generators for specified meta models. this allows to create domain-specific graphical modeling environments only by a few lines of specification. listing 1 gives an example specification of a simple meta model for control flow. a model has a name and can contain nodes of types start, activity and end of which start nodes are limited to exactly 1 and end nodes to at least one. all of these are connected by edges of type transition. the model elements activity, end and transition are labeled. a sample model is shown in figure 1. a proper addition to the meta model would be a possibility for model hierarchy. this enables a first level of model reuse similar to procedural programming on code level. to reuse a model a ”procedure call” has to be defined and used. because models in general do not always represent any sort of control flow these ”calls” are described by representational model elements. from a graphical modeling perspective, the creation of a reference is initiated by dragging a specific model or any of its inherent elements and dropping it on the active workbench. in 1 www.pure-systems.de 3 / 11 volume 74 (2017) towards reuse on the meta-level 1 graphmodel controlflow ( 2 attr estring as name 3 containableelements( 4 start(1,1),activity(0,*),end(1,*) 5 ) 6 node activity ( 7 attr estring as label 8 incoming(transition(1,*)) 9 outgoing(transition(1,*)) 10 ) 11 node start ( 12 outgoing(transition(1,1)) 13 ) 14 node end ( 15 attr estring as label 16 incoming(transition(1,*)) 17 ) 18 edge transition ( 19 attr estring as label 20 ) 21 ) listing 1: controlflow.mgl figure 1: a sample controlflow general, the reference to another model needs to be represented in the current model in some way. the cinco framework supports the automatic registration of a drop handler to be called whenever such a drag-and-drop action is performed. this drop handler represents the junction point between model references and defined semantics as it typically triggers code for creation of the model elements that represent specific artifacts. these may span the model itself, any of its inherent elements as well as any other artifact that is somehow associated with any of these. to model a switch into another model and handle possible outcomes the meta model is extended with an additional node and edge type. listing 2 shows necessary additions on the meta level. a new submodel node is introduced which represents the switch of control to the start node of another model. first, it inherits from activity and therefore has a label and incoming transitions. second, it overrides outgoing edges to type branch. a new edge type is introduced to represent possible outcomes of the underlying control flow model, which reference to according end nodes in the submodel symbolically. third, a prime attribute is defined of which a model element can only have one. this is the actual reference to another model element and establishes a link between these. in this case the attribute submodel references the other controlflow model. other specifications are also imaginable: the prime reference could point directly to the start node and additionally branch edges could have been referenced by a prime attribute to an end node. in figure 2 the control flow of figure 1 is reused as follows: the submodel node references the reused controlflow model in the prime attribute. the possible multiple outcomes in form of end nodes are represented by outgoing edges of type branch and labeling them accordingly. the simple example shown in listing 2 and figure 2 shows how references between model isola ds 2016 4 / 11 eceasst 1 ... 2 node submodel extends activity ( 3 prime controlflow as submodel 4 outgoing(branch(1,*)) 5 ) 6 7 edge branch extends transition ( 8 ) 9 ... listing 2: extending controlflow.mgl figure 2: sample of model hierarchy elements can be defined in cinco meta model specifications. note that it is possible to even define references across instances of different meta models. potential problems as well as possible solutions in this context are presented in the following sections. 4 solution approaches and recurring problems references to some artifact, e.g. to another model, need to be somehow represented in the current model. this usually means presence of a set of specific model elements, each of which representing the referenced artifact itself or another artifact that is somehow associated with the referenced one. however, as these representational model elements share a common relation to the referenced artifact they form associated components, as a whole referred to as a compound throughout the following discussion. handling these compounds on the model level comprises the following aspects: • initialization: as soon as the respective reference has been created the compound needs to be initialized by means of creating all associated model elements and setting up their attributes. • updating: the compound is to be updated if the referenced artifact changes, i.e. refactoring to rebuild structural integrity be it automatically or in a manual manner. • validation: the structural integrity of the compound should be checkable to generate a warning or error if incorrect. • deletion: the compound should be deletable by means of deleting all associated model elements at once. before a structured approach to the realization of these aspects is described, it should be mentioned that in practice we found that the realization of these aspects often is not pursued in a uniformly structured manner. one reason for this are developers that follow naive approaches 5 / 11 volume 74 (2017) towards reuse on the meta-level without a view of the whole. another reason is intentionally independent development effort due to separation of concerns. as an example, the validation of models might be developed independently from the structural logic or graphical representation of the model elements. however, along with a potential lack of consultation between the developers in the worst case this might lead to multiple separate implementations for each of these aspects with scattered semantics due to distributed logic as well as a substantial amount of code duplication. a structured approach to the realization of the compound-related aspects can be achieved with written code that relies on a suitable data structure. the latter should reflect both the inherent structure of a specific compound as well as the underlying reference to the respective artifact. the logic that covers each of the compound-related aspects (initialization, updating, validation and deletion) can build upon this data structure. this logic only slightly differs depending on each respective aspect: • initialization means creating all necessary model elements associated to a compound according to the current state of the referenced artifact. • updating first of all requires the identification of already existing compound-related model elements. these need to be synchronized with the model elements that should exist according to the current state of the referenced artifact. obsolete elements need to be deleted while missing elements need to be created. note that it is not an efficient solution to completely remove and re-insert the model elements associated with a compound because existing ones might have been customized by the user and deletion as well as creation might trigger additional model-transforming routines that probably should not (again) be triggered. • validation like updating requires the identification of already existing compoundrelated model elements. but instead of restoring structural integrity, it is only checked by means of comparing the model elements that exist to those that really should exist according to the current state of the referenced artifact. • deletion also rests upon the identification of existing compound-related model elements. altogether, the realization of each of these compound-related aspects in total makes up the synchronization logic. it can be defined in a generic manner as it is independent of the type of elements that actually are handled. in particular, it can be separated from the structural information about a specific compound as it all comes down to identifying and comparing model elements related to a compound. however, the implementation requires the existence of routines to compare, add and delete model elements as typically provided by the modeling framework, e.g. by cinco . the focus lies on reusability of compounds, and as already mentioned above a suitable data structure reflects the structural information about a compound, i.e. it defines the referenced artifacts for each compound-related element and provides a matching routine to determine whether an existing model element represents a specific artifact. applied to the example shown in listing 2 and figure 2, the data structure holds the information that the compound consists of a submodel node and various branches, i.e. edges of type branch. additionally, it holds the information that each branch represents an end node of the referenced model whereas the isola ds 2016 6 / 11 eceasst submodel node somehow represents the referenced model itself. a corresponding matching routine would check whether a suitable branch edge exists for each represented end node, i.e. the edge exists and its label equals that of the respective end node. in this example, the compound consists of a submodel node and branch edges. however, the focus lies on reusability of compounds and the actual type of represented artifacts may depend on the concrete reuse scenario. hence, the fact that the artifacts represented by branch edges are the end nodes of a referenced model is characteristic of the implementation in terms of the specific use case of this example. this circumstance can be reflected in code. consider an abstract data type submodelcompound to hold abstract methods getsubmodel that retrieves the referenced model as well as a method getbranchreferences that retrieves the artifacts to be represented by branch edges. the reuse of the compound would be achieved via multiple concrete implementations of the abstract data type, each of which pointing towards a specific type of model as well as specific type of branch references. this requirement can be tackled by means of type parameters in the context of generic programming. the depicted approach based on written code is scalable to an arbitrary number of model elements that make up the actual compound. from an abstract point of view, the depicted solution though it is implemented in source code already has a declarative tendency, because in essence the concrete parts of an implementation link representational elements to those that are actually represented. the logic regarding initialization, updating, validation and deletion can be defined on abstract types and is independent of the concrete reuse scenario. this observation leads to ideas for a realization on a higher level of abstraction to be described in the next section. 5 language improvements via linking model in the previous section reoccurring problems were presented in managing model elements and their representations and also a solution to this on source code level was introduced which made use of a general description of the relation between original and representational elements. although this works fine for the moment this idea still has some essential drawbacks on a practical level. one is that the description of how something is represented with model elements is hidden on source code level from the domain expert. without programming knowledge, it is not possible for him to change the behavior of the modeling tool when creating such representational elements. instead, this could also be understood as a model transformation from original elements to some representation, which leads to the various different ways to describe such one and some of them are likely known by domain experts. this gives them the possibility to specify application semantics with more detail by themselves. another problem relates to the strict linking between origin and representation by the existent prime attribute. the solution allows to reuse models or model elements, but some kind of reuse for the representation is denied. the model transformation implemented on source code level does only handle structural information. but because it creates a graphical representation there are more aspects that need to be considered, like creating a desired layout for these model elements. an example to a more complex transformation in the context of the dywa integrated model7 / 11 volume 74 (2017) towards reuse on the meta-level (a) process model (b) process representation figure 3: reuse of process models in dime ing environment (dime) [bfk+16] is given in figure 3. dime is a cinco product for specification and generation of web applications. on the left side a shortened process model of dime [bfk+16] is shown. a process in dime is much like the control flow in figure 1 but with added support for data flow aspects. as only the start and ends of such model are relevant for the discussion, the actual process logic in between is not show in the figure. it translates to its representation shown on the right side. a process not only has a start and multiple ends, but these also have so called ports which represent input parameters and return values. besides application behavior the user interface has also to be modeled. therefore, it offers the meta model gui which has to be integrated in process models to enable manipulation by program control flow. by design, to have a somewhat consistent look of process models, the representation of gui models in a process is very similar to the one shown in figure 3. although the transformation of a gui model in a process on a structural level is very different, the graphical layout for the representation should be reused both for process and gui models. therefore, it is necessary to decouple the strict linking of representational model elements by the prime attribute in a mgl to their origins. the solution proposed in this paper centers around expanding the cinco language family by a new meta model specialized on loose linking of model elements to replace the existing prime attribute. to satisfy the needs the new meta model has to offer a minimum set of properties to describe a reference: source it must be possible to define to which model or model element something is referenced to. on tool level this describes which object can be dragged on the modeling canvas. container possible areas need to be specified, where dragged objects can be dropped. in the context of cinco this means a set of container model elements. target the compound model element links to the source and can be reused in multiple scenarios. the actual definition of such compound will still be done in the mgl model. isola ds 2016 8 / 11 eceasst handler a description of how the source is translated to the target is needed. this basically replaces the source code solution of section 4 and extends to the possibility to use any transformation language. the linking between model elements is decoded in the transformation and can be dynamically created on runtime. applied to the example of listing 2 the prime attribute is removed and instead one description tuple of loose linking is created. based upon this information the familiar tool behavior can be maintained. cinco uses the source and container information to call the appropriate handler in the form of a transformation when a control flow is dropped on the canvas. the transformation outputs a structure of a submodel node with connected branch edges. additional functionality like layout-creation is separated from the transformation and could therefore be reused whenever elements of the target compound are created. with this model the translation semantic of model elements to their representation could possibly be done by a domain expert. if then another meta model is integrated e.g. in this control flow model, a second tuple of loose linking has to be created, which includes writing a specialized transformation for that case. but this could purely be done again by the domain expert as it is not hidden in the source code. also representational element definitions and related layouts are reused automatically and because the creation of representations and keeping them synchronized with their origin is generalized the benefits of the solution discussed in section 4 still apply. 6 conclusion in this article we proposed a solution for specification duplication in the context of inter-model references. exemplary shown for the cinco meta tooling suite the idea of a loose linking model has been introduced, which supports domain experts in their specification of modeling tool behavior and meta model description. the new model separates the concerns of model definition and tool behavior by means of shifting the specification of references and corresponding representations from the source code to a declarative language, hence making them accessible for domain experts without programming knowledge. additionally, it enables the reuse of representational compound elements for different purposes. it facilitates syntactical correctness in terms of valid references and thus reduces implementation effort. however, basic tool features like layout-creation for now remain on code-level. applying the same level of abstraction on these aspects might be a topic of future work. the depicted idea to enhance the cinco meta tooling suite towards optimizing development time of software product lines is only one out of many. the next steps will address the implementation of an actual linking model as well as the evaluation of the benefits on a more sophisticated product like dime. beyond that, more concepts of modern programming languages like polymorphic dispatching could be pushed up to the modeling level, especially in terms of connecting and reusing models. the goal is to maximize the flexibility on all levels of modeling, from the domain expert as a tool developer to the actual modeler, to enable the specification of products and product variants without the need for a programming expert to be present. 9 / 11 volume 74 (2017) towards reuse on the meta-level bibliography [bfk+16] s. boßelmann, m. frohme, d. kopetzki, m. lybecait, s. naujokat, j. neubauer, d. wirkner, p. zweihoff, b. steffen. dime: a programming-less modeling environment for web applications. pp. 809–832. springer international publishing, cham, 2016. doi:10.1007/978-3-319-47169-3 60 [bnns16] s. boßelmann, j. neubauer, s. naujokat, b. steffen. model-driven design of secure high assurance systems: an introduction to the open platform from the user perspective. in t.margaria and m.g.solo (eds.), the 2016 international conference on security and management (sam 2016). special track ”end-to-end security and cybersecurity: from the hardware to application”. pp. 145–151. crea press, 2016. [bps04] d. beuche, h. papajewski, w. schröder-preikschat. variability management with feature models. science of computer programming 53(3):333–352, 2004. doi:10.1016/j.scico.2003.04.005 [dsw14] f. damiani, i. schaefer, t. winkelmann. delta-oriented multi software product lines. proceedings of the 18th international software product line conference on splc ’14, 2014. doi:10.1145/2648511.2648536 [gv09] i. groher, m. voelter. aspect-oriented model-driven software product line engineering. lecture notes in computer science transactions on aspect-oriented software development vi, p. 111–152, 2009. doi:10.1007/978-3-642-03764-1 4 [kt08] s. kelly, j.-p. tolvanen. domain-specific modeling: enabling full code generation. john wiley & sons, 2008. [lh06] a. löh, r. hinze. open data types and open functions. proceedings of the 8th acm sigplan symposium on principles and practice of declarative programming ppdp ’06, 2006. doi:10.1145/1140335.1140352 [lns13] a.-l. lamprecht, s. naujokat, i. schaefer. variability management beyond feature models. computer 46(11):48–54, 2013. doi:10.1109/mc.2013.299 [ms09] t. margaria, b. steffen. business process modelling in the jabc: the one-thingapproach. handbook of research on business process modeling, pp. 1–26, 2009. [nlks17] s. naujokat, m. lybecait, d. kopetzki, b. steffen. cinco: a simplicity-driven approach to full generation of domain-specific graphical modeling tools. sttt, 2017. to appear. isola ds 2016 10 / 11 http://dx.doi.org/10.1007/978-3-319-47169-3_60 http://dx.doi.org/10.1016/j.scico.2003.04.005 http://dx.doi.org/10.1145/2648511.2648536 http://dx.doi.org/10.1007/978-3-642-03764-1_4 http://dx.doi.org/10.1145/1140335.1140352 http://dx.doi.org/10.1109/mc.2013.299 eceasst [nnl+13] s. naujokat, j. neubauer, a.-l. lamprecht, b. steffen, s. jörges, t. margaria. simplicity-first model-based plug-in development. softw. pract. exper. software: practice and experience 44(3):277–297, 2013. doi:10.1002/spe.2243 [nnms16] s. naujokat, j. neubauer, t. margaria, b. steffen. meta-level reuse for mastering domain specialization. pp. 218–237. springer international publishing, cham, 2016. doi:10.1007/978-3-319-47169-3 16 [nsm13] j. neubauer, b. steffen, t. margaria. higher-order process modeling: productlining, variability modeling and beyond. electron. proc. theor. comput. sci. electronic proceedings in theoretical computer science eptcs 129:259–283, 2013. doi:10.4204/eptcs.129.16 [rvms12] d. ratiu, m. voelter, z. molotnikov, b. schaetz. implementing modular domain specific languages and analyses. proceedings of the workshop on model-driven engineering, verification and validation modevva ’12, 2012. doi:10.1145/2427376.2427383 [sm09] b. steffen, t. margaria. continuous model driven engineering. 2009 14th ieee international conference on engineering of complex computer systems, 2009. doi:10.1109/iceccs.2009.58 [sn16] b. steffen, s. naujokat. archimedean points: the essence for mastering change. pp. 22–46. springer international publishing, cham, 2016. doi:10.1007/978-3-319-46508-1 3 [ssa14] c. seidl, i. schaefer, u. aßmann. integrated management of variability in space and time in software families. proceedings of the 18th international software product line conference on splc ’14, 2014. doi:10.1145/2648511.2648514 [tor04] m. torgersen. the expression problem revisited. ecoop 2004 – object-oriented programming lecture notes in computer science, p. 123–146, 2004. doi:10.1007/978-3-540-24851-4 6 [vv11] m. voelter, e. visser. product line engineering using domain-specific languages. 2011 15th international software product line conference, 2011. doi:10.1109/splc.2011.25 [wad98] p. wadler. the expression problem. http://homepages.inf.ed.ac.uk/wadler/papers/ expression/expression.txt/, nov 1998. [online; accessed 13-july-2017]. [wo16] y. wang, b. c. d. s. oliveira. the expression problem, trivially! proceedings of the 15th international conference on modularity modularity 2016, 2016. doi:10.1145/2889443.2889448 11 / 11 volume 74 (2017) http://dx.doi.org/10.1002/spe.2243 http://dx.doi.org/10.1007/978-3-319-47169-3_16 http://dx.doi.org/10.4204/eptcs.129.16 http://dx.doi.org/10.1145/2427376.2427383 http://dx.doi.org/10.1109/iceccs.2009.58 http://dx.doi.org/10.1007/978-3-319-46508-1_3 http://dx.doi.org/10.1145/2648511.2648514 http://dx.doi.org/10.1007/978-3-540-24851-4_6 http://dx.doi.org/10.1109/splc.2011.25 http://homepages.inf.ed.ac.uk/wadler/papers/expression/expression.txt/ http://homepages.inf.ed.ac.uk/wadler/papers/expression/expression.txt/ http://dx.doi.org/10.1145/2889443.2889448 research motivation related work meta tooling with cinco solution approaches and recurring problems language improvements via linking model conclusion the use of executable fit tables to support maintenance and evolution tasks electronic communications of the easst volume 8 (2008) proceedings of the third international ercim symposium on software evolution (software evolution 2007) the use of executable fit tables to support maintenance and evolution tasks filippo ricca, marco torchiano, massimiliano di penta, mariano ceccato and paolo tonella 12 pages guest editors: tom mens, ellen van paesschen, kim mens, maja d’hondt managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst the use of executable fit tables to support maintenance and evolution tasks filippo ricca1, marco torchiano2, massimiliano di penta3, mariano ceccato4 and paolo tonella5 1 filippo.ricca@disi.unige.it unit cini at disi, genova, italy 2 torchiano@polito.it politecnico di torino, italy 3 dipenta@unisannio.it university of sannio, benevento, italy 4 ceccato@fbk.eu 5 tonella@fbk.eu fondazione bruno kessler—irst, trento, italy abstract: acceptance testing is a kind of testing performed prior to software delivery. in the agile approach, acceptance test suites are specified by analysts and customers during the requirement elicitation phase and used to support the development/maintenance activities. this paper reports an experiment with master students that investigates on the usefulness of executable acceptance test cases, developed by using fit (framework for integrated test), during software maintenance and evolution activities. the preliminary results indicate that fit tables help students to correctly perform the maintenance/evolution tasks with no significant impact on time. keywords: experiment with students, acceptance testing, fit tables. 1 introduction fit (framework for integrated test) [mc05] is an open source framework used to express executable acceptance test cases in a simple way. fit lets analysts write acceptance tests (fit tables) using simple html tables. programmers write code (fixtures) to link the test cases with the system to verify. then, in a test-driven development scenario, they perform their development or maintenance task being supported by the execution of these test cases. in this paper we describe a controlled experiment aimed at assessing whether fit tables are helpful in maintenance tasks. we asked some master students to execute four maintenance/evolution tasks (two corrective maintenance tasks and two evolution interventions), providing them two java systems to be maintained with and without the fit tables. the development environment is eclipse with the plug-in fitnesse1 that implements the fit table-based 1 http://fitnesse.org/ 1 / 12 volume 8 (2008) mailto:filippo.ricca@disi.unige.it mailto:torchiano@polito.it mailto:dipenta@unisannio.it mailto:ceccato@fbk.eu mailto:tonella@fbk.eu the use of executable fit tables to support maintenance and evolution tasks approach. the research questions that we are interested in answering are: rq1: does the presence of fit tables help programmers to execute maintenance tasks? rq2: does the presence of fit tables improve the productivity in the execution of maintenance interventions? the dependent variable “correctness” was measured by exercising an alternative junit2 acceptance test suite, the variable “productivity” using time sheets where students annotated start and stop time expressed in minutes. preliminary results of our experiment show that fit tables help developers to correctly perform the four maintenance/evolution tasks given without affecting the productivity. the difference between the two groups considering the time to complete the tasks was similar and not significant. the paper is organized as follows: section 2, briefly, presents the framework for integrated test (fit) used in the experimental study. section 3 describes the definition, design and settings of the proposed experiment. results are presented in section 4 while related works, conclusions and future works are given, respectively, in section 5 and section 6. 2 fit tables, fixtures and test runner the fit tables serve as the input and expected output for the tests. figure 1 shows an example of column fit tables, a particular kind of table where each row represents a test case (inputs and output). the first five columns are input values (name, surname, address, date of birth and credit/debit) and the last column represents the corresponding expected output value (member number()). other than column fit tables, it is possible to specify action fit tables, to test user interfaces or work-flows. an action fit table represents a test case where the first column contains commands (start, enter, press, and check) used to simulate the actions that a user would perform on a screen while the other columns contain the parameters. others types of fit tables (see [mc05]) are: row fit tables, to validate collection of objects produced as the result of a query, and timedaction fit tables to deal with non functional requirements. with this large set of different types of fit tables we are fairly confident that each functional requirement of a traditional business systems (this could not to be true in other contexts, for example reactive systems) may be “transformed” in executable fit tables. developers write the fixtures to link the test cases with the system to verify. a component in the framework, the test runner, compares fit table data with actual values obtained from the system. the test runner highlights the results with colors (green = correct, red = wrong). see the relationships among fit tables, fixtures, test runner and system under test in figure 2. 2 http://www.junit.org/ proc. software evolution 2007 2 / 12 eceasst figure 1: example of column fit table. fit table column’s names without parenthesis represent input; parenthesis indicate output. 3 experiment definition, design and settings we conceived and designed the experiment following the guidelines by wohlin et al. [wrh+00]. the goal of the study is twofold: to analyze the use of fit tables with the purpose of evaluating their usefulness during maintenance tasks and to measure the effort (if any). the perspective is both of researchers, evaluating how effective are the fit tables during the maintenance activities, and of project managers, evaluating the possibility of adopting the fit tables in her/his organization. the context of the experiment consists of two objects – two java systems – and of subjects, 13 students from a master course. all the material of the experiment (sources, documents, questionnaire, etc.) will be available for replications on a website soon. 3.1 hypotheses the null hypotheses for the study are the following: • h0a the availability of fit test cases does not significantly improve the correctness of the maintained source code. • h0b the availability of fit test cases does not significantly affect the effort in the maintenance task. the context in which we investigate the above question has the following characteristics: (1) system requirements have been written in detail, (2) automated acceptance tests have been pro3 / 12 volume 8 (2008) the use of executable fit tables to support maintenance and evolution tasks figure 2: the complete testing process duced in the form of fit tables and (3) some change requirements are expressed only in textual form while other include also an automated fit test case. 3.2 treatments the treatments for the main factor (availability of test cases) are: (+) textual change requirements enhanced with fit tables and fixtures, thus enabling test case execution; (-) only textual change requirements. other independent variables (not accounted in this paper) to be considered could be: the objects, the labs and the subjects’ ability, if available. 3.3 objects the objects of the study are two simple java programs realized by students: latazza and avecalc. latazza is a coffee maker management support application. latazza helps a secretary to manage the sale and the supply of small-bags of beverages (coffee, tea, lemon-tea, etc.) for the proc. software evolution 2007 4 / 12 eceasst table 1: some requirements for latazza. r1 the secretary can sell small-bags of coffee, arabic coffee, tea, lemon-tea and camomile-tea. the cost of each small-bag is 0.62 euro. the secretary can select the kind of beverage and the number of small-bags and select the button sell. if there are enough small-bags then the sale is done, otherwise the sale can not be done. r2 the secretary can register a payment. she/he has to select the employee that perform the payment. this payment can extinguish a debt of the employee or it can used in future as advance fee. the payment must be > 0. r3 the secretary can buy boxes of beverages. a box contain 50 small-bags of beverages all of the same kind (i.e, 50 coffee or 50 arabic coffee, etc.). each box cost 31 euro. r4 the secretary can request the list of debtors with their debts. coffee-maker. the application supports two kinds of clients: visitors or employees (university employees and professors). employees can purchase beverage cash or on credit, visitors only cash. the secretary can: sell small-bags to clients, buy boxes of beverages (a box contains 50 beverage of the same kind), manage credit and debt of the employees, check the inventory and check the cash account. the system consists of 18 java classes for a total of 1121 locs. its requirement document comprises 9 requirements (see table 1 for the first four requirements) complemented with a total of 16 fit tables. avecalc is a simple “desktop application” that manages an electronic register (record book) for master students. a student can add a new exam to the register, remove an existing exam and remove all exams. an exam has a name, a cfu (a positive number that represent the university credits) and a (optional) vote. an exam without vote is an exam not taken. the vote must be included between 0 and 30 (or equal). if the vote is >= 18 then the vote is positive, otherwise it is negative. it is possible to save the register and to load it (all data or only positive exams). avecalc computes some statistics: average of the exams passed, total number of cfu, number of exams passed, (hypothetical) degree vote and whether the student has passed a number of exams sufficient to defend his/her thesis. the system consists of 8 java classes for a total of 1827 locs. its requirement document comprises 10 requirements complemented with a total of 19 fit tables. 3.4 population the subjects were 13 students from the course of laboratory of software analysis, in their last year of the master degree in computer science at the university of trento. the subjects had a good knowledge about programming, in particular java, and an average knowledge about software engineering topics (e.g. design, testing, software evolution). subjects have been trained in meaning and usage of fit tables and fitnesse3 with two theoretical lessons and two practical lessons (two hours each). 3 fitnesse is the tool that implement the fit table approach used in the experiment 5 / 12 volume 8 (2008) the use of executable fit tables to support maintenance and evolution tasks table 2: experimental design (s1 = latazza, s2 = avecalc; + = with fit tables, = without fit tables). group a group b group c group d lab 1 s1+ s1s2s2+ lab 2 s2s2+ s1+ s13.5 variables and experiment design the dependent variables to be measured in the experiment are the code correctness and the effort required to perform the maintenance task. the code correctness is assessed by executing a junit acceptance test suite — developed by someone different from who developed the fit tables — and measuring the percentage of test cases passed and failed. the effort was measured by means of time sheets (students marked start and stop time for each change requirements implemented). time is expressed in minutes. since the acceptance test suite for avecalc was made up of 25 test cases, while the one for latazza included just 24 test cases, a derived measure was adopted: the fraction of test cases passed. we adopt a balanced experiment design (see [wrh+00]) intended to fit two lab sessions (2hours each). subjects were split randomly into four groups, each one working in lab 1 on all tasks of a system with a treatment and working on lab 2 on the other system with a different treatment (see table 2 for a graphical representation of the design). 3.6 material and procedure as already mentioned, the test cases are written in the form of fit tables and the supporting environment is a fitnesse wiki. the development environment is based on the eclipse ide with the fitnesse plugin4. for each group we prepared an eclipse project containing the software and a fitnesse wiki with both requirements and change requirements. the projects were zipped and made available on a web server. the experiment was introduced as a lab assignment about fitnesse. every subject received: • summary description of the application • instructions to set-up the assignment (download the zipped eclipse project, import it, and start the embedded fitnesse server) • a post experiment questionnaire for each lab the subjects had two hours available to complete the four maintenance tasks: cr1 cr4 (see table 3) . the first two change requirements (corrective maintenance) are very easy to implement, while the third and fourth require more work to locate the code to be changed and implementing the change (evolution). the maintenance/evolution tasks, for the two different systems, are very similar and we think of comparable difficulty. 4 http://www.bandxi.com/fitnesse/ proc. software evolution 2007 6 / 12 eceasst table 3: change requirements for latazza. cr1 there is an error in show debtors. only employees with negative balance must be visualized. fix the error. cr2 there is an error in update employees. not all the fields are updated. fix the error. cr3 the vendor of boxes of beverages changed his selling policy. each five bought boxes one is added as a gift. cr4 change price of small-bags. now the total price of the beverages that an employee would like to buy depends on (i) the number of small bugs bought (ii) if the beverage is seasonal or not. if a employee buys a number of small bags minor than 5 no discount is applied. if a employee buys a number of small bags included between 5 and 10 of a seasonal beverage, no discount is applied; but if the beverages are not seasonal a 1 euro discount is applied. the post experiment questionnaire aimed at both gaining insights about the students’ behavior during the experiment and finding justifications for the quantitative results. it included questions about the task and systems complexity, the adequacy of the time allowed to complete the task and the perceived usefulness of the provided fit tables. before the experiment, subject were trained by means of introductory lectures (2 lessons 2 hours each) and laboratories (4 hours) on fit. after subject were randomly assigned to the four groups, the experiment execution followed the steps reported below: 1. we delivered a sheet containing the description of the system. 2. subjects had 10 minutes to read the description of the system and understand it. 3. subjects had to write their name and start time on the delivered sheet. 4. subjects had to download at the given url the eclipse project and import it. 5. subjects had to launch the fitnesse wiki of the application. 6. subjects had to write the stop time for installing the application. 7. for each change requirement (cr1-cr4): (a) subjects had to fix the application code (latazza or avecalc) in order to make the test cases pass (treatment +) or to satisfy the change requirement (treatment -). (b) subjects had to record the time they use to apply change task (start/stop time). 8. subjects were asked to compile the post experiment questionnaire. 7 / 12 volume 8 (2008) the use of executable fit tables to support maintenance and evolution tasks 4 experimental results there were 13 subjects divided into three groups of three and one group of four. they took a median of 5 minutes to set up the environment and they worked for a median of 73 minutes on the tasks. the subjects deemed as complete an average of 2.75 tasks over four tasks assigned. the subjects worked on each task for a time ranging from 11 to 39 minutes with an average of 21 minutes. the distributions of passed tests and time required to complete tasks are not normal (shapiro-wilk test p=0.026 and p=6.9 · 10−6 respectively) therefore we will use the mann-whitney test for both hypotheses. 4.1 data analysis to test the first hypothesis (h0a) we compared the number of acceptance tests passed by the program whose change requirements included fit tables or not. the boxplot summarizing the percentage (expressed as fraction) of passed test cases is presented in figure 3. the percentage of passed test cases for the fit group is about 60% while that one for the text only group is a little bit more than 40%. by applying a one-tailed mann-whitney test, we found this difference to be statistically significant (p-value=0.03), therefore we can reject the null hypothesis. no yes 0.0 0.2 0.4 0.6 0.8 1.0 fit tables present fraction of tests passed figure 3: boxplot of fraction of passed tests. the second hypothesis can be tested by looking at the time required to complete the tasks. since not all students completed all the tasks and since the tasks’s difficulty varied both among tasks and systems, we analyzed the time for each task. figure 4 shows the boxplot of times used by subjects to complete each task; filled boxes correspond to the presence of fit tables. to test the second hypothesis we used a mann-whitney test. table 4 reports the p-values of mannwhitney tests for each task. overall in 5 cases out of 8 (see figure 4) we observe a reduction of time (considering the median) when fit tables are present but the only significant difference (highlighted in boldface in 4) is found for the first task on system avecalc. with only these data we cannot reject the null hypothesis h0b. further experiments are necessary to answer our proc. software evolution 2007 8 / 12 eceasst second research question. no yes no yes no yes no yes no yes no yes no yes no yes 10 20 30 40 50 60 70 ti m e to c om pl et e ta sk [m in ] 1 2 3 4 1 2 3 4 avecalc latazza | | | fit present: task: system: figure 4: boxplot of time required to complete task. system avecalc latazza task p-value median yes median no p-value median yes median no 1 0.01 8 18 0.83 12 15.5 2 0.33 6 12 0.57 15 9 3 1.00 40 43 0.53 39 29 4 0.63 28 17 0.45 10 26 table 4: analysis results on times to complete tasks. 4.2 analysis of survey questionnaires the analysis of the survey questionnaires that the subjects filled-in after each experiment can be useful to better understand the experimental results. in this paper the analyses are supported only by descriptive statistics. answers are on a likert scale [opp92] from 1 (strongly agree) to 5 (strongly disagree). overall, all subjects agreed they had enough time to perform the tasks (i had enough time to perform the lab tasks, overall mean = 2.35) and the objectives were clear enough (the objectives 9 / 12 volume 8 (2008) the use of executable fit tables to support maintenance and evolution tasks of the lab were perfectly clear to me, overall mean = 1.73). the description of the systems were clear (overall mean = 2.08) as the change requirements (overall mean = 2.35). similarly to melnik et al. [mrm04], we can observe that the students deemed the fit tables and the capability of running tests automatically useful enough. the possibility of executing fit tables as tests was perceived useful for performing the change (running fit tables are useful in maintenance/evolution tasks, mean = 1.69). moreover, fit tables were also considered useful “per-se” to clarify change requirements (fit tables are useful to clarify change requirements, mean = 1.92). see [rtct07] for another experiment with students treating the research question: “fit tables are able to clarify (change) requirements?”. 4.3 threats to validity this section discusses the threats to validity that can affect our results: internal, construct, conclusion and external validity threats. internal validity threats concerns external factors that may affect a dependent variable (in our case code correctness and effort). since the subject had to perform a sequence of four tasks, a learning effect may intervene: the subjects were previously trained and the tasks were of progressively increasing difficulty, therefore we expect learning not to have influenced significantly the results. the experiment was proposed as an ungraded assignment within a university course, thus the student should not have been subject to evaluation apprehension. construct validity threats concern the relationship between theory and observation. it is possible that the junit test suite does not provides and adequate means to measure the quality of change requirement implementation. we mitigated this risk by having the test suite and the fit tables developed independently from different people. threats to conclusion validity can be due to the sample size (only 13 subjects) that may limit the capability of statistical tests to reveal any effect. however, attention was paid to not violate assumptions made by statistical tests and, whenever conditions necessary to use parametric statistics did not hold, we used non-parametric tests. threats to external validity can be related to (i) the simple java system chosen and (ii) to the use of students as experimental subjects. we do not expect the absolute performance of students being at the same level as professionals, but we expect to be able to observe a similar trend of improvement. another threat to external validity is that (iii) the results are limited to fit-based acceptance test suites, which may be rather different from other approaches to acceptance testing. we don’t think that obtained results could change using a different implementation of the fit table based approach5 or a different development environment. all the existing implementations of fit are very similar to fitnesse. further studies with larger systems and more experienced developers are needed to confirm or contrast the obtained results. 5 for example fitlibrary, mavenfit, etc. see http://fit.c2.com/wiki.cgi?fittools proc. software evolution 2007 10 / 12 eceasst 5 related work although there are several papers [aar06, rmm05] and books [mc05] describing acceptance testing with fit tables, only a few works report empirical studies about fit. the most related work is the paper by melnik et al. [mrm04]. it is a study focused on the use of fit user acceptance tests for specifying functional requirements. it has been conducted at the university of calgary and at the southern alberta institute of technology. in this experiment, the authors showed that the use of fit tables and the possibility to execute them improve the comprehension of requirements. melnik et al. [mmc06] investigated whether acceptance tests can be authored effectively by customers of agile projects. results show that customers can specify functional requirements clearly. the paper [rtct07] reports a controlled experiment with master students aimed at assessing the impact of fit tables on the clarity of requirements. the results obtained indicate that fit helps in the understanding of the requirements. in another preliminary study [trd07] some of the authors of the present paper found a statistically significant evidence that the availability of fit tables allows the programmers to complete more maintenance tasks. however, they did not measure, as we did in the present study, whether completed maintenance tasks were correct. 6 conclusion and future work this paper reported a controlled experiment with 13 master students aimed at assessing the use of fit executable acceptance test suites in the context of maintenance and evolution tasks. the obtained results indicates that fit tables significantly help developers to correctly perform the maintenance tasks. other than looking at requirements, developers continuously execute fit test cases to (i) ensure that fit tables related to the change requirements passed and (ii) use requirement fit tables to regression test the existing pieces of functionality. regarding productivity, fit tables may or may not help: on the one hand, they provide a guideline to perform the maintenance tasks; on the other hand, they require time to be understood and executed. further investigation is anyway necessary to answer our second research question. future work aims at replicating this study with a larger population of students, with professionals and by using larger and more realistic software systems. also, other metrics (e.g., number of change requirements completed) and other factors such as subjects’ ability and experience will be taken into account. acknowledgements: we thank all the students of the course of laboratory of software analysis at the university of trento who participated in the experiment. without them this work would not have been possible. 11 / 12 volume 8 (2008) the use of executable fit tables to support maintenance and evolution tasks bibliography [aar06] j. aarniala. acceptance testing. in whitepaper. www.cs.helsinki.fi/u/jaarnial/jaarnial-testing.pdf. october 30 2006. [mc05] r. mugridge, w. cunningham. fit for developing software: framework for integrated tests. prentice hall, 2005. [mmc06] g. melnik, f. maurer, m. chiasson. executable acceptance tests for communicating business requirements: customer requirements. in proceedings of agile 2006 conference (agile2006). pp. 35–46. ieee computer society, los alamitos, ca, usa, 2006. [mrm04] g. melnik, k. read, f. maurer. suitability of fit user acceptance tests for specifying functional requirements: developer perspective. in extreme programming and agile methods xp/agile universe 2004. pp. 60–72. august 2004. [opp92] a. n. oppenheim. questionnaire design, interviewing and attitude measurement. pinter, london, 1992. [rmm05] k. read, g. melnik, f. maurer. examining usage patters of the fit acceptance testing framework. in proc. 6th international conference on extreme programming and agile processes in software engineering (xp2005). pp. lecture notes in computer science, vol. 3556, springer verlag: 127–136 2005. june 18-23 2005. [rtct07] f. ricca, m. torchiano, m. ceccato, p. tonella. talking tests: an empirical assessment of the role of fit acceptance tests in clarifying requirements. in 9th international workshop on principles of software evolution (iwpse 2007). pp. 51–58. ieee, september 2007. [trd07] m. torchiano, f. ricca, m. di penta. ”talking tests”: a preliminary experimental study on fit user acceptance tests. in ieee international symposium on empirical software engineering and measurement. (to appear) 2007. [wrh+00] c. wohlin, p. runeson, m. höst, m. ohlsson, b. regnell, a. wesslén. experimentation in software engineering an introduction. kluwer academic publishers, 2000. proc. software evolution 2007 12 / 12 introduction fit tables, fixtures and test runner experiment definition, design and settings hypotheses treatments objects population variables and experiment design material and procedure experimental results data analysis analysis of survey questionnaires threats to validity related work conclusion and future work flowemu: an open-source flow-based network emulator electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) flowemu: an open-source flow-based network emulator daniel stolpmann, andreas timm-giel 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst flowemu: an open-source flow-based network emulator daniel stolpmann1, andreas timm-giel2 1daniel.stolpmann@tuhh.de, 2timm-giel@tuhh.de hamburg university of technology, institute of communication networks, germany abstract: when analyzing the impact of a communication system on the quality of experience (qoe) of an interactive application, simulation and mathematical modeling typically require reimplementation or modeling of the application under test and only provide insights in terms of selected key performance indicators (kpis), which raises the need for network emulation. in this demo, we present flowemu, an opensource flow-based network emulator that allows the user to manipulate the underlying model and analyze various statistics in real-time via an easy-to-use graphical user interface (gui), by using interactive game streaming as an example application. keywords: network emulation, flow-based programming, game streaming 1 introduction network emulation combines abstract models and real hardware / software components to mimic the behavior of a communication system. compared to simulation and mathematical modeling, which typically require reimplementation or modeling of the application under test and only provide insights in terms of selected key performance indicators (kpis), this makes it possible to showcase the impact of a communication system on the quality of experience (qoe) of a real interactive application. in this demo, we present flowemu, an open-source flow-based network emulator. flowemu provides an extendable set of modules that implement basic functions such as adding delay and introducing packet loss as well as packet queues and departure processes. these modules can be freely connected to model the impact of different types of communication systems on the network traffic of a real application. by providing an easy-to-use web-based graphical user interface (gui), flowemu allows the user to change the structure and parameters of the model at runtime. additionally, it gives insights on internal states by visualizing various statistics. this paper is structured as follows. in section 2, existing network emulators are presented and compared to flowemu. section 3 describes the architecture of flowemu and highlights some of its features. in section 4, we give an overview of our demo at netsys 2021. in the end, we provide a summary in section 5. 2 related work netem [hem05] is an open-source network emulator that is implemented as a queuing discipline (qdisc) in the linux kernel and can be set up on any network interface using the “tc” command 1 / 4 volume 080 (2021) mailto:daniel.stolpmann@tuhh.de mailto:timm-giel@tuhh.de flowemu: an open-source flow-based network emulator line tool. besides being able to emulate uncorrelated and correlated delay, packet loss, packet corruption, packet duplication and packet re-ordering, it also provides bit rate emulation [lin]. however, without workarounds, it can only be applied to the outgoing traffic. also, the included discrete gilbert-elliot model is evaluated for each packet, which makes the wall-clock sojourn times dependent on the packet rate. in contrast, flowemu provides a continuous-time gilbertelliot model that can be applied to both directions of the traffic simultaneously. facebook’s augmented traffic control (atc) [fac] is based on netem and targeted to application developers for testing. it features a web-based gui and a representational state transfer (rest) application programming interface (api) that allows the user to set up and switch between profiles that are parametrized for emulating different types of networks. the project is open-source, but was officially discontinued. compared to this, flowemu is under active development and provides an message queuing telemetry transport (mqtt) interface as well as a web-based gui and the option to save and load models. in [ste17], a framework for analyzing active queue management (aqm) algorithms was developed. it is build around netem and the linux kernel implementations of different aqm algorithms. the framework supports execution on a physical testbed consisting of multiple computers or a single machine using docker. in order to get statistics for queuing delay and packet drops, the implementations of the aqm algorithms in the linux kernel have to be modified to write these statistics into the 16 bit “identification” field of the internet protocol version 4 (ipv4) header. from there, these statistics can be read by the receiving application. flowemu also provides the flexibility of running experiments on a testbed or a single machine using docker, but allows each module to directly record statistics independently of the application under test. mahimahi [nsd+15] is an open-source record-and-replay framework for hypertext transfer protocol (http) traffic, which contains a set of network emulation tools called “shells”. each shell creates its own network namespace, in which any program can be executed, and operates on the incoming and outgoing traffic of it. multiple shells can be chained together by starting them in each other’s namespace. besides shells that are able to add fixed delay or uncorrelated packet loss, there is also a shell that emulates a link based on a trace file, which contains timestamps of packet transmission opportunities. the link shell has an internal queue and is able to create live graphs of the channel capacity, the throughput and the queuing delay. flowemu reimplements many of mahimahi’s features and is even able to read its trace files for link emulation. additionally, it is more flexible due to its modular design and the way modules can be connected while introducing less overhead by running everything in a single process. 3 architecture and features flowemu is written in c++ and runs on linux as a single user space process. the network interfaces are accessed using raw sockets, which provide layer 2 access, so the emulator is able to process the ethernet medium access control (mac) header and all higher layer information of the packets. all functions such as adding delay, introducing packet loss as well as the packet queues and departure processes are implemented in separate modules. each module has multiple “ports”, which are used to connect the modules to each other. depending on its functionality, a module netsys 2021 2 / 4 eceasst can have sending, receiving, requesting or responding ports, which can be connected independently for each direction of a data flow. packets are passed between modules as references, so no unnecessary copying overhead is introduced. additionally, each module can define a set of parameters that can be set by the user and a set of statistics to be recorded. flowemu provides an extensive toolchain for running experiments either in a virtual environment or on a physical testbed. running the emulator as well as the client and server components of the application under test on a single machine in separate docker containers, which are connected via bridge interfaces, enables fast and convenient development and testing. when running experiments and recording statistics, it is recommended to run the emulator and the application components on separate machines to improve the real-time behavior. message queuing telemetry transport (mqtt) interface flowemu is fully controllable via mqtt. this includes changing the structure of the model as well as setting the parameters of the individual modules and getting their statistics. by writing special driver applications, this also allows the emulator to interface with hardware devices, such as microcontrollers and musical instrument digital interface (midi) controllers. graphical user interface (gui) flowemu provides a web-based gui that communicates with the emulator via mqtt over a websocket. the gui features a graphical node-based editor, which allows the user to freely connect the modules by drag-and-drop, set the parameters of the individual modules and visualize their statistics. runtime reconfigurable model flowemu applies changes to the structure of the model and the parameters of the individual modules at runtime without resetting the internal state of the modules, which makes it possible to immediately see their effect on the application performance. besides being an impressive feature for demonstration purposes, this can also be used to quickly test various parameters and find value ranges that are worth to be investigated. real-time statistics each module in flowemu can provide statistics in real-time, which can give insights on internal states such as queue lengths or can be used if the application under test does not provide end-to-end packet level statistics such as throughput or delay. independent and seedable pseudo random number generators in flowemu, each module uses its own pseudo random number generator, which ensures that the behavior of a module is not influenced by other modules. additionally, the seed of each pseudo random number generator can be set, which allows results to be reproducible and comparable. 4 netsys 2021 demo at netsys 2021, we present flowemu in a demo. an overview of the setup is shown in figure 1. the demo is based on a cloud gaming scenario, where a video game is executed on a server and streamed as a video to a client while the inputs of the player are sent back from the client to the server. for this, we use steam’s remote play [val] feature to stream a game from a highend computer to another device over the network emulator. when changing the structure and parameters of the underlying model via the gui while the game is being played, the impact of these changes on the performance of the interactive game streaming application can be seen. 3 / 4 volume 080 (2021) flowemu: an open-source flow-based network emulator inputs video server clientflowemu figure 1: overview of the demo setup 5 summary in this demo, we present flowemu, an open-source flow-based network emulator that allows the user to manipulate the underlying model and analyze various statistics in real-time via an easy-to-use gui and offers advanced features such as independent and seedable pseudo random number generators. for demonstration purposes, we use interactive game streaming as an example application. the source code of flowemu is available at [sto], where we also provide further documentation and invite everyone to contribute improvements and new modules. bibliography [fac] facebook. augmented traffic control. (accessed 12.08.2021). https://github.com/facebookarchive/augmented-traffic-control [hem05] s. hemminger. network emulation with netem. in linux.conf.au 2005. canberra, australia, apr. 2005. [lin] linux foundation. netem. (accessed 12.08.2021). https://wiki.linuxfoundation.org/networking/netem [nsd+15] r. netravali, a. sivaraman, s. das, a. goyal, k. winstein, j. mickens, h. balakrishnan. mahimahi: accurate record-and-replay for http. in 2015 usenix annual technical conference (usenix atc ’15). pp. 417–429. santa clara, ca, usa, july 2015. [ste17] h. steen. destruction testing: ultra-low delay using dual queue coupled active queue management. master’s thesis, university of oslo, oslo, norway, 2017. [sto] d. stolpmann. flowemu. https://github.com/comnetshh/flowemu [val] valve corporation. steam remote play. (accessed 12.08.2021). https://store.steampowered.com/remoteplay netsys 2021 4 / 4 https://github.com/facebookarchive/augmented-traffic-control https://wiki.linuxfoundation.org/networking/netem https://github.com/comnetshh/flowemu https://store.steampowered.com/remoteplay introduction related work architecture and features netsys 2021 demo summary user space packet schedulers: towards rapid prototyping of queue-management algorithms electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) user space packet schedulers: towards rapid prototyping of queue-management algorithms ralf kundel, paul stiegele, dat tran, julian zobel, osama abboud, rhaban hark, ralf steinmetz 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst user space packet schedulers: towards rapid prototyping of queue-management algorithms ralf kundel1, paul stiegele1, dat tran1, julian zobel1, osama abboud2, rhaban hark1, ralf steinmetz1 ralf.kundel@kom.tu-darmstadt.de 1 multimedia communications lab (kom) technical university of darmstadt, germany 2 huawei technologies duesseldorf gmbh, germany abstract: quality of service indicators in computer networks reached tremendous importance over the last years. especially throughput and latency are directly influenced by the dimension of packet queues. determining the optimal dimension based on the inevitable tradeoff between throughput and latency tends to be a hard, almost infeasible challenge. several algorithms for active queue management have been proposed to address this challenge over the last years. however, the deployment and by that the development of such algorithms is challenging as they are usually located within the operation systems’ kernel or implemented in fixed hardware. in this work, we investigate how novel algorithms can be deployed in user space for rapid prototyping with tolerable effort. we provide core performance characteristics and highlight the viability and reasonability of this approach. keywords: aqm, user space, congestion control, bufferbloat introduction, background, and related work: internet applications have experienced an enormous upswing, most notably through intensive video streaming and conferencing applications realized by congestion-controlled tcp. besides steadily increasing requirements regarding throughput and availability, especially latency became more and more important as a quality of service (qos) criterion. however, there is a tradeoff between high bandwidth utilization and low latency packet forwarding for congestion-controlled flows when determining the dimension of packet queues on forwarding routers. throughput-intensive applications require sufficiently large packet queues in order to entirely utilize a link, whereas latency-sensitive applications suffer from too large packet queues, known as bufferbloat phenomenon [gn12]. the optimal queue size in terms of maximizing throughput without unnecessary increase in latency is considered to be b = rtt·c√n , where b is the maximum queue size, c the bottleneck link speed, and rt t the round trip time of n tcp flows [akm04]. however, this formula and its application on packet queue engineering has several challenges. first, this “optimal” queue size primarily focuses on full utilization of the subsequent bottleneck link and only secondary on latency reduction. second, constantly having n congestion-controlled tcp flows with a known rt t is not realistic as internet traffic changes dynamically and numerous different congestion control algorithms are present in parallel [kwm+19]. third, this rule of thumb considers only long data transmissions, so called “elephant flows”, although there is always a mixture of long and short flows in real networks. 1 / 4 volume 080 (2021) mailto:ralf.kundel@kom.tu-darmstadt.de user space packet schedulers to this end, numerous general but advanced active queue management (aqm) algorithms have been proposed [ada13][kbv+18]. they tackle the problem of bufferbloat by intelligently dropping packets or marking them with a congestion notification bit. however, no algorithm fitting all use cases has been found to this point and it is unlikely to exist. certainly, the development and experimental evaluation of such algorithms is challenging as they are either located within the operating system kernel, e.g., the linux kernel, or within hardware of packet forwarding chips (i.e., asics). with this work, we propose the idea of packet queuing and scheduling in the user space for rapid prototyping and preliminary evaluation of novel aqm algorithms. note that congestion control experiments are usually not performed by simulations, due to an easier reproduction of complexity and heterogeneity of real computer networks in emulation environments, mainly mininet. hence, our work focuses on network emulations. the most related approach to this work is the linux kernel extension libnetfilter queue1. this library enables a hybrid coexistence of queues in the kernel space and a packet scheduler in the user space. the scheduler decides whether a packet is dropped or not based on packet metadata information. however, advanced queuing and scheduling structures exceed the api capabilities, e.g., front drop instead of tail drop or push-in-extract-out queues cannot be investigated. prototype and preliminary evaluation results: in order to realize a user space queue emulator, we first investigate which programming languages should be used with regard to their packet i/o performance. for that, we create a simple mininet topology h1 ←→ h2 ←→ h3 consisting of three hosts. host h1 and h3 perform a bandwidth performance test with the iperf3 tool. host h2 is responsible to forward all packets from h1 to h3 and vice versa without queuing. to evaluate the performance of the packet forwarding, we measure the throughput of the iperf3 tcp flow and the observed round trip time (rtt). performance characteristics for six different packet forwarding implementations are displayed in figure 1. we use native linux kernel and the userspace implementation in c as baseline. forwarding incoming packets based on the linux kernel’s ipv4 routing table represents a theoretically achievable upper bound for throughput and a lower bound for rtt. the low-level user space implementation in c, compiled with -lpcap flags and no further optimizations, is supposed to be a even better upper bound for throughput. we realized four implementations in different programming languages with similar behavior to the baseline designs for the user space queuing system. note that the user space implementations are built upon a raw socket kernel module and thus the kernel is still involved. they are implemented as multi-threaded applications and use raw sockets to receive and send packets. one thread receives packets on the first interface and immediately sends them out on the second one. the second thread is responsible for similar packet forwarding in the opposite direction. more threads per direction are only meaningful if a hardware network interface card with load balancer is in use. the third and main thread is responsible to monitor these two workers. this multi-threading approach achieves almost a twofold increase in forwarding performance compared to a single-threaded approach. as depicted in figure 1, within the user space implementations, go achieves the best rtt. the c user space baseline-implementation, in contrast, achieves the best throughput, but surprisingly also experiences a significantly higher rtt. this is caused by a different interrupt handling 1 the netfilter.org libnetfilter queue project. www.netfilter.org/projects/libnetfilter queue/. accessed: 2020-10-13. netsys 2021 2 / 4 www.netfilter.org/projects/libnetfilter_queue/ eceasst linux kernel c go rust python2 python3 0 1000 2000 3000 4000 5000 6000 7000 tc p ra te [m bi t/s ] linux kernel c go rust python2 python3 0 1 2 3 4 5 6 o bs er ve d rt t [m s] figure 1: user space packet forwarding characteristics with disabled checksum offloading. the linux ipv4 forwarding is by default part of the linux kernel and represents an upper bound. all tests are compiled and running on a intel xeon d-1541 with ubuntu 16.04 (kernel 4.4.0). structure within the c raw socket implementation. note that this comparison is an evaluation of the raw socket performance within these languages only and not a benchmark of the languages itself. in total, go performed best considering both metrics of the four high-level languages. as prototyping in the queueing system should be as easy as possible, a high-level language such as go, rust or python is pursued instead of low-level c. as it performed best in our preceding tests, we decided to implement the user space queuing system in go, including queues with exchangeable schedulers and aqm algorithms. to showcase the benefits of such a user space queuing system, the prototype was integrated into the previous mininet topology, as depicted in figure 2a. for that, two experiments with different queue behavior were performed using (1) a tail drop queue and (2) a front drop queue with a rate limit of 50 mbit/s . in case of a full queue, when packet loss is unavoidable on receiving another packet, the former approach drops the new packet, while the latter drops the first in the queue and then adds the new packet at the tail. as the congestion control detects the packet loss earlier in the latter case, we assume a different behavior in both scenarios which shall be further investigated. exemplary results for both approaches are compared in figure 2b. in this concrete case, h1 queueing system fx gx h3 mininet network raw socket raw socket kernel space user space h2 (a) 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 0 25 ta il d ro p 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 time [s] 0 50 fr on t d ro p (b) figure 2: results of an exemplary user space queuing experiment. (a) integration of queueing system into simple mininet topology. (b) packet latency for dropping packets in the front/tail of the queue while overflowing. red bars mark dropped packets (22 tail drops, 15 front drops). 3 / 4 volume 080 (2021) user space packet schedulers we observe that the front drop queue creates less packet loss—15 compared to 22 lost packets— because tcp congestion control receives feedback earlier and thus adapts the sending rate sooner. this quickly realized experiment in combination with easily generated but yet expressive results highlight the major advantage of our user space queue emulator. in contrast to kernel or hardware implementations, the user space allows to log all dropping events. in essence, it is easy and convenient to determine why, where, and when a packet is dropped using user space packet schedulers for rapid prototyping. conclusion and outlook: active queue management is highly important in modern networks in order to meet the growing quality-of-service requirements. however, their development and evaluation is challenging due to their typical implementation in the kernel or in hardware. we therefore propose the idea of user space queuing systems for rapid prototyping of novel aqm algorithms. we performed experiments comparing four different high level programming languages for their applicability to implement such a system. the results have shown that raw sockets and user space packet queues achieve bandwidths above 1 gbit/s, which is sufficient for most aqm-experiments. additionally, two simple queuing algorithms were evaluated using a prototypical implementation of our user space queuing system. because aqm prototyping in user space allows to conveniently test and evaluate novel aqm approaches, future work will focus on the realization of programmable user space queues in real computer networks based on high performance packet i/o frameworks together with high-level programming languages. acknowledgements: this work has been supported by the federal ministry of education and research (bmbf, germany) within the software campus project ”5g-pci” and in parts by the german research foundation (dfg) as part of the projects b1 and c2 within the collaborative research center (crc) 1053 maki and the loewe initiative (hessen, germany) within the project nature 4.0 and the emergencity centre. bibliography [ada13] r. adams. active queue management: a survey. ieee communications surveys tutorials 15(3):1425–1476, 2013. [akm04] g. appenzeller, i. keslassy, n. mckeown. sizing router buffers. in conference on applications, technologies, architectures, and protocols for computer communications. p. 281–292. acm, 2004. [gn12] j. gettys, k. nichols. bufferbloat: dark buffers in the internet. communications of the acm 55(1):57–65, 2012. [kbv+18] r. kundel, j. blendin, t. viernickel, b. koldehofe, r. steinmetz. p4-codel: active queue management in programmable data planes. in proceedings of the conference on network function virtualization and software defined networks (nfv-sdn). pp. 1–4. 2018. [kwm+19] r. kundel, j. wallerich, w. maas, l. nobach, b. koldehofe, r. steinmetz. queueing at the telco service edge: requirements, challenges and opportunities. in workshop on buffer sizing. stanford, us, 2019. netsys 2021 4 / 4 reconfiguration of reo connectors triggered by dataflow electronic communications of the easst volume 10 (2008) proceedings of the seventh international workshop on graph transformation and visual modeling techniques (gt-vmt 2008) reconfiguration of reo connectors triggered by dataflow christian koehler, david costa, josé proença, farhad arbab 13 pages guest editors: claudia ermel, reiko heckel, juan de lara managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst reconfiguration of reo connectors triggered by dataflow christian koehler, david costa, josé proença, farhad arbab cwi, amsterdam abstract: reo is a language for coordinating autonomous components in distributed environments. coordination in reo is performed by circuit-like connectors, which are constructed from primitive, mobile channels with well-defined behaviour. while the structure of a connector can be modeled as a graph, its behaviour is compositionally defined using that of its primitive constituents. in previous work, we showed that graph transformation techniques are well-suited to model reconfigurations of connectors. in this paper, we investigate how the connector colouring semantics can be used to perform dynamic reconfigurations. dynamic reconfigurations are triggered by dataflow in the connector at runtime, when certain structural patterns enriched with dataflow annotations occur. for instance we are able to elegantly model dynamic reo circuits, such as just-in-time augmentation of singlebuffered channels to a circuit that models a channel with an unbounded buffer. additionally we extend reo’s visual notation and the reo animation language to describe and animate dynamically reconfiguring connectors. keywords: coordination, reconfiguration, graph transformation, animation. 1 introduction the coordination paradigm provides models for describing the communication among the components in a composed system. coordination languages, such as reo [arb04], describe the ‘gluing’ of loosely coupled components, such that a desired system behaviour emerges. the achieved separation of business logic and coordination of the active entities leads to a much cleaner design and helps to handle the greater complexity of large applications. reo can be applied in various distributed scenarios—from service-oriented to grid computing—as the coordination model is exogenous and independent from the actual component implementation and infrastructure. in a reo network, software components are autonomous, self-contained entities that communicate with the outside world via a published interface. to avoid dependencies and to achieve a truly modular, distributed system, reo proposes the notion of connectors which are used to coordinate the components without their knowledge. by this, the system is divided into two orthogonal aspects: 1) the computation, performed by the components and 2) the coordination of these independent components, performed by the connectors. a major advantage of this design is the ability of changing the topology of the connector, and thereby the behaviour of the system. the configuration of a reo connector consists on the interconnection between the structural elements of the connector, together with their states. communication with the components may change the state of the connector, but not its topology. in this paper, we consider reconfigurations of a connector as high-level transformations of its underlying graph structure. using the theory 1 / 13 volume 10 (2008) reconfiguration of reo connectors triggered by dataflow of typed, attributed graph transformation, we can directly apply many useful results, such as termination and confluence theorems [kla07]. in this paper, we explore the interplay between the data flow in a connector and its reconfigurations. for this, we include the connector colouring semantics into the patterns of transformation rules. transformations are automatically applied depending on the structure, the state and the context of a connector. connectors are reconfigured at run-time based on this information. this leads to a powerful notion of dynamic connectors. we illustrate the principles through the example of a dynamically growing buffer. related work the logic rectl* was introduced in [cla08] to reason about connector reconfiguration in reo. the reconfigurations are performed using the basic primitive operations of the reo api, e.g. channel creation or node splitting. up to now we have not provided a technique to reason about our dynamic high-level reconfigurations. however, we plan to look at model checking techniques for graph transformation as proposed in [ren03a]. architectural design rewriting (adr) [blmt07] is a framework for general reconfigurable software architectures. as in our approach, reconfigurations are modeled using graph transformation rules. reconfigurations occur at run-time whenever the system evolves to a configuration that violates the architectural style of the system. in our case we do not fix the use of the reconfiguration to any particular purpose. the dynamic f if o introduced in this paper can be seen as an example of the use of reconfiguration to guarantee dataflow. if a write operation is performed while the buffer is full, the buffer is reconfigured to allow data to be stored and dataflow to occur. adr is not tied to any architectural style (e.g. client-server, peer-to-peer) while in our case, reo determines the architecture. a systematic introduction to animations based on graph transformation concepts was given in [erm06]. the animation language for reo that we use in this paper was introduced in [cp07]. it is important to note that the animation language for reo is not based on graph transformation. instead the authors introduce an abstract animation language that can be used to compute animation descriptions for a connector compositionally out of the descriptions of its constituent primitives. structure of the paper this paper is organised as follows. section 2 gives a general introduction to reo by introducing the notions of channels, nodes and connectors. an overview of the colouring semantics for reo is given in section 3. we recall the concepts of graph-based reconfigurations and provide our contributions to dynamic reconfigurations in section 4. we discuss the proposed model in section 5. the status of the current implementation and plans for future work are given in sections 6 and 7, respectively. 2 reo overview reo is an exogenous coordination language where complex connectors are compositionally built out of simpler ones. the simplest (atomic) connectors in reo consist of a user defined set of channels, each of which with its particular constraint policy. proc. gt-vmt 2008 2 / 13 eceasst a channel is a medium of communication with exactly two directed ends. there are two types of channel ends: source and sink. a source channel end accepts data into its channel. a sink channel end dispenses data out of its channel. a channel can connect two components or be composed with other channels using reo nodes to build more complex connectors. reo nodes are logical places where channel ends coincide. a node with only source channel ends is a source node; a node with only sink channel ends is a sink node; and finally a node with both source and sink channel ends is a mixed node. we use the term boundary nodes to refer indistinguishably to source and sink nodes. boundary nodes define the interface of a connector. components connect to and interact anonymously with each other through the interface of the connector by performing i/o operations on its boundary nodes: take and read operations on sink nodes, and write operations on source nodes. reo fixes the constraint policy for the dataflow in reo nodes. data flows through a node only if at least one sink channel end is pushing data and all the source channel ends can accept a copy of the data. in case more than one sink channel end is pushing data, one is picked nondeterministically and all the others are excluded. data cannot be stored in a node, hence its constraints on dataflow and exclusion must propagate through the connector. resolving the composition of the constraint policies of a connector consisting of several channels and nodes is a non-trivial task. in figure 1 we present two examples of reo connectors that illustrate how non-trivial dataflow behaviour emerges from composing channels using reo nodes. the constraints propagate through the (synchronous regions of the) connector to the boundary nodes. the propagation enables a certain context-awareness in connectors. a detailed discussion of this can be found in [cca07]. (a) (b) figure 1: (a) exclusive router, (b) ordering connector. the two connectors in figure 1 involve, in total, four different types of channels. we represent mixed nodes as filled circles ( ), and boundary nodes as empty circles ( ). the sync channel ( ) synchronously takes a data item from its source end and makes it available at its sink end. this transfer can succeed only if both ends are ready to communicate. the lossysync ( ) has the same behavior, except that it does not block if the receiver cannot accept data. in this case, the written data item is accepted and destroyed by the channel. the fifo1 ( ) is an asynchronous channel that has a buffer of size one. unlike the prior channels, fifo1 is a stateful channel. the syncdrain channel ( ) has two source ends through which it can only consume data, and no sink ends. its behavior can be described as follows: if there are data 3 / 13 volume 10 (2008) reconfiguration of reo connectors triggered by dataflow items available at both ends, it consumes (and looses) both of them atomically. the exclusive router, shown in figure 1a, routes data from a to either b or c. the connector can accept data only if there is a write operation at the source node a, and there is at least one component attached to the sink nodes b or c, which is performing a take operation. if both b and c have a take operation, the choice of whether data is routed to b or c is made nondeterministically by the mixed node i. node i can accept data only from one of its sink ends. to the other end it gives an exclusion reason for data not to flow, which forces the lossysync to lose the data. the second connector, shown in figure 1b, imposes an ordering on the flow of the data from the input nodes a and b to the output node c. the syncdrain enforces that data flows through a and b synchronously. the empty buffer together with the syncdrain guarantee that the data item obtained from a is delivered to c whereas the data item obtained from b is stored in the fifo1 buffer. at this moment the buffer of the fifo1 is full and data cannot flow in through either a or b, but c can obtain the data stored in the buffer. the buffer is then empty again. these informal descriptions of the behavior of connectors can be formalised using the connector colouring semantics, introduced in [cca07]. the colouring semantics is used to generate animations and to implement reo, and we discuss it in section 3. reo offers a number of operations to reconfigure and change the topology of a connector at run-time. operations that enable the dynamic creation of channels, splitting and joining of nodes, hiding internal nodes and more. the hiding of internal nodes is important concerning reconfiguration, because it allows to fix permanently the topology of a connector, such that only its boundary nodes are visible and available. the resulting connector can be viewed as a new primitive connector, or primitive for short, since its internal structure is hidden and its behaviour is fixed. reconfiguration is impossible on a primitive. we have the basic primitives that include the user defined channels, the reo nodes, and the i/o operations. plus the non-basic primitives constructed through the use of the hiding operation. 3 connector colouring semantics connector colouring semantics is based on the idea of colouring a connector using a set of colours colour. we consider a set colour with three colours as in clarke et al. [cca07]. one dataflow colour ( ) to mark places in the connector where data flows and two colours for no-dataflow ( , ) to mark the absence of dataflow. the reason for having two distinct nodataflow colours is to be able to trace the exclusion constraints responsible for the no-flow back to their origins. graphically, the arrow indicates the direction of exclusion, i.e. it points away from the exclusion reason and in the direction that the exclusion propagates. colouring a reo connector in a specific state with given boundary conditions (i/o operations) provides a means to determine the route alternatives for dataflow. each colouring of a connector is a solution to the synchronization and exclusion constraints imposed by its channels and nodes. the dataflow allowed by a connector is collected in a colouring table whose elements— colourings—are functions mapping each node of the connector to a colour. the different colourings present in a colouring table of a connector correspond to the alternative ways that the connector can behave in the different contexts where it can be used. proc. gt-vmt 2008 4 / 13 eceasst we recall some essential definitions from [cca07] that formalise the notion of a colouring and of a colouring table. let node be a finite set of node names. definition 1 (colouring) a colouring c : n → colour for n ⊆ node is a function that assigns a colour to every node of a connector. definition 2 (colouring table) a colouring table t over nodes n ⊆node is a set of colourings with domain n. to give semantics to a reo connector using connector colouring one must provide the colouring tables for the user defined primitives, the channels, used in the construction of that connector. table 1 shows the channels we use in this paper and their respective colouring tables. we highlight a few points of interest in this table, focusing only on reasons to exclude dataflow. no dataflow at one end of a sync or syncdrain, is enough to prevent dataflow in the all channel. the reason is propagated to the other end. an empty fifo1 buffer does not enable data flow on its output end, giving a reason for no dataflow. dually, a full fifo1 buffer gives a reason for having no dataflow on its input end. the second entry of the table for a lossysync states that it will lose the data only when a reason for no dataflow is propagated into its output end, which amounts to saying that the channel is unable to transfer the data. reo fixes the colouring tables for the other primitives: nodes and the i/o operations. the table 2 gives a brief account of the connector colouring semantics for these primitives. to comply with the page limit we omit the general colouring table of reo mixed nodes. we give an example of one possible colouring for a reo mixed node with 3 source ends and 2 sink ends. for the purpose of this paper that should suffice without compromising the understanding of what follows. for a full description we refer to [cca07]. definition 3 (primitive) a labelled tuple (n j11 , . . . , n jk k )c represents a primitive connector, c, where for 0 < ` ≤ k, n` ∈ node, j` ∈{i, o}, k ≥ 1 is the arity of the primitive, and the labels i and o indicate a source node or a sink node respectively, such that a node n appears at most as ni and/or no in (n j11 ,··· , n jk k )c. a primitive with colouring is a pair of a primitive with a colouring table t whose domain ranges over the nodes of the primitive. (ni1, n o 2)sync (n i 1, n i 2)syncdrain (n i 1, n o 2)lossysync (n i 1, n o 2)fifo1 (n i 1, n o 2)fifo1[x] table 1: user defined channels, and their colouring tables. a connector is a collection of primitives composed together, satisfying some well-formedness conditions. as such, the colouring table of a connector is computed from the colouring tables of its constituents. 5 / 13 volume 10 (2008) reconfiguration of reo connectors triggered by dataflow (ni1, n i 2, n i 3, n o 4, n o 5)node (n i)write (no)take table 2: reo primitives, and their colouring tables. definition 4 (connector) a connector c is a tuple 〈n, b, e, t〉 where, n is the set of nodes that appear in e; b ⊆ n is the set of boundary nodes; e is a set of primitives; t is a colouring table over n; such that (1) n ∈ b if and only if n appears exactly once in e, and (2) n ∈ n \b if and only if n occurs exactly once as no and as ni in e. a primitive with a colouring table can straightforwardly be considered as a connector. a connector’s semantics is computed by joining the tables of its constituents. two colourings can only be composed if the common nodes in their domains are coloured with the same colour. definition 5 (join) let ck = 〈nk, bk, ek, tk〉 with k ∈{1, 2} be connectors such that (n1\b1)∩ (n2\b2) = /0, and for each n ∈ b1∩b2, ni appears in e1 and no appears in e2, or vice versa. the join of c1 and c2, is given by: c1 �c2 . = 〈n1 ∪n2, (b1 ∪b2)\(b1 ∩b2), e1 ∪e2, t1 ·t2〉, where · is the join operator for two colouring tables defined as: t1 ·t2 . = {c1 ∪c2 | c1 ∈ t1, c2 ∈ t2, n ∈ (dom(c1)∩dom(c2)) ⇒ c1(n) = c2(n)}. figure 2 depicts two colourings of the ordering connector of figure 1(b). in both colourings, a component is connected to each boundary node and performs an i/o operation: write on both source nodes and take on the sink node. the colouring in (a) describes the dataflow behaviour (a) (b) figure 2: (a) and (b) are two possible colourings of the ordering connector. of the connector when the buffer is empty, indicating that data flows through the entire connector except for the sink end of the fifo1 channel, data is stored in the buffer, and all three i/o operations succeed. this dataflow changes the state of the fifo1 channel, changing also its colouring table. the colouring in (b) describes the dataflow behaviour of the connector when the buffer is full. this colouring states that data flows in the connector only at the sink end of the proc. gt-vmt 2008 6 / 13 eceasst fifo1[x] channel, the buffer is emptied, the take operation on node c succeeds, and the write operations on nodes a and b are delayed. figure 3 depicts the two colourings of the exclusive router that are the valid behaviour alternatives when a component is connected to each boundary node and performs an i/o operation: write on the source node and take on both the sink nodes. the colouring in (a) describes the (a) (b) figure 3: (a) and (b) are two possible colourings of the exclusive router. dataflow behaviour of the connector when the mixed node i picks, non-deterministically, node b to route the data to. alternatively the colouring in (b) describes the dataflow behaviour of the connector when the mixed node i picks, non-deterministically, node a to route the data to. 4 connector reconfiguration it has been shown in [kla07] that the theory of graph transformation can be applied to model connector reconfigurations as high-level transformations of their underlying graph structures. the approach allows to define reconfigurations at a high level of abstraction and therefore can be used to model complex reconfigurations, e.g. refactorings. while in the previous work, we considered these techniques in the context of business process customisation, we combine now these transformations with the connector colouring semantics, described in section 3. 4.1 reconfiguration triggered by dataflow dynamic reconfigurations are transformations of connectors at run-time. in the following, we present a framework that allows to define such dynamic reconfigurations by annotating transformation rules with colourings, which leads to a notion of dynamic connectors. to use graph transformation for connector reconfiguration we make the following assumptions. connectors are considered as typed, attributed graphs in the following way: i) reo nodes are vertices of the graph and ii) channels are its edges. the typegraph in this scenario consists of a single node and one edge for each channel type. edge attributes are used to model channel properties, e.g. the content of a full fifo1. since channels in reo are not necessarily directed (cf. the syncdrain channel) we simply assert an underlying direction of the channel to fit the formal model of directed graphs that is usually assumed. note also that we have given a formal definition of connectors in [kla07] and showed that it indeed forms an adhesive high-level replacement category [eept06]. 7 / 13 volume 10 (2008) reconfiguration of reo connectors triggered by dataflow we use the double-pushout (dpo) approach [eept06] for our connector reconfigurations. our transformation rules are an extended version of the usual spans of morphisms in the dpo approach. we write a connector reconfiguration rule as p = (colour c←− l l←− k r−→ r) where l, k and r are connectors (typed, attributed graphs), l, r are connector (typed, attributed graph) homomorphisms and c is a (potentially partial) colouring for the left-hand side of the rule. the rationale behind this extension is that we do not just want to match the structure of a particular connector part, but also its state and the current execution strategy. note that this introduces a certain asymmetry to the rules, caused by the fact that only the left-hand side is coloured. this is also the reason why we do not model the colouring as attributes of the graphs. such an extended rule can be applied with respect to a given match l m→ m and a current colouring m k→ colour iff 1. the gluing condition holds (see [eept06] for more details); and 2. the rule colouring matches the current colouring: c = k◦m. with the latter constraint we extend the pattern of a rule, in the way that a specific colouring has to be matched as well. a transformation rule can and will be applied only if the structure can be matched and a specific behaviour occurs. the extended version of the dpo approach can be summarised as shown in the diagram colour l k r m c n (po) (po) = loo r // m �� �� �� oo // coo k ddiiiiiiiiii where colour is a fixed set of possible flow-colours and c and k are colourings. including the colourings of a connector part makes the pattern matching much more restrictive. in fact, it is so restrictive now that the reconfiguration rules can be invoked by the reo engine without ‘supervision’. the transformation system becomes in some respect autonomous. the connector is transformed when necessary, without the need for an external party to trigger the reconfiguration. figure 4: reconfiguration rule for a dynamic f if o. figure 4 shows a reconfiguration rule for a dynamic (unbounded) f if o. the matches from the lhs to the rhs are indicated by using the same node labels. the reconfiguration rule gives rise to a dynamic connector, which we call f if o∞. the f if o∞ consists of a sequence of f if o1 channels and a reconfiguration rule. the reconfiguration adds a new f if o1 in the proc. gt-vmt 2008 8 / 13 eceasst beginning whenever the f if o∞ is full and someone tries to write to it assuring that way that the write can succeed and the data can flow and be stored in the buffer. the left-hand side matches an arbitrary full f if o1 with content x where someone tries to write to. the reason for the no-flow is the fact that the f if o1 is full already, not that there is no data available. if this pattern can be matched, the rule states that the original channel is destroyed and two new f if o1 channels are created in its place. the second f if o1 is filled with the original content x. however, the rule as it is now does not reflect our initial requirement that only the first f if o1 in the sequence is replaced by two new ones. this can achieved by adding negative application conditions [eept06] as shown in figure 5. we need two extra nacs for the dynamic f if o∞ figure 5: negative application conditions for the f if o∞. that restrict where the rule can be applied. an empty f if o1 should always be added at the very beginning of the sequence. expressed as negative application conditions, this means that there must not be an empty or a full f if o1 in front of the one where the original rule (figure 4) applies. these additional restrictions allow us to apply the transformation rule automatically at runtime when the colouring occurs. for completeness we would have to define also an inverse rule that shrinks the connector again. we omit this here. 4.2 run of a connector connectors are executed in an abstract reo engine. the reo engine includes two independent components, one to compute colouring tables and to perform the dataflow, and one for computing reconfiguration matches and executing the transformations. we refer to these components as df and tr respectively. reconfiguration rules are applied locally only in specific regions of the connector. these regions can be formally viewed as disjoint sub-graphs that restrict the domain of the transformations. these regions are the reconfigurable parts of the connector. each of these regions has a number of reconfiguration rules attached to it, such as the one in figure 4. the reo engine utilises df and tr to execute dynamic connectors. in this scenario a run of the engine consists of performing the following actions: 1. invoke df to compute the colouring table ct of the connector for the actual boundary conditions. 2. choose non-deterministically a colouring k from the colouring table ct . 3. for each reconfiguration region, invoke tr to find pattern matches m1, . . . , mn for the colouring k. 4. invoke df to execute the dataflow according to k. the state of the connector is updated. 9 / 13 volume 10 (2008) reconfiguration of reo connectors triggered by dataflow (a) (b) (c) (d) (e) (f) (g) (h) (i) figure 6: run of the f if o∞. 5. for each pattern match in m1, . . . , mn, invoke tr to perform the transformations. since the steps 3 and 4 are independent from each other, they can be performed in parallel. note also that in general, multiple transformation policies could be supported, e.g. apply at most once or until it is not applicable anymore. figure 6 depicts a run according to the descriptions above, for the f if o∞. part (a) shows the basic connector that we want to reconfigure. it consists of a single empty f if o1 with a w rite operation. the region where the transformation rule from figure 4 should be applied is delimited by the dashed box. part (b) shows the colouring k1 and the corresponding dataflow. the transformation engine is invoked with a snapshot of the reconfiguration region and the colouring k1. since the rule does not match colouring k1 no transformation is performed. in part (c) we see how the dataflow has changed the state of the f if o1 and that the write operation disappeared after succeeding. in part (d) a new write operation is attached to the connector. at this point a new run of the engine starts. figure 6e shows that the new write operation cannot succeed because the f if o1 is already full. this event is formally observed by the no-flow colouring k2. the transformation engine is invoked again and returns a match m this time, because k2 matches the colouring of the reconfiguration rule. since no dataflow has to be performed, the transformation can immediately be applied, as shown in figure 6f. the reconfigured connector is shown in part (g). at this point, the write operation succeeds, and the corresponding colouring k3 is depicted in part (h). there is no rule that matches k3, since the only possible match with the full f if o1 is restricted by nac 1, defined in figure 5. in figure 6i we show the state of the connector after the dataflow, whereas the write operation is removed and the left f if o1 becomes full. the visual representation we use is based on an extension of the animation language introduced in [cp07]. we use the colour green to highlight the part of the connector that matches the left-hand side of the rule, as depicted in the region inside the dashed box in figure 6f, and proc. gt-vmt 2008 10 / 13 eceasst the connector that appears below corresponds to the right-hand side of the rule. the dashed blue arrows in figure 6f, pointing from the match to the substitute connector, indicate that the nodes a and b are preserved by the transformation. furthermore, the colour of the new data token matches the colour of the data token in the matched part of the connector, expressing that the argument of the full f if o1 channel is the same as the argument of the new full f if o1. 5 discussion up to now we have shown that by applying graph transformation techniques to reo, we are able to describe complex connector reconfigurations using a very compact notation. transformation rules are extended with colouring information so that they can be applied when a certain behaviour occurs. external invocation of the transformation is not required. in the current setup, the transformation engine is invoked in each run of the execution of the connector. even if the required colouring is not present, the engine first tries to match the structural pattern of the current connector and then validates its colouring. if the reconfiguration regions become larger, the pattern matching can cause a significant degradation of performance. note that general pattern matching for graphs is a problem that is known to be np-complete. figure 7: rule for a f if o∞ with an evolving reconfiguration region. to improve the performance of dynamically reconfiguring connectors, we suggest the following optimisation. we augment the transformation rules with information about the reconfiguration regions. for our example of the dynamic f if o we extend the rule as shown in figure 7. the dashed boxes mark the reconfiguration regions. with this extension we can state that the rule is always applied to the first f if o1. while before the reconfiguration region was growing with each application of the rule, it will now remain as consisting of exactly two nodes and one f if o1 only, all the time. this way, the pattern matching is much faster and (in the ideal case) always uniquely determined. 6 implementation reo as a modelling framework the reo modelling framework [ect] consists of a set of plug-ins for the eclipse1 development environment such as: a graphical editor, a model checker and a flash-based animation tool. executable instances of connectors can be derived by generating code from reo specifications or by interpreting these specifications. we generate code from abstract animation descriptions 1 http://www.eclipse.org 11 / 13 volume 10 (2008) http://www.eclipse.org reconfiguration of reo connectors triggered by dataflow when producing the flash animations [cp07]. we have implemented an interpreter2, based on the colouring semantics, for the application domain of web services. currently we are integrating the dynamic reconfiguration scheme into the interpreter. reo as a runtime architecture we are also developing a distributed reo engine [pro07], where each primitive is deployed and executed as an independent block running in parallel with the other primitives. there is no centralized entity computing the colouring table of the composite connector at each step, but instead the colouring table is obtained by distributed agreement. the engine has a mechanism to pause regions of the connector, making them suitable to be reconfigured. however, since the development is still in an early stage we are not able to integrate the dynamic reconfiguration yet. 7 future work the example of the dynamic f if o is certainly very basic. in general, there may be more than one reconfiguration rule (a graph grammar) attached to a region of a connector. the role that the colouring extension in our rules plays in these more complex scenarios must be investigated. furthermore, we need to make the notion of the evolving reconfiguration regions, as suggested in section 5, more specific. additional partial mappings from the lhs to rhs may be a possible approach to model this. we are interested in preservation of behavioural properties by transformations. two different classes of transformations are interesting in this setting: transformations that preserve the behaviour and transformations that change it. on the one hand, reconfigurations that do not change the behaviour are interesting in the area of automated refactoring and optimisation. on the other, reconfigurations that change the behaviour can be used to implement new sub-circuits that adapt the behaviour of a circuit based on dataflow. in this context, it will be interesting to do static analysis of reconfiguration rules to reason about behaviour preservation. finally, we want to integrate the dynamic reconfiguration triggered by dataflow into the reo tools. for this purpose, we will extend both the eclipse based development tools and the reo runtime engine for web services. since we model reconfiguration through graph transformation, and due to the fact that our implementation of the reo development tools is based on the eclipse modeling framework (emf)3, we plan to implement the reconfiguration extensions using the tiger emf transformation tools [emt]. bibliography [arb04] f. arbab. reo: a channel-based coordination model for component composition. mathematical structures in computer science 14:329–366, 2004. 2 http://www.cwi.nl/∼koehler/services 3 http://www.eclipse.org/emf proc. gt-vmt 2008 12 / 13 http://www.cwi.nl/~koehler/services http://www.eclipse.org/emf eceasst [bek+06] e. biermann, k. ehrig, c. koehler, g. kuhns, g. taentzer, e. weiss. graphical defnition of in-place transformations in the eclipse modeling framework. in model driven engineering languages and systems (models’06). 2006. [blmt07] r. bruni, a. lluch-lafuente, u. montanari, e. tuosto. style-based architectural reconfigurations. technical report tr-07-17, computer science department, university of pisa, 2007. [cca07] d. clarke, d. costa, f. arbab. connector colouring i: synchronization and context dependency. sci. comput. program. 66(3):205–22, 2007. doi:http://dx.doi.org/10.1016/j.scico.2007.01.009 [cla08] d. clarke. a basic logic for reasoning about connector reconfiguration. fundamenta informaticæ 82:1–30, 2008. [cp07] d. costa, j. proença. connector animation: a compositional framework to analyse reo connectors. december 2007. submitted. available online: http://www.cwi. nl/∼costa/publications/publications.htm. [ect] eclipse coordination tools. http://homepages.cwi.nl/∼koehler/ect. [eept06] h. ehrig, k. ehrig, u. prange, g. taentzer. fundamentals of algebraic graph transformation. eatcs monographs in theoretical computer science. springer, 2006. [emt] tiger emf transformation project. http://tfs.cs.tu-berlin.de/emftrans. [erm06] c. ermel. simulation and animation of visual languages based on typed algebraic graph transformation. phd thesis, 2006. [kla07] c. koehler, a. lazovik, f. arbab. connector rewriting with high-level replacement systems. in proceedings of foclasa 2007, to be published in electronic notes in theoretical computer science. 2007. [pro07] j. proença. towards distributed reo. talk presented at cic workshop, 2007. [ren03a] a. rensink. groove: a graph transformation tool set for the simulation and analysis of graph grammars. http://www.cs.utwente.nl/∼groove, 2003. [ren03b] a. rensink. towards model checking graph grammars. in leuschel et al. (eds.), workshop on automated verification of critical systems (avocs). technical report dsse–tr–2003–2, pp. 150–160. university of southampton, 2003. 13 / 13 volume 10 (2008) http://dx.doi.org/http://dx.doi.org/10.1016/j.scico.2007.01.009 http://www.cwi.nl/~costa/publications/publications.htm http://www.cwi.nl/~costa/publications/publications.htm http://homepages.cwi.nl/~koehler/ect http://tfs.cs.tu-berlin.de/emftrans http://www.cs.utwente.nl/~groove introduction reo overview connector colouring semantics connector reconfiguration reconfiguration triggered by dataflow run of a connector discussion implementation future work firewall-as-a-service for campus networks based on p4-sfc electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) firewall-as-a-service for campus networks based on p4-sfc marco häberle, benjamin steinert, michael menth 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst firewall-as-a-service for campus networks based on p4-sfc marco häberle1, benjamin steinert2, michael menth3 1marco.haeberle@uni-tuebingen.de 2benjamin.steinert@uni-tuebingen.de 3menth@uni-tuebingen.de university of tuebingen, chair of communication networks, tuebingen, germany * abstract: taking care of security is a crucial task for every operator of a campus network. one of the most fundamental security-related network functions that can be found in most networks for this purpose are stateful firewalls. however, deploying firewalls in large campus networks, e.g., at a university, can be challenging. hardware appliances that can cope with today’s high data rates at the border of a campus network are not cost-effective enough for most deployments. shifting the responsibility to run firewalls to single departments at a university is not feasible because the expertise to manage these devices is not available there. for this reason, we propose a cloud-like infrastructure based on service function chaining (sfc) and network function virtualization (nfv) that allows users to deploy network functions like firewalls at a central place while hiding most technical details from the users. keywords: service function chaining, software defined networking, firewall 1 introduction in recent years, network function virtualization (nfv) gained traction among internet service providers and operators of campus networks and data centers. nfv aims to reduce cost and improve flexibility by replacing hardware-based network functions with virtual equivalents, virtual network functions (vnf), that are run on commercial off-the-shelf servers. nfv is worked on actively by both academia and standardization bodies [ywl+18]. network traffic is often steered through vnfs with the help of service function chaining (sfc). the sfc architecture standardized by the ietf in rfc 7665 consists of classifiers, service functions, and service function forwarders. service functions are single network functions, e.g., firewall or nat appliances. these service functions may be vnfs. the classifier is the entry point to an sfc-enabled domain and assigns every packet to a service function path consisting of service functions that the packet needs to traverse. this path is encoded into the packet, e.g., by using the network service header or by pushing an mpls label stack. similar to nfv, sfc is worked on by academia and standardization bodies [bjse16]. in this work, we propose to use sfc and nfv to secure campus networks of universities in a flexible, yet simple to manage way. an easy-to-use self-service portal enables individual departments to define service function chains and configure service functions without the need ∗ this work was supported by the bwnet2020+ project which is funded by the ministry of science, research and the arts baden-württemberg (mwk). the authors alone are responsible for the content of this paper. 1 / 4 volume 080 (2021) mailto:marco.haeberle@uni-tuebingen.de mailto:benjamin.steinert@uni-tuebingen.de mailto:menth@uni-tuebingen.de firewall-as-a-service for campus networks based on p4-sfc to operate expensive and hard to configure network appliances. instead, service functions are deployed as nfvs in a local sfc cloud that is run in a central data center in the campus network. a prototype of the self-service portal and the sfc infrastructure is available at github1. in the following section, we give an overview of the local sfc cloud. in section 3, we describe the self-service portal. in section 4, we conclude our work. 2 local sfc cloud in the following, we give an overview of the setup of the local sfc cloud and explain how it can be integrated in an existing campus network with minimal changes. 2.1 sfc cloud infrastructure the infrastructure is largely based on p4-sfc [shhm20]. service function chaining is realized using mpls segment routing similar to [cxf+20]. the components are shown in figure 1(a). ingress switch orchestrator forwarder vnf 11 ... vnf 12 vnf 13 vnf 01 ... vnf 02 vnf 03 vnf 21 ... vnf 22 vnf 23 (a) sfc cloud components. campus network data center department internet sfc cloud (b) integration: routing campus network data center department internet sfc cloud (c) integration: tunnel figure 1: the setup of the sfc cloud and options to integrate it into a campus network. incoming traffic is classified by an ingress switch implemented in p4. classification is done according to the service function chain configured in the self-service portal. the classified traffic is then sent to the vnf hosts by service function forwarders. these forwarders may be any traditional off-the-shelf switches with support for mpls segment routing. several vnf hosts run the vnfs that have been configured in the self-service portal. the vnfs themselves are executed either in a dedicated lxc container or in a dedicated kvm virtual machine. the mpls router module of the linux kernel of the vms or containers serves as an sfc proxy for sfc-unaware vnf applications. a central orchestrator transforms the settings taken in the self-service portal into an appropriate network configuration and deploys the requested vnfs on the vnf hosts. hereby, mpls 1 https://github.com/uni-tue-kn/p4-sfc-faas netsys 2021 2 / 4 https://github.com/uni-tue-kn/p4-sfc-faas eceasst labels are assigned to the vnfs and the label stack for each service function chain is configured on the ingress switch for classification. 2.2 traffic classification in p4 p4 provides the ability to execute a single data plane definition on different software and hardware platforms. so far, p4-sfc supports the intel tofino switching asic and the software-based bmv2 switch for traffic classification. the intel tofino supports up to 64 100g ethernet ports, its successor up to 32 400g ethernet ports. on this platform, traffic classification happens in line speed. while this high performance is highly future-proof, a more cost-effective solution with less performance may be desirable for some use-cases. the bmv2 is an open-source software switch that provides an easy way to run p4 programs. however, it is not production-ready and is intended for development of p4 programs only. in a virtual machine with 8 virtual cpus and 15 gib of ram, it achieves a throughput of only around 1 gb/s when forwarding packets between two hosts [bmv]. 2.3 integration into campus networks the sfc cloud is deployed in a data center inside the campus network. depending on the structure of the campus network and the requirements of the departments that use the sfc cloud, several options exist to integrate it into existing infrastructure. the most straight forward option is to modify the routing of all traffic going to departments that use the sfc cloud. this option is shown in figure 1(b). while this results in minimal technical overhead, the administrative overhead may be cumbersome when deploying the system. alternatively, departments that want to use the sfc cloud may use the approach shown in figure 1(c). the traffic that enters and leaves the network of the department is tunneled to the sfc cloud. this approach needs modifications of the network configuration at the department only without modifying any other parts of the campus network. however, even lightweight tunneling protocols like gre have a protocol overhead that should not be ignored. as a consequence, this approach is suitable for departments with a low traffic volume only. 3 self-service portal the self-service portal is the central point to configure service function chains and vnfs. it features a user and permission model that allows to delegate responsibility for specific traffic classes, e.g., traffic to and from an individual department. defining traffic classes and template vnfs, as well as managing user rights is done by administrators of the sfc cloud system. users with delegated rights for specific traffic classes are able to define service function chains easily. a sample chain configuration is shown in figure 2(a). when configuring a service function chain, users can simply drag-and-drop vnfs from a vnf inventory. the vnf inventory contains vnfs that are predefined by administrators, as well as vnfs that have been configured by the user himself. for this purpose, the self-service portal provides configuration wizards for selected vnf types. an example, configuring a firewall vnf, is shown in figure 2(b). 3 / 4 volume 080 (2021) firewall-as-a-service for campus networks based on p4-sfc (a) sfc configuration with drag and drop. (b) configuration of firewall vnf. figure 2: self-service portal. 4 conclusion securing campus networks, e.g. at universities, becomes more and more difficult, complex, and expensive with increasing network traffic. service function chaining in combination with network function virtualization can help in solving these issues. in this work, we presented a cloud-like infrastructure that enables users in a campus network, e.g. single departments at a university, to deploy virtual network functions like firewalls at a central point in the network. a self-service portal enables users to easily configure these functions. references [bjse16] d. bhamare, r. jain, m. samaka, a. erbad. a survey on service function chaining. journal of network and computer applications 75:138 – 155, 2016. [bmv] performance of bmv2. https://github.com/p4lang/behavioral-model/blob/master/ docs/performance.md. accessed 26-10-2020. [cxf+20] f. clad, x. xu, c. filsfils, d. bernier, c. li, b. decraene, s. ma, c. yadlapalli, w. henderickx, s. salsano. service programming with segment routing. internetdraft draft-ietf-spring-sr-service-programming-03, internet engineering task force, sept. 2020. work in progress. [shhm20] a. stockmayer, s. hinselmann, m. häberle, m. menth. service function chaining based on segment routing using p4 and sr-iov (p4-sfc). in isc high performance international workshops. pp. 297–309. 2020. [ywl+18] b. yi, x. wang, k. li, s. k. das, m. huang. a comprehensive survey of network function virtualization. computer networks 133:212 – 262, 2018. netsys 2021 4 / 4 https://github.com/p4lang/behavioral-model/blob/master/docs/performance.md https://github.com/p4lang/behavioral-model/blob/master/docs/performance.md introduction local sfc cloud sfc cloud infrastructure traffic classification in p4 integration into campus networks self-service portal conclusion generating euler diagrams from existing layouts electronic communications of the easst volume 13 (2008) proceedings of the second international workshop on layout of (software) engineering diagrams (led 2008) generating euler diagrams from existing layouts gem stapleton, john howse peter rodgers and leishi zhang 16 pages guest editors: andrew fish, harald störrle managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst generating euler diagrams from existing layouts gem stapleton1, john howse2 peter rodgers3 and leishi zhang4 1 g.e.stapleton@brighton.ac.uk,2 john.howse@brighton.ac.uk www.cmis.brighton.ac.uk/reseach/vmg university of brighton, uk 3 p.j.rodgers@kent.ac.uk, 4 l.zhang@kent.ac.uk university of kent, uk abstract: euler diagrams have a wide variety of uses, from information visualization to logical reasoning. in the case of software engineering, they form the basis of a number of notations, such as state charts and constraint diagrams. in all of their application areas, the ability to automatically layout euler diagrams brings considerable benefits. there have been several recent contributions towards the automatic generation and layout of euler diagrams, all of which start from an abstract description of the diagram and produce a collection of closed curves embedded in the plane. in this paper, we are concerned with producing layouts by modifying existing ones. this type of layout approach is particularly useful in domains where we require an updated, or modified, diagram such as in a logical reasoning context. we provide two methods to add a curve to an euler diagram in order to create a new diagram. the first method is guaranteed to produce layouts that meet specified wellformedness conditions that are typically chosen by others who produced generation algorithms; these conditions are thought to correlate well accurate user interpretation. we also overview a second method that can be used to produce a layout of any abstract description. keywords: information visualization, diagram layout, venn diagrams 1 introduction automated diagram layout has the potential to bring huge benefits and it is unsurprising that, with the computing power now available, considerable research effort is focused on this topic. in software engineering, the prevalent use of diagrammatic notations makes this area an ideal candidate to benefit from state-of-the-art generation and layout techniques. many diagrams are based on collections of closed (usually simple) curves, such as state charts and class diagrams both of which are part of the array of languages that form the uml. various other languages are based on closed curves, such as constraint diagrams, that are designed for software specification; see [hs05, kc99] for examples of such specifications. to illustrate, the constraint diagram in figure 1 expresses that every store stocks at least two copies of some film in its collection. a well studied fragment of the constraint diagram language, called spider diagrams, that is also based on closed curves has been used in a variety of application areas; see, for example, [cla05, nie2006]. a finite collection of closed curves constitutes an euler diagram and, therefore, the languages 1 / 16 volume 13 (2008) mailto:g.e.stapleton@brighton.ac.uk mailto:john.howse@brighton.ac.uk www.cmis.brighton.ac.uk/reseach/vmg mailto:p.j.rodgers@kent.ac.uk mailto:l.zhang@kent.ac.uk generating euler diagrams sp q r store stock film storecopies collection allcopies sp q r figure 1: a constraint diagram. mentioned above can all be viewed as extending euler diagrams in some manner. thus, the automated layout of euler diagrams provides an essential basis for the automated layout of a large range of other diagrams. in addition to those mentioned above, euler diagrams have numerous other application areas; for example [des03, hes+05, kmg+05, lov02, tvv05]. various methods for generating euler diagrams have been developed, each concentrating on a particular class of euler diagrams; see, for example [cr05b, cr03, fh02, kmg+05, rzf08, vv04]. ideally, such generation algorithms will produce diagrams with desirable properties in an efficient way; such properties are sometimes called wellformedness conditions and will be more fully explained below. the generation algorithms developed so far produce euler diagrams that have certain sets of properties. each of these generation methods starts with an abstract description of the required diagram and proceeds to seek a layout. in this paper, we take a different approach to generation, in that we take an existing diagram layout and transform it in to another layout. in particular, we describe how to add a curve to an existing layout to create a new euler diagram using two methods. the first method is presented in two stages, with the first stage describing how to add a new curve to a so-called wellformed layout in such a way that each ‘minimal region’ is split in to two minimal regions (one inside and one outside the new curve) and wellformedness is maintained. the technique is extended to allow selected minimal regions to be split, others to be completely contained by the new curve and the rest to be completely outside the new curve. in fact, our technique guarantees to be able to find an embedding of the new curve in the required manner whenever this is possible given the existing layout. the second method (informally outlined in the paper) can be used to find a layout of any euler diagram description; we can decompose the layout problem in to a sequence of layout problems, where we add a new curve at each stage. thus, in this paper we provide two approaches to euler diagram generation that, in addition to contributing to the general generation problem, are particularly advantageous in any situation where we wish to modify a diagram by adding a curve and maintain the existing layout. in section 2, we provide motivation specific to our generation approach of adding a curve and section 3 overviews the syntax of euler diagrams and other necessary background material. section 4 defines the operation of adding a curve to euler diagrams and their descriptions. in section 5 we show how to add a curve to a so-called atomic wellformed euler diagram and prove that the resulting diagram is also atomic and wellformed. section 6 generalizes to the non-atomic (nested) case. section 7 shows how to add a curve to an arbitrary euler diagrams; the technique proc. led 2008 2 / 16 eceasst ensures that any abstract euler diagram can be generated by adding curves one at a time. the application of our layout technique to the general generation problem is discussed in section 8. finally, section 9 discusses a prototype implementation of the approach and presents some output from the software. 2 motivation euler diagram generation is hard and, as the number of curves increases, the layout problem becomes increasingly more difficult. moreover, we may have added requirements of our layouts in certain contexts. for example, euler diagrams often form the basis of visual logics and, in such settings, the automated layout of diagrams is essential when building theorem provers. a common operation in reasoning systems based on euler diagrams is to add a curve to a diagram in such a manner that each so-called zone (defined later) splits in to two new regions, one inside and the other outside the new curve [sk00, shi94, sa04]. to illustrate, in figure 2, we can add a curve to d1 to give d2. the diagram d2 has the same abstract description as d3 but looks rather different. at the abstract level, adding a curve to ab(d1), the abstract description of d1, would give ab(d2) = ab(d3). if we want to preserve the layout of d1 when adding the curve, we need some method to generate d2, rather than go via the abstract syntax, d1 7→ ab(d1) 7→ ab(d2), and then generate a concrete diagram with abstraction ab(d2), which could result in d3. moreover, sometimes we want to a b d1 a b d2 c c d b c d3 d a figure 2: adding a curve. add a curve to an euler diagram in such a manner that not every zone is split in two, such as in [smf+07]. in this, and other areas, it can be helpful to layout one diagram so that it looks similar to another, preserving as much of the user’s mental map as possible [mel+95]. our approach to layout gives this preservation for free, in that we take an existing layout and add to it a curve. another area where our approach to generation will be particularly helpful is when we utilize a library of nicely drawn examples (such as with circles) as a basis for producing further layouts; see [sfr07] for preliminary work towards building such a library. for example, such a library might include a layout for each abstract description where at most three curves are used. if we then wanted a layout of a diagram containing four curves, we can extract from the library a good layout of an appropriate three curve diagram and add the forth curve to produce the required diagram. 3 / 16 volume 13 (2008) generating euler diagrams 3 euler diagrams we now overview a formalization of euler diagrams and their descriptions. moreover, we also describe various concepts that will be required throughout the paper, in particular wellformed and atomic diagrams. 3.1 concrete diagrams as stated above, an euler diagram is a collection of closed curves drawn in the plane. we assume that each curve has a label chosen from some fixed set of labels, l . definition 1 a concrete euler diagram is a pair, d = (curve, l), where 1. curve is a finite collection of closed curves each with codomain r2, 2. l : curve → l is a function that returns the label of each curve. a b d4 c a b d5 c d a b d6 c d figure 3: concrete euler diagram syntax. for example, d4 in figure 3 contains three curves labelled a, b and c. to be more precise, d4 depicts the images of three simple closed curves. given a curve, c : [0, 1] → r2 say, we denote the image of c by im(c) (following the standard notation for the image of a function). the curves partition r2 − ⋃ c∈curve im(c) into connected regions of the plane, called minimal regions. a contour in an euler diagram is the set of curves in that diagram with the same label. a point is interior to a contour if it is inside an odd number of its curves, otherwise it is exterior. for formal definitions of the interior of curves in the non-simple case see [srh+07]. a zone in a diagram is a maximal set of minimal regions that can be described as being inside certain contours (possibly no contours) and exterior to the remaining contours. in figure 3, d4 has six zones, of which two are inside a. the diagram d5 has ten minimal regions but only eight zones, such as the disconnected zone inside b but outside the remaining curves. concrete euler diagrams may possess certain properties, sometimes called wellformedness conditions, such as containing no triple points (where three or more curves intersect at a single point) or no concurrency between curves (where curves intersect at a non-discrete set of points). typically, generation algorithms produce concrete diagrams that possess certain properties, in part for reasons of interpretability. we say that a concrete euler diagram, (curve, l) is wellformed if 1. the function l is injective (no pair of distinct curves have the same label), 2. all of the curves are simple (no curve self-intersects), proc. led 2008 4 / 16 eceasst 3. there are no triple points of intersection between curves, 4. the zones are connected (each zone consists of exactly one minimal region), and 5. every time two curves intersect they do so transversely (note that this implies that no curves run concurrently); see [srh+07] for formalizations of these properties. the generation algorithm in [fh02], for example, draws diagrams that are wellformed. in figure 3, both d4 and d6 are wellformed but d5 is not. whilst all of d5’s curves are simple (that is, they do not self-intersect), it has a triple point where a, b and c intersect, the zones are not all connected, and the curves c and d do not meet transversely at the point they intersect. the concept of nesting in diagrams is of particular importance in automated layout. the (images of the) curves in a concrete euler diagram form connected components of r2. if the curves give rise to exactly one connected component then the diagram is called atomic, otherwise the diagram is nested [fht04]. in figure 3, d4 and d5 are atomic whereas d6 is nested and comprises three atomic components. when laying out nested diagrams, we can automatically generate each of the atomic components separately and then merge the results together. in the case of wellformed diagrams, it has been shown that nestedness can be detected from diagram descriptions and the atomic components identified prior to layout [fht04]. concrete euler diagrams are associated with various graphs. these dual graphs play an instrumental role in their automated layout; see [cho07, fh02] for more details. first, we can take a concrete diagram, d1 = (curve, l) and construct its euler graph which has a vertex at each point two curves meet, these vertices are shown in d7, figure 4, and the edges are then the curve segments that connect the vertices. as a special case, the euler graph of a diagram containing a single curve has exactly one vertex placed on that curve. from the euler graph, we can construct an euler graph dual which is simply a dual graph of the euler graph, as shown in d8. finally, we have what is called a concrete dual. a concrete dual graph is a maximal subgraph of an euler graph dual that contains all of the vertices but no multiple edges. the diagram d9 shows one concrete dual that can be derived from the depicted euler dual. we note that all of the graphs described include layout information and are plane. for the purposes of this paper, we require a b d7 d8 c d9 a b c a b c figure 4: various graphs. the following results. lemma 1 let d be a concrete euler diagram. any two concrete dual graphs (duals of euler graphs) of d, are isomorphic. corollary 1 let d be a concrete euler diagram. given any two concrete dual graphs (duals of euler graphs) of d they are either both hamiltonian or neither are hamiltonian. 5 / 16 volume 13 (2008) generating euler diagrams in the context of layout, typical algorithms take a diagram description, convert it into an abstract dual graph (sometimes called the superdual) and subsequently seek subgraphs of this abstract dual as candidate concrete dual graphs. once an appropriate candidate concrete dual graph is found, the curves are embedded around the vertices. 3.2 diagram descriptions in order to generate an euler diagram, we start with a description of that diagram. to illustrate, d4 in figure 3 can be described as having three curves, a, b and c, which are also contours. these contours divide the plane in such a manner that there are six zones present. each zone can be described as being inside certain contours and outside the remaining contours. for instance, there is one zone inside a only and another zone inside precisely a and c. thus, each present zone can be described by the labels of the contours that the zone is inside. note that there is always a zone outside all of the contours (the infinite minimal region). definition 2 an abstract euler diagram description (or, simply, abstract description), d, is a pair, (l, z) where l = l(d) is a subset of l and z = z(d) ⊆ pl such that /0 ∈ z(d). elements of z are called (abstract) zones. definition 3 given a concrete euler diagram d = (curve, l), we map d to abstract description ab(d) = (im(l), z), called the abstraction of d, where z contains exactly one abstract zone for each concrete zone in d; in particular, given a concrete zone, z, in d, the abstract zone ab(z) = {l(c) : c ∈ c(z)} is in z where c(z) is the set of curves in d that contain z. the diagram d4 in figure 3 has abstraction (l, z) where l = {a, b,c} and z = {/0,{a},{a,c},{c},{b,c},{b}}. the generation problem can be summarized as given an abstract description, d1, find a concrete euler diagram, d2, such that ab(d2) = d1. sometimes, the generation problem is restricted by imposing certain conditions on d2, for instance requiring that the concrete diagram is wellformed. 4 adding a curve there are situations when we want to add a curve to a given layout in a specified manner, such as when generating diagrams. in order to do this, we need to know what is meant by adding a curve and how to specify its addition. in this section, we formalize the notion of adding a curve at both the abstract and concrete levels. at the concrete level this is easy: we simply take a diagram and add to it a curve and a label. in figure 5, we add c to d10, giving d11. definition 4 let d = (curve, l) be a concrete euler diagram. let l be a label in l and let c be a curve that is not in curve. then we defined d + (c, l) = (curve ∪{c}, l ∪ (c, l)). if l 6∈ im(l) proc. led 2008 6 / 16 eceasst and the number of zones doubles when adding c then d + (c, l) is said to be derived from d by splitting each zone. a b d10 d11 c a b figure 5: a more complex example. at the abstract level, things are not quite so straightforward. we note that each zone can be either completely contained by the new curve, completely outside the new curve, or split by the new curve. in figure 5, the diagram d11 has a curve c that completely contains the zone inside b, is completely outside the zone inside a and splits the zone outside both a and b. we can think of the split zone as being both inside and outside c. to describe how c is added at the abstract level, we state which zones are to be inside c and which are to be outside c; those which are split are stated as being both inside and outside. thus, the inside zones and outside zones between them include all of the zones. definition 5 let d = (l, z) be an abstract description. let l be a label in l and let in and out be two subsets of z such that in ∪ out = z and /0 ∈ out. then d + (l, in, out) is defined to be d + (l, in, out) = (l ∪{l}, zin ∪ zout ) where zin = {z∪{l} : z ∈ in} and zout = out. if in = z and out = z and l 6∈ l then d + (l, in, out) is said to be derived from d by splitting each zone. in figure 5, in = {{b}, /0} and out = {{a}, /0}; when adding c to d10 to give d11, c contains the zone {b} (in the set in − out), splits the zone /0 (in the set in ∩ out) and does not contain the zone {a} (in the set out − in). the following lemma shows that the notions of splitting each zone at the abstract and concrete level coincide. lemma 2 let d = (curve, l) be a concrete euler diagram. let l be a label in l and let c be a curve that is not in curve. then d + (c, l) is derived from d by splitting each zone if and only if ab(d + (c, l)) is derived from ab(d) by splitting each zone. 5 adding curves to wellformed atomic layouts intuitively, when adding the curve to an atomic diagram, we are seeking a path that cuts zones to be split in two, contains certain zones, excludes the remaining zones, and returns to its starting point. first, we consider the special case of splitting each zone. to illustrate, if we wish to add a curve to d4 in figure 3 that splits each zone then we can do so by finding a hamiltonian cycle 7 / 16 volume 13 (2008) generating euler diagrams d12 a b c a b d13 c d figure 6: splitting zones and hamiltonian cycles. in the concrete dual (figure 4, d9), as shown in figure 6, d12. this hamiltonian cycle then, rather nicely, gives us the new curve and wellformedness is maintained, as shown in d13. of course, there are concrete dual graphs that have different layouts, but corollary 1 establishes that this is not an issue when splitting each zone. the following theorem, importantly, provides a constructive method for embedding the new curve, namely find a hamiltonian cycle in an arbitrary concrete dual graph and use that cycle as the image of the new curve. theorem 1 let d = (curve, l) be an atomic wellformed concrete euler diagram containing at least two curves. let l be a label in l − im(l). there exists a curve, c, that is not in curve such that d + (c, l) is wellformed and derived from d by splitting each zone if and only if a concrete dual graph of d is hamiltonian. d14 a b c figure 7: an extended euler dual. there are topologically different ways of adding a curve that splits each zone1. in fact, given all concrete dual graphs (reduced by equivalence up to an isotopy of a subset of r2; we omit the details) we can exactly classify the number of topologically different ways of adding such a curve. to do this, we define a new type of dual graph for atomic diagrams that generalizes both the concrete dual graph and the euler graph dual: the extended euler dual. its construction is the same as the euler graph dual, except that it has additional edges as follows. given an edge in the euler graph dual that connects a vertex, v1, to the vertex, v2, in the infinite face, we had a choice about the direction that edge wraps around the curves. for each such edge, we add a new edge incident with v1 and v2 that wraps the opposite way around the curves. note that the extended euler dual is not necessarily planar; in fact, it is only planar when exactly one zone is topologically adjacent to the infinite face. we add the new edges in such a manner 1 given d and curves c1 and c2 added to d, d + (c1, l) is topologically different to d + (c2, l) if c1 and c2 are not isotopic in r2 −v (g) where v (g) is the set of (images of the) vertices in the euler graph. proc. led 2008 8 / 16 eceasst that a minimal number of edge crossings are introduced. the minimal number of crossings is 1 + 2 + ...+ ( deg(v2 )2 −1). the construction is illustrated in figure 7, where d14 shows the extended euler dual of d4 in figure 4. these extra edges are required in order to make the graph reflect all possible ways (up to some notion of equivalence) of adding a new curve and maintaining wellformedness. we note that we only add extra edges incident with v2 since the infinite face is the only zone that is not simply connected in an atomic wellformed diagram. in simply connected faces, there is essentially no choice about the ‘direction’ of the edges. theorem 2 the set of all plane hamiltonian cycles in the extended euler dual of an atomic wellformed concrete euler diagram, d, gives all the topologically different ways of introducing a curve that splits each zone in d and maintains wellformedness. the method of adding a curve that splits each zone generalizes. when we want to add a curve that splits a specified set of zones, we instead seek a plane, simple cycle2 in the extended euler dual that passes through exactly the vertices corresponding to the zones that are to be split. suppose that we wish to add a curve to d4 in figure 3 such that the zone inside exactly c is split as is the zone outside all curves. then there are several ways of doing this, resulting in diagrams with different abstract descriptions. the method is to find a simple cycle in the extended euler dual that contains precisely the vertices inside these two zones. one such cycle is shown in d15, figure 8, which results in the curve d being added as shown in d16. another cycle gives d17, which has an abstraction different from that of d16. typically, we want to specify which zones are to be contained by the new curve as well as those which are to be split. a method for adding an appropriate curve is captured by the following theorem. d17 d a b c d16 d a b c d15 a b c figure 8: splitting specified zones. theorem 3 let d = (curve, l) be an atomic wellformed concrete euler diagram. let l be a label in l − im(l). given ab(d) = (l, z), let in ⊆ z and out ⊆ z be such that in ∪ out = z and /0 ∈ out. then there exists a curve, c, that is not in curve such that d + (c, l) is atomic, wellformed and has abstraction ab(d) + (l, in, out) if and only if either 1. there exists a plane, simple cycle, c, in the extended euler dual, g, such that 2 a simple cycle is a cycle, containing at least one vertex, that does not pass through any vertex more than once. a plane cycle is a cycle in which no pair of edges cross. 9 / 16 volume 13 (2008) generating euler diagrams (a) the vertices in g that correspond to zones that are elements of the set in ∩ out are exactly those in c, (b) the vertices in g that correspond to zones that are elements of the set in − out are located inside c, and (c) the vertices in g that correspond to zones that are elements of the set out − in are located outside c, or 2. |in| = |in ∩ out| = 2 and the two concrete zones corresponding to the abstract zones that are elements of the set in ∩ out are topologically adjacent. again, it is the plane, simple cycle that provides the image of the curve to be added except in case 2, where we do not seek a such a cycle. instead, we are effectively seeking a path of length 1 (indicating the adjacency of the two zones) and can simply add a curve that is a circle, for example, in the appropriate manner. 6 adding a curve to wellformed nested layouts the previous section characterizes exactly when a curve can be added to an atomic wellformed diagram and maintain wellformedness. moreover, the characterization provides a constructive method to add a curve to give a new diagram with some specified abstract syntax. here we demonstrate how to extend the approach to the nested case by example only, due to space limitations. in figure 9, we may want to add a new curve to d18 that splits the zone outside all curves, that inside just b and that inside just d, and all remaining zones are outside the curve. to do this, we decompose the diagram into its atomic components, add the new curve, e , to each part in the required manner and then recompose the diagram, joining up the curves labelled e in each of the atomic parts to create a single curve labelled e , shown in d19. the curve e is called a disconnecting curve for d19, the theory of which is developed in [ff08]. a b d18 c d a b d c e e a b c d a b d19 c d e e e figure 9: adding curves to wellformed nested diagrams. in summary, the approach is to add the required curve to each atomic component and then join the pieces of the new curve together, to create one curve. the results of the previous section tell proc. led 2008 10 / 16 eceasst us when the curve can be added to the atomic pieces. the only further requirement is that these new curves can be joined together to create an appropriate curve in the original nested diagram. we note that being able to add a curve to each atomic part does not imply that it is possible to add that curve to the entire diagram. for example, suppose we had wanted to add a curve to d18 so that instead of splitting the zone inside just d we split the zone inside both c and d. it would not then be possible to join the curves labelled e in the atomic components together to form an appropriate curve in d18 whilst maintaining wellformedness. the following theorem characterizes the case when we wish to split each zone. theorem 4 let d be a wellformed concrete euler diagram containing exactly n atomic components, say d1, ... , dn. let l be a label in l − im(l). then there exists a curve, c, such that d + (c, l) is wellformed and derived from d by splitting each zone if and only if each di has a a hamiltonian concrete dual or contains exactly one curve. in the more general wellformed case (where we do not necessarily wish to split every zone), we must be able to add a curve, c, to each atomic component in the required manner (as illustrated above). we need to know when the curves added to the atomic components can be joined up to form c. suppose that c is to be added to a diagram, d, consisting of two atomic components d1 and d2, with d2 nested in a zone z1 of d1. then the curves c1 and c2 added to d1 and d2 respectively can be joined whenever they both pass through the zone z1: in other words, in d1 the curve c1 splits z1 and in d2 the zone outside all of the curves is split by c2. this observation generalizes to the case when there are more atomic components. 7 adding curves in general we aim to be able to generate an embedding of any abstract description, which is not possible when imposing the wellformedness conditions. thus, we no longer insist that the wellformedness conditions are met and allow ourselves to add a curve to arbitrary euler diagrams. there are many ways of adding curves in the general case and it can be shown that methods exist that allow the inductive construction of a concrete diagram (i.e. by successively adding contours) for any abstract description. here we informally outline one such method, but better layouts can be achieved by using more sophisticated techniques; for space reasons we do not provide details. since we are allowed multiple label use, the method may add many curves (which constitute a single contour) in order to achieve the correct zone set after addition. one point to note is how the interior of a contour is defined when multiple curves have the same label; such a definition is given in [srh+07] and extends work in [vv04]. to illustrate, in figure 10, d20 contains three curves labelled a, which we will refer to as the contour a. a point is interior to the contour a if the number of curves labelled a to which it is interior is odd, otherwise it is exterior to the contour a. thus, the zones in the diagram are z(d20) = {/0,{a},{a, b},{b},{c},{a,c}} and the zone /0 is disconnected. suppose, to the abstract description ab(d20), we wish to add a contour labelled d, given in = {/0,{a},{c}} and out = {/0,{a},{a, b},{b},{a,c}. the diagram d21 has the required 11 / 16 volume 13 (2008) generating euler diagrams d20 a b c a a d21 a b c a a d d d d figure 10: inside contours and adding curves in general. abstract description and is obtained from d20 by following the following method. given a concrete diagram d = (curve, l), a label l ∈ l − im(l), and sets in, out ⊆ z(ab(d)) such that in ∪ out = z(d), we can add a curve to d such that: 1. for each abstract zone in the set in ∩ out, add a curve labelled l properly inside some minimal region of which the corresponding concrete zone consists, 2. for each abstract zone in the set in − out, add a curve labelled l for each boundary of any minimal region of which the corresponding concrete zone consists (note that minimal regions may have many boundaries) such that the image of the curve is that boundary, and 3. if in = /0 then add a curve labelled l whose image is a straight line segment. the resulting diagram has abstraction ab(d) + (l, in, out). 8 application to the generation problem as stated earlier, the generation problem is to find an embedding of any given abstract description (or, at least, each atomic component). several approaches have been devised to date and they all proceed to find a layout of the entire diagram and do not always succeed. our results provide a new approach to generation, in that we can inductively produce the required diagram. suppose we have an abstract description, d1 = (l, z), that we wish to draw. in both the wellformed and non-wellformed cases, we can decompose d1 into a sequence of layout problems, starting with the diagram ( /0,{/0}) and successively adding curves in the required manner to build d1 = (l, z). of course, there are choices about how to inductively add the curves to achieve a good final layout, but there are some obvious methods one can use to help here. if we want to produce a wellformed layout then we can detect whether the diagram is atomic at the abstract level and consider the abstract components separately, as discussed above. moreover, there may be some label, l, and abstract description, d2, such that d2 + (l, in, out) = d1, for some sets in and out, and d2 contains more atomic components than d1; in such cases, it may be more efficient to add the curve labelled l last. to illustrate, we provide an example in figure 11, which produces a wellformed embedding of d = (l, z) where l = {a, b,c, d} and z = {/0,{a},{b},{a, b},{a,c},{a, d},{a, b,c},{a, b, d},{a,c, d},{a, b,c, d}}. proc. led 2008 12 / 16 eceasst c d4 ba d c d5 ba d1 a d2 b d3 a figure 11: application to the general generation problem. there are limits to the inductive construction method. first, we know of wellformed concrete diagrams that cannot be produced using the inductive approach. one such diagram can be seen in figure 12; the removal of any curve results in a diagram that is not wellformed. in this case, a wellformed diagram with the same abstract description can be generated using the inductive method even though this particular layout cannot be achieved. it remains the subject of future work to establish whether any abstract description that has a wellformed embedding can be drawn using our inductive approach that utilizes the extended euler dual. however, the general method for adding a contour, outlined in section 7, can be shown to yield an embedding of any abstract description using such an inductive construction. a b cd e figure 12: a wellformed venn-5 embedding. 9 implementation to prototype the generation mechanism, we have started implementing the method as a java program. this draws closed curves with polygons, detects the extended euler dual using geometric algorithms and routes the edges around this dual as shown in figure 13, where the polygons are regular and form venn-3. note that the routing mechanism used for constructing the extended euler dual means that some edges are very close together and can be mistaken as tangential. in figure 13, we show the result of finding a hamiltonian cycle in the extended euler dual, and using that cycle to add a new polygon, labelled ‘d’ to the diagram, resulting in venn-4. in this case we chose the new curve so that it intersects with every zone, but we could have 13 / 16 volume 13 (2008) generating euler diagrams figure 13: using the extended euler dual graph to add a curve. used any simple cycle in the extended euler dual to add a curve and maintained wellformedness (of course, the chosen cycle impacts the abstract description). we can enumerate every simple cycle by finding sets of faces in the extended euler dual that are connected; the cycle formed by traveling around such a set of faces then gives rise to a new curve. layout improvements, such as those applied in [frm03], are required in order to improve the appearance of the diagram; this is currently being implemented. our intention is to use this generation mechanism to enumerate through possible diagrams, looking for those that can be drawn ‘nicely’, for instance where a regular polygon can be added to a diagram already consisting of regular polygons. the process of generating the extended euler dual and discovering a single simple cycle within it is reasonable efficient and works in real time. however, the time complexity of enumerating every cycle is exponential relative to the number of edges and so will be infeasible as the size of the diagrams increases beyond the small diagrams shown in this paper; heuristics will need to be developed for this task. 10 conclusion in this paper we have presented several methods for generating euler diagrams by modifying existing layouts. the technique we have presented to add a curve in the wellformed case guarantees to preserve wellformedness. moreover this novel technique of using inductive generation methods can be used to produce embeddings of a class of abstract description. indeed, our general method of adding a curve in the non-wellformed case can be used to generate an embedding of any abstract description. we plan to use these inductive embedding methods (concentrating initially on wellformed diagrams) to populate a library of drawn examples from which we can subsequently create further embeddings by adding further curves. we anticipate that such a library will contain a concrete diagram for each abstract description with up to three labels, and many with four labels (there are 216 abstract descriptions with four labels). we will then be able to take abstract descriptions and select sub-diagrams from the library and add curves to them to produce the required concrete diagram. proc. led 2008 14 / 16 eceasst further work also includes extending the generation algorithms to allow subsets of the wellformedness conditions to be imposed. we anticipate using a hybrid of the euler graph and the extended euler dual to allow, for example, concurrency or triple points to be present in the created layouts. this will enable a wider variety of layouts to be produced and will allow us to take user preference more fully into account, for example. acknowledgements: this work is supported by the uk epsrc grants ep/e011160/1 and ep/e010393/1 for the visualization with euler diagrams project. thanks also to john taylor for helpful discussions on aspects of this research. bibliography [cho07] s. chow. generating and drawing area-proportional euler and venn diagrams. phd thesis, university of victoria, 2007. [cla05] r. clark. failure mode modular de-composition using spider diagrams. proc. euler diagrams 2004 elsevier, entcs vol. 134, pages 19–31, 2005. [cr03] s. chow, f. ruskey. drawing area-proportional venn and euler diagrams. proc. graph drawing 2003, perugia, italy, springer, 466–477, september 2003. [cr05b] s. chow, f. ruskey. towards a general solution to drawing area-proportional euler diagrams. proc. euler diagrams, elsevier, entcs vol 134, pages 3–18, 2005. [des03] r. dechiara, u. erra, v. scarano. a system for virtual directories using euler diagrams. proc. information visualisation, ieee computer society, pages 120126, 2003. [fh02] j. flower, j. howse. generating euler diagrams. proceedings of 2nd international conference on the theory and application of diagrams, springer, pages 61–75, april 2002. [fht04] j. flower, j. howse, j. taylor. nesting in euler diagrams: syntax, semantics and construction. software and systems modelling 3:55–67, march 2004. [frm03] j. flower, p. rodgers, p. mutton. layout metrics for euler diagrams. 7th international conference on information visualisation ieee computer society press, pages 272-280, 2003. [ff08] a. fish, j. flower. euler diagram decomposition. accepted for diagrams 2008, springer, 2008. [hes+05] p. hayes, t. eskridge, r. saavedra, t. reichherzer, m. mehrotra, d. bobrovnikoff. collaborative knowledge capture in ontologies. proc. 3rd international conference on knowledge capture, pp. 99–106, 2005. 15 / 16 volume 13 (2008) generating euler diagrams [hs05] j. howse, s. schuman. precise visual modelling. journal of software and systems modeling 4:310–325, 2005. [kc99] s.-k. kim, d. carrington. visualization of formal specifications. 6th asia pacific software engineering conference, ieee computer society press, pages 102–109, 1999. [kmg+05] h. kestler, a. muller, t. gress, m. buchholz. generalized venn diagrams: a new method for visualizing complex genetic set relations. journal of bioinformatics 21(8):1592–1595, 2005. [lov02] j. lovdahl. towards a visual editing environment for the languages of the semantic web. phd thesis, linkoping university, 2002. [mel+95] k. misue, p. eades, w. lai, k. sugiyama. layout adjustment and the mental map, journal of visual languages and computing, 2(6):183-210, 1995. [nie2006] l. niebrój. defining health/illness: societal and/or clinical medicine? journal of physiology and pharmacology, 57(4):251-262, 2006. [rzf08] p. rodgers, l. zhang, a. fish. general euler diagram generation, accepted for diagrams 2008, springer, 2008. [sk00] h. sawamura, k. kiyozuka. jvenn: a visual reasoning system with diagrams and sentences. proc. 1st international conference on the theory and application of diagrams springer, pages 271–285, 2000. [shi94] s.-j. shin. the logical status of diagrams. cambridge university press, 1994. [sfr07] g. stapleton, a. fish and p. rodgers. abstract euler diagram isomorphism. accepted for visual languages and computing, knowledge systems institute, 2008. [smf+07] g. stapleton, j. masthoff, j. flower, a. fish, j. southern. automated theorem proving in euler diagrams systems. journal of automated reasoning, june 2007. [srh+07] g. stapleton, p. rodgers, j. howse and j. taylor. properties of euler diagrams. proc. of layout of software engineering diagrams, easst, pages 2–16, 2007. [sa04] n. swoboda, g. allwein. using dag transformations to verify euler/venn homogeneous and euler/venn fol heterogeneous rules of inference. journal on software and system modeling 3(2):136–149, 2004. [tvv05] j. thiévre, m. viaud, a. verroust-blondet. using euler diagrams in traditional library environments. euler diagrams 2004 elsevier, entcs vol 134, pages 189– 202, 2005. [vv04] a. verroust, m.-l. viaud. ensuring the drawability of euler diagrams for up to eight sets. proc. 3rd international conference on the theory and application of diagrams springer, pages 128–141, 2004. proc. led 2008 16 / 16 introduction motivation euler diagrams concrete diagrams diagram descriptions adding a curve adding curves to wellformed atomic layouts adding a curve to wellformed nested layouts adding curves in general application to the generation problem implementation conclusion towards scion-enabled ixps: the scion peering coordinator electronic communications of the easst volume 080 (2021) conference on networked systems 2021 (netsys 2021) towards scion-enabled ixps: the scion peering coordinator lars-christian schulz and david hausheer 4 pages guest editors: andreas blenk, mathias fischer, stefan fischer, horst hellbrueck, oliver hohlfeld, andreas kassler, koojana kuladinithi, winfried lamersdorf, olaf landsiedel, andreas timm-giel, alexey vinel eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst towards scion-enabled ixps: the scion peering coordinator lars-christian schulz1 and david hausheer2 networks and distributed systems lab otto-von-guericke-university magdeburg, germany email: 1lschulz@ovgu.de, 2hausheer@ovgu.de abstract: internet exchange points (ixps) around the world bring thousands of isps together to form dense peering fabrics. since bilateral bgp peering sessions alone would result in large overhead, many ixps offer route servers enabling the exchange of routing information with the entire peering population over a single multilateral bgp session. route servers also perform rpki validation to combat the lack of authentication and security in bgp. scion is a novel inter-domain routing architecture addressing the security flaws of bgp by replacing it with a security and reliability centric clean-slate approach. we envision that operators of scion ases will be just as open to peering at internet exchanges as they are today with bgp. moreover, to fully utilize scion’s multipath capabilities scion as operators tend to deploy more different as numbers than in bgp, further increasing the potential number of unique peering links at an ix. since scion has no native multilateral peering support, we propose the scion peering coordinator, an ixp-hosted service automating scion peering link setup based on per as policies. as such, the scion peering coordinator provides an open peering platform similar to bgp route servers to scion. keywords: scion, ixp, peering, route server 1 introduction route servers at ixps solve two important problems. first, they improve bgp’s scalability by replacing densely meshed bilateral peering with hub-based multilateral peering. secondly, they filter out invalid prefix announcements and perform rpki validation to prevent network outages due to misconfigured border routers and malicious prefix hijacking. scion, which aims to replace bgp in a future internet, addresses the latter issue with its fully authenticated control plane, but does not provide a built-in solution to the scalability concerns of fully meshed peering. the notion of multilateral peering does not exist in scion. since scion’s control plane is designed to be very efficient, full mesh peering might not pose a scalability issue like it does in bgp, but multilateral peering at bgp route servers has another advantage. route servers offer instant connectivity to a large part of the ixp’s peering population to anyone joining [rsf+14]. scion lacks a mechanism to get connected quickly and easily at ixps. all peering links have to be manually negotiated and set up by human operators. the aim of our proposed peering coordinator is to provide an automatic as configuration service to scion ases joining a scion-enabled ixp. the peering coordinator accepts a set of policy rules, which are set by each connected as, and takes care of configuring the scion border routers to form rule conforming links. 1 / 4 volume 080 (2021) mailto:lschulz@ovgu.de mailto:hausheer@ovgu.de towards scion-enabled ixps: the scion peering coordinator the policies the peering coordinator supports are inspired by bgp prefix redistribution rules as made available by special bgp community attributes at some ixp route servers. with bgp communities, peers can limit the distribution of their prefixes to certain ases or geographical regions [dec, ams]. similarly, the peering coordinator can limit peering advertisements to certain ases or administrative domains. for ixps, scion with a peering coordinator offers two direct advantages over bgp. first, the need for rpki validation at the ixp is eliminated, because scion’s control plane is authenticated by default. second, scion is a native multipath protocol, whereas a bgp route server advertises only a single route per destination and peer. the ability to expose multiple paths through the ixp opens up unused internal link capacity, for example from backup links, increasing the bandwidth available to scion over bgp. 2 background and related work scion is an inter-domain network architecture aiming to improve the internet’s scalability, reliability and security [bcp+17]. it achieves isolation of administrative concerns on a large scale by grouping ases into isolation domains (isds), which fall under a common jurisdiction. an isd is administered by its core, a consortium of densely interconnected core ases. scion’s path discovery process based on flooding so called path-segment construction beacons (pcbs) through the network gives rise to three distinct inter-as link types. core links connect core ases of the same or different isds. peering links connect non-core ases of the same or different isds, but in contrast to core links do not carry beacons. finally, provider-customer links attach non-core ases to their non-core transit providers and ultimately the isd core. to enable peering irrespective of whether an as belongs to the core or not, the peering coordinator is able to negotiate all three types of links. core and peering links allow ases of the same type to peer, but are forbidden between ases of different types by the current scion specification. however, in the special case of peering between a core and non-core as within the same isd, a provider-customer link can substitute for a native peering link. operators of core ases connecting to a peering coordinator should keep in mind that peering in the core is transitive by default, and might want to implement a more restrictive peering policy than in non-core ases. originally, the peering coordinator was designed as an extension to the scionlab coordinator, which is a web service maintaining a central configuration repository for the scionlab research network [kgl+20]. we decided to fork the peering coordinator as a standalone application for the following reasons: (1) we expect that ixps want full control over their own coordinator instance. extending the scionlab coordinator would put control over the peering coordinator in the hands of the scionlab administrators. (2) the scionlab coordinator manages the configuration of all as-internal scion services. such deep access is unnecessary for the peering coordinator. (3) a separate peering coordinator should simplify deployment in private networks. 3 scion peering coordinator the peering coordinator is designed to alleviate the lack of multilateral peering in scion by automating the discovery of peers. upon joining an ixp, a connection to one or, for redundancy netsys 2021 2 / 4 eceasst reasons, multiple instances of the coordinator must be set up manually. after that, the coordinator will mediate the automatic creation of peering links between the connected ases. as operators can limit to which other ases links are formed by defining a set of filtering rules. more specifically, the peering coordinator manages two pieces of information about every connected as: (1) the set of ases the as in question offers peering to. (2) the ip address and udp port range reserved for peering links in the scion underlay of every border router connected to the ixp. the later piece of information is required for technical reasons since scion is currently implemented as an ip/udp overlay. for the actual link establishment, the peering coordinator’s only responsibility is to pick unused udp ports in the underlay at the two border routers forming the link’s endpoints, allowing for a fairly unsophisticated implementation. the set of ases to whom peering is offered is constructed from a set of peering rules installed at the coordinator by every as. in the common case of completely open peering like at a bgp route server, the only rule an as has to set at the start of its peering session is “peer with everyone”. peering rules in the coordinator both offer peering to other ases and accept offers from other ases, so an as with a default “peer with everyone” rule will have links to as many other ases participating at the peering coordinator as possible. in addition to open peering, the peering rules can express a limited set of selective peering policies. selective peering we define four kinds of accept/reject rules, with a scope of either any as (the default rule), all ases of an isd (often coincident with a geographical region), all ases owned by the same entity (company, etc.), or individual ases. more specific rules have precedence over less specific ones. we introduce rules based on the as owner, because scion as operators are likely to employ multiple logically distinct ases to better exploit the path control offered by scion. selecting potential peer ases grouped by the underlying administrative domain seems like an obvious use case for the coordinator’s filtering rules. however, scion itself does not provide owner information, so the mapping from as to owner must be maintained out-of-band. as policies b accept c } ⇒ c-b c accept b,�d d accept �c, e ⇒ d-e e reject g, h core as core link peering link provider-customer link a c f isd 1 b d e g h isd 2 owner policies f accept [f, g] } ⇒ f-g g accept [f, g] isd policies e accept isd 2 ⇒ d-e g accept isd 2 } ⇒ g-h h accept isd 2 figure 1: example topology. ases a to h all peer at an instance of the scion peering coordinator. the black links are fixed (e.g., private peering or non-ixp links). peering links negotiated by the peering coordinator and the corresponding as-, owner-, and isd-level rules are colored. as mentioned previously, open peering is achieved by setting the default rule to accept. assuming the default rules is set to reject instead, figure 1 illustrates a few example policies expressible in the coordinator. in the example, as e accepts peering with everyone in its own isd (isd 2), 3 / 4 volume 080 (2021) towards scion-enabled ixps: the scion peering coordinator but disallows peering with g and h, which are its customers. since as-level rules have higher priority than less specific isd-level policies, only the peering link to d (dashed) is created. as mentioned in section 2, due to limitations in scion’s current design, it is not possible to connect core and non-core ases from different isds like c and d, consequentially such pairs are ignored. implementation due to its legacy as an extension to the scionlab coordinator, the peering coordinator is a django web application [spc]. it provides a grpc api to the peering configuration client which is installed alongside all scion border routers connecting to the ixp. the purpose of the peering client is to perform the actual configuration changes needed to instruct the scion infrastructure services to establish or drop links. additionally, the coordinator serves a web interface for monitoring and management purposes. 4 preliminary conclusions and future work the scion peering coordinator brings the ease of public bgp peering at an ixp-hosted route server to scion’s clean-slate internet routing approach. it supports the basic use case of open peering as well as a set of selective peering rules modeled after the prefix distribution control offered by bgp communities. a remaining challenge with the peering coordinator is that it operates completely out-of-band. like the scionlab coordinator, the peering coordinator aims at getting scion interconnection started today, but is not integrated with the scion control plane. in future work, we will investigate the possibility of an in-band neighbor discovery protocol in scion which could supersede the peering coordinator. another topic of future interest are the questions posed to the economics of public peering by scion’s inherent path control and multipath capabilities. bibliography [ams] ams-ix route servers. https://www.ams-ix.net/ams/documentation/ams-ix-routeservers. [bcp+17] d. barrera, l. chuat, a. perrig, r. m. reischuk, p. szalachowski. the scion internet architecture. in communications of the acm. volume 60(6), pp. 56–65. 2017. [dec] de-cix: operational bgp communities. https://www.de-cix.net/en/resources/routeserver-guides/operational-bgp-communities. [kgl+20] j. kwon, j. a. garcı́a-pardo, m. legner, f. wirz, m. frei, d. hausheer, a. perrig. scionlab: a next-generation internet testbed. in ieee international conference on network protocols. 2020. [rsf+14] p. richter, g. smaragdakis, a. feldmann, n. chatzis, j. boettger, w. willinger. peering at peerings: on the role of ixp route servers. in proceedings of the internet measurement conference. pp. 31–44. acm, 2014. [spc] peering coordinator. https://github.com/netsys-lab/scion-peering-coordinator. netsys 2021 4 / 4 https://www.ams-ix.net/ams/documentation/ams-ix-route-servers https://www.ams-ix.net/ams/documentation/ams-ix-route-servers https://www.de-cix.net/en/resources/route-server-guides/operational-bgp-communities https://www.de-cix.net/en/resources/route-server-guides/operational-bgp-communities https://github.com/netsys-lab/scion-peering-coordinator introduction background and related work scion peering coordinator preliminary conclusions and future work divide and conquer -organizing component-based adaptation in distributed environments electronic communications of the easst volume 11 (2008) proceedings of the first international discotec workshop on context-aware adaptation mechanisms for pervasive and ubiquitous services (campus 2008) divide and conquer – organizing component-based adaptation in distributed environments ulrich scholz and romain rouvoy 12 pages guest editors: romain rouvoy, mauro caporuscio, michael wagner managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst divide and conquer – organizing component-based adaptation in distributed environments ulrich scholz1 and romain rouvoy2 1 european media laboratory gmbh schloß-wolfsbrunnenweg 33 69118 heidelberg, germany ulrich.scholz@eml-d.villa-bosch.de 2 university of oslo, dept. of informatics p.o.box 1080 blindern 0316 oslo, norway rouvoy@ifi.uio.no abstract: this paper introduces a divide and conquer approach for organizing the adaptation of distributed applications in a potentially large number of interacting middleware instances. in such an environment, a centralistic and static adaptation reasoning i) is inadequate and ii) gives the same priority to all applications. the divide and conquer method aims at minimizing the interference between running applications, allowing users to weight the priority of applications, and organizing the adaptation and the reasoning about the adaptation in a decentralized and flexible way. keywords: adaptive middleware, distributed adaptation reasoning 1 introduction this work is concerned with the task of adapting a number of large, distributed applications in mobile environments subject to frequent context changes. we consider this problem within music [mus], an initiative to develop a comprehensive open-source platform that facilitates the development of self-adaptive software in ubiquitous environments. one aim of music is a large-scale deployment of multiple middleware instances. some of these instances host two or more applications, some applications are distributed on two or more instances. furthermore, the topology of middleware instances and applications is transient—i.e., they can appear and disappear at any time. for such a collection of middleware instances, users, applications, devices, connections, and other artefacts related to adaptation, we assign the term theatre. current solutions to the adaptation problem use a centralized and coarse-grained approach, which is not suited for large theatres: if one part of an application needs adaptation, the whole application—or even a set of applications—is adapted in combination by a dedicated solver. the solver considers all alternative configurations of all these applications at once and chooses the configuration that yields the best utility. this approach is not feasible for large theatres due to the combinatorial explosion of alternative configurations and because it interferes with parts of the theatre even if such interference is not required. as a solution, we propose the divide and conquer (d&c) approach to organize the adaptation tasks of a theatre in a decentralized and distributed way. d&c considers units that are smaller than single applications and provides techniques that allow the adaptation of partial applications. 1 / 12 volume 11 (2008) ulrich.scholz@eml-d.villa-bosch.de rouvoy@ifi.uio.no divide and conquer – organizing component-based adaptation in the following, we describe the adaptation problem and introduce the basic d&c ideas (cf. section 2). then we detail these ideas, namely the application packs (cf. section 3), resource distribution (cf. section 4), the decomposition tree (cf. section 5), and the strategies (cf. section 6). before listing related work (cf. section 8) and giving concluding remarks (cf. section 9), we explain d&c with an example scenario (cf. section 7). 2 the adaptation problem and the d&c approach the music project’s focus is the adaptation of component-based applications. applications are assembled of components—i.e., pieces of code—and several different collections of components, each called a variant, can realize the same application. many variants provide the same function to the user (e.g., participation in a picture sharing community), but often with different nonfunctional properties (e.g., quality of service and cpu usage). the degree to which the nonfunctional properties of a variant satisfy the user is called the utility of that variant [fhs+06]. adapting an application means choosing and commissioning one of its variants, while the adaptation problem means adapting an application such that it has the highest—or a sufficiently high—utility in a given situation. an adaptation middleware, such as music, aims at a solution to the adaptation problem such that every application maintains its functionality and a high utility despite of changes in the theatre. not all variants of an application are valid: they have to satisfy architectural constraints [krg07]. two examples for such constraints are the existential dependency between two components that holds if choosing one only makes sense if the other is chosen too, and a dependency where two components are mutually exclusive or require each other. adaptations can introduce new dependencies to an application and remove existing ones. existing adaptive systems usually consider entire applications: if some part of an application requires an adaptation then the whole application has to be adapted. in madam [mad06] the unit of adaptation is even wider, as it comprises all applications on a local device and all parts of these application deployed on other devices. adapting large units guarantees an optimal utility but has several drawbacks. one of them is the combinatorial explosion in the number of variants that have to be considered. if an application always consists of 5 components and there are 5 variants for each then the application has 55 = 3125 variants. for an application comprising 10 components, or for the combination of 2 applications with 5 components each, there are already about 10 million of them. although increasing processing power, restrictions on valid choices of variants, and heuristics allow solving large adaptation problems, even medium sized collections of applications are often infeasible for a global method. another reason for the inadequateness of current approaches is that they affect parts of the theatre that are better left untouched. adapting an application involves stopping it and re-starting it after some time. nevertheless, some application parts, such as video stream receivers, may not support to be suspended and resumed dynamically. in addition, users are willing to accept such interruption of service only if the disruption is very short or they observe a clear advantage. as the adaptation time is often pronounced, adapting large parts of theatres can lead to many such undesired outages for a user. in contrast, d&c forgoes globally optimal solutions by adopting a more fine-grained approach proc. campus 2008 2 / 12 eceasst towards adaptation. applications are divided into smaller units, called application parts, and d&c organizes the adaptation and distribution of collections of such parts, called packs, in a decentralized and flexible manner. then, the adaptation of each part is treated independently as black box by d&c. furthermore, current approaches do not distinguish between the application adaptation and its organization. as a result, the realization of adaptation control requires in-depth knowledge about the logic of its application and the environment to foresee possible adaptation situations. d&c provides a clear separation between both aspects of adaptation reasoning. in detail, d&c comprises five concepts to adaptation organization: (1) the concept of application parts and packs, (2) the splitting and merging of packs, (3) the reasoning about pack layout, (4) the resource negotiation between packs, and (5) the decentralized, flexible coordination of the adaptation. the first two points are covered by the following section, resource negotiation by section 4, and the decomposition tree by section 5. reasoning about pack layout is not an essential part of d&c and we omit it here for space reasons. 3 packs – adapting collections of application parts the d&c units of adaptation are parts and packs. applications are divided into parts that can be adapted independently or in combination; a pack is a collections of such parts. the handling of packs—i.e., their adaptation, division, aggregation, and relocation, as well as the organization of these operations—forms the essence of the d&c approach. with respect to complexity and autonomy, application parts are positioned between components and full applications. like applications, they are built of components and have their own utility function. their overall utility is the product of the ones of their parts. as for components, architectural constraints between parts can restrict their variant space at a certain time. note that the division of an application into parts and packs does not increase its adaptation complexity compared to the same application built of components: adapting all parts in combination takes the same time as adapting all components in combination and choosing to not adapt some parts can only reduce the required effort. packs are a purely logical assembly of application parts. parts in a pack are usually handled as a whole. in particular, all parts of the same application that are in one pack are always adapted in combination, eliminating the need for d&c to handle the architectural constraints between these parts explicitly. each pack can be adapted independently of other packs and only the applications of the part in a pack have to be stopped during its adaptation. the division into packs reflects the results of d&c’s reasoning about which application parts are likely to be adapted together and which independently. if two application parts a and b of different applications in the same pack adapt in combination then the adaptation mechanism considers each element of the cross product pa × pb. the time to adapt them is ta · tb, where pa and pb are the sets of variants of a and b, respectively, and the time to establish these sets is ta and tb. in case a and b are in different packs, the elements of pa and pb are considered independently and the adaptation time is ta + tb in the worst case. in other words, by placing a and b in different packs, we go from an exponential alternative space to a linear one. the aggregation of application parts to packs allows to organize the adaptation and to adjust 3 / 12 volume 11 (2008) divide and conquer – organizing component-based adaptation the reasoning effort against the quality of adaptation. on the one hand, the larger the packs on a machine are, i) the higher the expected utility after their individual adaptation and ii) the easier to find a good resource distribution among them (cf. section 4). on the other hand, the smaller the packs—-i.e., the higher the number of packs on the machine—the faster the adaptation of individual packs. thus, changing the composition of packs allows the middleware to balance reasoning time and adaptation quality. in more details, the motivations for merging two packs are as follows. (1) optimization of utility: in case an application was distributed over two or more of the initial packs, the merged pack might offer a better utility function for this application. (2) lower resource usage: the improved utility function can lead to an adaptation that uses less resources, but does not decrease utility. (3) decrease the need for and increase the quality of resource negotiation: the fewer packs there are, the easier it is to distribute resources between them. packs are split for two reasons. (1) to minimize adaptation time: smaller packs have a potential smaller adaptation time. prerequisites are that resources are sufficient and the new packs have few architectural dependencies. (2) to change the layout of packs: in a d&c setting, packs are the unit of relocation. we assume that the realization of splitting and merging of packs is transparent with respect to time and memory. in particular, they do not change the resource consumption of the involved packs and packs resulting from a split have the same combined resource usage like the initial pack. although splitting and merging packs takes time, we assume that this time is negligible such that packs can be split and merged without interfering with the applications. the question of where to split a pack, which packs to merge, and when to do so is a question of strategies, which we discuss in section 6. 4 negotiation: balancing the resource consumption among packs applications on a machine do not run in isolation: they share the device resources. with resource negotiation, d&c tries to prevent the combined adaptation of all applications in case a single one of them has to adapt. the main idea is that each pack is assigned a specific amount of every resource under negotiation. when changes in the theatre trigger an adaptation, the affected packs adapt locally within the allocated resource budgets. the hope is that such a local adaptation of packs result in new variants that are “good enough”, while the other packs can remain untouched. currently, we assume that an estimate within ±20% of reported utility value suffices. 4.1 weighting the priority of applications d&c gives the user additional control about the resource distribution. it allows her/him to rate applications according to her/his interests by assigning a number between 0 (irrelevant) and 1 (highly important). the priority is independent of the utility and the current variant of the application, thus an adaptation does not change it. priority and utility allow the middleware to find the applications that mostly contribute to the user satisfaction: these are those having the highest product of application priority × current application utility. note that the priority of one application is independent of other applications. as an effect, the proc. campus 2008 4 / 12 eceasst memory < 20 < 40 ≥ 40 cpu < 100 0.0 0.0 0.0 cpu < 200 0.0 v3, 0.3 v5, 0.4 cpu ≥ 200 0.0 v3, 0.3 v6, 0.7 happinessv3mem(x) =      0.0 if x < 20 0.3 if 20 ≤ x < 40 0.4 if 40 ≤ x happinessv3cpu(x) = { 0.0 if x < 100 0.3 if 100 ≤ x figure 1: example of happiness functions. the table on the left give the utilities (and associated variants) of an application part for different assignments of resources. the two functions on the right are the happiness functions for variant v3 with cpu less than 200 units. sum of priority values for a user’s applications can result in a number higher than 1. the reason is that we expect the user to manually set the priority and we do not expect her/him to normalize the values. 4.2 happiness – estimating utility resource negotiation is based on estimating the utility of packs given an allocated resource budget. optimal utility values and assignment of resources require global adaptation reasoning. because the aim of resource negotiation is to decide whether adaptation reasoning is necessary, it can only be based on estimates. d&c estimates utility by happiness functions. let us first detail happiness functions for application parts. assume that a part p depends on resource types r that d&c negotiates and other context information c that is not negotiated by d&c. if the context in c remains unchanged, then the utility of p depends on the amount of resources in r that it can consume. each variant v of p has a fixed utility uv if its resource needs are met by the assignment to r; if not, the part p is not able to run properly. to find the optimal utility of p for a specific resource assignment, we have to explore all variants of p, rejecting the invalid ones, and pick the one with the highest utility among the remaining. consider figure 1 as an example. assume that r consists of the resources memory and cpu, and an application part p has six variants that all require different amounts of these resources and yield different utilities. with the current context c, three of these six variants are realizable and are not dominated by another variant, i.e., there is no other variant that can be realized with the same resources and that has a higher utility. if, for example, p is assigned 50 units of memory and 150 units of cpu then v3 is the optimal variant of p yielding an utility of 0.3. now, if the availability of resources changes, then finding the new optimal variant requires recalculating the table or its stored copy. but, if we assume that only one resource can change, then it suffices to store all the different values for memory, given cpu = 100, and all different values for cpu, given memory = 40. these values are combined to piecewise define the happiness functions. happiness functions of parts can be calculated as by-product of adaptation reasoning. in particular, if the adaptation reasoning enumerates all variants, then the happiness functions can be constructed during the search. happiness functions for packs are the combination of those of parts. while happiness functions for parts always give the correct results, those for packs only approximate the real utility. the reason is that two or more parts can adapt individually without an adaptation of the whole 5 / 12 volume 11 (2008) divide and conquer – organizing component-based adaptation ≥ 80% of last part utility inititaion failure success pack utility < 80% of last < 80% of last ≥ 80% of < 80% of last part utility combined utility ≥ 80% of last pack utility local adaptation of part within its pack assignment of resources global adaptation, adaptation of pack individual pack estimates new pack utility last pack utility figure 2: life cycle of a resource negotiation node. note that only the regular state transitions are shown. the figure does not show transitions taken in exceptional situations, e.g., when new packs are established and when large amounts of free resources become available. transitions labeled with “80%” refer to estimates compared to the last calculated utility. pack. although each new happiness function is correct for the given resource assignment, a change of this assignment itself is not considered. therefore, the pack happiness functions after a global adaptation can differ from the locally adjusted ones. 4.3 distributing resources the life cycle of a resource negotiation node is shown in figure 2. it is an interplay of global and local adaptation. the automaton starts with a global adaptation of all participating packs. if it fails then local adaptation cannot find a solution either and the resource use on the considered device has to be reduced, e.g., by pack relocation. on success, the resulting initial resource assignments are handed to the packs. the automaton changes to the lower right state, where parts that are affected by a context change adapted individually. if the estimate by the happiness function indicates that the result is insufficient, then the automaton visits the states clockwise and the scope of adaptation widens: first the pack utility is estimated, then the whole pack adapts, and finally a global adaptation is performed. if in any of the states the current utility estimate is “good enough”, then the automaton goes into the lower right state again. for reasons of readability, figure 2 does not show three transitions of the automaton that connect the upper right state from the remaining three. these state changes are taken if an increase in the amount of free resources results in a combined expected utility of the packs that exceeds 120% of the current value, if a pack is relocated onto the machine, and after starting up an application. for reasons of brevity, we also left out the handling of priorities in the explanation of resource distribution. 5 decomposition tree: controlling the organization of packs d&c organizes the adaptation of applications in a theatre in a distributed and self-organized way without using a central controller. this organization is performed by a so-called decomposition tree—i.e., a weakly-connected, directed acyclic graph (dag) with a single root. we use the term “tree” instead of dag because, usually, we do not regard pack nodes as part of the graph. proc. campus 2008 6 / 12 eceasst nn1 nn2 nn3 nn4 tn2 tn3 tn1 b2a1 a2 b1p1 a3 p2 d1 d2 figure 3: a decomposition tree. square nodes denote packs while negotiation nodes have a diamond shape. the remaining tree nodes are depicted circular. the packs and the nodes are hosted by two devices d1 and d2, separated by the dotted border. physically, the nodes of the tree are data structures manipulated by the middleware instances; some part of the data constitutes the internal state of the node. the tree is distributed and there is no central middleware instance that has complete knowledge about all nodes. nodes are controlled by rules and a node being active means that its rules are matched against its internal state. the physical decomposition of the nodes allows several nodes to be active simultaneously. logically, the decomposition tree represents the result of the organization at a certain point in time. the nodes of the tree are annotated with local information about the organization process, which is the internal state of the node, such as the node’s parent and children, information about the node’s device, and past decisions. the collection of all such internal states is the current state of the d&c method. operationally, the nodes dynamically change the tree according to rules. on a certain context change, e.g., the appearance of a new middleware instance, nodes become active and update the tree until a new final form is reached and activity ceases. thus, the partitioning reflects the evolution of the application configuration. usually, only a few nodes are active and activity is deliberately handed from node to node. 5.1 nodes of the decomposition tree the nodes of the dag are of one of the three types: tree node, negotiation node, and pack node. the root is always a tree node and tree nodes can have children of all three kinds. negotiation nodes are restricted to have pack nodes as children and pack nodes are always leaves of the dag. each device has one negotiation node that distributes its system resources; a pack on a device is automatically child of this node. d&c uses a predefined list of resource types that a device can offer, such as memory and cpu usage. each application has a direct negotiation node that holds information about the dependencies between the application’s parts. different resource and application negotiation node have no direct relation to each other. figure 3 illustrates a decomposition tree that controls five application parts belonging to two applications: application a is divided into parts a1 to a3 and application b is divided into parts b1 and b2. parts a1, a2, and b1 form pack p1 that is under control of the node tn2 while the pack p2, consisting of parts a3 and b2, is handled by node tn3. negotiation nodes nn2 and nn3 handle the direct dependencies between the respective packs. node tn1 is the root of the decomposition 7 / 12 volume 11 (2008) divide and conquer – organizing component-based adaptation tree, tn2 and tn3 are leaf nodes (negotiation nodes and pack nodes are not considered regular nodes of the distribution tree). the packs and the nodes are distributed over two devices d1 and d2, denoted by the dotted border. the resources of d1 are negotiated by node nn1, while nn4 does the same for device d2. 5.2 working of the decomposition tree the operations of the decomposition tree can be divided into three classes: atomic operations, complex operations, and strategies. atomic operations are realized directly by the middleware. each node has available a set of atomic operations used to examine and alter the decomposition tree. examples are the migration of its children to another node or the participation in a process for electing a common root node. strategies decide which complex operation to perform in which situation (cf. section 6). complex operations correspond to reactive actions to apply on the decomposition tree. they are composed of basic operations of one or more nodes. examples are “join two decomposition trees” and “react to a failing negotiation node”. different compositions can result in similar complex operations (e.g., for the first example) and for the same purpose there can be complex operations with different outcomes (e.g., for the second). we have realized four classes of complex operations. (1) operations that organize the adaptation of applications: these are the splitting and merging of packs and their re-distribution. (2) operations resulting from changes in the status of an application: a starting application gets its own pack on the machine it is started on. on termination, application parts are removed from their packs. if a new application part is created as a result of an adaptation—i.e., not by the replacement of one part by another—then it is placed in a new pack. if a pack becomes empty because of the termination of an application or because of an adaptation, then it is removed. (3) operations that handle the sudden disappearance of a connection or of a device. (4) decomposition tree maintenance operations: these balance the tree, split nodes that have too many children, and merge those with too few. 6 strategies: selecting the adaptation heuristics the previous sections explained techniques that allow to reason about and to control the organization of the adaptation of theatres, the principles of this reasoning are given by strategies. they are implemented by rules and each node has its own rule-based control loop. strategies are currently under investigation and we develop a simulator to identify and compare different strategies. in detail, each node has an internal state that holds all the knowledge that the node has about the theatre. for example, a tree node knows about its parent, its children, its hosting middleware instance, and its past decisions. the internal state is updated either by one of its rules or by the middleware. the rules are condition-action pairs that match the internal state to state changes and to atomic operations as explained in subsection 5.2. strategies control the organization of the adaptation—i.e., which packs to split and where, which packs to merge, and when to relocate a pack. regarding the decomposition tree, strategies proc. campus 2008 8 / 12 eceasst have to determine the overall structure of the tree and the location of the tree nodes. they also handle effects that result from the localness of the reasoning within the decomposition tree: for example, independent decisions can yield to a deadlock. here, strategies have either to prevent these situations or have to provide a mechanism that overcomes the problem. other problems are desired states of the tree that are not reachable by other states and oscillating tree behavior. an example for a strategy is the following: when splitting a pack, we have two choices: (1) do not separate parts of the same application, e.g., {a1, a2, b1, b2} → {a1, a2},{b1, b2}. this option can result in a linear reduction of the adaptation time. it does not increase the structural negotiation effort and does not decrease the utility of the applications. (2) separate parts of the same application, e.g., {a1, a2, b1, b2} → {a1, b1},{a2, b2}. this option can result in an exponential reduction of the adaptation time but might decrease the utility of the applications. one strategy is to always try the first option if possible and try the second one only with packs where all contained parts are from the same application. 7 use case: the instantsocial scenario the following instantsocial (is) scenario [fhs08] demonstrates the capabilities of the d&c approach in organizing the adaptation in larger theatres. one general aim of adaptive middleware systems, and thus an aim of d&c, is to be transparent to the applications and to the user. therefore, we report two views: the user view typed in normal font and system view typed in cursive. paul is visiting a large rock festival. during a björk concert, he is not able to take a good shot, others could have done better. an adaptation is triggered and restructure the is decomposition tree with other discovered is nodes using a strategy which maximizes the quality of the pictures. paul is willing to share his pictures with others. he instructs his pda to look out for other visitors with the same interest. unfortunately, the internet connection is down and there is no immediate success. the decomposition tree is configured with a strategy that maximizes björk-related multimedia content. for the time being, the decomposition tree is restricted to a single node (paul’s pda) due to the lack of connectivity. back at his tent, paul listens to some music when his pda notifies him about the presence of a media sharing group. he happily joins, gives high priority to this application, and a moment later his display shows a selection of pictures, each representing a collection of shots. he browses through the content, selects the ones he likes, and begins to download. the pda runs a mp3 player with high priority and is with low. after a picture sharing community becomes available, the priorities get reversed. thus, the negotiation group sharing resources between the mp3 player and is is updated and the adaptation process allocates more resources to is in order to list and download the content provided by the community. among the adaptations performed, the media replicator and ontology components of is are replaced by similar services provided by two other pdas. suddenly, the current download aborts prematurely: one of the group members has left without prior notification. but only some of the pictures of this user disappear, others are still available. 9 / 12 volume 11 (2008) divide and conquer – organizing component-based adaptation the connection to the weak pda is lost and the is decomposition tree adapts with the help of the music middleware. the current download aborts, but some of the björk pictures have been seamlessly replicated, so their availability does not change. some time later, paul notices that the selection he sees becomes more precise: some topics he does not care about are no longer shown and some others, unusual but interesting ones, appear. he checks the internet connection and yes, the festival’s wifi network is up again. the internet connection is re-established. the decomposition tree adapts is by replacing the ontology component on paul’s pda by a much accurate one at a remote server. the selections are re-evaluated. unnoticed by him, paul’s pda now hosts a media replicator component. he decides to see the next concert and indicates his wish to leave the group. the pda asks him to wait a few seconds. after getting the acknowledge, paul returns to the stage for some more good music. some of the pictures kept on paul’s pda are moved to the remaining media replicator. after success, paul is notified and his community service terminates. the potential gains of using d&c in this example are numerous. its distributed nature allows applications to handle the frequent changes in the theatre gracefully: first, paul’s instantsocial application cannot provide useful service because it is alone. but after other instantsocial nodes become available, the social group is set up automatically. arbitrary, unanticipated changes alter the offered functionality but keep the group operational. a centralized control mechanism would fail in such situations. the use of packs allows instantsocial to make full use of the available resources by controlling which application parts are to be adapted in combination and which are not: some part of the weak pda’s instantsocial application is hosted by paul’s pda but they run with little interference. the fine-grained control over the priority of applications allows paul to search for a picture sharing group while listening to high quality music. when the group becomes available, the music is played with less quality. but paul is distracted anyway, so he does not realize it. 8 related work the greedy approach [bhre07] is another way of improving the adaptation organization. it adapts applications one by one, beginning with the one that offers highest expected utility. each application is given the remaining available resources until they are used up. the greedy approach reduces the overall adaptation time by adapting individual applications and has the potential of yielding a high overall utility. dacar [dm07] uses rule-based policies to monitor an environment, to deploy applications, and to react to changes in the environment. the use of generic rules allows the developer to formulate fine-grained policies that allow to reason about and verify the rule base. nevertheless, the control mechanism of dacar requires an entity with complete knowledge about the environment and the applications, which poses an error-prone bottleneck in dynamic theatres. in contrast, d&c builds on distributed, incomplete knowledge that is more suitable in this case. safran [dl06] is a framework for building self-adaptive, component-based applications that separates the application logic from the adaptation. it is very high level and, in principle, allows proc. campus 2008 10 / 12 eceasst for the implementation of techniques similar to the distribution tree. although safran supports distributed adaptation by allowing each component to decide upon which reconfiguration to operate, it does not support the coordination of adaptations that are carried out and can lead to unstable behavior, in certain cases. in [bt08], authors introduce a distributed architecture for coordinating autonomous agents. the proposed approach defines supervisors as coordinating entities for clusters of autonomous agents. supervisors can interact to aggregate and analyze context data retrieved by agents. each supervisor is responsible for implementing system-wide adaptations on agents associated to its cluster. according to authors, the clusters can be dynamically created and updated using dedicated techniques [edn07]. if, similarly to d&c, this approach tackles the coordination of large theatres, the proposed decomposition is rather static and does not support application driven organization of the topology. according to [mk], d&c is a combination of meta-level control-based planning and social law-based design: applications adhere to the distribution of resources provided by the negotiation nodes because of social laws. it is also control-based planning because each node in the decomposition tree “is guided by high level strategic information for cooperation”. minsky et. al. [mu00] develop principles of law-governed interactions, of which many hold for the d&c approach, too. the main difference to d&c is that they assume independent agents with their own priorities—i.e., whose decisions have to be controlled whether they are within the law— while in d&c the agents—i.e., the nodes—follow the law by design. 9 conclusions and future work this work proposes the divide and conquer (d&c) approach for organizing the adaptation of a theatre—i.e., of a number of large, distributed applications in mobile environments with frequent context changes. this organization is independent of the application logic and relieves the application developer from providing the organization himself. d&c considers packs—i.e., collections of parts of applications—and thus gives the middleware a more fine-grained control over the adaptation than what is achievable by operating with full applications. by dividing the overall task of adapting the theatre into the tasks of adapting individual packs, d&c allows the adaptation middleware to parallelize the required work. by allowing the user to assign priority to applications, d&c enables to balance the perceived quality of service according to her/his needs. the realization of this balance is independent of the application logic and does not require provisions by the developer. d&c uses distributed reasoning activities to decide upon and to change the division, as well as to react to expected and unexpected changes in the theatre. this approach yields a more decentralized and flexible organization of the adaptation as achievable by centralized reasoning. although the work on d&c is ongoing and the algorithms and heuristics of d&c are not rigorously validated, we believe that they will overcome the shortcomings of a global adaptation approach. we currently develop strategies that tell how and when the different options of organizing the adaptation should be applied. we plan to investigate if and how the results of other rule-based approaches to distributed adaptation could be applied in a d&c setting, e.g., the rules of dacar and safran. currently, we are developing a simulator that allows to investigate and compare different strategies. 11 / 12 volume 11 (2008) divide and conquer – organizing component-based adaptation acknowledgements: the work is funded by the klaus tschira foundation and by the european commission for the music project (# 035166). the authors would like to thank yun ding, frank eliassen, gunnar brataas, eli gjørven, and bernd rapp for their helpful comments. references [bhre07] g. brataas, s. hallsteinsen, r. rouvoy, f. eliassen. scalability of decision models for dynamic product lines. in proceedings of the international workshop on dynamic software produc line. sept. 2007. [bt08] l. baresi, g. tamburrelli. loose compositions for autonomic systems. in 7th international symposium on software composition (sc). lncs 4954, pp. 165–172. springer, budapest, hungary, mar. 2008. [dl06] p.-c. david, t. ledoux. an aspect-oriented approach for developing self-adaptive fractal components. in 5th international symposium on software composition. lncs 4089, pp. 82–97. springer, 2006. [dm07] j. dubus, p. merle. applying omg d&c specification and eca rules for autonomous distributed component-based systems. in kühne (ed.), international models workshop on models @ runtime (mrt’06). lncs 4364, pp. 242–251. springer, 2007. [edn07] r. m. elisabetta di nitto, daniel dubois. self-aggregation algorithms for autonomic systems. in 2nd international conference on bio-inspired models of network, information, and computing systems (bionetics). budapest, hungary, dec. 2007. [fhs+06] j. floch, s. hallsteinsen, e. stav, f. eliassen, k. lund, e. gjørven. using architecture models for runtime adaptability. ieee software 23(2):62–70, mar./apr. 2006. [fhs08] l. fraga, s. hallsteinsen, u. scholz. “instant social” – implementing a distributed mobile multi-user application with adaptation middleware. in 1st discotec workshop on context-aware adaptation mechanisms for pervasive and ubiquitous services (campus). easst, this volume. 2008. [krg07] m. u. khan, r. reichle, k. geihs. applying architectural constraints in the modeling of self-adaptive component-based applications. in ecoop workshop on model driven software adaptation (m-adapt). berlin, germany, july/aug. 2007. [mad06] madam ist. theory of adaptation. deliverable d2.2 of the project madam: mobility and adaptation enabling middleware, dec. 2006. [mk] a. mali, s. kambhampati. distributed planning. unpublished. [mu00] n. minsky, v. ungureanu. law-governed interaction: a coordination and control mechanism for heterogeneous distributed systems. tosem 9(3):273–305, 2000. [mus] ist music project. www.ist-music.eu. proc. campus 2008 12 / 12 www.ist-music.eu introduction the adaptation problem and the d&c approach packs -adapting collections of application parts negotiation: balancing the resource consumption among packs weighting the priority of applications happiness -estimating utility distributing resources decomposition tree: controlling the organization of packs nodes of the decomposition tree working of the decomposition tree strategies: selecting the adaptation heuristics use case: the instantsocial scenario related work conclusions and future work parsing of hyperedge replacement grammars with graph parser combinators electronic communications of the easst volume 10 (2008) proceedings of the seventh international workshop on graph transformation and visual modeling techniques (gt-vmt 2008) parsing of hyperedge replacement grammars with graph parser combinators steffen mazanek and mark minas 14 pages guest editors: claudia ermel, reiko heckel, juan de lara managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst parsing of hyperedge replacement grammars with graph parser combinators steffen mazanek1 and mark minas2 1 steffen.mazanek@unibw.de 2 mark.minas@unibw.de institut für softwaretechnologie universität der bundeswehr münchen, germany abstract: graph parsing is known to be computationally expensive. for this reason the construction of special-purpose parsers may be beneficial for particular graph languages. in the domain of string languages so-called parser combinators are very popular for writing efficient parsers. inspired by this approach, we have proposed graph parser combinators in a recent paper, a framework for the rapid development of special-purpose graph parsers. our basic idea has been to define primitive graph parsers for elementary graph components and a set of combinators for the flexible construction of more advanced graph parsers. following this approach, a declarative, but also more operational description of a graph language can be given that is a parser at the same time. in this paper we address the question how the process of writing correct parsers on top of our framework can be simplified by demonstrating the translation of hyperedge replacement grammars into graph parsers. the result are recursive descent parsers as known from string parsing with some additional nondeterminism. keywords: graph parsing, functional programming, parser combinators, hyperedge replacement grammars 1 introduction graph languages are widely-used nowadays, e.g., for modeling and specification. for instance, we have specified visual languages using graph grammars [min02]. in this context we are particularly interested in solving the membership problem, i.e., checking whether a given graph belongs to a particular graph language, and parsing, i.e., finding a corresponding derivation. however, while string parsing of context-free languages can be performed in o(n3), e.g., by using the well-known algorithm of cocke, younger and kasami [kas65], graph parsing is computationally expensive. there are even context-free graph languages the parsing of which is np-complete [dhk97]. thus a general-purpose graph parser cannot be expected to run in polynomial time for arbitrary grammars. the situation can be improved by imposing particular restrictions on the graph languages or grammars. anyhow, even if a language can be parsed in polynomial time by a general-purpose parser, a special-purpose parser tailored to the language is likely to outperform it. 1 / 14 volume 10 (2008) mailto:steffen.mazanek@unibw.de mailto:mark.minas@unibw.de graph parser combinators unfortunately the development of a special-purpose graph parser is an error-prone and timeconsuming task. the parser has to be optimized such that it is as efficient as possible, but still correct. backtracking, for instance, has to be prevented wherever possible. therefore, in a recent paper [mm08] we have proposed graph parser combinators, a new approach to graph parsing that allows the rapid construction of special-purpose graph parsers. further we have introduced a haskell [pey03] library implementing this approach. it provides the generic parsing framework and a predefined set of frequently needed combinators. in [mm08] we further have demonstrated the use of this combinator framework by providing an efficient special-purpose graph parser for vex [chz95] as an example. vex is a graph language for the representation of lambda terms. the performance gains mainly have resulted from the fact, that vex is context-sensitive and ambiguous – properties many general-purpose graph parsers do not cope well with. the structure of vex graphs is quite simple though, i.e., they basically are trees closely reflecting the structure of lambda terms. only variable occurrences are not identified by names as usual; they rather have to refer to their binding explicitly by an edge. nevertheless, the parser for vex could be defined quite operationally as a tree traversal. however, the operational description of languages like, e.g., structured flowgraphs is much more difficult. parsers get very complex and hard to read and verify. this brings up the question, whether graph parser combinators actually are powerful enough to express standard graph grammar formalisms. one such formalism are hyperedge replacement grammars [dhk97], which allow such languages to be described in a declarative and natural way. therefore, the main contribution of this paper is a method for the straightforward translation of hyperedge replacement grammars [dhk97] to parsers on top of our framework. the resulting parsers are readable and can be customized in a variety of ways. they are quite similar to top-down recursive descent parsers as known from string parsing where nonterminal symbols are mapped to functions. unfortunately, in a graph setting we have to deal with additional nondeterminism: besides different productions for one and the same nonterminal, we also have to guess particular nodes occurring in the right-hand side of a production. we can use backtracking, but performance naturally suffers. however, our approach can be used to build an initial, yet less efficient parser. languagespecific performance optimizations can then be used to improve the parser’s efficiency step by step. moreover, the presented approach offers the following benefits: • combination of declarative and operational description of the graph language. • an application-specific result can be computed.1 • context information can be used to describe a much broader range of languages. • robust against errors. the largest valid subgraph is identified. this paper is structured as follows: we discuss the combinator approach to parsing in sect. 2 and introduce our graph model in sect. 3. we go on with the presentation of our framework in sect. 4 and discuss the actual mapping of a hyperedge replacement grammar in sect. 5. finally, we discuss related work (sect. 6) and conclude (sect. 7). 1 a general-purpose parser normally returns a derivation sequence or a parse tree, respectively. several systems, however, provide support for attributed graph grammars. proc. gt-vmt 2008 2 / 14 eceasst 2 parser combinators our approach has been inspired by the work of hutton and meijer [hm96] who have proposed monadic parser combinators for string parsing (although the idea of parser combinators actually is much older). the basic principle of such a parser combinator library is that primitive parsers are provided that can be combined into more advanced parsers using a set of powerful combinators. for example, there are the sequence and choice combinators that can be used to emulate a grammar. however, a wide range of other combinators are also possible. for instance, parser combinator libraries often include a combinator many that applies a given parser multiple times, while collecting the results. parser combinators are very popular, because they integrate seamlessly with the rest of the program and hence the full power of the host language can be used. unlike yacc [joh75] no extra formalism is needed to specify the grammar. functional languages are particularly well-suited for the implementation of combinator libraries. here, a parser basically is a function (as we will see). a combinator like choice then is a higher-order function. higher-order functions, i.e., functions whose parameters are functions again, support the convenient reuse of existing concepts [hug89]. for instance, consider a function symb with type char->parser that constructs a parser accepting a particular symbol. then we can easily construct a list pl of parsers, e.g., by defining pl=map symb [’a’..’z’] (applies symb to each letter). a parser lcl that accepts an arbitrary lower-case letter then can be constructed by folding pl via the choice operator, i.e., lcl=foldr choice fail pl. thereby, fail is the neutral element of choice. at this point, we provide a toy example to give an impression of how a parser constructed with monadic combinators looks like. for that purpose we compare the implementation of a parser for the string language {akbkck|k > 0} and a graph parser for the corresponding language of string graphs as defined in [dhk97]. an important advantage of the combinator approach is that a more operational description of a language can be given. for instance, our exemplary language of strings akbkck is not contextfree. hence a general-purpose parser for context-free languages cannot be applied at all, although parsing this language actually is very easy: “take as many a characters as possible, then accept the same number of b characters and finally accept the same number of c characters.” using polyparse [wal07], a well-known and freely-available parser combinator library for strings, a parser for this string language can be defined as shown in fig. 1a. the type of this parser determines that the tokens are characters and the result a number, i.e., k for a string akbkck. abc::parser char int abc = do as<-many1 (char ’a’) let k=length as exactly k (char ’b’) exactly k (char ’c’) return k abcg::node->grappa int abcg n = do (n’,as)<-chain1 (diredge "a") n let k=length as (n’’,_)<-exactchain k (diredge "b") n’ exactchain k (diredge "c") n’’ return k figure 1: parsers for a) the string and b) the graph language akbkck 3 / 14 volume 10 (2008) graph parser combinators figure 2: the string graph “aabbcc” if the given word does not begin with a member of the language one of the calls of exactly fails. the code is written in the functional programming language haskell [pey03]. the given parser uses the do-notation, syntactic sugar haskell provides for dealing with monads. monads in turn provide a means to simulate state in haskell. in the context of parsers they are used to hide yet unconsumed input. otherwise, all parsers in a sequence would have to pass this list as a parameter explicitly. users of the library, however, do not have to know how this works in detail. they rather can use the library like a domain-specific language for parsing nicely embedded into a fully-fledged programming language. in order to motivate our combinator approach to graph parsing, we provide the graph equivalent to the previously introduced string parser abc. strings generally can be represented as directed, edge-labeled graphs straightforwardly. for instance, fig. 2 provides the graph representation of the string “aabbcc”.2 a graph parser for this graph language can be defined using our combinators in a manner quite similar to the parser discussed above. it is shown in fig. 1b. the main difference between the implementations of abc and abcg is, that we have to pass through the position, i.e., the node n we currently process. 3 graphs in this section we introduce hypergraphs and the basic haskell types for their representation. our graph model differs from standard definitions as found in, e.g., [dhk97], that do not introduce the notion of a context. let c be a set of labels and type : c → in a typing function for c. in the following, a hypergraph h over c is a finite set of tuples (lab, e, ns), where e is a (hyper-)edge3 identifier unique in h, lab ∈ c is an edge label and ns is a sequence of node identifiers such that type(lab) = |ns|, the length of the sequence. the nodes represented by the node identifiers in ns are called incident to edge e. we call a tuple (lab, e, ns) a context in analogy to [erw01]. the position of a particular node n in the sequence of nodes within the context of an edge e represents the so-called tentacle of e that n is attached to. hence the order of nodes matters. 2 in contrast to the string language there is a context-free hyperedge replacement grammar describing this language. however, it is quite complicated despite the simplicity of the language (cf. [dhk97]). an earley-style parser for string generating hypergraph grammars like this is discussed in [sf04]. 3 we call hyperedges just edges and hypergraphs just graphs if it is clear from the context that we are talking about hypergraphs. proc. gt-vmt 2008 4 / 14 eceasst figure 3: an exemplary flowchart a) and its hypergraph representation b) the same node identifier also may occur in more than one context indicating that the edges represented by those contexts are connected via this node. note, that our notion of hypergraphs is slightly more restrictive than the usual one, because we cannot represent isolated nodes. in particular the nodes of h are implicitly given as the union of all nodes incident to its edges. in fact, in many hypergraph application areas isolated nodes simply do not occur. for example, in the context of visual languages diagram components can be represented by hyperedges, and nodes just represent their connection points, i.e., each node is attached to at least one edge [min02]. the following haskell code introduces the basic data structures for representing nodes, edges and graphs altogether: type node = int type edge = int type tentacle = int type context = (string, edge, [node]) type graph = set context for the sake of simplicity, we represent nodes and edges by integer numbers. we declare a graph as a set of contexts, where each context represents a labeled edge including its incident nodes.4 throughout this paper we use flowcharts as a running example. in fig. 3a a structured flowchart is given. syntax analysis of structured flowcharts means to identify the represented structured program (if any). therefore, each flowchart must have a unique entry and a unique exit point. 4 in the actual implementation these types are parameterized and can be used more flexibly. 5 / 14 volume 10 (2008) graph parser combinators flowcharts can be represented by hypergraphs that we call flowgraphs in the following. in fig. 3b the hypergraph representation of the exemplary flowchart is given. hyperedges are represented by a rectangular box marked with a particular label. for instance, the statement n:=0 is mapped to a hyperedge labeled “text”. the filled black circles represent nodes that we have additionally marked with numbers. a line between a hyperedge and a node indicates that the node is visited by that hyperedge. the small numbers close to the hyperedges are the tentacle numbers. without these numbers the image may be ambiguous. for instance, the tentacle with number 0 of “text” hyperedges always has to be attached to the node the previous statement ends at whereas the tentacle 1 links the statement to its successor. the flowgraph given in fig. 3b is represented as follows using the previous declarations: fcg = {("start",0,[1]), ("text",1,[1,2]), ("cond",2,[2,7,3]), ("cond",3,[3,4,5]), ("text",4,[4,6]), ("text",5,[5,6]), ("text",6,[6,2]), ("end",7,[7])} the language of flowgraphs can be described using a hyperedge replacement grammar in a straightforward way as we see in the next section. we provide a special-purpose parser for flowgraphs on top of our framework in sect. 5. 4 parsing graphs with combinators in this section we introduce our graph parser combinators. however, first we clarify the notion of parsing in a graph setting. 4.1 graph grammars and parsers a widely known kind of graph grammar are hyperedge replacement grammars (hrg) as described in [dhk97]. here, a nonterminal hyperedge of a given hypergraph is replaced by a new hypergraph that is glued to the remaining graph by fusing particular nodes. formally, such a hrg g is a quadruple g = (n, t, p, s) that consists of a set of nonterminals n ⊂ c, a set of terminals t ⊂ c with t ∩ n = /0, a finite set of productions p and a start symbol s ∈ n. the graph grammar for flowgraphs can be defined as gfc = (nfc, tfc, pfc, fc) where nfc = {fc, stmts, stmt}, tfc = {start, end, text, cond} and pfc contains the productions given in fig. 4a. left-hand side lhs and right-hand side rhs of each production are separated by the symbol ::= and several rhs of one and the same lhs are separated by vertical bars. node numbers are used to identify corresponding nodes of lhs and rhs. the derivation tree of our exemplary flowgraph as introduced in fig. 3b is given in fig. 4b. its leaves are the terminal edges occurring in the graph whereas its inner nodes are marked with nonterminal edges indicating the application of a production. the direct descendants of an inner node represent the edges occurring in the rhs of the applied production. the numbers in parentheses thereby identify the nodes visited by the particular edge. a general-purpose graph parser for hrgs gets passed a particular hrg and a graph as parameters and constructs a derivation tree of this graph according to the grammar. this can be proc. gt-vmt 2008 6 / 14 eceasst figure 4: flowgraphs, a) grammar and b) derivation tree of the example done, for instance, in a way similar to the well-known algorithm of cocke, younger and kasami [kas65] known from string parsing (indeed, all hrgs that do not generate graphs with isolated nodes can be transformed to the graph equivalent of the string notion chomsky normal form). this approach has been elaborated theoretically by lautemann [lau89] and proven to be useful in practical applications, e.g., in [min02] for the syntax analysis of diagrams. flowgraphs can be parsed with such a general-purpose graph parser in a straightforward way. however, as mentioned in the introduction there are graph languages that are not context-free (and thus cannot be described by a hrg) or that are highly ambiguous (thus causing most general-purpose parsers to perform poorly). furthermore, here we are not interested in the derivation tree, but rather in the program represented by the graph, i.e., its semantics. for these reasons graph parser combinators are beneficial either way. we now briefly introduce the framework and describe how the hrg of flowgraphs (and other hrgs similarly) can be translated into a graph parser on top of our framework. 4.2 the combinator library due to space restrictions in the following we focus on those types and functions that are needed to translate hyperedge replacement grammars schematically. further information and a more general version of the framework can be found in [mm08]. first we provide the declaration of the type grappa representing a graph parser: newtype grappa res = p (graph -> (either res error, graph)) this type is parameterized over the type res of the result. graph parsers basically are functions from graphs to pairs consisting of the parsing result (or an error message, respectively) and the graph that remains after successful parser application. 7 / 14 volume 10 (2008) graph parser combinators name type description context (context->bool)-> grappa context a context satisfying a particular condition. labcontext string->grappa context a context with a particular label. connlabcontext string->[(tentacle,node)]-> grappa context a labeled context connected to the given nodes via the given tentacles. edge tentacle->tentacle->string-> node->grappa (node,context) a labeled context connected to the given node via a particular tentacle also returning its successor (via the other, outgoing tentacle). diredge string->node-> grappa (node,context) a directed edge, edge 0 1. table 1: graph-specific primitive parsers name type description oneof [grappa res]->grappa res returns the first successful parser of the input list, corresponds to | in grammars. chain (node->grappa (node, res))-> node->grappa (node, [res]) a chain of graphs, a node is passed through. bestnode (node->grappa res)-> grappa res identifies the node from which the best continuation is possible, very expensive. nodangledgeat node->grappa () succeeds if the given node is not incident to an edge, handy for ensuring dangling edge condition. alldifferent [node]->grappa () succeeds if the given nodes are distinct, handy for ensuring identification condition. conncomp grappa res->grappa [res] applies the given parser once per connected component, while collecting the results. table 2: some graph parser combinators in general, the most primitive parsers are return and fail. both do not consume any input. rather return succeeds unconditionally with a particular result whereas fail always fails; thereby, backtracking is initiated. in table 1 we provide some important graph-specific primitive parsers. they are all nondeterministic, i.e., support backtracking on failure, and consume the context they return. additionally we provide the primitive parser anode::grappa node that returns a node of the remaining graph (with backtracking). in table 2 we briefly sketch some of the graph parser combinators provided by our library. variations of chain have already been used in the introductory example, e.g., chain1 that demands at least one occurrence. proc. gt-vmt 2008 8 / 14 eceasst fc::grappa program fc = do (_,_,[n1])<-labcontext "start" (_,_,[n2])<-labcontext "end" stmts (n1,n2) stmts::(node, node)->grappa program stmts (n1, n2) = oneof [stmts1, stmts2] where stmts1 = do s<-stmt (n1, n2) return [s] stmts2 = do n’<-anode s<-stmt (n1, n’) p<-stmts (n’, n2) return (s:p) stmt::(node, node)->grappa stmt stmt (n1, n2) = oneof [stmt1, stmt2, stmt3] where stmt1 = do connlabcontext "text" [(0,n1),(1,n2)] return text stmt2 = do (_,_,ns)<-connlabcontext "cond" [(0,n1)] p1<-stmts ((ns!!1), n2) p2<-stmts ((ns!!2), n2) return (ifelse p1 p2) stmt3 = do (_,_,ns)<-connlabcontext "cond" [(0,n1),(1,n2)] p<-stmts ((ns!!2), n1) return (while p) figure 5: a parser for flowgraphs 5 parsing flowgraphs in this section we directly translate the grammar given in fig. 4a to a parser for the corresponding language using our framework. our goal is to map a flowgraph to its underlying program represented by the recursively defined type program: type program = [stmt] data stmt = text | ifelse program program | while program in fig. 5 the parser for flowgraphs is presented. it is not optimized with respect to performance. rather it is written in a way that makes the translation of the hrg explicit. for each nonterminal edge label l we have defined a parser function that takes a tuple of nodes (n1, ..., nt ) as a parameter such that t = type(l). several rhs of a production are handled using the oneof 9 / 14 volume 10 (2008) graph parser combinators combinator. terminal edges are matched and consumed using primitive parsers. thereby their proper embedding has to be ensured. we use the standard list operator (!!) to extract the node visited via a particular tentacle from a node list ns. for instance, stmt3 represents the while-production. first, a “cond”-edge e visiting the nodes n1 and n2 via the tentacles 0 and 1, respectively, is matched and consumed. thereafter the body of the loop is parsed, i.e., the stmts starting at the node visited by tentacle 2 of e, i.e., ns!!2, ending again at n1. finally the result is constructed and returned. if something goes wrong and backtracking becomes necessary, previously consumed input is released automatically. the parser is quite robust. for instance, redundant components are just ignored and both the dangling and the identification condition are not enforced. these relaxations can be canceled easily – the first one by adding the primitive parser eoi (end of input) to the end of the definition of the top-level parser, the others by applying the combinators nodangledgeat and alldifferent, respectively, to the nodes involved. note, that the implementation of stmts follows a common pattern, i.e., a chain of graphs between two given nodes. so using a combinator the parser declaration can be further simplified to stmts=chain1betw stmt. here, chain1betw ensures at least one occurrence as required by the language. its signature is chain1betw::((node,node)->grappa a)->(node,node)->grappa [a] and it is defined exactly as stmts except from the fact that it abstracts from the actual parser for the partial graphs. performance this parser is not very efficient. a major source of inefficiency is the use of anode that binds a yet unknown node arbitrarily thus causing a lot of backtracking. this expensive operation has to be used only for the translation of those productions, where inner nodes within the rhs are not incident to terminal edges visiting an external node, i.e., a node also occurring in the lhs.5 however, even so there are several possibilities for improvement. for instance, we currently try to make this search more targeted by the use of narrowing techniques as known from functional-logic programming languages [han07]. performance can be further improved if particular branches of the search space can be cut. for instance, we can prevent backtracking by committing to a (partial) result. in [mm08] we have demonstrated how this can be done in our framework. finally, we can apply domain-specific techniques to further improve the performance. for instance, a basic improvement would be to first decompose the given graph into connected components and apply the parser to each of them successively. we provide the combinator conncomp for this task. however, this step can only be applied to certain languages and at the expense of readability. so we can start with an easy to build and read parser for a broad range of languages. it may be less efficient, however, it can be improved step by step if necessary. further it can be integrated and reused very flexibly, since it is a first-class object. 5 the function anode can also be used to identify the start node in our introductory example abcg. proc. gt-vmt 2008 10 / 14 eceasst 6 related work our parser combinator framework basically is an adaptation of the polyparse library [wal07]. the main distinguishing characteristics of polyparse are that backtracking is the default behavior except where explicitly disallowed and that parsers can be written using monads. there is an abundance of other parser combinator libraries besides polyparse that we cannot discuss here. however, a particularly interesting one is the uu parser combinator library of utrecht university [sa99]. it is highly sophisticated and powerful, but harder to learn for a user. its key benefit is its support for error correction. hence a parser does not fail, but a sequence of correction steps is constructed instead. approaches to parsing of particular, restricted kinds of graph grammar formalisms are also related. for instance, in [sf04] an earley parser for string generating graph languages has been proposed. the diagram editor generator diagen [min02] incorporates an hrg parser that is an adaptation of the algorithm of cocke, younger and kasami. and the visual language compilercompiler vlcc [clot97] is based on the methodology of positional grammars that allows to parse restricted kinds of flex grammars (which are essentially hrgs) even in linear time. these approaches have in common that a restricted graph grammar formalism can be parsed efficiently. however, they cannot be generalized straightforwardly to a broader range of languages like our combinators. we have demonstrated that semantics can be added very flexibly in our framework. the graph transformation system agg also provides a flexible attribution concept. here, graphs can be attributed by arbitrary java objects [tae03]. rules can be attributed with java expressions allowing complex computations during the transformation process. agg does not deal with hypergraphs. however, it can deal with a broad range of graph grammars. these are given as so-called parse grammars directly deconstructing the input graph. critical pair analysis is used to organize reverse rule application. in [rs95] a parsing algorithm for context-sensitive graph grammars with a top-down and a bottom-up phase is discussed. thereby first a set of eventually useful production applications is constructed bottom-up. thereafter viable derivations from this set are computed top-down. parser combinators generally follow a top-down approach, although in a graph setting bottomup elements are beneficial from a performance point of view. finally there are other approaches that aim at the combination of functional programming and graph transformation. schneider, for instance, currently prepares a textbook that provides an implementation of the categorical approach to graph transformation with haskell [sch07]. since graphs are a category, a higher level of abstraction is used to implement graph transformation algorithms. an even more general framework is provided in [ks00]. the benefit of their approach is its generality since it just depends on categories with certain properties. however, up to now parsing is not considered. 11 / 14 volume 10 (2008) graph parser combinators 7 concluding remarks in this paper we have discussed graph parser combinators, an extensible framework supporting the flexible construction of special-purpose graph parsers even for context-sensitive graph grammars. it already provides combinators for the parsing of several frequently occurring graph patterns. we even may end with a comprehensive collection of reusable parser components. parser combinators are best used to describe a language in an operational way. for instance, we have provided a parser for the graph language akbkck as a toy example. similar situations, however, also appear in practical applications as, e.g., discussed in [kör08]. we further have provided a schema for the straightforward translation of hyperedge replacement grammars into parsers on top of our framework. we have demonstrated this using the language of flowgraphs as an example. the resulting parser is not efficient. it is rather a proof of concept. languages like flowgraphs can be parsed very efficiently using a standard bottom-up parser. however, the main benefit of our framework is that language-specific optimizations can be incorporated easily in existing parsers, e.g., by providing additional information, using special-purpose combinators, heuristics or even a bottom-up pass simplifying the graph. parsing generally is known to be an area functional languages excel in. in the context of string parsing a broad range of different approaches have been discussed. however, in particular the popular combinator approach has not been applied to graph parsing yet. with the implementation of our library we have demonstrated that graph parser combinators are possible and beneficial for the rapid development of special-purpose graph parsers. future work our approach is not restricted to functional languages though. for instance, in [ad01] the translation of string parser combinators to the object-oriented programming language java is described. we plan to adapt this approach in the future to, e.g., integrate graph parser combinators into the diagram editor generator diagen [min02]. this hopefully will allow the convenient description of even more visual languages. the parsers presented in this paper suffer from the fact that purely functional languages are not particularly dedicated to deal with incomplete information. for instance, we have discussed why inner nodes occurring in the right-hand sides of productions have to be guessed. multiparadigm declarative languages [han07] like curry [han], or even logic libraries like [nar07], are well-suited for such kinds of problems. we currently reimplement our library in a functionallogic style to overcome these limitations. this work will also clarify the relation to proof search in linear logic [gir87]. here, the edges of a hypergraph can be mapped to facts that can be connected to a parser via so-called linear implication ((). during the proof the parser consumes these facts and at the end none of them must be left. we further plan to investigate error correction-strategies in a graph setting. for instance, in the context of visual language editors based on graph grammars this would allow for powerful content assist. whereas in a string setting error-correcting parser combinators are well-understood already [sa99], not much has been done with respect to graphs yet. admittedly, we do not expect to find an efficient solution to the problem at large. with certain restrictions some kinds of corrections might be possible though. proc. gt-vmt 2008 12 / 14 eceasst bibliography [ad01] d. s. s. atze dijkstra. lazy functional parser combinators in java. technical report uu-cs-2001-18, department of information and computing sciences, utrecht university, 2001. [chz95] w. citrin, r. hall, b. zorn. programming with visual expressions. in haarslev (ed.), proc. 11th ieee symp. vis. lang. pp. 294–301. ieee computer soc. press, 5–9 1995. [clot97] g. costagliola, a. d. lucia, s. orefice, g. tortora. a parsing methodology for the implementation of visual systems. ieee trans. softw. eng. 23(12):777–799, 1997. [dhk97] f. drewes, a. habel, h.-j. kreowski. hyperedge replacement graph grammars. in rozenberg (ed.), handbook of graph grammars and computing by graph transformation. vol. i: foundations. chapter 2, pp. 95–162. world scientific, 1997. [erw01] m. erwig. inductive graphs and functional graph algorithms. j. funct. program. 11(5):467–492, 2001. [gir87] j.-y. girard. linear logic. theoretical computer science 50:1–102, 1987. [han] hanus, m. (ed.). curry: an integrated functional logic language. available at http://www.informatik.uni-kiel.de/˜curry/. [han07] m. hanus. multi-paradigm declarative languages. in proc. of the intl. conference on logic programming (iclp 2007). pp. 45–75. springer, 2007. [hm96] g. hutton, e. meijer. monadic parser combinators. technical report nottcs-tr96-4, department of computer science, university of nottingham, 1996. [hug89] j. hughes. why functional programming matters. comput. j. 32(2):98–107, 1989. [joh75] s. c. johnson. yacc: yet another compiler compiler. technical report 32, bell laboratories, murray hill, new jersey, 1975. [kas65] t. kasami. an efficient recognition and syntax analysis algorithm for context free languages. scientific report af crl-65-758, air force cambridge research laboratory, bedford, massachussetts, 1965. [kör08] a. körtgen. modeling successively connected repetitive subgraphs. in proc. of the 3rd intl. workshop on applications of graph transformation with industrial relevance (agtive’07). lncs. springer, 2008. [ks00] w. kahl, g. schmidt. exploring (finite) relation algebras using tools written in haskell. technical report 2000-02, fakultät für informatik, universität der bundeswehr, münchen, 2000. 13 / 14 volume 10 (2008) graph parser combinators [lau89] c. lautemann. the complexity of graph languages generated by hyperedge replacement. acta inf. 27(5):399–421, 1989. [min02] m. minas. concepts and realization of a diagram editor generator based on hypergraph transformation. science of computer programming 44(2):157–180, 2002. [mm08] s. mazanek, m. minas. graph parser combinators. in proc. of the 19th intl. symposium on the implementation and application of functional languages. lncs. springer, 2008. [nar07] m. naylor, e. axelsson, c. runciman. a functional-logic library for wired. in proc. of the acm sigplan workshop on haskell. pp. 37–48. acm, 2007. [pey03] s. peyton jones. haskell 98 language and libraries. the revised report. cambridge university press, 2003. [rs95] j. rekers, a. schürr. a parsing algorithm for context sensitive graph grammars. technical report 95-05, leiden university, 1995. [sa99] s. d. swierstra, p. r. azero alcocer. fast, error correcting parser combinators: a short tutorial. in pavelka et al. (eds.), 26th seminar on current trends in theory and practice of inform. lncs 1725, pp. 111–129. 1999. [sch07] h. j. schneider. graph transformations an introduction to the categorical approach. 2007. http://www2.cs.fau.de/˜schneide/gtbook/. [sf04] s. seifert, i. fischer. parsing string generating hypergraph grammars. in ehrig et al. (eds.), graph transformations. lncs 3256, pp. 352–367. springer, 2004. [tae03] g. taentzer. agg: a graph transformation environment for modeling and validation of software. in pfaltz et al. (eds.), agtive. lncs 3062, pp. 446–453. springer, 2003. [wal07] m. wallace. polyparse. 2007. http://www.cs.york.ac.uk/fp/polyparse/. proc. gt-vmt 2008 14 / 14 introduction parser combinators graphs parsing graphs with combinators graph grammars and parsers the combinator library parsing flowgraphs related work concluding remarks first-order logic for safety verification of hedge rewriting systems electronic communications of the easst volume 72 (2015) proceedings of the 15th international workshop on automated verification of critical systems (avocs 2015) first-order logic for safety verification of hedge rewriting systems alexei lisitsa 14 pages guest editors: gudmund grov, andrew ireland eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst first-order logic for safety verification of hedge rewriting systems alexei lisitsa1 department of computer science, the university of liverpool abstract: in this paper we deal with verification of safety properties of hedge rewriting systems and their generalizations. the verification problem is translated to a purely logical problem of finding a finite countermodel for a first-order formula, which is further tackled by a generic finite model finding procedure. we show that the proposed approach is at least as powerful as the methods using regular invariants. at the same time the finite countermodel method is shown to be efficient and applicable to the wide range of systems, including the protocols operating on unranked trees. keywords: hedge rewriting, safety verification, first-order logic 1 introduction hedges, or arbitrary width trees over unranked alphabets, also known as unranked trees or forests provide with the abstractions useful in several verification contexts. hedges and hedge transformations (rewritings) have been used for specification and verification of at least the following: • protocols working in tree-shaped networks of unbounded degree [27]; • xml transformations [30, 23, 27]; • multithreaded recursive programs [8, 29]. the most challenging task of infinite state or parameterized verification appears when one would like to establish the correctness of a system in a widest possible context either for all possible sizes of the system or for unbounded computations. in general such a problem is undecidable and the only way to address it is to focus on restricted classes of systems and properties. in this paper we consider the problem of safety verification for hedge rewriting systems and explore the applicability of the very general method based on disproving the first-order formulae by finding a countermodel. the basic intuition behind the method is that if the evolution of the system of interest can be faithfully modelled by the derivations within first-order logic then the safety (non-reachability of bad states) can be naturally reduced to disproving the formulae (non-provability). such an approach to the safety verification has been proposed in the early work on the verification of cryptographic protocols [28, 25, 12] and later has been extended to various classes of parameterized and infinite state verification tasks [16, 18, 19, 20, 21]. two attractive properties of the approach have emerged. on the one side it has turned out to be practically efficient in many applications. on the other side it has proved to be relatively complete with respect to the methods using regular invariants, in particular regular model checking, regular tree model checking and tree completion. 1 / 14 volume 72 (2015) first-order logic for safety verification safety verification for hedge rewriting systems has been already addressed e.g. in [14, 11, 27] with the regular invariants playing the major role. we show in this paper the finite countermodel method (fcm) provides a viable alternative to the methods using regular invariants for the verification of hedge rewriting systems. we show that theoretically the proposed approach is at least as powerful as the methods using regular invariants. at the same time the finite countermodel method is shown to be very flexible and applicable to the wide range of systems. the practical efficiency of the method is illustrated on a set of examples on verification of (1) parameterized protocols operating on arbitrary width tree topology – for most of them an automated verification is reported for the first time; and (2) dynamic multithreaded recursive programs. the paper is organized as follows. the next section provides with necessary preliminaries. in section 3 we formulate basic verifcation problem and its translation into a logical problem of first-order formulae disproving. in section 4 the verification method is discussed and its relative completeness is demonstrated. section 5 presents the experimental results and section 6 concludes the paper. 2 preliminaries 2.1 first-order logic a first-order vocabulary is defined as a finite set σ = f ∪p where f and p are the sets of functional and predicate symbols, respectively. each symbol in σ has an associated arity, and we have f = ∪i≥0fi and p = ∪i≥1pi, where fi and pi consist of symbols of arity i. the elements of f0 are also called constants. a first-order model over vocabulary σ, or just a model is a pair m = 〈d,[[σ]]d〉 where d is a non-empty set called domain of m and [[σ]]d denotes the interpretations of all symbols from σ in d. for a domain d and a function symbol f of arity n ≥ 1 an interpretation of f in d is a function [[ f ]]d : dn → d. for a constant c its interpretation [[c]]d is an element of d. for a domain d and a predicate symbol p of arity n an interpretation of p in d is a relation of arity n on d, that is [[p]]d ⊆ dn. the model m = 〈d,[[σ]]d〉 is called finite if d is a finite set. we assume that the reader is familiar with the standard definitions of first-order formula, firstorder sentence, satisfaction m |= ϕ of a formula ϕ in a model m , deducibility (derivability) φ ` ϕ of a formula ϕ from a set of formulae φ, first-order theory and equational theory. 2.2 hedge rewriting we borrow the definitions from [14] with minor modifications. let σ be a finite alphabet and x be a countable set of variables. then the set of terms t (σ,x ) and the set of hedges h (σ,x ), both over σ and x , are defined inductively as the least sets satisfying • t (σ,x ) = x ∪{ f (h) | f ∈ σ,h ∈ h (σ,x )} • h (σ,x ) = ⋃ n>0{t1,...,tn | t j ∈ t (σ,x ), j = 1,...,n}∪{λ} thus, a hedge is a finite (possibly empty) sequence of terms, whereas a term is obtained by applying an unranked functional symbol to a hedge. we denote by λ the empty sequence of proc. avocs 2015 2 / 14 eceasst terms. we do not make a difference between a term and a hedge of length one, i.e. consider that t (σ,x )⊂h (σ,x ). we will also do not distinguish the hedges of the form fi1(λ),..., fik (λ) and words fi1,..., fik ∈ σ ∗. the sets of ground terms and ground hedges ( i.e., terms and hedges without variables) are denoted t (σ) and h (σ). a variable x ∈ x is called linear in a hedge h ∈ h (σ,x ) if it has exactly one occurrence in h. the set of variables occurring in a term t ∈ t (σ,x ) is denoted var(t). a substitution σ is a mapping from x to h (σ,x ). the application of a substitution σ to a hedge h, denoted hσ , is defined inductively as follows. for t1,...,tn ∈ t (σ,x ) we have (t1,...,tn)σ = t1σ ...tnσ and f (h)σ = f (hσ). a context is a hedge h ∈ h (σ,x ) with a distinguished variable x linear in h. we write c[x] to denote a context with a distinguished variable x. the application of a context c[x] to a hedge h is defined by c[h] = c{x 7→ h}. a hedge rewriting system (hrs) is a set of rewriting rules of the form l → r, where l,r ∈ h (σ,x ). the rewrite relation →r (resp. outer rewrite relation r ) of an hrs r is defined as follows: h →r h′ iff there is a context c[x] (resp. a trivial context c[x] = x), a rule l → r ∈ r and a substitution σ such that h = c[lσ] and h′ = c[rσ]. the reflexive and transitive closure of →r (resp. r ) is denoted by →∗r (resp. ∗ r ). given a set of ground hedges l ⊆ h (σ) and an hrs r we denote by post∗r(l) the set of all hedges reachable from l, that is {h ∈ h (σ) | ∃g ∈ l,g →∗r h}. similarly post ∗ r, (l) denotes {h ∈ h (σ) | ∃g ∈ l,g ∗r h}. 2.3 forest automata and regular hedge languages the following definition is taken from [5]. definition 1 a forest automaton over an unranked alphabet σ is a tuple a = ((q,e,∗),σ,δ : (σ×q → q),f ⊆ q) where (q,e,∗) is a finite monoid, δ is a transition function and f is a set of accepting states. for every ground hedge h the automaton assigns a value ha ∈q which is defined by induction: • λa = e; • f (h)a = δ ( f ,ha ) • (t1,...tn)a = ta1 ∗ ...∗t a n a hedge h is accepted by the forest automaton a iff ha ∈ f . the hedge language la of the forest automaton a = ((q,e,∗),σ,δ : (σ×q → q),f ⊆ q) is defined as la ={h | ha ∈ f}. a hedge language l is called regular iff it is a hedge language la of some forest automaton a . alternative but equivalent definitions of regular hedge languages using hedge automata and unranked tree automata have been considered e.g. in [14, 27, 26, 15]. 3 / 14 volume 72 (2015) first-order logic for safety verification 2.4 finitely based sets of hedges for a finite set of hedges b ⊆ h (σ,x ) the set of ground instances of all hedges from b is denoted by gi(b), that is gi(b) ={h | ∃h′ ∈ b∧h = h′θ ; h′θ is ground}. example 1 for σ ={a,b, f} and b ={ f (x,b,b,y)} the language gi(b) is the set of all ground hedges with the outer symbol f and containing two consecutive symbols b at leaves. notice that finitely based sets of hedges are not necessarily regular; on the other hand, it is easy to see that if every hedge in a set b is linear then gi(b) is regular. 3 safety verification: from hedge rewriting to fo logic in this section we define the basic verification problem and its translation into a purely logical problem of disproving the first-order formula. 3.1 basic verification problem the general form of safety verification problems we address in this paper is as follows. given: an unranked vocabulary σ, a hedge rewriting system r over σ, a language i ⊆ h (σ) of initial ground hedges, a language u ⊆ h (σ) of unsafe ground hedges. question: is it true that ∀h ∈ i∀h′ ∈u h 6→∗r h ′? we will also consider a variant of the basic verification problem for outer rewriting, where →∗ above is replaced by ∗. notice that in the definition of the basic verification problem the sets i and u , in general, may be infinite. in that case we assume that the sets are defined by some finitary and constructive means, for example as regular languages given by forest automata, or as finitely based sets of hedges. 3.2 from hedge rewriting to first-order logic for an unranked alphabet σ we denote by σ f o a ranked vocabulary σr ∪{e,∗}∪{r(2)} where σ r = { f̃ | f ∈ σ}, with all f̃ being unary functional symbols, e is a constant (0-ary functional) symbol, ∗ is a binary functional symbol, which we will use in infix notation and r is a binary predicate symbol. the constant e will denote the empty hedge, i.e. the hedge of length 0, ∗ will denote the concatenation of hedges and the semantics of r is going to capture reachability for hedge rewriting. then we define the translation τ : h (σ,x ) → t (σ f o,x ) from hedges to terms over the extended alphabet inductively as follows: • τ(x) = x for x ∈ x proc. avocs 2015 4 / 14 eceasst • τ(λ) = e • τ(t1,...,tn) = πni=1τ(ti) • τ( f (h)) = f̃ (τ(h)) here πni=1τ(ti) denotes the ∗-product of τ(ti) defined as • π1i=1τ(ti) = τ(t1) • πk+1i=1 τ(ti) = (π k i=1τ(ti))∗τ(tk+1) notice that we will specify associativity of ∗, so exact association of brackets in the product will be immaterial. for a hedge rewriting system r over alphabet σ and a set of variables x we define its translation φr as the set of the following universally closed first-order formulae: 1. (x∗y)∗z = x∗(y∗z) 2. e∗x = x 3. x∗e = x 4. r(τ(h1),τ(h2)) for all (h1 → h2)∈ r 5. r(x,y)→ r((z∗x)∗v,(z∗y)∗v) 6. r(x,y)→ r( f̃ (x), f̃ (y)) for all unary f̃ ∈ σ f o 7. r(x,x) 8. r(x,y)∧r(y,z)→ r(x,z). proposition 1 (adequacy of the first-order translation) h1 →∗r h2 ⇔ φr ` r(τ(h1),τ(h2)) for all ground h1 and h2. proof. ⇒ due to transitivity of r specified in φr it is sufficient to show that if h1 →r h2 then φr ` r(τ(h1),τ(h2)). assume h1 →r h2 then h1 = c[lσ] and h2 = c[rσ] for some context c[x], some substitution σ and a rewriting rule (l → r) ∈ r. now the argument proceeds by a simple induction on the context construction. indeed, for the base case of the simplest context c[x] ≡ x we have h1 ≡ lσ and h2 ≡ rσ and r(τ(h1),τ(h2)) is a ground instance of the formula r(τ(l),τ(r)) ∈ φr (item 4 in the translation). it follows φr ` r(τ(h1),τ(h2)). there are two step cases. assume c[x] ≡ g1c′[x]g2 where g1 and g2 are ground hedges and c′[x] is a context. then we have h1 ≡ g1c′[lσ]g2, h2 ≡ g1c′[rσ]g2 and c′[lσ]→r c′[rσ]. by induction hypothesis φr ` r(τ(c′[lσ]),τ(c′[rσ])). that together with the congruence axiom (5) being in φr implies φr ` r(τ(g1)∗τ(c′[lσ])∗τ(g2),τ(g1)∗τ(c′[rσ])∗τ(g2)), and therefore, by definition of τ , φr ` r(τ(h1),τ(h2)). the second step case with c[x] ≡ f̃ (c′[x]) is dealt with in a similar way using the congruence axiom (6). 5 / 14 volume 72 (2015) first-order logic for safety verification ⇐ consider the following first-order model m in the vocabulary σ f o. the domain of the model is a set h (σ) of all ground hedges over σ. the interpretations of functional symbols and constants are defined inductively as • [[e]] = λ; • [[x∗y]] = [[x]][[y]]; • [[ f (x)]] = f ([[x]]) for all f ∈ σ. the interpretation of r is given by [[r]] = {(h,h′) | h,h′ ∈ h (σ)∧h →∗r h ′}. by simple structural induction we have [[τ(h)]] = h. that concludes the construction of m. a straightforward check shows now that for such defined m we have m |= φr . assume now φr ` r(τ(h1),τ(h2)). it follows that m |= r(τ(h1),τ(h2)), that is ([[τ(h1)]],[[τ(h2)]])∈ [[r]], and therefore (h1,h2)∈ [[r]], which in turn means h →∗r h ′ by definition of [[r]]. � taking a contraposition of proposition 1 we get φr 6` r(τ(h1),τ(h2))⇒ h1 6→∗r h2 which expresses the essence of the proposed verification method: in order to prove non-reachability(≈ safety) it is sufficient to disprove a first-order formula. in order to apply this observation to the solution of basic verification problems we need to provide also the first-order representations for the sets of initial and unsafe terms. we start with very general definition 2 given an unranked alphabet σ and a set of ground hedges h ⊆h (σ) a first-order formula ϕh(x) with one free variable and in vocabulary σ f o, possibly extended by relational and fucntional symbols, is called first-order representation of h if h ∈ h implies ` ϕh(τ(h)). 3.2.1 first-order representations of regular hedge languages let la be a regular hedge language given by a forest automaton a = ((q,e,∗),a,δ : (a×q → q),f ⊆ q). the vocabulary of the first-order representation of a consists of unary predicates pq for each q ∈ q and unary functional symbols ã for each a ∈ a. define φa as the set of the following universally closed formulae: 1. (x∗y)∗z = x∗(y∗z); 2. (x∗e) = x; 3. (e∗x) = x; 4. pδ (a,e)(ã(e)) for every a ∈ a; 5. pq(x)→ pδ (a,q)(ã(x)) for every a ∈ a and q ∈ q. 6. pq1(x)∧pq2(y)→ pq3(x∗y) for all q1,q2,q3 ∈ q such that q1 ∗q2 = q3 in q. now for the forest automaton a we denote by φla the formula φa → ∨ q∈f pq(x) proc. avocs 2015 6 / 14 eceasst proposition 2 φla is a first-order representation of la proof (sketch) straightforward induction on the hedge construction shows that φa `pha (τ(h)). the proposition statement follows immediately. � 3.2.2 finitely based sets of hedges for a finite set of hedges b ⊆ h (σ,x ) the formula ∨ h∈b(∃ȳ(x = τ(h))) is a first-order representation of gi(b). here ȳ denotes all variables in τ(h) and x is different from all variables in ȳ. 3.3 outer rewriting and unary reachability encoding in many specification and verification scenarii outer rewriting is sufficient to model all essential aspects of the evolution of the system of interest. in that case the first-order translations can be simplified as there is no need in the congruence axioms and the unary reachability predicate does suffice. let r be a hedge rewriting system over alphabet σ. we define its unary translation φur as the set of the following universally closed first-order formulae: 1. (x∗y)∗z = x∗(y∗z); 2. (x∗e) = x; 3. (e∗x) = x; 4. r(τ(h1))→ r(τ(h2)) for all (h1 → h2)∈ r . such a simplified translation captures reachability by the outer rewrite relation r as the following proposition states. proposition 3 (adequacy of the unary first-order translation) h1 ∗r h2 ⇔ φ u r ∧r(τ(h1)) ` r(τ(h2)) for all ground h1 and h2. proof (hint) the proof follows the arguments in the proof of proposition 1 appropriately modified. to prove ⇒ entailment an easy induction on the length of rewriting is used. to prove ⇐ a semantical argument taking as the interpretation of unary predicate r the set of the term encodings of all reachable from h1 hedges, is applied. � 4 back to safety verification now we have defined all necessary concepts and the basic verification problem defined in sect. 3.1 gets its full specification. 7 / 14 volume 72 (2015) first-order logic for safety verification given: an unranked vocabulary σ, a hedge rewriting system r over σ, a first-order representation ϕi of the language i ⊆ h (σ) of initial ground hedges, a first-order representation ϕu of the language u ⊆ h (σ) of unsafe ground hedges. question: is it true that ∀h ∈ i∀h′ ∈u h 6→∗r h ′? proposition 4 if the verification problem above has a negative answer, that is ∃t1 ∈ i∃t2 ∈ ut1 →∗r t2 then φr `∃x ∃y (ϕi(x)∧ϕu (y)∧r(x,y)). proof. the proposition statement follows immediately from proposition 1 and definition 2. � 4.1 verification method taking contraposition of proposition 4 we have φr 6` ∃x ∃y (ϕi(x)∧ϕu (y)∧r(x,y)) implies ∀h ∈ i∀h′∈u h 6→∗r h ′, that is a positive answer to an instance of the basic verification problem. thus, the essence of the finite countermodels (fcm) verification method we advocate in this paper is: to demonstrate a positive answer to an instance of a basic verification problem apply an automated generic finite model procedure to find a countermodel to φr → ∃x∃yϕi(x)∧ϕu (y)∧r(x,y), or equivalently a model for φr∧¬∃x∃yϕi(x)∧ϕu (y)∧ r(x,y). the next theorem, which is a generalization of similar theorems in [17, 21] (the case of word languages) and [19] (the case of tree languages), shows the applicability of fcm method and its relative completeness with respect to the methods based on regular invariants [14, 27]. theorem 1 let r be a hedge rewriting system over unranked alphabet σ, ϕi and ϕu be first-order formulae representing the regular sets i and u of initial and unsafe hedges, respectively. let inv be a regular set of hedges such that post∗r(i) ⊆ inv and inv ∩u = /0, that is a regular invariant separating i and u . then there is a finite model m such that m |= φr ∧¬∃x,y(r(x,y)∧ϕi(x)∧ϕu (y)). proof. (sketch) assume the condition of the theorem holds. let ai = ((qi,ei,∗i),a,δi,fi ⊆ qi), ainv = ((qinv,ei,∗inv),a,δinv,finv ⊆ qinv) and au = ((qu ,ei,∗u ),a,δu ,fu ⊆ qu ) be forest automata recognizing regular hedge languages i,inv and u , respectively. then the required finite model m is constructed as follows. the domain d of the model is qi×qinv×qu . the interprettaion of e is [[e]] = (ei,einv,eu ). the interpretation of ∗ is given by (q1,q2,q3)[[∗]](q′1,q ′ 2,q ′ 3) = (q1 ∗i q′1,q2 ∗inv q ′ 2,q3 ∗u q ′ 3). for all a ∈ a the interpretations are given by [[a]](q1,q2,q3) = (δi(a,q1),δinv(a,q2),δu (a,q3)). once we defined the interpretations of all functional symbols (including constants) any ground term t gets its interpretation [[t]] ∈ d in a standard way. define the interpretation of r as [[r]] = {([[τ(h)]],[[τ(h′)]]|h →∗r h ′}. for all predicates pq with q ∈ qi used in ϕi the interpretations are defined as [[pq]] ={(q,x,y)|x ∈ qinv,y ∈ qu}. similarly, [[pq]] = {(x,q,y)|x ∈ qi,y ∈ qu} for q ∈ qinv and [[pq]] = {(x,y,q)|x ∈ qi,y ∈ qinv}. now it is straightforward exercise to show that the such defined model is indeed as required. �. proc. avocs 2015 8 / 14 eceasst 5 experiments in this section we present the experimental results of applications of the fcm method for safety verification of hedge rewriting systems. in the experiments we used the finite model finder mace4 [24] within the package prover9-mace4, version 05, december 20071. 5.1 tree arbiter protocol and other protocols on unranked trees we consider here as a case study the verification of parameterized tree-arbiter protocol working on the trees of unbounded branching degree. we take the description of the protocol from [4] where its automated verification has been demonstrated for the case of binary trees. the protocol supervises the access to a shared resource of a set of processes arranged in a tree topology. the processes competing for the resource are located in the leaves of the trees. the safety property to be verified is that of mutual exclusion at no point, two or more processes at leaves of the tree can get an access to the resource (obtain a token). a process in the protocol can be in state idle (i), requesting (r), token (t) or below (b). all the processes are initially in state i. a node is in state b whenever there is a node below (descendant) in state t. when a leaf is in state r, the request is propagated upwards until it encounters a node in state t of b (aware of the presence of the token). a node state t can choose to pass it upwards or pass it downwards to a requesting node in state r. we model the tree arbiter protocol as a hedge rewriting system rta consisting of the rules: 1. i(x r(y) z)→ r(xr(y)z) (request propagated upwards) 2. t(x r(y) z)→ b(xt(y)z) (token passed downwards) 3. b(x t(y) z)→ t(xi(y)z) (token passed upwards) 4. i(λ)→ r(λ) (idle leaf node becomes requesting) the set of initial configurations is a hedge language consisting of all hedges in which all leaves are either r(λ) or i(λ), all intermediate nodes (neither leaves, nor the top) are i(...) and the top node is t(...). the hedge language of unsafe configurations consists of all hedges with two or more leaves with t(λ) (tokens). now we represent the verification problem in the format required by fcm method. the automata ai and au reconginizing the sets of initial, respectively unsafe states and the first-order translation φ of the hedge rewriting system and the automata can be found in the appendix a of the extended version [22] of this paper. after translation of the protocol specification and forest automata recognizing the initial and unsafe configurations into a first order formula φ and (un)safety condition into a formula ψ the safety is established automatically by mace4 finding a countermodel for φ → ψ of size 3 in 0.03s. 1 the system configuration used in experiments: intel(r) core (tm)2 duo cpu, t7100 1.8ghz 1.79 ghz, 1.00 gb of ram. 9 / 14 volume 72 (2015) first-order logic for safety verification we have applied fcm to the verification of other protocols operating on the unranked trees. the specification of the protocols are taken from [4, 6] where their automated verification has been demonstrated for the binary trees case. in [19] we have considered the application of fcm for the verification of binary tree case protocols. the table below lists the parameterized tree protocols and shows the time it took mace4 to find a countermodel and verify a safety property for the case of unranked trees (first column). for comparison the times it took to verify the binary tree versions of the same protocols by fcm [19] and by regular tree model checking [6] are given in the second and third columns, respectively. protocol time time from [19] time from[6]∗ token 0.04 0.02s 0.06s two-way token 0.04 0.03s 0.09s percolate 0.06 0.09s 2.4s tree arbiter 0.03 0.03s 0.31s ∗ the system configuration used in [6] was intel centrino 1.6ghz with 768mb of ram further details of the experiments can be found in [16]. as far as we are aware the automated verification of the above parameterized protocols over trees of unbounded degree is reported here for the first time, with an exception being two-way token protocol verified in [27]. 5.2 verification of synchronised pad systems the verification techniques based on disproving in first-order logic are very flexible and can accommodate various constraints on the properties of the systems of interest. to illustrate this point we consider an automated verification of a program which involves dynamic creation of processes. we take as an example a program from [29] specified in terms of synchronised pad systems [29]. such systems can be seen as an extension of hedge rewriting systems to the case of two different associative constructors, one of which being commutative. furthermore the rewriting is constrained to capture the effects of modelled synchronisation. let sync = {a,b,c,...} be a set of actions such that every action a ∈ sync corresponds to a co-action ā ∈ sync s.t. ¯̄a = a. let act = sync∪{τ} be the set of all the actions, where τ is a special action. let var = {x,y,...} be a set of process variables and t be the set of process terms t over var defined by: t ::= 0 | x | t ·t | t||t definition 3 a synchronised pad (spad) is a finite set of rules of the form x ↪→a t or x ·y ↪→a t, where x,y ∈var,t ∈ t and a ∈ act the process terms are considered modulo the following equational theory se of structural equivalence: a1: t ·0 = 0·t = t||0 = 0||t = t proc. avocs 2015 10 / 14 eceasst figure 1: an example from [29] a2: (t ·t′)·t′′ = t ·(t′ ·t′′) a3: t||t′ = t′||t a4: (t||t′)||t′′ = t||(t′||t′′) a spad r induces a transition relation �a over t by the following inference rules: θ1 : t1↪→at2∈r t1�at2 ; θ2 : t1�at′1 t1·t2�at′1·t2 ; θ3 : t1=0,t2�at′2 t1·t2�at1·t′2 θ4 : t1�at′1 t1||t2�at′1||t2;t2||t1�at2||t ′ 1 θ5 : t1�at2;t2�āt′2;a,ā∈sync t1||t2�τ t′1||t ′ 2 the equational theory se induces a transition relation �ase over t defined by ∀t,t ′∈t ,t �ase t′ iff ∃u,u′ s.t. t =se u,u �a u′ and u′ =se t′. for w ∈ act∗ the transition relations �w and →wse are defined in a standard way. 5.2.1 example consider the program represented in fig 1 which involves dynamic creation of processes. the figure represents the flow graph of a program having two procedures π1 and π2 such that: • π1 calls itself in parallel with another procedure π2. • π2 calls itself recursively, • π1 and π2 communicate via the synchronizing actions a,b, and their corresponding coactions ā and b̄, and the program starts at point n0 the safety property to be verified is “starting from n0, the program never reaches a configuration where the control point m1 is active.” the program is modelled by the syncronized pad r [29] which includes the following rules: 11 / 14 volume 72 (2015) first-order logic for safety verification r1 : n0 ↪→a n1 r2 : n1 ↪→b̄ n2 r3 : n0 ↪→τ (n0 ‖ m0)·n2 r4 : m0 ↪→b m1 r5 : m1 ↪→ā m2 r6 : m0 ↪→τ m0 ·m4 r7 : m3 ·m4 ↪→τ m2 r8 : m2 ·m4 ↪→τ m3 the above verification task is formulated in terms of r as follows. show that there are no w ∈ τ∗ and t ∈ t such that m0 is a subterm of t and n0 �wse t. notice that the transition relation �wse with w ∈ τ ∗ encodes the reachability by a rewriting process, where the rules with ↪→τ can be applied without any restrictions, whereas the rules with ↪→a with a ∈ sync can be applied only if a some rule ↪→ā with a co-action superscript is applied simultaneously ”in parallel” subterm (cf. the rule θ5). we claim now that one can specify the first-order formulae: • φr describing the reachability by �wse , the initial configuration and the set of unsafe configurations, and • ψ specifying unsafety condition such that if φr ` ψ then the program is unsafe. see the details in appendix b of the extended version [22] of this paper. we apply mace4 and a countermodel of the size 3 for φr → ψ is found in 2s. 6 conclusion we have shown in this paper that the simple encoding of hedge rewriting systems by first-order logic and using available finite model finders provides with an interesting and viable alternative to the existing methods for the verification of systems working on the trees of unbounded degree. on the one side it is relatively complete w.r.t. the methods using regular invariants. on the other side it is either practically efficient, as our automated verification of parameterized protocols working on the trees of unbounded degree illustrates well, or at least looks promising to explore the boundaries of its practical applicability, as our spad example shows. the advantages of the fcm method are (1) it is simple, (2) it is flexible, (3) it is modular and (4) it reuses existing tools. the fcm method has a natural limitation that it can be used only to establish the safety. if safety actually does not hold looking for the countermodels does not help. one can use then automated theorem provers to confirm that indeed the safety is violated by searching for the proofs of formulas of the form φ → ψ. one direction for the future work here is the development of the systematic procedures which would extract the unsafe traces from first-order proofs. bibliography [1] parosh aziz abdulla, jonsson b. verifying programs with unreliable channels. information and computation, 127(2):91-101, june 15, 1996. proc. avocs 2015 12 / 14 eceasst [2] abdulla, parosh aziz and jonsson, bengt and mahata, pritha and d’orso, julien, regular tree model checking, in proceedings of the 14th international conference on computer aided verification, cav ’02, 2002, 555–568 [3] abdulla, p.a., jonsson,b., nilsson, m., & saksena, m., (2004) a survey of regular model checking, in proc. of concur’04, volume 3170 of lncs, pp 35–58, 2004. [4] parosh aziz abdulla, noomene ben henda, giorgio delzanno, frdric haziza and ahmed rezine, parameterized tree systems, in formal techniques for networked and distributed systems, forte 2008, lecture notes in computer science, 2008, volume 5048/2008, 69-83. [5] mikolaj bojanczyk, igor walukiewicz, forest algebras, in logic and automata, history and perspectives (j.flum, e.gradel, t. wilke eds.), amsterdam university press, 2007, 107–132. [6] a. bouajjani, p. habermehl,a. rogalewicz, t. vojnar, abstract regular tree model checking, electronic notes in theoretical computer science, 149, (2006), 37–48. [7] parosh aziz abdulla, giorgio delzanno, noomene ben henda, ahmed rezine. monotonic abstraction: on efficient verification of parameterized systems. int. j. found. comput. sci. 20(5): 779-801 (2009) [8] a. bouajjani and t. touili. on computing reachability sets of process rewrite systems, in proc. 16th int. confenerence on rewriting techniques and applications (rta’05), lncs 3467, 2005 [9] r. caferra, a. leitsch, n. peltier, automated model building, applied logic series, 31, kluwer, 2004. [10] g. delzanno. constraint-based verification of parametrized cache coherence protocols. formal methods in system design, 23(3):257–301, 2003. [11] julien d’orso and tayssir touili. regular hedge model checking, proc. of the 4th ifip international conference on theoretical computer science (tcs’06), 2006. [12] goubault-larrecq, j., (2010), finite models for formal security proofs, journal of computer security, 6: 1247–1299, 2010. [13] guttman, j., (2009) security theorems via model theory, proceedings 16th international workshop on expressiveness in concurrency, express, eptcs, vol. 8 (2009) [14] f. jacquemard and m. rusinowitch, closure of hedge-automata languages by hedge rewriting, in a. voronkov (ed.): rta 2008, lncs 5117, pp 157–171, 2008. [15] l. libkin, logics for unrankeed trees: an overview. logical methods in computer science,2:131,2006. [16] a. lisitsa verification via countermodel finding http://www.csc.liv.ac.uk/˜alexei/countermodel/ [17] lisitsa, a., (2010c), finite model finding for parameterized verification, corr abs/1011.0447: (2010) [18] lisitsa, a., (2010b), reachability as deducibility, finite countermodels and verification. in proceedings of atva 2010, lncs 6252, 233–244 13 / 14 volume 72 (2015) first-order logic for safety verification [19] lisitsa, a., (2011), finite countermodels for safety verification of parameterized tree systems, corr abs/1107.5142 (2011) [20] lisitsa, a, (2012) finite models vs tree automata in safety verification, in proc. rta’12, pp 225–239 [21] lisitsa, a., (2013) finite reasons for safety. parameterized verification by finite model finding, journal of automated reasoning, 51(4): 431-451 (2013) [22] lisitsa, a., (2015) first-order logic for safety verification of hedge rewriting systems, extended version of this paper, available at www.csc.liv.ac.uk/˜alexei/countermodel/avocs15.pdf [23] s. maneth, a. berlea, t. perst, and h. seidl, xml type checking with macro tree transducers, pods, 2005 [24] w. mccune prover9 and mace4 http://www.cs.unm.edu/˜mccune/mace4/ [25] selinger, p., (2001), models for an adversary-centric protocol logic. electr. notes theor. comput. sci. 55(1) (2001) [26] j.w. tatcher, characterizing derivation trees of context-free grammars through a generalization of finite automata theory. j. comput. syst. sci., 1:317-322, 1967. [27] tayssir touili, computing transitive closures of hedge transformations, ijccbs, volume 3, 1/2, 2012, 132–150 [28] weidenbach, c., (1999), towards an automatic analysis of security protocols in first-order logic, in h. ganzinger (ed.): cade-16, lnai 1632, pp. 314–328, 1999. [29] tayssir touili. dealing with communication for dynamic multithreaded recursive programs. invited paper in proc. of vissas’05. [30] silvano dal zilio and denis lugiez, xml schema, tree logic and sheaves automata, rta’03, 2003. proc. avocs 2015 14 / 14 introduction preliminaries first-order logic hedge rewriting forest automata and regular hedge languages finitely based sets of hedges safety verification: from hedge rewriting to fo logic basic verification problem from hedge rewriting to first-order logic first-order representations of regular hedge languages finitely based sets of hedges outer rewriting and unary reachability encoding back to safety verification verification method experiments tree arbiter protocol and other protocols on unranked trees verification of synchronised pad systems example conclusion security and privacy challenges in the internet of things electronic communications of the easst volume 17 (2009) workshops der wissenschaftlichen konferenz kommunikation in verteilten systemen 2009 (wowkivs 2009) security and privacy challenges in the internet of things christoph p. mayer 12 pages guest editors: m. wagner, d. hogrefe, k. geihs, k. david managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst security and privacy challenges in the internet of things christoph p. mayer http://www.tm.uka.de/itm institute of telematics universität karlsruhe (th), germany abstract: the future internet of things as an intelligent collaboration of miniaturized sensors poses new challenges to security and end-user privacy. the itu has identified that the protection of data and privacy of users is one of the key challenges in the internet of things [int05]: lack of confidence about privacy will result in decreased adoption among users and therefore is one of the driving factors in the success of the internet of things. this paper gives an overview, categorization, and analysis of security and privacy challenges in the internet of things. keywords: global sensor networks, security, privacy, future internet 1 introduction the internet has undergone severe changes since its first launch in the late 1960s as an outcome of the arpanet. the initial four-node network has quickly grown into a highly interconnected and self-organized network that builds the daily basis for business, research, and economy. the number of people using this worldwide network has exponentially grown up to about 1.5 bn and hereby makes up about 20% of the world population. this sheer number of end users – that does not even comprise servers and routers inside the networks – has changed our daily life and habits. with the miniaturization of devices, increase of computational power, and reduction of energy consumption, this trend will continue – the internet of things. one of the most challenging topics in such an interconnected world of miniaturized systems and sensors are security and privacy aspects: without sureness that safety of private information is assured and adequate security is provided, users will be unwilling to adopt this new technology that invisibly integrates into their environment and life. besides technical solutions to provide privacy and security, further instruments – like governmental and ethical institutions, that we will not cover here – need to get established and applied. having every ‘thing’ connected to the global future internet and ‘things’ communicating with each other, new security and privacy problems arise, e. g., confidentiality, authenticity, and integrity of data sensed and exchanged by ‘things’. privacy of humans and things must be ensured to prevent unauthorized identification and tracking. further, the more autonomous and intelligent things get, problems like the identity and privacy of things, and responsibility of things in their acting will arise. up to now, corrupted digital systems were mostly not able to act in the physical world. this will change dramatically in a dangerous way that corrupted digital systems can now operate in and influence the physical world. what happens, once a corrupted thing killed a person? 1 / 12 volume 17 (2009) http://www.tm.uka.de/itm security and privacy challenges in the internet of things the sequel of this paper is structured as follows: section 2 performs an analysis of the components in the internet of things, their sensitivity to security and privacy, as well as an analysis of the state in research for topics considered as highly sensitive. in section 3 two major components in the internet of things – global sensor networks and rfid – are introduced and detailed on related security and privacy work. three research results from other fields that we believe are worth investigating for the internet of things are introduced in section 4. finally, concluding remarks are given in section 5. 2 analysis of security and privacy as the internet of things is a large field with diverse technologies used, we provide a categorization of topics and technologies in section 2.1. the categorization serves as base to detail on the security and privacy sensitivity in the respective fields. section 2.2 then looks into the state of research in the identified categories and details on topics that have insufficient research from our point of view. 2.1 categorization and sensitivity figure 1 shows a categorization of topics – inner items – and respective technologies used in each topic – outer items – that make up the internet of things. in our opinion the internet of things can be categorized into eight topics: • communication to enable information exchange between devices • sensors for capturing and representing the physical world in the digital world • actuators to perform actions in the physical world triggered in the digital world • storage for data collection from sensors, identification and tracking systems • devices for interaction with humans in the physical world • processing to provide data mining and services • localization and tracking for physical world location determination and tracking • identification to provide unique physical object identification in the digital world each topics has different technologies attached (outer items) that are used in the respective topic. note, that the categorization given in this work is not strictly hierarchical in terms of topics and technologies. identification, e. g., is actually a form of processing that results from the use of sensors. as we believe that identification has a special role in the internet of things that is independent of physical world sensing, it is handled as a separate topic. some technologies appear multiple times: rfid, e. g., is used as communication technology, provides identification, localization and tracking, rfid readers act as sensors, and finally rfid tags and readers make up devices in the internet of things. the manifold usage of rfid assigns it a special role that is detailed in section 3.2. the topics introduced are listed again in table 1 and rated with respect to properties of security and privacy. the properties are taken from the cia triad (without non-repudiation) and the parkerian hexad (without possession or control and utility). the additional property regulation represents the need for laws and regulations in this topic. for each topic the table contains the sensitivity for the respective property. as our categorization is not strictly hierarchical, sensitivity proc. wowkivs 2009 2 / 12 eceasst internet of things identification communication sensors storage devices localization and tracking actuators processing rfid gsm gps sensors services sensor networks global sensor networks in-network processing rfid barcodes 2d tags biometry video rfid tags rfid reader mobile phones sensors laptops things actuators rfid cellular wireless wired overlays infrared databases dht video audio positioning acceleration temperature proximity rfid reader figure 1: categorization of topics and technologies in the internet of things xxxxxxxxxtopic property integrity authenticity confidentiality privacy availability regulation communication +++ +++ +++ ++ +++ + sensors +++ ++ + +++ + +++ actuators + + + + ++ storage +++ ++ +++ +++ + +++ devices +++ + + ++ ++ ++ processing ++ + + +++ + +++ localization/tracking + + +++ +++ +++ +++ identification ++ + +++ +++ +++ +++ table 1: sensitivity of topics in the internet of things to different security and privacy properties, and the need for laws and regulations (+ low sensitivity, ++ middle sensitivity, +++ high sensitivity) 3 / 12 volume 17 (2009) security and privacy challenges in the internet of things sometimes is based on sensitivity of other properties. we will now describe the decisions for the chosen sensitivity values. communication research in communication protocols has come up with solutions that provide integrity, authenticity, and confidentiality – examples are tls, or ipsec. privacy needs have been tackled by different routing schemes like onion routing, or freenet but unfortunately are not in wide use. an open issue – despite strong research – is disturbed availability through ddos attacks. in regulations we currently see a low need. sensors the integrity and authenticity of sensor data is a current research target that can be handled, e. g., in the form of watermarking [jkk08]. confidentiality of sensor data is a weak requirement, as an attacker can just place its own sensor physically near and sense the same values. therefore, the need for sensor confidentiality at the sensor itself is low and confidentiality therefore relies on communication confidentiality. privacy in sensors mainly targets the physical world that is getting sensed. mechanisms like face blurring in video data need to be employed to preserve the privacy of humans and objects in the physical world. the availability of sensors mostly depends on the communication infrastructure. regulations are necessary to preserve the privacy of people who are currently most often unaware of sensors – like video cameras – in their environment. actuators the integrity, authenticity, and confidentiality of data send to an actuator mostly depends on the communication security, therefore low sensitivity of the actuator itself is necessary. what must be assured, is that an attacker can not control the actuator (we will come back to this property when looking at devices). privacy in actuators is highly specific to the scenario, therefore we don’t give a general rating on sensitivity. whether availability of an actuator is critical highly depends on the kind of actuator, but can generally be rated as sensitive. regulations are similar to sensors and must assure that the use of actuators does not disturb privacy. storage security mechanisms for storage devices are well-established, but employment is still weak. as storage of data is highly privacy-sensitive and reports on data breaches are common, regulations must be extended to provide adequate protection of user privacy. availability of storage mostly depends on availability of communication infrastructure and well-established mechanisms for storage redundancy. devices in the scope of devices integrity means that a device is free of malware. this property has also been called ‘admissibility’ [sch06]. ensuring admissibility is an open issue currently researched in trusted platform computing (tpm) and highly sensitive. the authenticity of a device is mostly handled in the communication part and there seen as connection endpoint. confidentiality in a device goes with the integrity in ensuring that no third party has access to the devices internal data. this is normally ensured in case of device integrity. privacy of devices depends on physical privacy and communication privacy. availability of a device depends on the devices integrity and reliability, and availability of the communication part that connects the device. processing integrity in the processing of data for higher services and correlation is based on device integrity and integrity of communication. furthermore, it depends on the correct design and implementation of algorithms for processing. as processing can often be followed by actuator actions it is sensitive in that an actuator may get incorrect commands. the authenticity of proc. wowkivs 2009 4 / 12 eceasst xxxxxxxxxtopic property integrity authenticity confidentiality privacy availability communication 2 2 3 1 sensors 2 1 actuators storage 3 3 1 devices 1 processing 1 localization/tracking 3 1 2 identification 3 1 2 table 2: state of research for highly sensitive properties (1 research needed, 2 basic research available, 3 adequate research available) processing solely depends on the authenticity of the device and authenticity of communication, and therefore in itself is not sensitive to processing. the property of confidentiality in processing is only dependent on the integrity of the device, and – in case of distributed processing – dependent on the integrity of the communication. processing is of major privacy and critical to storage. privacy preserving data mining is available, but regulations must be employed to make sure they are applied and applied correctly. the availability of processing depends on the device and communication availability solely. localization and tracking integrity of localization and tracking is especially based on communication integrity. furthermore, the integrity of reference signals used in localization, e. g. gsm or gps cell, need to be ensured. likewise, the authenticity depends on communication authenticity and device integrity. confidentiality and privacy of localization and tracking data are of high importance to ensure user privacy and therefore highly sensitive. confidentiality in this context means that an attacker is not able to reveal localization data and therefore is mainly based on communication confidentiality. privacy in localization data means that (1) there is no way for an attacker to reveal the identity of the person or object the localization data is attached to and (2) that localization and tracking is not possible without the explicit agreement or knowledge. availability of localization is important to ensure that the reference signals for localization are robust and can not be manipulated by an attacker. we think that regulations in localization and tracking are of high importance mainly in terms of privacy, as mentioned above. identification for identification we see mainly the same sensitivities as for localization and tracking. one difference is the higher sensitivity in integrity. we think it is easier for an attacker to manipulate the identification process as it is to manipulate the localization process. this results mainly due to the technology used (e. g. rfid or biometry) that we think is more feasible for an attacker to manipulate than localization technologies (e. g. gsm). 2.2 state of research we will now look into the state of research for the properties rated highly sensitive in section 2.1. table 2 shows the properties rated as highly sensitive in section 2.1 along with our rating of the state of research. for highly sensitive properties with a research rating of 1 we will explain in more detail why we think that research in this area is currently insufficient. 5 / 12 volume 17 (2009) security and privacy challenges in the internet of things communication mechanisms for securing communication are well-established but unfortunately seldom applied. especially in small devices with weak processing power communication security is often weak or missing. availability of communication is a big problem that is caused by botnets and ddos attacks that exploit the best-effort service provided by ip. sensors a major problem in sensors is privacy. this is mainly caused by people not knowing that they are being sensed. langheinrich [lan01] defines several guidelines to handle this problem in the design phase: (1) users must be aware that they are being sensed (‘notice’), (2) users must be able to choose whether they are being sensed and be able to opt-out (‘choice and consent’), and (3) users must be able to remain anonymous (‘anonymity and pseudonymity’). as the user has no way to tell whether a ubiquitous system integrates these guidelines we believe that regulations must be employed. storage mechanisms for integrity and confidentiality in storage are well-established, but unfortunately often complex to employ. privacy issues, however, are – besides sensor privacy – one of the main privacy problems in the internet of things. anonymization and pseudonymization mechanisms must be used to ensure that data does not contain information sensitive to privacy. often, too much information is stored that is not necessary for the actual system, as mentioned lately by schneier [sch08]. devices the integrity of devices – called ‘admissibility’ in [sch06] – is an unresolved issue. research in trusted platform computing aims at protecting the integrity of devices. although tpm modules have been built into laptops for some time now, fully tpm-capable operating systems are still missing. processing mechanisms for data processing must assure sure that no sensitive information is available in processed data that is forwarded to untrusted devices or storage. mechanisms for privacy preserving data mining exist [vbf+04] – e. g. adding noise – but are applied seldom. regulations need to define a standard set of privacy preserving mechanisms that must be applied in processing. localization and tracking, and identification for localization and tracking, as well as identification we see the same research requirements: the privacy of users that are being localized, identified, or tracked. in all cases the user must be able to opt-out and be notified of the process. this has been defined by langheinrich as ‘choice and consent’. to summarize, we see specific need for research in the availability of communication due to ddos and the best-effort service provided by ip. furthermore, the integrity of devices to make sure they are free from malware like spyware or rootkits needs more research. finally, nearly all areas miss applicable mechanisms in privacy for the internet of things. the guidelines by langheinrich are very helpful for system designers, but we suggest that (1) regulations are needed to ensure systems conform to these guidelines, and (2) mechanisms must be developed that provide users with possibilities in actively protecting their privacy instead of only relying on that systems in the internet of things respect their privacy and implement respective mechanisms. proc. wowkivs 2009 6 / 12 eceasst 3 major players in the internet of things we will now shortly detail on gsn middleware and – more in-depth – on rfid technology. besides background on rfid we will detail on gsn and rfid security in section 3.1 and section 3.2, respectively. 3.1 global sensor network middleware a lot of work has been performed in the last years about middleware architectures that connect sensors and sensor networks into the global infrastructure of the internet [oll07, ahs06, fjk+05, gkk+03] and therewith enable advanced sensing applications. security aspects in the work on middleware have mostly been derived from security needs that arise when connecting multiple heterogeneous networks over the internet – the topic of connection in our categorization. however, the new security and privacy issues that arise when integrating the physical with the digital world in the internet of things need to be covered by future research. in the following, work in security and privacy in global sensor network middleware systems is detailed on shortly. irisnet the irisnet architecture [gkk+03] for distributed sensing uses webcams as sensors. security is based on the assumption that ‘the entire worldwide sensor web is administered by a single, universally trusted authority’. it therefore can be secured using communication security mechanisms. to counter privacy concerns in the video data irisnet implements face blurring. as one scenario in irisnet is monitoring of free parking-lots, if is a good example that anonymization does not necessarily limit data utility. hifi the hifi [fjk+05] specifies a hierarchical architecture of ‘levels’. data is forwarded from the lowest level – actual sensor data sources like rfid or sensor networks – over intermediate levels upwards. levels perform specific tasks: the lowest level ‘cleans’ the data and only forwards data items that comply to a specific quality standard (like rfid signal strength). higher levels ‘smooth’ and ‘validate’ data. privacy and access control is implemented using sql views for the specification of authorization policies. etri ubiquitous sensor network an analysis of security threats in the etri project is given in [klr07]. the following security requirements are identified: threats toward applications like unauthorized users acquiring sensing data, applications disrupting the functionality of the sensor network by reconfiguring sensor nodes, and performing denial-of-service attacks through large numbers of sensing requests. threats from corrupted sensor networks that provide invalid sensing data, therewith distorting application results. threats from external objects like eavesdropping. furthermore, replay attacks and man-in-the-middle attacks are possible when an attacker positions itself between the application and a layer of the middleware system. 3.2 rfid an rfid tag is a small integrated circuit that contains a unique id that identifies this special item – not only the item group as done with barcodes. rfid readers can query the tags and receive the unique item id. the id is then the entry key into a database that contains additional information about the item. often, the id is built in a hierarchical form that contains, e. g. the manufacturer, the group class, and the item id. the rfid market is growing rapidly with 1.02 bn tags sold alone in 2006, and a $5.20 bn 7 / 12 volume 17 (2009) security and privacy challenges in the internet of things market value in 2008 [dh08]. these numbers are predicted to grow and rfid tags posing an important technology in the internet of things. with current passive rfid tags having no sensing capabilities, the sensors in an rfid scenario are the rfid readers. future rfid tags will incorporate advanced sensing and communication capabilities, so the sensor part will also be available in the rfid tag itself. as rfid tags identify unique items, privacy issues arise as the tracking of items – and with this tracking of the person who carries, wears, has implanted the item – becomes possible. therefore, different cryptographic techniques have been proposed with the goal that only authorized parties are able to reveal the real id of the tag. a good overview of such techniques is given in [pm07]. as different rfid tags exist (e. g. active, passive, semi-active) with different computational capabilities, different mechanisms exist for security and privacy. security and privacy in rfid systems has been defined as [ban08]: • security: ‘the ability of the rfid system to keep the information transmitted between the tag and the reader secure from non-intended recipients.’ • privacy: ‘the ability of the rfid system to keep the meaning of the information transmitted between the tag and the reader secure from non-intended recipients.’ rfid tags have the ability to perform basic operations like xor, simple hashing, calculating pseudorandom functions (prfs), and to participate in challenge-response protocols. therewith a number of protocols have been proposed that employ challenge-response protocols between a rfid tag and rfid reader. an rfid reader has access to a database system that includes all tag ids and additional information, depending on the protocol. physical mechanisms to preserve rfid privacy is detailed in section 3.2.1 and cryptographic rfid protocols in section 3.2.2. 3.2.1 physical mechanisms kill codes kill codes permanently disable the tag and therefore prevent reading and tracking. this is especially useful for items that need tracking in the supply-chain only and not after customer purchase. rfid applications like easy item reshipment are made impossible. faraday cage putting an rfid tag into a faraday cage makes it impossible to read the tag. enables the user to decide when reading should be possible. a faraday cage renders ubiquitous services impossible. blocker tag the blocker tag [jrs03] performs jamming that makes unauthorized readers think that a large number of different rfid tags are present. a blocker tag is placed besides the actual rfid tag, therefore it can be easily applied and removed. 3.2.2 cryptographic protocols randomized hash-lock protocol a database contains the ids of all tags idi, i ∈ {0 . . . n}. the rfid reader queries the tag a to start the protocol. the tag calculate x = h(ida|r) using a hash function h(x), his id ida, a random number r, and transmits {x, r} to the server. as the server knows the ids of all keys idi, he calculates y = h(idi|r) for every i and compares x =? y. if x matches y the correct id ida is found. as the protocol runs in o(n) it can put heavy load at the server. proc. wowkivs 2009 8 / 12 eceasst hash chain based protocols the ya-trap [tsu06] protocol requires loose time synchronization between server and readers, tags aren’t required to have clocks. in initialization every tag i is assigned a triple {ki, t0, tmax}, ki is a tag-specific identifier used as tag id and cryptographic key, t0 is an initial timestamp, e. g. the time of manufacture, and tmax is the top value of timestamps and corresponds to the maximum lifetime of the tag. every timestamp tr with r ∈ {0 . . . max} gets a hashtable hasht abler created in the server database. for every tag i the values of hmacki (tr) are inserted into the tables hasht abler, i. e., for every tag i with key ki every hashvalue in every timestamp tr is precomputed and stored. the protocol goes as follows: the reader sends the current timestamp tr to the tag. the tag remembers the current timestamp as tt = tr, computes hr = hmacki (tt ) and sends back hr to the reader. the server now knows tr and hr which is the current timestamp and the hashvalue for the tag in the current timestamp. it looks into the hashtable hasht abler that contains all hashes for the current timestamp tr. by querying hr of the hasht abler the server can retrieve ki which corresponds to the id of tag i. 4 research from other domains the current internet has failed in many ways to provide adequate security and privacy. we present three research results that are worth considering in the internet of things. we shortly present these approaches and motivate in investigating them for the use in the internet of things. 4.1 information accountability since first information systems have been set up and the web has taken its way to reach millions of people, the dilemma of privacy in the digital world has begun. using the same techniques to protect privacy of people – and maybe the privacy of ‘things’ in the internet of things – will maybe end in the same results: uncontrolled information flow and uncontrolled privacy. the current large-scale databases storing personal data will get filled up even more in the days of the internet of things and record our every steps. as schneier warns in [sch08] we have quite no way of controlling the collection and use of personal data. worse, lots of data is linked to personal information – which is often not necessary. all of this data is collected and stored, but not deleted, which inevitably result in data garbage that goes uncontrolled. weitzner et al. present a new concept to privacy which they call information accountability [wab+08]. the main principle of information accountability is not to try to prevent the leakage of data – and being helpless once data leaks – but rather being able to control the usage of the data. therewith being able to call persons to account that misuse the data – which is not able with the current concept of privacy that is based on keeping information secret. 4.2 cryptographic identifiers cryptographic identifiers [mc04] are used within several newer networking protocols to prove ownership of an address. the ipv6 secure neighbor discovery (send), e. g., uses cryptographically generated addresses to prevent address spoofing, as possible in the address resolution protocol (arp) used in lans. furthermore, given the large size of overlay identifiers, the use of cryptographic identifiers can there be used to prove the ownership of ones identifier. the host identity protocol (hip), e. g., bases its security highly on cryptographic identifiers. the cryptographic identifiers as rfid ids would enable tags to prove that they really own 9 / 12 volume 17 (2009) security and privacy challenges in the internet of things the id. with current rfid solutions mainly deployed in self-contained systems, the need to ownership proof does hardly arise. having public databases that store all information about a tags and are publicly queriable, brings up the problem of tag id spoofing as an attacked can gather all tag information from the database and then prepare a tag that spoofs its identity as some other tag. cryptographic identifiers can help detect tags that spoof their id as other tags. furthermore, the scheme can be deployed for sensor nodes that take part in an overlay network where identifiers are long enough to use cryptographic identifiers. these nodes can then prove ownership of their identifier. this allows to detect rogue sensors that spoof as another tag and possible give out corrupted sensing data. cryptographic identifiers are based on asymmetric-key cryptography and therefore have a large overhead compared to symmetric-key cryptography in terms of computational power and key-size. as it has been shown that sensor nodes can be able to perform asymmetric-key cryptography [bz05], the use of cryptographic identifiers in sensor nodes is possible. rfid tags are quite some time away from performing asymmetric-key cryptography, but will eventually be able. therefore, interesting results are to arise when using the rfid tags id in combination with cryptographic identifiers. 4.3 key extraction from wireless channel characteristics as a large part of communication in the internet of things will occur over wireless channels – that are susceptible to eavesdropping – key establishment is necessary to provide confidential communication. the work of mathur et al. [mtm+08] provides the establishment of a common cryptographic key for two users by the use of characteristics of the wireless channel. as the wireless channel characteristics for a communication context between a and b are the same only for exactly a and b, it is possible to use this characteristic to extract bits from stochastic processes. these bits can then be used to form a symmetric cryptographic key. so, a and b independently calculate the same symmetric key for the communication between a and b – solely through the fact that a talks to b and b talks to a. this scheme seems promising when it comes to wireless communication in the internet of things, because (1) it is based only on symmetric-key cryptography, and (2) it would be expensive to establish key infrastructures or distribute keys in the internet of things that is made up of such large numbers of ‘things’. 5 conclusions the internet of things is quickly coming closer. the incremental deployment of the technologies that will make up the internet of things must not fail what the internet has failed to do: provide adequate security and privacy mechanisms from the start. the introduction of e-passports, e. g., has been pushed by politics into deployment with – back then – insufficient privacy mechanisms [jmw05]. we must be sure that adequate security and privacy is available before the technology gets deployed and becomes part of our daily live. in this paper we presented a categorization of topics and technologies in the internet of things with analysis of sensitivity and state in research to different security and privacy properties. we see this (1) as a basis for coming up with an integrated systems approach for security and privacy in the internet of things, and (2) as stimulator for discussion on the categorization and sensitivity rating in the internet of things. furthermore, we presented research in security and privacy for proc. wowkivs 2009 10 / 12 eceasst two major technologies in the internet of things – gsn and rfid – and finally pointed out research from other fields in computer science that is worth considering for use in the internet of things. acknowledgements: the author thanks oliver p. waldhorst for comments on an early version of this paper and the anonymous reviewers of gsn09 for their valuable comments. the work presented in this paper was done as part of the spovnet project that is funded by the landesstiftung baden-württemberg under the initiative bw-fit. bibliography [ahs06] k. aberer, m. hauswirth, a. salehi. a middleware for fast and flexible sensor network deployment. in proceedings of the 32nd international conference on very large databases. pp. 1199–1202. sept. 2006. [ban08] j. banks. understanding rfid part 9: rfid privacy and security. http://www.rfidnews.org, may 2008. [bz05] e. blaß, m. zitterbart. towards acceptable public-key encryption in sensor networks. in proceedings of 2nd international workshop on ubiquitous computing. pp. 88–93. may 2005. [dh08] r. das, p. harrop. complete rfid analysis and forecasts 2008-2017. http://www.idtechex.com/forecasts, 2008. [fjk+05] m. franklin, s. jeffery, s. krishnamurthy, f. reiss, s. rizvi, e. wu, o. cooper, a. edakkunni, w. hong. design considerations for high fan-in systems: the hifi approach. in proceedings of the cidr conference. pp. 290–304. jan. 2005. [gkk+03] p. gibbons, b. karp, y. ke, s. nath, s. seshan. irisnet: an architecture for a worldwide sensor web. ieee pervasive computing 2(4):22–33, dec. 2003. [int05] international telecommunication union. the internet of things. itu report, nov. 2005. [jkk08] h. juma, i. kamel, l. kaya. on protecting the integrity of sensor data. in proceedings of 15th ieee international conference on electronics, circuits and systems. pp. 902–905. sept. 2008. [jmw05] a. juels, d. molnar, d. wagner. security and privacy issues in e-passports. in proceedings of first international conference on security and privacy for emerging areas in communications networks. pp. 74–88. sept. 2005. [jrs03] a. juels, r. l. rivest, m. szydlo. the blocker tag : selective blocking of rfid tags for consumer privacy. in proceedings of 10th acm conference on computer and communications security. pp. 103–111. 2003. 11 / 12 volume 17 (2009) security and privacy challenges in the internet of things [klr07] m. kim, y. lee, j. ryou. what are possible security threats in ubiquitous sensor network environment? lecture notes in computer science 4773:437–446, 2007. [lan01] m. langheinrich. privacy by design principles of privacy-aware ubiquitous systems. in proceedings of ubicomp. pp. 273–291. oct. 2001. [mc04] g. montenegro, c. castelluccia. crypto-based identifiers (cbids): concepts and applications. acm transactions on information and system security 7(1):97–127, feb. 2004. [mtm+08] s. mathur, w. trappe, n. mandayam, c. ye, a. reznik. radio-telepathy: extracting a secret key from an unauthenticated wireless channel. in proceedings of mobicom. pp. 128–139. sept. 2008. [oll07] ollero et al. aware: platform for autonomous self-deploying and operation of wireless sensor-actuator networks cooperating with unmanned aerial vehicles. in proceedings of ieee international workshop on safety, security and rescue robotics. pp. 1–6. june 2007. [pm07] r. d. pietro, r. molva. information confinement, privacy, and security in rfid systems. in proceedings of esorics. pp. 187–202. dec. 2007. [sch06] b. schneier. updating the traditional security model. http://www.schneier.com/blog/archives/2006/08/updating the tr.html, aug. 2006. [sch08] b. schneier. the future of privacy. presentation at rsa conference europe, oct. 2008. [tsu06] g. tsudik. ya-trap: yet another trivial rfid authentication protocol. in proceedings of fourth annual ieee international conference on pervasive computing and communications workshops. pp. 196–200. mar. 2006. [vbf+04] v. s. verykios, e. bertino, i. n. fovino, l. p. provenza, y. saygin, y. theodoridis. state-of-the-art in privacy preserving data mining. acm sigmod record 33(1):50–57, mar. 2004. [wab+08] d. j. weitzner, h. abelson, t. berners-lee, j. feigenbaum, j. hendler, g. j. sussman. information accountability. communications of the acm 51(6):82–87, june 2008. proc. wowkivs 2009 12 / 12 introduction analysis of security and privacy categorization and sensitivity state of research major players in the internet of things global sensor network middleware rfid physical mechanisms cryptographic protocols research from other domains information accountability cryptographic identifiers key extraction from wireless channel characteristics conclusions formal verification of synchronisation, gossip and environmental effects for wireless sensor networks electronic communications of the easst volume 076 (2019) automated verification of critical systems 2018 (avocs 2018) formal verification of synchronisation, gossip and environmental effects for wireless sensor networks matt webster, michael breza, clare dixon, michael fisher and julie mccann 19 pages guest editors: david pichardie, mihaela sighireanu eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst formal verification of synchronisation, gossip and environmental effects for wireless sensor networks matt webster1, michael breza2, clare dixon1, michael fisher1 and julie mccann2 1department of computer science, university of liverpool, liverpool, l69 3bx, uk {matt,cldixon,mfisher}@liverpool.ac.uk https://www.liverpool.ac.uk/computer-science/ 2department of computing, imperial college london, london, sw7 2az, uk {mjb04,j.mccann}@doc.ic.ac.uk https://www.imperial.ac.uk/computing abstract: the internet of things (iot) promises a revolution in the monitoring and control of a wide range of applications, from urban water supply networks and precision agriculture food production, to vehicle connectivity and healthcare monitoring. for applications in such critical areas, control software and protocols for iot systems must be verified to be both robust and reliable. two of the largest obstacles to robustness and reliability in iot systems are effects on the hardware caused by environmental conditions, and the choice of parameters used by the protocol. in this paper we use probabilistic model checking to verify that a synchronisation and dissemination protocol for wireless sensor networks (wsns) is correct with respect to its requirements, and is not adversely affected by the environment. we show how the protocol can be converted into a logical model and then analysed using the probabilistic model-checker, prism. using this approach we prove under which circumstances the protocol is guaranteed to synchronise all nodes and disseminate new information to all nodes. we also examine the bounds on synchronisation as the environment changes the performance of the hardware clock, and investigate the scalability constraints of this approach. keywords: internet of things, critical systems, wireless sensor networks, formal verification 1 introduction in this paper we use formal verification, through the logical method of probabilistic modelchecking [fis11], to analyse and verify critical communication protocols used for the internet of things (iot) [yin14]. iot systems often involve networks of small, resource-constrained, computer devices embedded in an environment. these devices have low-power sensors, radios for communication, and can potentially control motors and other devices to perform actuation to change their environment. a common class of iot systems, called wireless sensor networks (wsn), enable the monitoring and control of critical infrastructures made up of large, complex 1 / 19 volume 076 (2019) https://www.liverpool.ac.uk/computer-science/ https://www.imperial.ac.uk/computing formal verification of synchronisation for wireless sensor networks systems such as precision agriculture or smart water networks. such systems require control software that can synchronise the events of the nodes in the system, and disseminate parameters and code updates. wsn and iot deployments are increasingly mobile, allowing for wider applications and new challenges in their design and deployment [nln+16, mrj+07]. a key problem with the development of critical iot systems is ensuring that they will function correctly, or at least, fail in a way that is non-destructive to the systems that they monitor and control. the use of probabilistic models is crucial because it allows us to quantitatively analyse the system with the dynamical effects caused by the environment — one of the most significant causes of failure for wsns [lbv06]. wsns deployed on critical infrastructure suffer from the effects of cyber-physical interactions in a way not seen with office or domestic computing. environmental conditions such as precipitation or changes in temperature will affect the performance of the sensor nodes, and can degrade the wsn potentially causing node failure. the control software that provides event synchronisation and controls message dissemination needs to be programmed to be reliable in the light of these potential problems. errors here can make the infrastructure itself inefficient at best, or even unstable and failing in the worst case. 1.1 formal verification and probabilistic model checking in prism formal methods are a family of techniques used to verify software and hardware, typically using mathematical, proof-based approaches [fis11]. these include techniques such as automated theorem proving [fit96], in which full mathematical proof is carried out, and model checking [bk08], in which every state of a model (also known as the model’s state space) can be examined exhaustively. formal methods allow for formal verification, where models of software and hardware systems can be proved to satisfy certain requirements. these requirements are typically provided using a precise formal logical language such as temporal logic [fis11]. in this paper we use probabilistic model-checking [knp11], a variant of traditional model-checking that allows for probabilities to be incorporated into a model, and for quantitative analyses to be carried out on such models. the probabilistic model checker, prism [knp11, par18], consists of two parts: a modeling language, and a model checker. the prism modeling language can be used to specify the behaviour of a probabilistic finite state automaton (p-fsa), which can then be formally verified via the model checker. for example, we can model a simple sensor node in the prism version 4.4 modelling language as follows: module sensornode state: [0..1] init 0; [] state=0 -> 0.99: (state’=0) + 0.01: (state’=1); [] state=1 -> 0.99: (state’=1) + 0.01: (state’=0); endmodule this sensor node is modelled as a module in prism. we have one variable, ‘state’, which can be set to 0 or 1 (which we define as representing ‘transmit’ and ‘idle’ respectively). note that we define an initial state of 0 for this variable. there are two lines denoting commands. the first command says that if the state is 0, then remain in state 0 with probability 0.99 or transition to state 1 with probability 0.01. the second command is similar, but with 0 and 1 reversed. in avocs 2018 2 / 19 eceasst general, commands take the form [s] guard -> p1 : u1 +...+pn : un; where pi are probabilities and ui are lists of variable updates. in the case where only one list of updates is made with probability 1.0, a simpler form is used (e.g., [s] guard -> u;). the letter s denotes an optional synchronisation. synchronised commands execute simultaneously with synchronisation commands from other modules that share the same label, and can be used for inter-module communication. another way for modules to communicate is via the use of local variables, which can be read by all modules, as well as global variables which can be read by, and written by, all modules. multiple modules can be specified within a prism model. models are executed by selecting non-deterministically a command (from any module) whose guard evaluates to true. if there are no commands whose guards are true, then the model has reached a fixed point and will stop executing. once a formal model has been developed in the prism language, it can be formally verified, with respect to some requirement, using the prism model checker. prism requirements can be formalised as properties using logical languages such as probabilistic computation tree logic (pctl∗) and probabilistic linear temporal logic (p–ltl) [bk08]. different logics allow different kinds of properties to be specified. in this paper we will use pctl∗ to specify properties. pctl∗ is based on a discrete formulation of time as a tree-like structure, starting from a particular point and extending into the future. the following are well-formed pctl∗ formulae: ‘ p’, meaning that p is true; ‘¬p’, meaning that p is false; ‘ p =⇒ q’, meaning if p is true then q is true; p∧q’, meaning that both p and q are true; ‘f p’, meaning p is true now or at some point in the future; and ‘g p’, meaning p is true now and at every point in the future. prism also allows the use of standard numerical operators such as =, ≥ and ≤. formal verification works by analysing the entire state space of a model in order to determine whether a particular property holds. for example, for the sensor node model above, we can use pctl∗ to specify the probability that sensor node is eventually in the ‘idle’ state: p=?[f (state = 1)] we can then use prism model checker to determine that this probability is 1.0: p=?[f (state = 1)] = 1.0 more complex properties can be formed, e.g., the following property says that the probability that the model will always be in the ‘idle’ state eventually is 1.0: p=?[g f (state = 1)] = 1.0 this kind of property is said to specify the probability that the model is in the ‘idle’ state infinitely often. 3 / 19 volume 076 (2019) formal verification of synchronisation for wireless sensor networks 2 related work formal methods have been used previously for design and analysis of wsn and iot. for example, chen et al. [czz+13] provide a survey of a number of approaches to formal verification of routing protocols for wsns. kim et al. [kklb17] conduct a formal security analysis of an authorization toolkit for the internet of things using the alloy verification tool. mouradian & augé–blum [ma13] describe the formal verification of real-time wsn protocols using the uppaal model checker. tobarra et al. [tcc09] use the avispa model checking tool to formally verify a security protocol for wsns. usman et al. [umw13] demonstrate formal verification of mobile agent-based anomaly detection for wsns using the symbolic analysis laboratory model checking tool. dong et al. [dss+08] use a formal specification language for sensor networks and perform formal verification using sat-solvers. however, none of these approaches uses a probabilistic model checker, as is the case in this paper, to determine the probability of success or failure for particular requirements. fruth [fru11] used prism to analyse contention resolution and slot allocation protocols for wsns, but not synchronisation or dissemination protocols. synchronization [gld+17b, gld+17a, psh99] and gossip protocols [bbfh07, fg06, knp08, hss08, kat08, cpr08] have been formally verified but not together, and not accounting for environmental effects. mohsin et al. [msha17] used prism to formally assess security risks in iot systems, but not risks due to the environment. modelling of embedded systems and the environment have been explored by baresi et al. [bbk+15], who used a uml-based mades approach to model a system, and by basile et al. [bdg17], who performed statistical model checking of an energysaving cyber-physical system using uppaal smc. these approaches can find when constraints are not met, but do not perform an exhaustive search of the entire state space, as is the case here. boano et al. explored the effects of temperature on cpu processing time and transceiver performance though templab, a wsn test-bed which allows for the manipulation of the temperature of each individual sensor node [bzb+14]. lenzen et al. [lsw09] studied the effect of temperature on the hardware clocks chips used as timers on many common wsn sensor node platforms. 3 modelling a wsn protocol in prism it is possible to model various wsn protocols in prism. in order to illustrate the approach, we create a model of a non-trivial decentralised wsn management protocol known as figo (an abbreviation of firefly–gossip) [bm08]. figo enables synchronisation of different sensors’ clocks in order to unify the measurement of time across the network based on firefly-like synchronisation. figo also enables consensus on key information between nodes via gossiping, in which nodes pass on new information to their neighbors. figo was chosen because it is a simple protocol that contains elements, such as epidemic propagation [jmb05], found in more complex protocols like trickle [llwc03] and rpl [bra12]. avocs 2018 4 / 19 eceasst 3.1 the firefly-gossip (figo) protocol current techniques for large-scale computer management are not suitable for wsns due to the unreliable nature of the nodes and their networks. a potential solution is to use management protocols, such as figo, that scale well and are robust to the failure of individual nodes. in applications such as precision agriculture [uais14, omr15], wireless nodes need to be synchronised to be able to deliver time-correlated samples of data such as moisture levels and temperature, and to analyse the data. if the analysis shows a problem, control messages need to be sent to nodes with actuators, e.g., to increase irrigation in a drought, or decrease it if a particular disease is discovered. synchronisation of wsns is essential in many applications, for example in adaptive sensing for smart water networks [kyam16]. wsns allow urban water providers to monitor the water flow to match customer demand. synchronisation enables the sensor nodes to measure, communicate and aggregate the flow rates and water pressure data. a control algorithm on the actuator nodes can open or close valves to stabilise water flow for the network, or re-route water in the case of a major leak. importantly, the control software can also disseminate new control algorithms or critical security updates to all the sensing and actuation nodes via gossiping. figo is typical of a class of algorithms that combine firefly synchronisation [wtp+05] and gossip protocols [jmb05] into a single epidemic process [bm08]. this mixture of synchronisation and dissemination processes is used to bring the internal states of wsn nodes to a stable, global equilibrium where all nodes are synchronised with respect to both time and metadata. experiments have shown such protocols to be both scalable and resilient to individual node failure [bm08, bre13, bm17]. a typical figo algorithm is shown in figure 1. figo algorithms have been deployed for the synchronisation and management of several wsn deployments run by the adaptive emergent systems engineering group at imperial college1. for example, they were used to organise pollution sensors for an experiment with mobile data mules as part of an imperial college grand challenge project, and to synchronise and control the sampling rate for a rainfall monitoring sensor network as part of a floodplain monitoring project done in collaboration with the imperial college department of civil engineering. they are currently undergoing evaluation for deployment across the liverpool sensor city iot-x/lorawan network2. 3.2 a prism model of figo a prism model of figo was developed precisely capturing the control flow of the algorithm in figure 1. the algorithm begins with a number of variable assignments which are directly translated into variable assignments in prism. some of the variables are not updated at all in the model, so these are set as global constants in prism, e.g.: const int cyclelength = 100; const int refractoryperiod = floor(cyclelength/2); the main loop of the algorithm is then divided into a number of phases. for example, the transmit phase corresponds to the if-statement in lines 9 to 14. the next if-statement consists of 1 http://wp.doc.ic.ac.uk/aese/ 2 http://www.sensorcity.co.uk/ 5 / 19 volume 076 (2019) http://wp.doc.ic.ac.uk/aese/ http://www.sensorcity.co.uk/ formal verification of synchronisation for wireless sensor networks figure 1: phases of the figo gossip–synchronisation algorithm. a number of nested if-statements called clockcheck, listen, sync1, sync2, and so on. the final phase corresponds to the final if-statement in the main loop and is called updateclock. these phases are defined as global constants and are used as the values of a local variable s1phase which contains the currently-executing phase: s1phase : [0..7] init transmit; note that s1phase refers to the phase of the first sensor node module, which is called s1. the phases of other sensors (sensors s2, s3, etc.) are called s2phase, s3phase, etc. when one phase has finished executing the next phase is chosen according to the control flow of the algorithm in figure 1. for example, during the sync1 phase in lines 17 to 19 the algorithm checks whether the clock is outside a ‘refractory period’ set to half of the cycle length. if it is, then the sensor updates its clock to the average of its own clock and the clock of the other sensor. avocs 2018 6 / 19 eceasst the ‘circular average’ is used, in which the average of 90 and 10 are 0 (with respect to the clock cycle of length 100), rather than 50. the circular average ensures that the update to the clock variable moves it closer to the clock of the other sensor. in the prism model, this behaviour is shown in the following three commands: [] s1phase=sync1 & s1localclock>=refractoryperiod & diff<=floor(cyclelength/2) -> (s1localclock’=s1avg1) & (s1phase’=sync2); [] s1phase=sync1 & s1localclock>=refractoryperiod & diff>floor(cyclelength/2) -> (s1localclock’=s1avg2) & (s1phase’=sync2); [] s1phase=sync1 & !( s1localclock>=refractoryperiod ) -> (s1phase’=sync2); the first two commands say that if the sensor is in the sync1 phase and the clock is greater than or equal to refractoryperiod, then set s1’s clock to the circular average of s1’s clock and s2’s clock. the third command says that if these conditions are not set, then proceed to the next phase of the algorithm, sync2. the sensor which we have modelled here is called s1. to model communication between sensor nodes we need at least one more sensor in the model, s2. the sensor s2 is exactly the same as s1, except all references to ‘s1’ in the code are modified to ‘s2.’ communication in the model is achieved asynchronously through the use of inboxes: when a sensor sends a message to another sensor it does so by leaving the message in an inbox, which can then be read by the receiving sensor when it is ready to do so. the resulting combined model is around 140 lines of code long including variable declarations, and can be found in the online repository3. this prism model is an almost direct translation from the pseudocode to prism and has not been optimised for formal verification. 4 formal verification of figo using prism we build a formal model in prism, in a manner analogous to compiling a program: the source code, in this case the prism model, is automatically converted into a mathematical model, essentially a finite state structure. during this construction, prism calculates the set of states reachable from the initial state and the transition matrix which represents a probabilistic finite state automaton. building revealed that the full model consisted of 4,680,914 reachable states, with 9,361,828 transitions between those states, and took 21 minutes on an intel core i7-3720qm cpu @ 2.60ghz laptop, with 16 gb of memory, running ubuntu linux 16.04. as we shall see in section 4.1, it was possible to reduce the size of this model significantly. one of the key features of prism is that it can find the probability of a particular property holding through some path through a computation tree. for example, we can create a property to determine the probability that eventually the two sensors are synchronised: p=?[f (s1clock = s2clock)] = 1.0 [23.8s] (1) in this case the probability is 1.0, meaning that on all paths through the model the clocks will eventually synchronise. (the time taken for model checking was 23.8 seconds.) that is not to say that they remain synchronised, or that they become synchronised again once they are no 3 http://livrepository.liverpool.ac.uk/3021710/ 7 / 19 volume 076 (2019) http://livrepository.liverpool.ac.uk/3021710/ formal verification of synchronisation for wireless sensor networks longer synchronised. if we wish to verify the latter, that synchronisation happens repeatedly, then we can create a probability with a different formula: p=?[g f s1clock = s2clock] = 1.0 [100s] (2) this probability, in which synchronisation occurs infinitely often, is 1.0. we can strengthen the property further: we can determine the probability that, once the clocks are synchronised, they remain synchronised: p=?[f g s1clock = s2clock] = 0.0 [75.6s] (3) in this case the probability of this property being true is 0.0, meaning that it is never the case that the two clocks synchronise and then remain synchronised forever. the reason this is so can be seen by examining a simulation, or trace, of the model. (a simulation is a sample path or execution of the model [par18].) below is a simulation of the model showing how de-synchronisation occurs after synchronisation: action s1phase s1clock s2phase s2clock s1 updateclock 4 updateclock 4 s2 updateclock 4 transmit 5 s2 transmit 5 transmit 5 the table shows the values of certain state variables during an execution of the model. the leftmost column, ‘action’, shows which module, s1 or s2, is currently executing. in the first state, both clocks have the value ‘4’ and are synchronised. however, a transition occurs in which one of the sensors, in this case, s2, increments its clock value resulting in de-synchronisation. however, in the next state we can see that the sensor s1 updates its clock as well, resulting in synchronisation. we might postulate that once synchronisation occurs, then de-synchronisation will occur at some point. this can be encoded as the following property: p=? [ g ( s1clock = s2clock =⇒ f ¬(s1clock = s2clock) ) ] = 1.0 [123s] (4) we can also verify whether once de-synchronisation has happened, that synchronisation will eventually happen: p=? [ g ( ¬(s1clock = s2clock) =⇒ f s1clock = s2clock ) ] = 1.0 [175s] (5) property 1 tells us that synchronisation will occur at some point during the execution of the model and property 2 tells us that synchronisation will occur infinitely often. properties 4 and 5 tell us even more: that periods of synchronisation are separated by periods of de-synchronisation, and vice versa. avocs 2018 8 / 19 eceasst 4.1 increasing the model’s accuracy examining simulations using prism reveals that clocks will rapidly de-synchronise after synchronisation, as we saw in the previous section. this is a result of the way clocks were handled in this model: we allowed for clocks to tick at any rate. therefore it is possible for clocks to tick unevenly, as in this case. in fact, it is possible for one clock to tick indefinitely without the other clock ticking. this assumption of the model can be seen to correlate with a real-world sensor system in which clocks are unreliable and may vary widely in comparative speeds. the figo sensor network we are modelling is based on the ‘micaz’ sensor mote developed by memsic inc. [mem18] the network is homogeneous across nodes, meaning that the same hardware and software is present on each node. this includes the microcontroller, in this case the ‘atmega128l’ developed by atmel corporation [atm18]. this microcontroller has a clock speed of 16 mhz and operates at up to 16 million instructions per second. as the network is homogeneous we can model the clock speed as constant across different nodes. in practice, and as we shall see in section 5, clock speeds are never exactly the same. however, treating the clock speeds as constant is much closer to reality than one clock being able to tick indefinitely without the other ticking. clock speeds were made constant by introducing synchronisations in the updateclock phase: [tick] s1phase=updateclock & s1clock<cyclelength -> (s1clock’=s1clock+1) & (s1phase’=transmit); [tick] s1phase=updateclock & s1clock=cyclelength -> (s1clock’=0) & (s1samecount’=0) & (s1phase’=transmit); the first command says that if the clock is less than the cycle length (equal to 99 in this model), then increment the clock, but if the clock is equal to 99, then reset the clock to zero. these commands both use a synchronisation label, tick, and correspond to a similar set of commands in the s2 sensor module, which use the same label. the label means that one of these commands must execute at the same time as one of the corresponding commands in the s2 module. since these commands handle clock updates, this ensures that the clocks will update synchronously, and therefore it is impossible for one clock to tick faster than the other. this models more closely the homogeneous network on which figo is used. one advantage of constant clock speeds is that it reduces the total number of states of the probabilistic model. in this case the model reduced in size from 4,680,914 states with 9,361,828 transitions to 8,870 states and 13,855 transitions. the time taken for model building also decreased, from 21 minutes to 17 minutes. property 1 was formally verified for this revised model: p=?[f s1clock = s2clock] = 1.0 [5.4s] (6) in the definition of the figo algorithm, the variable nextbroadcast is assigned a random value between 0 and 99. during model translation, however, this random value was modified to a constant integer value. we used prism variables to automatically check every possible value of nextbroadcast. this is done by removing the values of the global constants that represent the next broadcast value. then, prism can be used to perform automatic, and exhaustive, modelchecking of a property across a range of values for these constants, by automatically building and verifying a model for each value. however, the prism model has a large verification time 9 / 19 volume 076 (2019) formal verification of synchronisation for wireless sensor networks of 17 minutes. prism needs to build a model for each value of the two variables above, meaning that 10,000 models would need to be constructed, each taking 17 minutes. to reduce the size of the model the duty cycle length was reduced from 100 to 20. this reduces the size of the model to 1,947 states and 3,040 transitions, and takes 16.8 seconds to build. the duty cycle length can be reduced from 100 to 20 without significantly affecting the accuracy of the model, as there is still a large enough range of possible values to allow for an accurate depiction of clock synchronisation via circular averaging. property 2 was verified with a range of [0,20] for both variables, modelling every possible combination of the two nextbroadcast values. the results showed that the probability that synchronisation will occur infinitely often is always 1.0. 4.2 gossip and synchronisation the properties examined thus far have concerned clock synchronisation. the other main function of the figo algorithm is to spread information across a network using a gossip protocol in which sensors tell their neighbours about new information. in the case of the figo algorithm, this is represented by an integer variable whose initial value is zero, but which may increase when a node is updated with a new piece of information. this captures a common function of wsns that must share new information, roll-out software updates, etc. in order to analyse metadata synchronisation the model was modified to allow new metadata values. this was done by creating a branching point during the updateclock phase of the algorithm: [tick] s1phase=updateclock & s1clock=cyclelength & s1metadata<3 -> (1-pupdatemetadata): (s1clock’=0) & (s1samecount’=0) & (s1phase’=transmit) + pupdatemetadata: (s1clock’=0) & (s1samecount’=0) & (s1metadata’=s1metadata+1) & (s1phase’=transmit); the metadata can take any value from 0 to 3, representing a sequence of three possible updates. this updated command allows the metadata to be incremented at the point the duty cycle ends. this happens with probability pupdatemetadata which is equal to 0.5, a value chosen to represent that new metadata will happen, on average, every other duty cycle. therefore the probability that the metadata will not be updated at the end of the duty cycle is also 0.5. this functionality is included in s1, but not in s2, to model a sensor node that receives updates first. for example, this could be the sensor node located closest to an engineer who is updating node software, which will therefore receive an update first. adding this branch point to the model introduces new states for the various values of the local metadata variables. this increased the size of the model from 1,947 states and 3,040 transitions to 4,776 states and 7,467 transitions for a model with a duty cycle of 20. it is now possible to form properties that verify the gossip part of the figo algorithm. for example: p=?[f s1metadata = s2metadata] = 1.0 [0.041s] (7) this formula says that the probability that the metadata is eventually synchronised across nodes is 1.0. as is the case with software version numbers, the metadata increases but never decreases, avocs 2018 10 / 19 eceasst i.e., once it reaches 3 it stays at 3. therefore it can also be verified that at some point the metadata is synchronised (e.g., when it is equal to 3) and remains so: p=?[f g s1metadata = s2metadata] = 1.0 [2.0s] (8) furthermore, we can verify that the firefly and gossip parts of the algorithm both work, and that eventually the two sensors will by synchronised on both time and metadata, and will remain so: p=? [ f g ( s1metadata = s2metadata ∧ s1clock = s2clock ) ] = 1.0 [1.6s] (9) to examine the scalability of the model, the two-sensor network was extended to three and four sensors. a complete graph topology was used, so that every node can communicate with every other node. a range of clock duty cycle lengths was examined for 2–, 3– and 4–sensor networks. the aim was to see how the total time to verify property 2 (including build and verification time) was affected. the results are summarised in figure 2. figure 2: total time for formal verification of property 2 for 2– and 3–sensor networks. the 2– and 3–sensor networks could be verified formally with a clock cycle length of up to 100 for 2-sensor networks, and 28 for 3–sensor networks. however, the 4–sensor network could not be analysed at all. the amount of time taken to verify this property increases with cycle length, and increases significantly with the number of sensors (see figure 2). this is due to a state space explosion [cknz12] occurring as a result of a larger number of large variables occurring in the model (e.g., the duty cycle has a range of up to 100 for each sensor). the state space also increases with cycle length due to increased non-determinism in the model: the larger the duty cycles for the clocks of each sensor, the more combinations of these clock values there are in the model. all of the probabilities for property 2 for the different network and duty cycle sizes were found to be 1.0, showing that synchronisation happens infinitely often in all the cases examined. it should be noted that these results only pertain to the model examined in this paper, and other models and protocols may permit larger sensor networks to be analysed. while state space explosion is a recurrent theme in model checking, it can be mitigated through abstraction and re-modelling to reduce the size of the state space. 11 / 19 volume 076 (2019) formal verification of synchronisation for wireless sensor networks -10 0 10 20 30 temperature (°c) 921.809 921.810 921.811 921.812 921.813 921.814 921.815 f re q u e n cy ( kh z) node 1 figure 3: mica2 hardware clock frequency for different ambient temperatures [lsw09]. 5 environmental effects on hardware microcontrollers such as the atmega128l [atm18] are often set to process instructions at a particular speed, known as the clock speed. (here, the clock speed refers to the clock internal to the microcontroller, not the clock used in the figo algorithm.) these clock speeds can vary slightly due to environmental conditions (principally temperature). laboratory tests with synchronised micaz [mem18] sensor nodes, which use the atmega128l controller, have revealed that the drift in clock speed can be pronounced over a period of hours. lenzen et al. [lsw09] studied the effect of varying ambient temperature on the clock speed of a ‘mica2’ node, which uses the same processor as the micaz node used in this paper. it was found that drift was up to one microsecond per second for a difference of five degrees celsius (see figure 3). using the raw data from [lsw09] it was determined that at 0.0 degrees celsius the operating frequency was 921,814 hz, and at 30.0 degrees celsius the frequency was 921,810 hz. therefore, for each tick of the clock, the amount of time taken per tick for a processor at 30.0 degrees celsius will be 1.000004339 times longer than for a clock at 0.0 celsius. eventually the warmer clock will lag the colder clock by one whole tick, i.e., the colder clock will have ticked twice and the warmer clock will have ticked once. suppose that clock c1 has ticked n1 times, with each tick having length l1. then, after a period of time, the total time elapsed is n1l1. similarly for clock c2, after n2 ticks the total time elapsed is n2l2. after a period of time, the clocks will tick in unison, so that n1l1 = n2l2. suppose that clock c2 has ticked exactly once more than c1, so that n1 = n2 + 1. therefore we know that (n2 + 1)l1 = n2l2. if we let c1 be the colder clock, and c2 be the warmer clock, then we know that c2’s tick is 1.000004339 times longer that the tick of c1, so that l2 = 1.000004339l1. therefore (n2 + 1)l1 = 1.000004339l1n2. therefore n2 = 230,467, and we know that after 230,468 ticks of c2’s clock it will be exactly one tick behind c1’s clock. therefore, on average, every 230,468 ticks, the warmer clock will lag behind the colder one by one whole tick. we can convert this to a probability, 1 in 230,468, or 0.000004339, which can be incorporated into the prism model: [tick] s1phase=updateclock & s1clock=1 -> (1-pclockdrift): (s1clock’=s1clock+1) & (s1phase’=start) + pclockdrift: (s1clock’=s1clock+2) & (s1phase’=start); this command says that if it is time to update the clock, then increase the clock value by 1 with probability 1−pclockdrift, or by 2 with probability pclockdrift, where pclockdrift = avocs 2018 12 / 19 eceasst 0.0004339. note that pclockdrift is 100×0.000004339. this is because clock drift is modelled as happening once per duty cycle (specifically, when s1clock = 1), which is every hundred clock ticks. this helps reduce the state space because this branching point can only happen once per duty cycle, rather than on every tick. note that the clock is increased by 2 when clock drift occurs. this is to ensure that the clock drifts only once per duty cycle — if the clock was increased by 0 (representing a slower clock rather than a faster one) then the precondition of this command would be true on the next iteration of the algorithm meaning that the clock could drift more than once in the duty cycle. as clock drift can be modelled either by one clock slowing by one tick, or the other clock speeding up by one tick, the accuracy of the model is not affected. it is possible to calculate the effect of clock drift on the stability of clock synchronisation. one way to do this is use a steady-state probability in prism, denoted s=?[s], which is the probability that a model is in a particular state s at any given time. for example it was found that: s=?[s1clock = s2clock] = 0.996709321 [0.5s] (10) i.e., the probability that the model is in a synchronised state is equal to 0.996709321. that is to say, 99.67% of the time the model is in a synchronised state. it should be noted that the numerical methods normally used to determine the steady state probabilities in prism were not suitable in this case, as they either did not converge or returned a value of 1.0 after a very short execution time, indicating a possible problem with the use of the numerical method. one possible reason for this is the closeness of the probability of clock drift to zero. instead, ‘exact model checking’ was used, a technique in which the model checker builds the state space explicitly, and returns a probability based on the number of states matching the specified formula divided by the total number of states. exact model checking is not enabled by default as it requires a lot of time and memory [par18], but in this case the model was sufficiently small to allow its use. experiments with different values for pclockdrift showed that the steady state probability of synchronisation is dependent on the clock drift rate. if the clock drifts more often, then the model will spend less time in a synchronised state. the varying clock drift rates due to ambient temperature were examined to determine the effect on synchronisation of operating at varying temperatures. various clock speeds were taken from the data in lenzen et al. [lsw09] corresponding to different temperatures. these were compared against a base clock speed of 921,814.624 hz. this value was chosen as it was the highest frequency observed, and it occurred at approximately zero degrees celsius. therefore the drift rates in our experiment were relative to a reference node operating at that temperature. figure 4 shows the effect on synchronisation between two nodes when one node is at zero degrees celsius, and a second node is at a varying ambient temperature between −12.48 degrees celsius and 30.48 degrees celsius. it can be seen that the steady-state probability never drops below 0.9959677566, and decreases with increased difference in temperature between the two nodes. the shape of the curve closely matches that in figure 3, as expected. 13 / 19 volume 076 (2019) formal verification of synchronisation for wireless sensor networks figure 4: probability of synchronisation for varying temperatures of a second node. 6 conclusions and future work we have shown how formal methods, in particular probabilistic model checking using prism, can be used to model and verify protocols used in critical iot systems. models were developed based on a straightforward translation from a pseudocode-style language into the prism modelling language. key requirements of a gossip–synchronisation algorithm were encoded using probabilistic computation tree logic (pctl∗) and then verified formally using prism. these requirements included clock synchronisation, metadata synchronisation and steady-state probability of synchronisation. environmental effects, such as temperature, can affect a wsn node’s hardware and cause clock drift. we have explored the use of formal verification to quantify the extent to which clock drift affects the synchronisation of wsn nodes. results such as these can be useful for system designers who may wish to adjust the parameters of figo, or even develop new algorithms, to better cope with sources of unreliability such as clock drift. these new algorithms can then be verified formally in a similar way to that described in this paper. we have also demonstrated that state space explosion is a key challenge in the formal verification of wsns. state space explosion issues are common when using model checkers like prism [cknz12], and the results in figure 2 are typical. however, it is often possible to compensate for state space issues through the use of abstraction and re-modelling. for example, rather than modelling the algorithm completely for each sensor, we could model it in detail for a single sensor, and model the rest of the network of n nodes with a second module in prism. in doing so the module size would be kept to a minimum, but would still allow for verification of the behaviour of the node in response to a network. a possible application of this approach would be to verify how long a particular sensor node takes to synchronise with an already-synchronised network. another possibility is to use a population model (e.g., [gld+17b, gld+17a]), in which sensors are not modelled in detail, but rather the whole network, or several sub-networks, are modelled in order to verify properties concerning overall sensor network behaviour. these approaches, which could also be applied to investigate different sensor network topologies, are intended for future work. another way to compensate for state space explosion is to complement model checking with other verification methods, e.g., simulation. for example, sensor networks consisting of thousands of nodes can be analysed by simulation software [bre13]. of course, the disadvantage of simulation is that it does not allow exhaustive examination of the state space, and is therefore prone to missing highly improbable events that can be detected using model checking: so-called avocs 2018 14 / 19 eceasst ‘black swans’ [tal07]. however, this can be mitigated through analysis of sufficiently large numbers of simulations, as is the case with statistical model checking [ldb10, ap18]. naturally, we advocate the use of a range of different methods of verification for critical iot systems, as their different characteristics are often complementary. our intention is to extend this approach beyond specific synchronisation and distribution algorithms, through the generation of a more general approach to critical iot systems design. such systems have commonly-used programming archetypes, such as sense–compute–send cycles for sensor nodes, or clock duty cycles. formal modelling of these elements is often straightforward and could potentially be automated. in addition, simulation, algorithm animation, testing and a range of formal verification elements could all be included in a single tool to provide a strong and useful apparatus for the exploration and analysis of a range of design decisions. while there is much work still to be done to facilitate this, the research reported in this paper shows how certain design choices can be explored in a more precise, formal way. acknowledgements: the authors would like to thank philipp sommer for the experimental data from [lsw09]. this work was supported by the epsrc-funded programme grant s4 (ep/n007565/1) and the fair-space (ep/r026092/1), rain (ep/r026084/1). and orca (ep/r026173/1) rai hubs. bibliography [ap18] g. agha, k. palmskog. a survey of statistical model checking. acm transactions on modelling and computer simulation 28(1):6:1–6:39, 2018. doi:10.1145/3158668 [atm18] atmel corporation. atmega128l: 8-bit atmel microcontroller with 128 kbytes insystem programmable flash. http://www.atmel.com/images/doc2467.pdf, 2018. last accessed 6/4/18. [bbfh07] r. bakhshi, f. bonnet, w. fokkink, b. haverkort. formal analysis techniques for gossiping protocols. acm sigops operating systems review 41(5):28–36, oct. 2007. doi:10.1145/1317379.1317385 [bbk+15] l. baresi, g. blohm, d. kolovos, n. matragkas, a. motta, r. paige, a. radjenovic, m. rossi. formal verification and validation of embedded systems: the uml-based mades approach. software & systems modeling 14(1):343–363, feb 2015. doi:10.1007/s10270-013-0330-z [bdg17] d. basile, f. di giandomenico, s. gnesi. statistical model checking of an energysaving cyber-physical system in the railway domain. in proceedings of the symposium on applied computing. sac ’17, pp. 1356–1363. acm, new york, ny, usa, 2017. doi:10.1145/3019612.3019824 15 / 19 volume 076 (2019) http://dx.doi.org/10.1145/3158668 http://www.atmel.com/images/doc2467.pdf http://dx.doi.org/10.1145/1317379.1317385 http://dx.doi.org/10.1007/s10270-013-0330-z http://dx.doi.org/10.1145/3019612.3019824 formal verification of synchronisation for wireless sensor networks [bk08] c. baier, j.-p. katoen. principles of model checking. mit press, 2008. [bm08] m. breza, j. a. mccann. lessons in implementing bio-inspired algorithms on wireless sensor networks. in 2008 nasa/esa conference on adaptive hardware and systems. pp. 271–276. june 2008. doi:10.1109/ahs.2008.72 [bm17] m. breza, j. mccann. polite broadcast gossip for iot configuration management. in 3rd international workshop on sensors and smart cities. 2017. [bra12] brandt, a., et al. rpl: ipv6 routing protocol for low-power and lossy networks. rfc 6550, 2012. doi:10.17487/rfc6550 [bre13] m. breza. bio-inspired tools for a distributed wireless sensor network operating system. phd thesis, imperial college, london, 2013. [bzb+14] c. boano, m. zúñiga, j. brown, u. roedig, c. keppitiyagama, k. römer. templab: a testbed infrastructure to study the impact of temperature on wireless sensor networks. in proceedings of the 13th international symposium on information processing in sensor networks. pp. 95–106. 2014. [cknz12] e. m. clarke, w. klieber, m. nováček, p. zuliani. model checking and the state explosion problem. in tools for practical software verification. volume 7682, pp. 1–30. springer lncs, 2012. [cpr08] p. crouzen, j. van de pol, a. rensink. applying formal methods to gossiping networks with mcrl and groove. sigmetrics perform. eval. rev. 36(3):7–16, nov. 2008. doi:10.1145/1481506.1481510 [czz+13] z. chen, d. zhang, r. zhu, y. ma, p. yin, f. xie. a review of automated formal verification of ad hoc routing protocols for wireless sensor networks. sensor letters 11(5):752–764, 2013. [dss+08] j. s. dong, j. sun, j. sun, k. taguchi, x. zhang. specifying and verifying sensor networks: an experiment of formal methods. in liu et al. (eds.), 10th international conference on formal engineering methods, icfem 2008, kitakyushu-city, japan, october 27-31, 2008. pp. 318–337. springer, 2008. http://dx.doi.org/10.1007/978-3-540-88194-0_20 [fg06] a. fehnker, p. gao. formal verification and simulation for performance analysis for probabilistic broadcast protocols. pp. 128–141. springer, 2006. http://dx.doi.org/10.1007/11814764_12 [fis11] m. fisher. an introduction to practical formal methods using temporal logic. wiley, 2011. avocs 2018 16 / 19 http://dx.doi.org/10.1109/ahs.2008.72 http://dx.doi.org/10.17487/rfc6550 http://dx.doi.org/10.1145/1481506.1481510 http://dx.doi.org/10.1007/978-3-540-88194-0_20 http://dx.doi.org/10.1007/11814764_12 eceasst [fit96] m. fitting. first-order logic and automated theorem proving. springer, 1996. [fru11] m. fruth. formal methods for the analysis of wireless network protocols. phd thesis, university of oxford, 2011. [gld+17a] p. gainer, s. linker, c. dixon, u. hustadt, m. fisher. investigating parametric influence on discrete synchronisation protocols using quantitative model checking. in proceedings of qest 2017. volume 10503, pp. 224–239. springer lncs, 2017. [gld+17b] p. gainer, s. linker, c. dixon, u. hustadt, m. fisher. the power of synchronisation: formal analysis of power consumption in networks of pulse-coupled oscillators. arxiv e-prints, 2017. https://arxiv.org/abs/1709.04385. last accessed 23/5/18. [hss08] b. r. haverkort, m. siegle, m. van steen. quantitative analysis of gossiping protocols. sigmetrics perform. eval. rev. 36(3):2, 2008. doi:10.1145/1481506.1481508 [jmb05] m. jelasity, a. montresor, o. babaoglu. gossip-based aggregation in large dynamic networks. acm transactions on computer systems 23(3):219–252, 2005. [kat08] j.-p. katoen. how to model and analyze gossiping protocols? sigmetrics perform. eval. rev. 36(3):3–6, nov. 2008. doi:10.1145/1481506.1481509 [kklb17] h. kim, e. kang, e. a. lee, d. broman. a toolkit for construction of authorization service infrastructure for the internet of things. in proceedings of the second international conference on internet-of-things design and implementation, iotdi 2017, pittsburgh, pa, usa, april 18-21, 2017. pp. 147–158. acm, 2017. doi:10.1145/3054977.3054980 [knp08] m. kwiatkowska, g. norman, d. parker. analysis of a gossip protocol in prism. sigmetrics performance evaluation review 36(3):17–22, nov. 2008. doi:10.1145/1481506.1481511 [knp11] m. kwiatkowska, g. norman, d. parker. prism 4.0: verification of probabilistic real-time systems. in proc. 23rd international conference on computer aided verification (cav’11). lncs 6806, pp. 585–591. springer, 2011. [kyam16] s. kartakis, w. yu, r. akhavan, j. a. mccann. adaptive edge analytics for distributed networked control of water systems. in 2016 ieee first international conference on internet-of-things design and implementation (iotdi). pp. 72–82. 2016. [lbv06] k. langendoen, a. baggio, o. visser. murphy loves potatoes: experiences from a pilot sensor network deployment in precision agriculture. in 20th international parallel and distributed processing symposium. 2006. 17 / 19 volume 076 (2019) https://arxiv.org/abs/1709.04385 http://dx.doi.org/10.1145/1481506.1481508 http://dx.doi.org/10.1145/1481506.1481509 http://dx.doi.org/10.1145/3054977.3054980 http://dx.doi.org/10.1145/1481506.1481511 formal verification of synchronisation for wireless sensor networks [ldb10] a. legay, b. delahaye, s. bensalem. statistical model checking: an overview. in barringer et al. (eds.), runtime verification. pp. 122–135. springer, 2010. [llwc03] p. levis, n. lee, m. welsh, d. culler. tossim: accurate and scalable simulation of entire tinyos applications. in proceedings of the 1st international conference on embedded networked sensor systems. pp. 126–137. 2003. [lsw09] c. lenzen, p. sommer, r. wattenhofer. optimal clock synchronization in networks. in proceedings of the 7th acm conference on embedded networked sensor systems. sensys ’09, pp. 225–238. acm, new york, ny, usa, 2009. doi:10.1145/1644038.1644061 [ma13] a. mouradian, i. augé-blum. formal verification of real-time wireless sensor networks protocols with realistic radio links. in proceedings of the 21st international conference on real-time networks and systems. pp. 213–222. 2013. [mem18] memsic, inc. micaz wireless measurement system. http://www.memsic.com/ userfiles/files/datasheets/wsn/micaz_datasheet-t.pdf, 2018. last accessed 6/4/18. [mrj+07] s. a. munir, b. ren, w. jiao, b. wang, d. xie, j. ma. mobile wireless sensor network: architecture and enabling technologies for ubiquitous computing. in proceedings of the 21st international conference on advanced information networking and applications workshops. ainaw ’07 2, pp. 113–120. ieee, washington, dc, usa, 2007. doi:10.1109/ainaw.2007.257 [msha17] m. mohsin, m. sardar, o. hasan, z. anwar. iotriskanalyzer: a probabilistic model checking based framework for formal risk analytics of the internet of things. ieee access 5:5494–5505, 2017. [nln+16] k. nahrstedt, h. li, p. nguyen, s. chang, l. h. vu. internet of mobile things: mobility-driven challenges, designs and implementations. in first ieee international conference on internet-of-things design and implementation, iotdi 2016, berlin, germany, april 4-8, 2016. pp. 25–36. 2016. doi:10.1109/iotdi.2015.41 [omr15] t. ojha, s. misra, n. s. raghuwanshi. wireless sensor networks for agriculture: the state-of-the-art in practice and future challenges. computers and electronics in agriculture 118:66–84, 2015. [par18] d. parker. prism 4.4 manual. department of computer science, university of oxford, april 2018. http://www.prismmodelchecker.org/manual/main/welcome. last accessed 18/4/18. [psh99] h. pfeifer, d. schwier, f. w. von henke. formal verification for time-triggered clock synchronization. in 7th ifip international working conference on dependable computing for critical applications (dcca-7). pp. 207–226. ieee, 1999. avocs 2018 18 / 19 http://dx.doi.org/10.1145/1644038.1644061 http://www.memsic.com/userfiles/files/datasheets/wsn/micaz_datasheet-t.pdf http://www.memsic.com/userfiles/files/datasheets/wsn/micaz_datasheet-t.pdf http://dx.doi.org/10.1109/ainaw.2007.257 http://dx.doi.org/10.1109/iotdi.2015.41 http://www.prismmodelchecker.org/manual/main/welcome eceasst [tal07] n. n. taleb. the black swan: the impact of the highly improbable. penguin books, 2007. [tcc09] l. tobarra, d. cazorla, f. cuartero. security in wireless sensor networks: a formal approach. in from problem toward solution: wireless sensor networks security. chapter 8, pp. 145–164. nova, 2009. [uais14] a. ur-rehman, a. z. abbasi, n. islam, z. a. shaikh. a review of wireless sensors and networks’ applications in agriculture. computer standards & interfaces 36(2):263–270, 2014. [umw13] m. usman, v. muthukkumarasamy, x.-w. wu. formal verification of mobile agent based anomaly detection in wireless sensor networks. in 8th ieee workshop on network security. pp. 1001–1009. ieee, 2013. doi:10.1109/lcnw.2013.6758544 [wtp+05] g. werner-allen, g. tewari, a. patel, m. welsh, r. nagpal. firefly-inspired sensor network synchronicity with realistic radio effects. in proceedings of the 3rd international conference on embedded networked sensor systems. pp. 142–153. 2005. [yin14] yinbiao, s. et al. internet of things: wireless sensor networks. international electrotechnical commission white paper, july 2014. http://www.iec.ch/whitepaper/ pdf/iecwp-internetofthings-lr-en.pdf. last accessed 23/5/18. 19 / 19 volume 076 (2019) http://dx.doi.org/10.1109/lcnw.2013.6758544 http://www.iec.ch/whitepaper/pdf/iecwp-internetofthings-lr-en.pdf http://www.iec.ch/whitepaper/pdf/iecwp-internetofthings-lr-en.pdf introduction formal verification and probabilistic model checking in prism related work modelling a wsn protocol in prism the firefly-gossip (figo) protocol a prism model of figo formal verification of figo using prism increasing the model's accuracy gossip and synchronisation environmental effects on hardware conclusions and future work type checking c++ template instantiation by graph programs electronic communications of the easst volume 10 (2008) proceedings of the seventh international workshop on graph transformation and visual modeling techniques (gt-vmt 2008) type checking c++ template instantiation by graph programs karl azab and karl-heinz pennemann 14 pages guest editors: claudia ermel, reiko heckel, juan de lara managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst type checking c++ template instantiation by graph programs karl azab and karl-heinz pennemann azab@informatik.uni-oldenburg.de, pennemann@informatik.uni-oldenburg.de carl v. ossietzky universität oldenburg, germany abstract: templates are a language feature of c++ and can be used for metaprogramming. the metaprogram is executed by the compiler and outputs source code which is then compiled. templates are widely used in software libraries but few tools exist for programmers developing template code. in particular, error messages are often cryptic. during template instantiation, a compiler looks up names that depend on a template’s formal parameters. we use graphs to represent the relevant parts of the source code and a graph program for the name lookup and type checking for expressions involving such names. this technique provides compiler writers with a visual way of writing algorithms that generate error messages and forms the basis for a visual inspection of type problems and suggested remedies for the programmer. our graph program terminates and emits correct error messages. keywords: graph programs, type checking, c++ 1 introduction templates are a feature of the c++ programming language for generic programming, i.e. programmed code generation. generic source code is written by omitting the specific data types of variables and instead supplying those as parameters (parameterized types). a parameterized type and variable of that type can be used as any other type or variable, e.g. the type name can be used to resolve names and the variable’s members can be accessed. this way, templates separate types from algorithms in design, and combines them into new class-types and functions at compile time. compared to non-template code which uses a generic type like void *, an immediate advantage from templates is improved static type checking. templates are used extensively in the standard template library and boost libraries [jos99, ag04]. they have also found use in performance critical domains, such as scientific computing and embedded systems [vel98, str04]. an introduction to templates can be found in e.g. [str00]. a class type or function containing generic source code is called a template definition. a list of type parameters for a particular template definition is called a declaration. for each unique declaration, the template instantiation mechanism generates a specialization of that template definition. a specialization is a copy of the definition where the parameterized types are replaced by the declaration’s actual type parameters. non-types, i.e. constants, are allowed as template parameters, allowing e.g. array sizes to be set at compile time. templates form a computationally complete metalanguage [ce00], a sub-language of c++ executed during compilation. consider the following example: a parameterized type is used to resolve the name size in the template definition in figure 1. the first specialization is for the declaration icon<char> and will not compile since the provided type char has no field named size and can therefore 1 / 14 volume 10 (2008) mailto:azab@informatik.uni-oldenburg.de mailto:pennemann@informatik.uni-oldenburg.de type checking c++ template instantiation by graph programs not be used for the expression defining the array size. for the second specialization, if the type resolution<128> contains a static field named size of an unsigned integer type, then the second specialization will compile. t e m p l a t e<t y p e n a me resolution> s t r u c t i c o n { p i x e l m i c o n [ resolution : : s i z e ] [ resolution : : s i z e ] ; /∗ . . . ∗/ }; template definition i c o n<c h a r> w r o n g d e c l a r a t i o n ; i c o n<r e s o l u t i o n <128>> c o r r e c t d e c l a r a t i o n ; declaration metaprogram s t r u c t i c o n<c h a r> { p i x e l m i c o n [ c h a r : : s i z e ] [ c h a r : : s i z e ] ; /∗ . . . ∗/ }; s t r u c t i c o n<r e s o l u t i o n <128>> { p i x e l m i c o n [ r e s o l u t i o n <128>:: s i z e ] [ r e s o l u t i o n <128>:: s i z e ] ; /∗ . . . ∗/ }; specializations t em plate instantiation compiler figure 1: c++ template instantiation. even though templates are a useful technique, they can be complex and difficult to read and write. in particular, error messages are often cryptic. this has led to the development of methods and tools to analyze template code. the usage of specializations can be analyzed by debuggers, software patterns like tracers [vj02], and tools like tuanalyzer [gpg04]. for the metaprogram itself, research is being done on a debugging framework templight [pms06]. to improve error messages, we suggest modeling definitions and declarations by graphs, while name lookup and type checking of such graphs is made by graph programs that emit error messages as graphs instead of text. graphs allow an abstract and visual representation of all necessary information, while graph programs provide an intuitive way of writing programs that detect problems and suggests remedies. in combination with presentation techniques such as focusing (on relevant parts) and hierarchical depiction, we believe that our model is usable as a basis for a visual inspection of type problems and suggested remedies. graph transformation systems is a well investigated area in theoretical computer science. an overview on the theory and applications is given in the book fundamentals of algebraic graph transformation [eept06]. graph transformation systems rewrite graphs with (graph transformation) rules. a rule describes a leftand right-hand side. a transformation step is done by matching the left-hand side to a subgraph of the considered graph and modifying that subgraph according to the difference of the leftand right-hand side. graph programs [hp01, ps04] provide a computationally complete programming language based on graph transformations. graph conditions [hp05] can be used to express properties of graphs by demanding or forbidding the existence of specific structures. in a similar way, graph conditions can limit the applicability of rules in a graph program, by making demands on elements local to the subgraph matched by a rule. in this paper, we use graphs to represent the template source code necessary for name lookup and type checking during template instantiation. we refer to those graphs as source-code graphs. proc. gt-vmt 2008 2 / 14 eceasst a graph program ttc (template-type checker) looks up dependent names and detects type clashes in expressions for a subset of the c++ template features. ttc attempts to solve type clashes by implicit type casts. if such a cast loses precision, a warning message is generated. if no appropriate cast is found, an error message is generated, indicating the location of the error and suggesting a remedy for the programmer. ttc outputs a message graph, where errors and warnings are embedded. the message graph is interpreted by the programmer with the help of graph conditions. graph conditions detect warning and error messages in graphs and when an error is present, they can determine for which declarations a definition can successfully be instantiated. figure 2 gives an overview. graph program ttc source-code-graph transformer programmer write / update interpretation with graph conditions message graph source-code graph source code figure 2: ttc type checks graphs and outputs error messages. the paper is structured as follows. graph programs are introduced in section 2. section 3 informally describes how c++ source code is transformed into source-code graphs and defines type safety for graphs. in section 4 we present the graph program ttc for transforming a sourcecode graph into a message graph. in section 5 we give proof ideas for how to show that ttc terminates and that the error messages generated by it correctly indicate that the input is not type safe. we conclude our results in section 6. a long version of this paper, with complete proofs and more examples, is available as a technical report, see [ap07]. 2 graph programs in this section, we review conditions, rules, and programs, in the sense of [hp05] and [hp01]. in the following, we consider the category of directed labeled graphs with all injective graph morphisms. labels distinguish different types of nodes and edges and directions model relationships between nodes. we use the standard definition of labeled graphs and labeled graph morphisms, see [eept06] or [ap07] for details. for expressing properties on graphs we use so-called graph conditions. the definition is based on graph morphisms. definition 1 (graph conditions) a graph condition over an object p is of the form ∃a or ∃(a, c), where a : p → c is a morphism and c is a condition over c. moreover, boolean formulas over conditions (over p) are conditions (over p). a morphism p : p → g satisfies a condition ∃a (∃(a, c)) over p if there exists an injective morphism q : c → g with q◦a = p (satisfying c). an object g satisfies a condition ∃a (∃(a, c)) if all injective morphisms p : p → g satisfy the 3 / 14 volume 10 (2008) type checking c++ template instantiation by graph programs condition. the satisfaction of conditions over p by objects or morphisms with domain p is extended to boolean formulas over conditions in the usual way. we write p |= c (g |= c) to denote that morphism p (object g) satisfies c. in the context of rules, conditions are called application conditions. we rewrite graphs with rules in the double-pushout approach [eept06]. application conditions specify the applicability of a rule by restricting the matching morphism. definition 2 (rules) a plain rule p = l ← k → r consists of two injective morphisms with a common domain k. l is the rule’s left-hand side, and r its right-hand side. a left application condition ac for p is a condition over l. a rule p̂ = p, ac consists of a plain rule p and an application condition ac for p. l k r g d h m m∗(1) (2) given a plain rule p and injective morphism k → d, a direct derivation consists of two pushouts (1) and (2) where the match m and comatch m∗ are required to be injective. we write a direct derivation g ⇒p,m,m∗ h . given a graph g together with an injective match m : l → g, the direct derivation g ⇒p,m,m∗ h can informally be described as: h is obtained by deleting the image m(l−k) from g and adding r−k. given a rule p̂ = p, ac and a morphism k → d, there is a direct derivation g ⇒p̂,m,m∗ h , if g ⇒p,m,m∗ h , and m |= ac. we now define graph programs as introduced in [hp01]. definition 3 (graph programs) every rule p is a (graph) program. every finite set s of programs is a program. if p and q are programs, then (p; q), p∗ and p↓ are programs. the semantics of a program p is a binary relation jpk⊆gc ×gc on graphs: (1) for every rule p, jpk = { g, h | g ⇒p h}. (2) for a finite set s of programs, js k = ∪p∈s jpk. (3) for programs p and q, j(p; q)k = jqk◦jpk, jp∗k = jpk∗ and jp↓k = { g, h ∈ jpk∗ |¬∃m. h, m ∈ jpk}. programs according to (1) are elementary and a program according to (2) describes the nondeterministic choice of a program. the program (p; q) is the sequential composition of p and q. p∗ is the reflexive, transitive closure of p, and p↓ the iteration of p as long as possible. programs of the form (p; (q; r)) and ((p; q); r) have the same semantics and are considered as equal; by convention, both can be written as p; q; r. we use p↓+ as a shortening of p; p↓. notation. when the label of an element is either a or b we use the notation a|b. l ⇒ r is used a short form of l ← k → r , where k consists of the elements common to l and r. for an application condition with morphism a : p → c, we omit p as it can be inferred from the left-hand side. we omit the application condition if it is satisfied by any match. to distinguish nodes with the same label, we sometimes add an identifier in the form of “label:id”. we use source-code fragments as identifiers and therefore print them in a fixed-width font. proc. gt-vmt 2008 4 / 14 eceasst 3 from source code to source-code graphs in this section, we introduce source-code graphs, the input for our type-checking program, and informally describe how source code is transformed into such graphs. a source-code graph is a graph representation of template definitions, declarations, and expression’s specializations. the type signature of every declared method, function, and operator in a template definition is represented in the graph by an overload forest, see below. expressions that involve parameterized types are represented in the graph by expression paths, explained shortly. for every declaration, the above mentioned graph representations are copied and the parameterized types are replaced by the actual types provided by the declaration. the basic elements of our source-code graphs are nodes for template definitions, declarations, data types, names, type signatures, and expression trees. for quick reference, node and edge labels together with a short explanation are listed below. note that a visualization of an edge as dashed or solid denotes a difference in labels. nodes edges d declaration p actual parameter = comparison e error message t data type c cast without precision loss ex (sub)expression t template definition d deduced type of expression ol overloaded operator w warning message p parameter op operator name pc cast with precision loss r return type r recovered comparison template definitions are represented by t-nodes. two declarations are equivalent if they are based on the same template and have equal lists of template parameters. each class of equivalent declarations are represented by a d-node and denotes a future specialization. each d-node has an incoming edge from the t-node representing the template definition the declaration means to instantiate. possible template parameters are represented by t-nodes. such parameters include classes, structures, fundamental types, and constants. operator-, functionand method names are represented by op-nodes. in [ap07] we show a graph program that generates source-code graphs. example 1 consider the source code with two class-type templates, line 1 and 18, in figure 3. the principal data type is the icon structure with a template parameter for its resolution type. the resolution structure has a constant template parameter for a (quadratic) resolution. for the two unique declarations in main, name lookup and type checking is needed for the expressions on lines 3, 20, 23, 24, 25, 40, and 41. in section 4 we will show how the graph program ttc reports the type clash in the expression on line 41. note that figure 4 shows the source-code graph of the source code example from figure 3 where the above mentioned lines are represented as expression paths. that source-code graph also shows the overload trees for the operators on line 3 and 22. for completeness, some overload paths representing used operations native to c++ are also included, e.g. comparison (<) of integers. we will now introduce some necessary graph-theoretic notions. in particular, we introduce and use expression paths and overload trees to define type safety for graphs. expression paths rep5 / 14 volume 10 (2008) type checking c++ template instantiation by graph programs 1 te m p la te < u n si g n e d sh o r t m y si z e> s tr u c t re s o lu ti o n { 3 c o n st s t a t ic u n si g n e d sh o r t s iz e = m y si z e ; } ; 5 s tr u c t p ix e l { 7 u n si g n e d c h a r re d , g re e n , b lu e ; 9 p ix e l o p e r a to r + ( p ix e l o v e rl a y ) { p ix e l r e s u lt ; 11 r e s u lt . re d = ( re d + o v e rl a y . re d ) / 2 ; r e s u lt . g re e n = ( g re e n + o v e rl a y . g re e n ) / 2 ; 13 r e s u lt . b lu e = ( b lu e + o v e rl a y . b lu e ) / 2 ; r e tu r n r e s u lt ; 15 } } ; 17 te m p la te < ty p e n a m e r e s o l u t io n> 19 s tr u c t ic o n { p ix e l m ic o n [r e s o l u t io n :: s iz e ][ r e s o l u t io n :: s iz e ] ; 21 ic o n < r e s o l u t io n> & o p e r a to r + = ( ic o n < r e s o l u t io n> & o v e rl a y ) { 23 fo r ( in t i = 0 ; i < r e s o l u t io n :: s iz e ; i + + ) { fo r ( in t j = 0 ; j < r e s o l u t io n :: s iz e ; j + + ) { 25 m ic o n [ i ] [ j ] = m ic o n [ i ][ j ] + o v e rl a y . m ic o n [ i ] [ j ] ; } 27 } r e tu r n ∗ th is ; 29 } } ; 31 # d e fi n e l a r g e r e s re s o lu ti o n < 12 8> 33 # d e fi n e s m a l l r e s re s o lu ti o n < 32 > 35 in t m a in ( ) { ic o n < l a r g e r e s> p ic ; 37 ic o n < l a r g e r e s> o v e rl a y ; ic o n < s m a l l r e s> lo w re s ; 39 p ic + = o v e rl a y ; 41 p ic + = lo w re s ; 43 r e tu r n 0 ; } figure 3: two class-type templates. resent the type information from an expression tree and are modeled by exand p-nodes. the root of the tree becomes an ex-node and has an incoming edge from the d-node that represents the specialization in which it will exist. each ex-node has an edge to the op-node denoting the operation’s name. we allow for operators with an arbitrary number of operands, so the children of the root in an expression tree are modeled by a path of p-nodes. if such a child is a subexpression, then the corresponding p-node has an edge to a new expression node. if it is not, then it denotes a type and its p-node has an edge to the t|d-node denoting that type. definition 4 (expression paths) given a graph g and a natural number i, an i-expression path in g is a path ex p0 . . . pi, where the head, ex, is an ex-node and p0, . . . , pi are p-nodes such that, from every node pk, 0 ≤ k < i, the only edge to another p-node is to pk+1. example 2 figure 5 shows (to the left) an expression tree denoting line 41 in example 1 together with its corresponding 2-expression path (to the right). we represent the type signatures of methods with overload forests, trees and paths. a method named method declared in class type class with n parameters is represented by a path of n + 2 ol-nodes. the head of that path has an edge to the op-node representing the name method proc. gt-vmt 2008 6 / 14 e c e a s s t op : < d : icon<resolution<32>> d : icon<resolution<128>> op : += op : = t : functions & operators t : unsigned short t : int t : bool t : unsigned long long op : ::size d : resolution<128> d : resolution<32> t : const 128 t : const 32 t : main d : main ex pp p ex pp p t : icon<resolution> ex pp p ex p ex pp p ex p ex pp p ex p ex pp p ex p t : resolution<size>ex pp p ex pp p op : < d : icon<resolution<32>> d : icon<resolution<128>> op : += op : = t : functions & operators t : unsigned short t : int t : bool t : unsigned long long op : ::size d : resolution<128> d : resolution<32> t : const 128 t : const 32 cpc c pc c pc cpc c pc c pc c pc c c c pc cc ol p ol p ol p ol r ol p ol p ol r ol p ol p ol p ol r ol p ol p ol p ol r ol p ol r ol p ol r f igure 4: a source-code graph split in tw o for sim pler representation, but note that the tw o subgraphs are not disjoint: the nodes w ith identical ids (see the center colum n) are identifi ed. 7 / 1 4 v o lu m e 1 0 (2 0 0 8 ) type checking c++ template instantiation by graph programs += pic low res op : += d : icon<resolution<128>> d : icon<resolution<32>> t : functions & operators ex p p p p p p figure 5: expression paths represent expression trees. and another edge to the t|d-node representing class. the ol-node at position k (2 ≤ k ≤ n + 1) in the path has an edge to the node denoting the type of the variable at parameter position k−1. the last ol-node in the path has an edge to the t|d-node denoting the return type of method. functions are modeled as methods but as declared in a special class with a name not allowed in the source language, e.g. functions & operators. operator overloading is modeled as functions. in the following operators, methods, and functions are collectively referred to as operators. definition 5 (overload forest) a graph g contains an overload forest iff all ol-nodes in g are part of exactly one overload tree and there exist no pair of overload trees with equivalent roots, see below. an overload tree is a maximal connected subgraph t in g, consisting of only olnodes. t is maximal in the sense that, if an ol-node in t has an edge to an ol-node, then that node is also in t . furthermore, t must have a tree structure, i.e. no cycles and every node has one parent, except for the root. for nodes in t the following holds for them in g: (1) each internal (leaf) node has exactly one p-edge (r-edge) to a t|d-node, one edge from its parent, and no other incoming edges. (2) the root of t has an additional edge to an op-node. (3) no two siblings have a p-edge to the same t|d-node. (4) every node has at most one child that is a leaf. requirements 3 and 4 are necessary to prevent ambiguous type signatures. two roots are equivalent iff there exists an op-node o and t|d-node t, such that both roots have edges to o and t. an i-overload path o0 . . . oi+1 is a path in t from the root to a leaf. the t|d-node to which an r-edge exist from oi+1 is called the return type of the i-overload path. example 3 the overload tree in figure 6 has two 2-overload paths, representing the type signatures of two overloaded operators. the tree represents the operator template on line 22 and the two paths are generated for the two declarations on line 40 and 41 in figure 3. op : += d : icon<resolution<128>> t : functions & operators d : icon<resolution<32>> ol p ol p ol p ol r ol p ol p ol r figure 6: two overload paths. proc. gt-vmt 2008 8 / 14 eceasst remark 1. the size of a source-code graph grows linearly with the size of the source code that would be output by the template instantiation mechanism in a c++ compiler. an expression or declared operator that exists in such source code is represented only by a single expression path or overload path, respectively. the main property in this paper is the one of type safety. a graph is type safe if for every expression path, there exists an overload path with the same type signature. this property corresponds to type safety for template instantiation in c++ programs, where every generated expression must be type checked against the existing and generated operators. definition 6 (type-safe graphs) a graph g is type safe iff it contains an overload forest and is i-type safe for all natural numbers i. g is i-type safe iff every i-expression path in g is type safe. an i-overload path o0 . . . oi+1 makes the i-expression path ex p0 . . . pi type safe iff: 1. there exists an op-node op and two edges: one from ex to op, the other from o0 to op. 2. for all k, where 0 ≤ k ≤ i, there exists a t|d-node t and two edges, one from ok to t and the other is from pk to either t or the head of a type safe j-expression path such that t is the deduced type of the that j-expression path. the deduced type of the i-expression path is the t|d-node with an incoming r-edge from oi+1. it is easy to see that no overload path from figure 6 makes the expression path in figure 5 type safe. 4 the type-checking program this section describes the graph program ttc which performs the name lookup and type checks source-code graphs. the section also shows how message graphs are interpreted with graph conditions. a schematic of how the subprograms of ttc interact is shown in figure 7. intuitively, ttc works as follows: the input is a source-code graph, each expression path is marked by markexpression, and compare moves this marker through the path until it reaches the tail or a type clash. at a type clash, recover either solves it or generates an error message. for markers at a tail, resolve finds the deduced type of the expression path (i.e. it resolves the type of a subexpression). this chain is then iterated as long as new work is made available by recover and resolve for compare. the yield of ttc is a message graph. programs and rules are described in more detail below. definition 7 (ttc) let the graph program ttc = markexpression↓;typecheck↓ with the subprograms: typecheck = compare↓+;recover↓;afterrecover↓;resolve↓ compare = { lookup,comparenext,findtype } recover = {cast,warning,error} resolve = { resolvesubexpression, resolveexpression1,resolveexpression2 } 9 / 14 volume 10 (2008) type checking c++ template instantiation by graph programs markexpression↓ compare↓+ recover↓ afterrecover↓ resolve↓ source-code graph message graph typecheck↓ figure 7: structure of ttc. markexpression↓ is executed before the actual type checking starts and markss each exnode with an =-edge to show where the evaluation starts. to avoid duplicate evaluation, a loop is also placed at the ex-node and checked for in the application condition. markexpression: op ex ⇒ op = ex , ¬∃ ( op ex ) compare↓+ consists of the rules lookup, comparenext, and findtype, see figure 8. they move the =-edge generated by markexpression through the expression path as long as a matching overload path can be found. the program halts when it completes this matching or encounters a problem: that no overload path makes this expression path type safe or that this node in the path depends on the deduced type of a subexpression. the rule lookup finds the overload tree for a marked expression’s name. comparenext matches the type signature of the expression path to an overload path parameter by parameter. the rule findtype is applied at the tail of the expression path and deduces the expression’s type via the matched overload path’s return type. the rule’s application condition makes sure that this is actually the tail of the expression path. lookup: ex p t|d olop p p = ⇒ ex p t|d olop p p = comparenext: ex p p t|d olol p p = ⇒ ex p p t|d olol p p = findtype: ex p t|d olol r= ⇒ ex p t|d olol d r , ¬∃     ex p p t|d olol r=     figure 8: rules in the program compare. proc. gt-vmt 2008 10 / 14 eceasst recover↓ consists of the rules figure 9 and tries to find alternative overload paths by implicit type casts. the rule cast finds a type cast that causes no loss of precision. warning works as cast but uses a cast with a possible loss of precision. for this we generate a warning, a w-node with three outgoing edges: the location of the problem, the original type, and the cast type. the application condition make sure that cast is preferred. the rule error is applied when there is no solution by implicit type casts. an error node is therefore generated. it has three outgoing edges, to the p-node where it occurred, the faulting type, and a suggested type. the application condition limits error from being applied where cast or warning could be applied instead. cast: p|ex ol|op p ol t|d t|d = p p c ⇒ p|ex ol|op p ol t|d t|d r p p c warning: p|ex ol|op p ol t|d t|d = p p pc ⇒ p|ex ol|op p ol t|d t|d r p p pc w from to ,¬∃          p|ex ol|op p ol t|d t|d = p p pc ol t|d p c          error: p|ex ol|op p ol t|d t|d = p p|r ⇒ p|ex ol|op p ol t|d t|d p p|r e on suggestion , ¬∃          p|ex ol|op p ol t|d t|d = p p|r ol t|d p c|pc          ∧¬∃        p|ex ol|op p ol t|d t|d = p p|r c|pc        figure 9: rules for the program recover. afterrecover↓ performs some cleanup work for recover↓, generated r-edges are reset to =-edges. afterrecover: ex|p r op|ol ⇒ ex|p = op|ol resolve↓ consists of three rules, as shown in figure 10: resolvesubexpression replaces a subexpression with its deduced return type. resolveexpression1 marks an expression as evaluated with a dashed edge. resolveexpression2 does the same in the special case when the return type is the same as the specialization in where the expression occurs. example 4 the graph in figure 11 is the yield of ttc applied to the overload tree from figure 6 and the expression path from figure 5. see [ap07] for more examples. 11 / 14 volume 10 (2008) type checking c++ template instantiation by graph programs resolvesubexpression: p t|d ex d ⇒ p t|d ex p resolveexpression1: d t|d ex d ⇒ d t|d ex resolveexpression2: d d ex ⇒ d ex figure 10: rules in resolve. op : += t : functions & operators d : icon<resolution<128>> d : icon<resolution<32>> ol pol ol ol ol ol ol p p r p p r ex p p p p p p e on suggestion = = = figure 11: portion of a message graph. remark 2. after the termination of ttc, graph conditions can help to interpret the message graph. a graph is an error (warning) graph iff it satisfies the condition ∃( /0 → e ) (∃( /0 → w )). a particular declaration can safely be instantiated if its corresponding d-node satisfies the condition ¬∃( d → d ex ). if that condition is not satisfied, then one of its expressions could not be resolved and the programmer must take appropriate actions. a message graph will contain all the detected errors for the corresponding source code. graph conditions can therefore help programmers to locate the areas of the graph that contain such errors. an implementation of this approach should be able to highlight the areas containing the errors. remark 3. the size of the resulting message graph will not grow more than linearly with the size of the corresponding source-code graph. this is so since every expression tree can at most be marked by one error message. for size of source-code graphs, see remark 1 5 correctness and termination we now define correctness with respect to errors and termination for graph programs. we give the ideas for proving that ttc terminates and is correct w.r.t. errors. definition 8 (correctness and completeness) a graph program p is correct with respect to errors if for every pair g, h ∈ jpk, h is an error graph implies g is not a type-safe sourcecode graph. if the converse of the implication holds, we say that p is complete w.r.t. errors. proc. gt-vmt 2008 12 / 14 eceasst theorem 1 (correctness) the graph program ttc is correct with respect to errors. proof idea. errors are only generated by recover and consumes an =-edge that should have been consumed by compare↓+, given that ttc were initially dealing with a type-safe sourcecode graph. a complete proof is given in [ap07]. fact 1. the graph program ttc is not complete with respect to errors. e.g. recover uses implicit type casts, and thereby avoids generating errors for some non-type-safe graphs. it has not yet been investigated whether or not other counterexamples exist. definition 9 (termination) termination of a graph program is defined inductively on the structure of programs: (1) every rule p is terminating. (2) for a finite set s of terminating programs, s is a terminating program. (3) for terminating programs p and q, (p; q) is terminating. moreover, p∗ and p↓ is terminating if for every graph g, there is no infinite chain of derivations g ⇒p g1 ⇒p . . . where ⇒p denotes the binary relation jpk on graphs. theorem 2 (termination) the graph program ttc is terminating. proof idea. compare is applied at least once for every iteration of typecheck and consumes solid edges that are not generated by the other subprograms. a complete proof is given in [ap07]. 6 conclusions we considered the template instantiation mechanism in c++ and showed how to write visual rules for type checking and error message generation. we informally described how source code was transformed into source-code graphs and defined type safety for graphs. we transformed source-code graphs into message graphs, a transformation given by the graph program ttc which type checked source-code graphs. the program automatically corrected some errors by implicit type casts. it emitted error messages for type clashes that it could not correct. proof ideas were given for termination and correctness w.r.t. errors. further topics include: 1. analysis of source-code graphs by generalized graph conditions. graph properties like “there exists a warning or error node” can be expressed by graph conditions in the sense of [hp05]. it would be interesting to generalize graph conditions so more complex graph properties like “the graph is type safe” becomes expressible. 2. debugging and a transformation from message graphs to source code. the error messages generated by ttc contained suggestions for remedies. in the double-pushout approach to graph transformation, a central property is the existence of an inverse rule, that when applied reverses the rewrite step of the rule [eept06]. in this way, the inverse rule allows for back tracking to a previous graph which can be manipulated to experiment with suggested remedies. the changes are logged in the output graph (message/change graph) and used by a source-code transformer to update the source code. 13 / 14 volume 10 (2008) type checking c++ template instantiation by graph programs 3. implementation of the approach. this would include a formalization of the transformation from source code to source-code graphs and an extension of the set of considered template features. acknowledgements: this work is supported by the german research foundation (dfg) under grant no. ha 2936/2 (development of correct graph transformation systems). we thank annegret habel for constructive suggestions that improved the paper. bibliography [ag04] d. abrahams, a. gurtovoy. c++ template metaprogramming: concepts, tools, and techniques from boost and beyond. addison-wesley professional, 2004. [ap07] k. azab, k.-h. pennemann. type checking c++ template instantiation (long version). technical report 04/07, university of oldenburg, 2007. available at http://formale-sprachen.informatik.uni-oldenburg.de/%7eskript/fs-pub/templates long.pdf. [ce00] k. czarnecki, u. w. eisenecker. generative programming: methods, tools, and applications. addison-wesley, 2000. [eept06] h. ehrig, k. ehrig, u. prange, g. taentzer. fundamentals of algebraic graph transformation. eatcs monographs of theoretical computer science. springer, 2006. [gpg04] t. gschwind, m. pinzger, h. gall. tuanalyzer–analyzing templates in c++ code. in proc. of wcre’04. pp. 48–57. ieee computer society, 2004. [hp01] a. habel, d. plump. computational completeness of programming languages based on graph transformation. lncs 2030, pp. 230–245. springer, 2001. [hp05] a. habel, k.-h. pennemann. nested constraints and application conditions for high-level structures. lncs 3393, pp. 293–308. springer, 2005. [jos99] n. m. josuttis. the c++ standard library: a tutorial and reference. addison-wesley, 1999. [pms06] z. porkoláb, j. mihalicza, á. sipos. debugging c++ template metaprograms. in proc. of gpce’06. pp. 255–264. acm, 2006. [ps04] d. plump, s. steinert. towards graph programs for graph algorithms. lncs 3256, pp. 128– 143. springer, 2004. [str00] b. stroustrup. the c++ programming language. addison-wesley, 3rd edition, 2000. [str04] b. stroustrup. abstraction and the c++ machine model. lncs 3605, pp. 1–13. springer, 2004. [vel98] t. l. veldhuizen. arrays in blitz++. lncs, pp. 223–230. springer, 1998. [vj02] d. vandevoorde, n. m. josuttis. c++ templates: the complete guide. addison-wesley, 2002. proc. gt-vmt 2008 14 / 14 http://formale-sprachen.informatik.uni-oldenburg.de/%7eskript/fs-pub/templates_long.pdf introduction graph programs from source code to source-code graphs the type-checking program correctness and termination conclusions observations for assertion-based scenarios in the context of model validation electronic communications of the easst volume 15 (2008) proceedings of the 8th international workshop on ocl concepts and tools (ocl 2008) at models 2008 observations for assertion-based scenarios in the context of model validation emine g. aydal, richard f. paige and jim woodcock 16 pages guest editors: jordi cabot, martin gogolla, pieter van gorp managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst observations for assertion-based scenarios in the context of model validation emine g. aydal, richard f. paige and jim woodcock aydal@cs.york.ac.uk, paige@cs.york.ac.uk, jim@cs.york.ac.uk department of computer science university of york, york, uk abstract: certain approaches to model-based testing focus on test case generation from assertions and invariants, e.g., written in the object constraint language. in such a setting, assertions and invariants must be validated. validation can be carried out via executing scenarios wherein system operations are applied to detect unsatisfied invariants or failed assertions. this paper aims to improve our understanding of how to write useful validation scenarios for assertions in ocl. to do so, we report on our experiences during the creation and execution of 237 scenarios for validating assertions for the mondex smart card application. we also describe key factors that must be considered in transforming scenarios into test cases. keywords: model-based testing, ocl, mondex, scenario validation 1 introduction in model-driven engineering (mde), models can be used for systematically deriving other artefacts needed within the engineering process, such as code and test cases. applying mde in practice requires well-defined modelling languages, scalable and practical tools for constructing and managing models, as well as means for validating models. in this paper, we analyse the applicability and suitability of using uml and the object constraint language (ocl) for model validation within the context of model-based testing (mbt). some of the research in this area focus on determining a set of test targets and translating them into abstract test cases [bgl+07, bl05]. these studies are valuable in introducing requirements traceability to mbt and finding test targets against these requirements, however, how model validation is achieved against these criteria is kept outside of the scope. the main focus, in this study is on validating models by constructing snapshots representing system states at a particular point in time with objects, attribute values, and links. there are other research studies based on the expressive power of snapshots such as [gbr03, gbr06, zg03]. the difference of this work is that we base our scenarios both on invariants and assertions of the system operations. therefore, the scenarios not only check whether there is a state where all the system invariants are satisfied, but they also check whether there are states that allow system operations to run. for instance, if one of the preconditions of an operation can never be satisfied, then either the operation is redundant or the operation is ill defined. the benefit of this approach in addition to model validation is that, potentially, the scenarios can also be used for abstract test case generation. 1 / 16 volume 15 (2008) mailto:aydal@cs.york.ac.uk mailto:paige@cs.york.ac.uk mailto:jim@cs.york.ac.uk scenario observations for model validation 1.1 mondex smart card application the software system used as the basis of this validation experiment was the mondex smart card application. mondex is a global electronic payment scheme that provides digital form of cash [cla97]. the card holds values in several different currencies and provides direct money transfer without signature, pin or transaction authorization between card holders [lim99]. mondex was the first case study carried out in the implementation of the grand challenge program that aims to populate a repository of formally specified and verified codes that are useful in practice and serve as examples for the future applications [jw07]. unlike other research studies based on the monograph outlined in [scw00], we follow a different path in the sense that we created the model of the system from the informal requirements detailed in [lim99]. in doing so, we covered some of the functional requirements omitted in [scw00] and in all the other studies that have been based on this monograph. in this experiment, we modelled the system using uml and ocl in use tool. use allows users to specify system models, invariants, and preand postconditions textually, and allows assertions to be checked. the class diagram of the mondex smart card application populated in this study is given in figure 1. the system has 30 invariants, the classes given in figure 1 have 31 operations and 197 assertions were written in order to cover these operations. these numbers exclude utility classes such as date and their associated operations. figure 1: class diagram mondex 1.2 test scenarios in uml specification environment (use) the use tool provides a multi-level platform where the model is defined in a .use file, the generation of an instance of the model is managed by an .assl file, the extra optional invariants are imposed in a .invs file and all these files as well as other use-related commands are executed by calling .cmd files in command prompt of the tool. an example operation definition written in .use file for creating pending logs in mondex smart card application is given in table 1. proc. ocl 2008 2 / 16 eceasst context mondexpurse::creatependinglog( p pendinglog: pendinglog ) : boolean pre creatependinglogpre1: self.lockingstate = ’unlocked’ pre creatependinglogpre2: p pendinglog.isdefined() pre creatependinglogpre3: pendinglog->isempty() post creatependinglogpost1: pendinglog->includes(p pendinglog) post creatependinglogpost2: pendinglog->size() = 1 post creatependinglogpost3: self.lockingstate = self.lockingstate@pre table 1: creatependinglog() once the operation definition is defined, there are other tasks to be completed before writing a scenario. an instance of a system can be generated in use by using assl (a snapshot and sequence language) [gbr03, gbr06] once the instance is generated, the output can be written into a command (.cmd) file and executed in the command prompt of the tool. depending on the strategy to be followed, the system and its environment may need adjustment. this is also called preamble in model-based testing terminology, i.e. bringing the system into a specific state before executing an operation. the tasks carried out as part of an operation are written in a .cmd file. finally, by using openter and opexit features of use, the preconditions and postconditions of the operation are checked. the following is an example scenario written for creatependinglog operation. open c:/<root>/mondex.use gen start -b c:/<root>mondexinstance.assl pursegenerator{2} gen result read c:/<root>/snapshot.cmd read c:/<root>/pre_creatependinglog.cmd !openter pendinglog1 creatependinglog(pendinglog1) read c:/<root>/creatependinglog.cmd !opexit true table 2 gives the structure of these scenarios with associated tasks. in step 1, a basic, valid, stable instance of the system model is created. this step is ideally the same for all the scenarios. step 2 prepares the system for the operation under investigation. at this point, we determine the set of variables that the operation reads/writes from/to; this set is called frame variable set (fvs). observation 6 in section 2.3 explains this concept in detail. during step 3, the operation call is put into the call stack and the preconditions of the operation are checked. step 4 is where the actions of the operation are carried out. if the value of an attribute in the set of frame variables should not change, it is also important to explicitly state this at this step. note that if we aim to write a scenario that creates a conflict with one of the postconditions of the operation under investigation, it is step 4 where we need to produce this conflict. in the final step, we exit the operation and the postconditions are checked if the preconditions were satisfied. 3 / 16 volume 15 (2008) scenario observations for model validation step no. step description 1 initial loading of object model an instance of the model is loaded at this stage. 2 environment setting (preamble) determine frame variables creation of objects setting attribute values 3 access to operation the preconditions are checked. 4 modification of the model creation/deletion of objects setting/modifying attribute values coverage of frame variables 5 exit from the operation the postconditions are checked. table 2: the steps of a basic scenario 1.3 validation of assertions and generation of abstract test cases validation is a crucial phase of the software development process in demonstrating that the system under investigation meets its requirements. assertion validation ensures that invariants, preand postconditions of operations are expressed as intended. there are three ways of accomplishing this: 1. checking that the preconditions, invariants and postconditions of the operation under investigation (opui) are satisfied at least once. 2. ensuring that the opui does not execute when at least one of the preconditions fails. 3. checking that at least one of the postconditions of the opui fails when a mutant is inserted into the operation, provided that the preconditions and the state invariants are satisfied. the first approach guarantees that the postconditions of the operation are not too strong and that there is at least one case where preconditions and state invariants are satisfied. the second approach checks the response of the system when the preconditions are not satisfied. finally, the last approach verifies that the harmful modifications to the operation are caught by the postconditions and that the postconditions are not too weak. note that the third approach needs mutants to be generated and inserted into the system, and therefore constitutes white-box testing. we included this for the sake of completeness in showing how the assertion can be used in different ways for scenario generation, however, in the context of model-based testing, this approach cannot be used. table 3 summarises the statistics for all the scenarios we created in our experiment. 237 scenarios were created of which 32 presented the ideal cases, 94 violated a precondition, 104 violated one postcondition and 7 violated more than one postcondition. of these 237 scenarios, 64 also contradicted an invariant. this means that if this scenario were to lead to an error in the system, invariant failure would provide an alert as well. in addition to these, 98 scenarios require creation/deletion of objects and 71 scenarios are involved in nested calls. the importance of these numbers is made clear in the rest of this paper. proc. ocl 2008 4 / 16 eceasst scenarios total number of scenarios 237 scenario presenting ideal case 32 scenario conflicting preconditions 94 scenario conflicting postconditions 111 invariant conflict scenarios conflicting one invariant 55 scenarios conflicting two invariants 9 scenarios satisfying all the invariants 170 state change scenarios that requires state change within operation 40 scenarios that requires state change outside the op. 23 scenarios that requires no state change 171 other information scenarios requiring object deletion/creation 98 scenarios with nested calls 71 scenarios that changes the value of at least one attribute 181 table 3: statistics for the scenarios created in this study 1.4 contribution the discipline of writing scenarios that allow developers both to validate models and generate test cases from the scenarios is not yet fully understood. thus, the main aim of this work is to improve the understanding of how to validate assertions by reporting on the results of a largescale experiment in which validation scenarios were written and used to check assertions. by doing so, we provide a structured set of observations from this set of experiments that provide advice on how to go about validating assertions in a scenario-based way and to use them as a basis for test case generation. one of the novelties of this work is that it explains the issues encountered from three different perspectives: tool-related, scenario-related and assertion-related. this classification not only facilitates the understanding the root cause of certain problems, but also foresee the benefits/disadvantagesof the techniques applied. one of the guidelines followed during the study was to keep the scenarios as general enough as possible, so that they can be applied to different instances of the model with little or no modification. the rationale behind this is to ease the transformation of these scenarios to test cases in a systematic manner with a good degree of automation. this principle has pointed out many different issues that need consideration, but have been overlooked, such as frame variables, test case sequencing, dependency of scenarios, etc. the rest of the paper explains our observations in a structured format. 5 / 16 volume 15 (2008) scenario observations for model validation 2 observations on scenario creation in the context of mbt in this section, we report our experiences in creating the scenarios described above. we structure our observations into three categories: tool-related observations, assertion-related observations, general scenario formation-related observations. the tool-related observations aim to reveal the drawbacks and advantages of using the use tool. the assertion-related observations address the usage of the pre/postconditions as well as the invariants in the process of scenario creation. the observations in this category focus on how to detect some of the errors related to assertions and how to write better assertions. the final category outlines the observations regarding the scenarios that either show the ideal execution of an operation or violate a pre/postcondition. each observation is described using the template given below. short description of the observation aim: the aim of the action that produced the observation. context: in which environment the observation is made issue encountered: the unexpected behaviour detailed observation: a clear explanation of the issue conclusion: the effects of the problem. extra notes: extra comments about the issue and its impacts 2.1 tool-related observations in this section, tool-related observations during the validation of assertions are discussed in detail. observation 1: how is the call stack managed in use? aim: the aim is to find a scenario that contradicts a precondition of an operation. context: in use, the preconditions of an operation are checked when the command for the operation call -!openteris executed. if the preconditions of the operation are satisfied, then the operation is put into the call stack. if the operation is defined by ocl expressions, these are executed and the expected value is returned when the !opexit command is run. the postconditions of the operation are also verified at this point. issue encountered: if the operation is not defined by using ocl, i.e. when the operation changes the value of some variables, creates/deletes some objects, etc., then these changes are performed via the commands introduced in use between the !openter and !opexit commands. !set, !create, !destroy are examples of such commands. the problematic situation occurs if the operation is defined by use commands and the preconditions of the operation are not satisfied. in such a case, the operation is not put into the call stack due to precondition failure, however, the commands after the operation call are still executed, i.e. the changes that the operation would do are still performed. then, when the !opexit command is called, the tool states that the call stack is empty and do not check the postconditions assuming that the operation is not executed anyway. the concern here is that the system may be in a different state than it was in the beginning although the operation is not executed according proc. ocl 2008 6 / 16 eceasst to the tool. observations: one of the operations in which we observed this behaviour of the tool is given in table 4. this operation is called when the personal code is entered more than the number of times stated by the attribute personalcodeattempts. the attribute numberofincorrectentries counts the number of incorrect entries. context mondexpurse::changethestatetolockedout() : boolean pre changethestatetolockedoutpre1: self.lockingstate = ’unlocked’ or self.lockingstate = ’locked’ pre changethestatetolockedoutpre2: personalcodeattempts<= numberofincorrectentries post changethestatetolockedoutpost1: self.lockingstate = ’lockedout’ post changethestatetolockedoutpost2: personalcodeattempts=personalcodeattempts@pre table 4: changethestatetolockedout the following generates a contradiction in the second precondition. <changethestatetolockedout_pre2.cmd> read basemodel.cmd !set p1.lockingstate := ’unlocked’ --setting the frame var. !set p1._numberofincorrectentries := 1 --conflict creation !set p1.personalcodeattempts := 4 !openter p1 changethestatetolockedout() --enter the operation !set p1.lockingstate := ’lockedout’ --modification of frame var. !opexit true --exit the operation conclusion: in this example, we managed to reach our aim in the sense that we found a scenario where the second precondition fails, however, the system is now in a different state than it was and this may cause unexpected results when the next scenario is run. extra notes: the workaround we followed for this problem is that we executed the commands that neutralise the modifications occurred during the execution of such a scenario. observation 7 explains the process of neutralisation in more detail. observation 2: reducing the dependency of scenarios on the model instance aim: in the creation of the scenarios, one of the rules we need to follow is that we need to be reasonably independent of the instance of the model we created. what we mean by this is that the instance of the model created for the test environment is just one of many possible, and the objects that exist in one instance may not exist in another. in other words, the number of objects, the number of links, the name of the objects, and the attribute values may all be different. since the scenarios in our approach are based on the instance of the model, it is true that they are somewhat dependent on the instance chosen. having said that, the level of this dependency can be decreased by using more general statements in the scenarios. context: if deleting an object is one of the operations handled by an operation, some of the postconditions of this operation must make sure that the deletion really occurs. the observation 7 / 16 volume 15 (2008) scenario observations for model validation that is explained in this section is made whilst trying to find a scenario that is related to one of these postconditions and that deletes an object. issue encountered: some of our attempts to write more general statements in our scenarios have caused exceptions in use especially during the deletion of an object. observations: the enquirecurrencyinfo() function defined in our system provides information about a given currency. one of the postconditions of this function states that that the operation should not modify currency objects. in our approach, we created scenarios that present the ideal functioning of an operation and that also generates conflicts with each different pre/postcondition of the operation. the scenario that targets the postcondition above deletes a currency object and observes the reaction of the system. our first attempt to delete the currency object had the following code: let objectdel : mondexcurrency = purse1.avcurrencies-> select(isocurrencycode =’gbp’)->assequence()->first() !destroy objectdel however, the objectdel object behaves like a pointer to the actual object and therefore the object requested to be deleted is not removed and we receive a runtime exception unbound variable error. our next attempt was to define the object to be deleted as defined in objectdel: !destroy purse1.avcurrencies-> select(isocurrencycode = ’gbp’)->assequence()->first() this expression states that the the first element in the set of currencies in purse1 whose isocurrencycode is gbp should be deleted. this command deletes the first object in the selected collection. however, when we execute the !opexit command to exit the operation, we received a null pointer exception and the postconditions are not checked. our final attempt was to execute the delete command by calling object’s name: !destroy c_gbp1 after this, the object is deleted, the postconditions are evaluated and no exception is thrown. conclusion: when we examine the scenario statements written above, we see that the first two need the same amount of information in order to proceed, i.e. the currency that is under investigation. this information is passed to the function as a parameter anyway and therefore the user does not necessarily have to know a lot about the current instance of the model. the third one, on the other hand, needs knowledge of an object name which makes the scenario extremely dependent on the selected instance of the model. we tried these options in different scenarios and noticed that in some situations the second and the more general option works, however we were unable to find a pattern that explains the rationale behind this varying behaviour. the main message we here is that the tool must allow the user to write reasonably general, ocl-based scenarios, and the rules related to creation and deletion of the objects must be clear. 2.2 assertion-related observations the observations made during the creation of pre/post-conditions are given in this section. the situations where a scenario highlights the importance of a change in the assertions and in invariants are also presented. proc. ocl 2008 8 / 16 eceasst observation 3: writing an invariant instead of an assertion tuple aim: some of the most important characteristics our model should possess are clarity, consistency and simplicity. the elements forming the model such as diagrams and ocl expressions must also have these characteristics and should not introduce further complexity. context: whilst creating the pre and postconditions of the operations, it is important to understand the context of the operation and the scope of the frame variables. in certain cases, the change in one variable may affect another variable and these changes must be presented in postconditions. issue encountered: for two conditions x and y, if y must be true each time x holds, then this relationship must be shown each time x appears in a pre/postcondition. if x is a widely-used variable, i.e. is an element of frame variable set of many operations, then y must be repeated as many times as x appears. observations: we first noticed this whilst writing scenarios for changethestatetounlocked() function. initially, the definition of the function had the pre/postconditions shown in table 5. context mondexpurse::changethestatetounlocked() : boolean pre changethestatetounlockedpre1: lockingstate = ’nonlocking’ or lockingstate = ’locked’ pre changethestatetounlockedpre2: lockingstate = ’nonlocking’ implies (not personalcode.isdefined or personalcode = 0) post changethestatetounlockedpost1: lockingstate = ’unlocked’ post changethestatetounlockedpost2: personalcode.isdefined and personalcode <> 0 table 5: changethestatetounlocked first version we then realised that each time the state is in nonlocking state or changes from nonlocking state to any other possible state, we have to check the value of personal code even if the operation itself does not necessarily state a change in the value of personal code. this not only creates many duplicates of the same requirement, but also creates a hole in the system when it is forgotten. to avoid this, we created the following invariant: inv ipercode_nonlocking: (lockingstate = ’nonlocking’ implies (personalcode = 0 or not personalcode.isdefined())) and (lockingstate <> ’nonlocking’ implies (personalcode <> 0 and personalcode.isdefined())) this invariant states that if the system is in nonlocking state, the personal code is either zero or not defined. when the system is in any other state, the personal code has a value other than zero. after the introduction of this invariant, the pre/postcondition definition of the above function became as given in table 6. conclusion: if there is a repetition of pre/postconditions, it is worth looking at the relationship between the variables under investigation to see whether such a relationship would hold for all cases. the example given above is a case where we noticed the need for an invariant due to this repetition. 9 / 16 volume 15 (2008) scenario observations for model validation context mondexpurse::changethestatetounlocked() : boolean pre changethestatetounlockedpre1: lockingstate=’nonlocking’ or lockingstate =’locked’ post changethestatetounlockedpost1: lockingstate = ’unlocked’ table 6: changethestatetounlocked second version observation 4: incorrect invariant detection aim: the rationale behind creating scenarios that violates a pre/postcondition is to be able to construct abstract test cases to challenge our system. one of the advantages of the process of scenario-creation is that it also allows us to test a model for correctness and consistency. context: it is possible to find a set of scenarios that violates an assertion of an operation. the choice of scenario to be used can be done simply by the user or a set of criteria can be defined and the scenarios may be expected to comply with some/all of these criteria. issue encountered: sometimes, we observed that the set of scenarios that aim to contradict a pre/postcondition of an operation must also violate an invariant. in one of these situations, the invariant that the scenario was supposed to violate seemed to be satisfied. after close examination, we found out that the predicate in the invariant was incorrectly written. observations: one of the preconditions of the function readpaymentlogs() is that the system must either be in state unlocked or locked. there are two possible scenarios that would violate this precondition: the scenario that sets the system state to nonlocking and the scenario that sets it to lockedout. the invariant that deals with the cases where the state is nonlocking -ipercode nonlockingwas given in observation 3. the invariant that checks the suitability of system variables for the lockedout state -ilockedoutstateis given below. inv ilockedoutstate : lockingstate = ’lockedout’ implies (purseexhaustionflag = true or _numberofincorrectentries >= personalcodeattempts) both of these invariants require that some other variables of the system are set to certain values. in the scenarios created to contradict the precondition about the state of the system, we first followed the approach where the system state is set to lockedout or nonlocking and all the relevant variables such as personal code, purseexhaustionflag, etc. are excluded. this is why we expected that each scenario would conflict at least with the relevant invariant. however, to our surprise, the invariant ipercode nonlocking was satisfied in both cases. when we analysed the problem further, we realised that the invariant in the form of (p implies q) and (not p implies r) was written as (p implies q) and not p implies r which was interpreted as ((p implies q) and not p) implies r by the tool and therefore the invariant was satisfied although it should not have been. the issue was resolved when the fault in the invariant was corrected. conclusion: the example above presents a case where a scenario written with the aim of violating a pre/postcondition also detects a fault in the model itself. this means that the approach supports the process of correctness and consistency check for the model itself. further research is needed to assess to what degree this support extends. extra notes: note that, whilst creating the scenarios discussed above, the variables that the invariants are associated to are excluded, i.e. the scenario did not deal with the setting of variables such as personalcode, purseexhhaustionflag, etc. as an alternative, we also created scenarproc. ocl 2008 10 / 16 eceasst ios that add these variables to the fvs of the operation and that handles the correct setting of these extra variables for the given scenario. by doing this, we bring the system in a different stable state and then observe the cases where the precondition under investigation fails, but all the invariants are satisfied. this issue is briefly discussed again in observation 6 in section 2.3. observation 5: overlapping postconditions aim: in our experiments, we created the scenarios that ideally violated only one assertion at a time. we believed that the possibility of finding an error increases when each scenario dealt with a different assertion, since the domain of a possible error differs when the subject matter is different in a scenario. context: in ocl, there may be several ways of navigating to reach an object. some operations may create even more links and increase the number of navigation possibilities. if such operations have postconditions, the postconditions may also use different navigation routes. issue encountered: we noticed that when postconditions use the links created by the operation under investigation, they happen to assume that the links are created successfully. in other words, they also check the creation of links in addition to their main goal. this conflicts with our initial aim. observations: following assertions were two of the postconditions initially written for the function sendvalue(). post sendvaluepost1 : pockets->select(default = true and currency.isocurrencycode = p_isocurrcode)->size() = 1 post sendvaluepost4 : pockets->select(default = true)-> assequence()->first().value@pre pockets->select(default = true)-> assequence()->first().value = p_paymentvalue sendvaluepost1 states that the pocket that carries the requested currency given by the parameter p isocurrcode is the default. this postcondition ensures that the pocket that carries the currency in which the amount will be transferred is set as the default pocket. the postcondition sendvaluepost4 ensures that the value held by the pocket from which the amount is transferred is decreased by p paymentvalue. both conditions seem to match their definitions as given above. however, when analysed further, we noticed that when the first condition fails, both postconditions fail even if the money is transferred in correct currency. this is because sendvaluepost4 assumes that the default pocket is set properly and therefore tries to reach the object through a newly defined link. for instance, if the transfer is made in gbp and the pocket that holds gbp is not set as default, but transfer is made in gbp successfully, then we expect sendvaluepost1 to fail and post sendvaluepost4 to pass. we observed that both of the above postconditions fail for such a scenario. this may seem as an advantage at first sight, but when we try to detect the root cause of such a failure, it is more difficult to find and we are unable to say which assertion is the main target of such a scenario. the solution to this is to separate the concerns as much as possible for each pre / postcondition. this also requires withdrawing the sequential way of thinking to reach the outcome of an operation. in the above example, one may think that the first action is the change of the default pocket and then the value transfer occurs, so the rationale behind writing a postcondition like 11 / 16 volume 15 (2008) scenario observations for model validation sendvaluepost4 may be the result of such reasoning. to overcome this issue, we changed the sendvaluepost4 as below: post sendvaluepost4 : pockets->select(currency.isocurrencycode@pre = p_isocurrcode) ->assequence()->first().value@pre pockets->select(currency.isocurrencycode = p_isocurrcode) ->assequence()->first().value = p_paymentvalue this new version of sendvaluepost subtracts the previous and current values of the pocket that holds the currency in which the payment is made. in this version, there is no assumption about any of the actions that the function under investigation must take prior to money transfer. as a result of this, the failure in sendvaluepost1 does not necessarily mean a failure in sendvaluepost4. conclusion: there are several conclusions we can reach by looking at this example. the first one is the importance of demonstrating the independent effect of each pre/postcondition. this is similar to that of modified condition/decision coverage (mc/dc), which is a structural coverage criterion that requires that the effect of all conditions in a program are demonstrated and that there is no condition that does not affect the outcome of a decision [mj94]. analogous to this criterion, in our approach, we create scenarios that would then form the abstract test cases based on the pre/postconditions and therefore, it is important to be able to see the independent effect of each unit. by showing the independent effect, we not only avoid the possibility of an assertion being masked by another assertion, but also let each assertion contribute to the final test suite. in addition to this, separation of concerns also makes root cause analysis easier when a fault is detected. in other words, if an error occurs during the execution of a test case that is based on a scenario created by using our approach, we can backtrack and reason about the error by looking at the part of the program that deals with the postcondition under investigation. 2.3 general scenario formation-related observations scenarios form the core of our approach and therefore it is crucial to investigate carefully the way they are written, how well they achieve the aim of contradicting an assertion, and the factors that affect their execution. this section presents the issues encountered during the scenario creation and execution. observation 6: frame variables and treatments to frame variables aim: in most cases, the operations of a system do not have to read/write from/to all the attributes of the model. the set of attributes that an operation is in contact with is called frame variables [kas06]. the frame variable set (fvs) serves as a completeness check for the operation in the sense that it includes all the variables that must appear in the definition of the operation. the elements of fvs can be analysed under two categories: static elements (those that are only read), dynamic elements (those that are modified by the operation). correct determination of frame variables is essential in order to be able to systematically monitor the state of the system after the execution of an operation, therefore in our observations we aim to find proc. ocl 2008 12 / 16 eceasst guidelines to reach a reasonably complete set of frame variables. context: by definition, if a variable is read/modified during the execution of an operation, it is considered to be included in the fvs. issue encountered: during the course of scenario creation, we noticed that the above definition alone is not sufficient to cover all the frame variables. there are various external factors such as invariants, nested calls, etc. that require the use of other variables than those listed in the initial form of fvs of an operation. observations: following is a list of some of the cases that must be considered in the determination of fvs: • if an operation calls another operation from within, either the elements of the fvs of the called function is added to that of callee function, or the fvss do not change. note that, in theory, a static element of callee function’s fvs should not be a dynamic element of the called function’s fvs. • as explained in observation 4, the change of a variable value may cause an invariant conflict. if the conflict is due to another variable that is not even considered in the scenario, then we have two routes to follow. we either count these scenarios that conflict with invariants as test cases, or frame variable set will be extended in order to cover the variables required by the invariant and new scenarios that do not conflict with the invariants will be created. so, the point to consider is whether to include the variables that are associated to already existing frame variables through an invariant. • another issue that must be taken into account in fvs determination is the case of derived attributes. ocl supports the definition of derived attributes that are prefixed with / in uml and we explained briefly how we dealt with derived attributes in [apw07]. currently, there is no automatic way of assigning the value of these attributes. in the context of frame variables, this brings extra work since the operation may not directly use the derived attribute, but if at least one of the dynamic variables included in its fvs set is the variable that affects the value of the derived attribute, then the derived attribute should also be modified accordingly. an example to this can be given by explaining the changes in numberofunusedexceptions. this derived attribute is calculated by subtracting the exceptionlogs− > size() value from cmaxexceptionno constant. when an exception occurs, the system calls createexceptionlog() function. this function creates an extra exception log if the current number of exceptions does not exceed a certain value. since the value of cmaxexceptionno is constant, but the size of exception logs changes during the course of this operation, we have to include numberofunusedexceptions in the fvs and change the value of the attribute accordingly. this allows us to use it as a counter in other functions. conclusion: the list presented above is not a complete list by any means, but we believe it demonstrates the need for the extension of fvs through different channels and if this concept is to be integrated into ocl tools, it is necessary that the aforementioned issues are considered. 13 / 16 volume 15 (2008) scenario observations for model validation observation 7: running sequences of scenarios aim: it is important to find the patterns that would help us automate our technique and/or integrate it with other techniques. in the context of scenario creation, automation is not only important in forming scenarios, but also in executing them one after the other. context: when the instance of the system model is exercised by a scenario, the scenario brings the system into a certain state. this new state may be the same as the old one if the scenario does not perform any changes on any of the existing objects and does not create/delete any objects. in this case, a second scenario can be run straight after the first one, since the system is in its initial, stable state. clearly, each scenario assumes that the system is in a state that accepts the further requests to be made by itself. this assumption comes from step 1 in table 2. issue encountered: if the first scenario makes modifications on the initial instance of the model, then the assumptions made by the second scenario may not hold. thus, the question that arise in the context of scenario execution is that how the process of running several scenarios one after the other can be achieved especially in the presence of those that modify the system. the following are two solutions to this problem: • system reset: after each scenario, the system can be reset to its initial state by performing step1 on table 2. • neutralisation: actions that erase the effects of the last scenario can be carried out. observations: during the execution of our scenarios, we used both of the above techniques. neutralisation requires undoing the actions taken by the scenario under investigation. for instance, if the scenario creates an object, the object and all its links to other objects must be deleted during the neutralisation process. this may seem straightforward, but it enforces to analyse all sorts of different actions and executing commands that ultimately has the opposite effect of these actions. this is a rigorous process especially if the scenario has nested calls and object deletions. in order to apply the neutralisation technique, we need to undo the actions in the reverse order. adjusting the value in default pocket by addition, deleting the payment log that holds the details for a transaction, changing the default pocket to its previous form, creating the previously existing exception logs are some of the actions carried out in order to neutralise the effect of the scenarios. this list only gives a rough idea about the actions that needs to be done before executing the next scenario. when we examine further, we realised that the elaboration of neutralisation process requires thorough analysis of program semantics and the process itself includes: storage of the previous values (in order to restore them later); implementation of reverse functions, i.e., functions that have the opposite effect of existing functions. on the other hand, system reset only requires the re-compilation of the model and the execution of the first line in the scenario -read basemodel.cmd-. conclusion: neutralisation and system reset are two solutions to the problem of running scenarios one after another. system reset may mean the initialisation of the whole model and depending on the technology used and the system environment, this may require extra memory space and adjustment of certain environment variables. however, during this research, system reset option has been used extensively in order to save time. proc. ocl 2008 14 / 16 eceasst 3 future work and conclusion in this paper, the observations made during the creation and execution of scenarios for validation of assertions have been outlined. the importance of independent scenarios, the effect of overlapping postconditions, the importance of the frame variable set and the ability to carry out successive executions of scenarios are some examples of observations that are not only applicable in the context of the use tool, but are also relevant for other tools that support scenario creation and execution. we believe that these observations explain how scenarios can be defined in a structured manner and what sort of obstacles can be experienced, thus leading us to a better model, and helping to form the basis for test case generation through model artifacts. the lessons learned during this study lead the work that compares several state-based modelling tools and concretises the tasks performed during modelling and model validation [auw08]. the outcome of this study also shows that the tasks carried out during these stages of modelbased testing may vary from one tool to another, however, the main concerns about scenarios (validity, generality, multi-platform applicability) that form the base of test cases are similar, if not the same. by using the comparison results and the lessons learned during this study, we now focus on on automatic generation, and concretisation of abstract test cases. bibliography [apw07] e. aydal, r. paige, j. woodcock. evaluation of ocl for large-scale modelling: a different view of the mondex smart card application. ocl4all: modelling systems with ocl, workshop at models’07,nashville, usa, 2007. [auw08] e. aydal, m. utting, j. woodcock. a comparison of state-based modeling tools for model validation. tools-europe’08, switzerland, 2008. [bgl+07] f. bouquet, c. grandpierre, b. legeard, f. peureux, n. vacelet, m.utting. a subset of precise uml for model-based testing. proceedings of the 3rd international workshop on advances in model-based testing, isbn:978-1-59593-850-3, 2007. [bl05] e. bernard, b. legeard. requirements traceability in automated test generation: application to smart card software validation process. a-most, 2005. [cla97] r. clarke. the mondex value-card scheme: a mid-term report. chip-based payment schemes: stored-value cards and beyond, 1997. [gbr03] m. gogolla, j. bohling, m. richters. validation of uml and ocl models by automatic snapshot generation. proc. uml 2003, springer, lncs 2863, 2003. [gbr06] m. gogolla, m. buettner, m. richters. use: uml specification environment for validating uml and ocl. science of computer programming, 2006. [jw07] r. b. j. woodcock. the verification grand challenge. csic, 2007. [kas06] i. kassios. dynamic frames: support for framing,dependencies and sharing without restrictions. proc. fm 2006, springer, lncs, 2006. 15 / 16 volume 15 (2008) scenario observations for model validation [lim99] m. i. limited. introduction to mondex purse operation, tech. report. mondex international limited, 1999. [mj94] s. miller, j.j.chilenski. applicability of modified condition/decision coverage to software testing. software engineering journal, 1994. [scw00] s. stepney, d. cooper, j. woodcock. an electronic purse: specification, refinement and proof. oxford university computing laboratory, tech report, 2000. [zg03] p. ziemann, m. gogolla. validating ocl specifications with the use tool: an example based on the bart case study. volume 80, elsevier science, 2003. http://www.elsevier.nl/locate/entcs/volume80.html proc. ocl 2008 16 / 16 http://www.elsevier.nl/locate/entcs/volume80.html introduction mondex smart card application test scenarios in uml specification environment (use) validation of assertions and generation of abstract test cases contribution observations on scenario creation in the context of mbt tool-related observations assertion-related observations general scenario formation-related observations future work and conclusion conditional lemma discovery and recursion induction in hipster electronic communications of the easst volume 72 (2015) proceedings of the 15th international workshop on automated verification of critical systems (avocs 2015) conditional lemma discovery and recursion induction in hipster irene lobo valbuena and moa johansson 15 pages guest editors: gudmund grov, andrew ireland eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst conditional lemma discovery and recursion induction in hipster irene lobo valbuena and moa johansson lobo@chalmers.se, moa.johansson@chalmers.se department of computer science and engineering chalmers university of technology, gothenburg, sweden. abstract: hipster is a theory exploration tool for the proof assistant isabelle/hol. it automatically discovers lemmas about given recursive functions and datatypes and proves them by induction. previously, only equational properties could be discovered. conditional lemmas, for example required when reasoning about sorting, has been beyond the scope of theory exploration. in this paper we describe an extension to hipster to also support discovery and proof of conditional lemmas. we also present a new automated tactic, which uses recursion induction. recursion induction follows the recursive structure of a function definition through its termination order, as opposed to structural induction, which follows that of the datatype. we find that the addition of recursion induction increases the number of proofs completed automatically, both for conditional and equational statements. keywords: theory exploration, automated induction, interactive theorem proving 1 introduction theory exploration is a technique for automatically discovering new interesting lemmas in a formal mathematical theory development. these lemmas are intended to help constructing a richer background theory about the concepts at hand (e.g. functions and datatypes) which can be useful both to enhance the power of automation as well as being of use in interactive proofs [mba07, jdb11, mmdb12]. theory exploration has proved particularly useful for automation of inductive proofs [cjrs13]. this work builds on hipster [jrsc14], an interactive theory exploration system for the proof assistant isabelle/hol [npw02]. it can be used in two modes, either exploratory mode to generate a set of basic lemmas about given datatypes and functions, or in proof mode, where it assists the user by searching for missing lemmas needed to prove the current subgoal. to generate conjectures, hipster uses as a backend the hipspec system, a theory explorer for haskell programs [cjrs13]. proofs are then performed by specialised tactics in isabelle/hol. hipster has been shown capable of discovering and proving standard lemmas about recursive functions, thus speeding up theory development in isabelle. however, lemma discovery by theory exploration has previously been restricted to equational properties. in this paper we take the first steps towards lifting this restriction and exploring also conditional conjectures. conditional lemmas are necessary if we for example want to prove properties about sorting algorithms. as an example, consider the proof of correctness for insertion sort: theorem isortsorts: ”sorted (isort xs)” 1 / 15 volume 72 (2015) mailto:lobo@chalmers.se mailto:moa.johansson@chalmers.se conditional lemma discovery and recursion induction in hipster to prove this theorem by induction will in the step-case require a lemma telling us that if a list is sorted, it remains so after an additional element is inserted: lemma ”sorted xs =⇒ sorted (insert x xs)” discovering this kind of conditional lemmas introduces a big challenge for theory exploration. first of all, the search space greatly increases: what statements should be picked as potentially interesting side-conditions to explore? secondly, as our theory exploration system relies on generation of random test-cases, we also need to ensure that we perform tests where the condition evaluates to true, otherwise the system might miss some conditional equations (example 2, p. 4). as hipster is designed as an interactive system, we avoid the first problem by asking the user to specify under which condition theory exploration should occur. in the example above, this would require the user to tell hipster that the predicate sorted is an interesting pre-condition, in addition to which function symbols should be explored in the bodies of lemmas. the rest of the process is however automatic. we describe it in more detail in §3 the second contribution of this paper is a new automated tactic for recursion induction (see e.g. §3.5.4 of [npw02]). previously, hipster only supported structural induction over the datatypes, but has now been extended with a new tactic that uses recursion induction, following the termination order of function definitions instead of the datatype. this has shown to be useful for many proofs that previously failed, but can also provide shorter proofs in some cases. the new recursion induction tactic is described in §3.2. it is used by hipster during automated theory exploration, but can equally well be applied as a powerful regular tactic by a human user working in isabelle. 2 hipster this section provides a description of how hipster works and how its subsystem quickspec generates conjectures. 2.1 theory exploration in hipster figure 1 gives an overview of the hipster system. starting from an isabelle theory file that defines a set of datatypes and functions, the user calls hipster on a list of functions about which she is interested in finding lemmas. the workings of hipster can be divided up into three stages: 1) generation of haskell code. 2) theory exploration in haskell. 3) proof in isabelle. hipster uses isabelle’s code generator [hn10], to translate the theory to a haskell program. hipster then employs the theory exploration system hipspec as a backend for generating conjectures. while hipspec can be used also as a fully fledged theorem prover, hipster only uses its conjecture generation subsystem quickspec [csh10], and performs proofs inside isabelle. isabelle is an lcf-style prover, which means that it is based on a small core of trusted axioms, upon which subsequent proofs must be built. therefore, any proofs found outside isabelle, e.g. by hipspec, would have to be reconstructed inside isabelle anyway. hence it is easier for hipster to simply use isabelle for proofs in the first place. proc. avocs 2015 2 / 15 eceasst isabelle theory haskell programcode generator theory exploration conjectures difficult reasoning theorems proved failed routine reasoning trivially proved? discard figure 1: overview of hipster not all conjectures returned from quickspec are interesting. hipster is parametrised by two tactics, which can be set by the user: one for routine reasoning and one for difficult reasoning. conjectures solved by routine reasoning are deemed trivial and discarded, while those requiring more difficult reasoning are displayed to the user and included in the isabelle theory so they can be used in subsequent proofs if necessary. in the context of this paper, routine reasoning is first-order equational reasoning and simplification, whilst difficult reasoning involves some kind of induction. if a conjecture is not immediately provable, hipster will place it at the end of the list of open conjectures and will try it again if it has found some additional lemmas. occasionally, hipster might discover some conjecture which it does not manage to prove automatically, because not even its tactic for difficult reasoning is strong enough. such an open conjecture would also be displayed to the user, who can then choose to perform an interactive proof in isabelle, perhaps employing other tactics or lemmas than those currently available to hipster. 2.2 conjecture generation in quickspec quickspec takes as input a set of functions and variables (by default three per type), and generates all type-correct terms up to a given limit (by default depth three). the number of variables and term-depth limit can be adjusted by the user. quickspec then proceeds to divide the generated terms into equivalence classes, so that each equivalence class eventually represents a set of equations. initially, all terms of the same type are in the same equivalence class. quickspec then uses quickcheck [ch00], to generate random ground values for the variables in the terms, and evaluates the result. if two terms in an equivalence class turn out to evaluate differently, the equivalence class is split accordingly. the process is then repeated until the equivalence classes stabilise (after several hundred different random tests), which means that we usually have quite a high confidence in that the conjectures produced are probably true, even though they are not yet proved. 3 / 15 volume 72 (2015) conditional lemma discovery and recursion induction in hipster example 1. as a small example, consider a theory exploration attempt where we have asked hipster for lemmas about a function isort implementing insertion sort. among the terms generated by quickspec are those in the table below. initially, all terms are placed in the same equivalence class. suppose quickspec generates the random value xs → [3,1]. term ground instance value 1 isort xs isort [3,1] [1,3] 2 isort (isort xs) isort (isort [3,1]) [1,3] 3 xs [3,1] [3,1] as not all terms evaluate to the same value, they should no longer be in the same equivalence class. we thus split the terms into two new equivalence classes: terms 1 and 2 evaluate to the same value and remain together, while term 3 is separated. after this, no subsequent tests further split these equivalence classes, and we can read off the equation: isort(isort xs) = isort xs. 3 conditional lemmas and recursion induction we now demonstrate how to employ hipster interactively for theory exploration of conditional lemmas in the development of a theory.we first explain how conditional conjectures are generated in quickspec. we then explain our new automated induction tactic for recursion induction, and finally show how hipster combines these in a case study proving the correctness of insertion sort. 3.1 generating conditional conjectures the support in quickspec for generating conditional conjectures (implications) is still rather basic. in this case, quickspec will in addition to the other input require the user to specify a predicate to use as the premise of an implication. term generation proceeds as described above, but testing takes the given predicate into account. here, we are only interested in tests with values that make the premise true, otherwise we may split the equivalence classes when they should not be split. quickcheck uses special functions called generators to produce random values of a given type. if using quickspec directly in haskell, the user can program special purpose generators that could be made to only produce values satisfying a given predicate. in hipster, however, these generator functions are simpler as they have to be automatically derived together with the haskell code. tests not satisfying the premise are simply discarded during conditional exploration, which means that we typically must generate more tests than for equational conjectures. also, the risk of some non-theorem slipping through is slightly higher, but as hipster then attempts to prove all conjectures, such a statement would be caught in the proving phase. automatically generating customised generator functions is further work. example 2. in example 1, we showed how quickspec generated equational conjectures about the insertion sort function isort. we are furthermore interested in the case with the condition that the predicate sorted holds (for one variable). in this case, quickspec first performs one pass looking for plain equations, as in example 1, then a second where it considers the condition sorted xs. in this second phase, quickspec performs a new exploration, this time requiring the proc. avocs 2015 4 / 15 eceasst predicate sorted xs to hold for all test values. suppose we test with the sorted list: xs → [1,2] (other non-sorted values for xs would be discarded). term ground instance value 1 isort xs isort([1,2]) [1,2] 2 isort (isort xs) isort (isort [1,2]) [1,2] 3 xs [1,2] [1,2] this time, all terms evaluate to the same value on all tests where the list is sorted, so all three terms remain in the same equivalence class. quickspec realises that there is no point producing the conjecture sorted xs =⇒ isort (isort xs) = xs, as this is subsumed by the non-conditional equation discovered in the first phase. it will however produce the additional conjecture sorted xs =⇒ isort xs = xs, which clearly only holds if the list is already sorted. 3.2 automating recursion induction a recursion induction scheme is derived from a function’s recursive definition. unlike structural induction, the recursion induction scheme corresponds to the originating definition, and hence, the cases considered in its simplification rules. when defining a recursive function over an inductive datatype one might traverse arguments following a different pattern to the strictly structural one (the one arising from a datatype’s definition). this pattern could be more specific, or even less so, than following the datatype. for instance, take the functions on lists sorted and last: fun sorted :: ”nat list ⇒ bool” where ”sorted [] = true” | ”sorted ([x]) = true” | ”sorted (x1 # (x2 # xs)) = (x1 ≤ x2 & sorted (x2 # xs))” fun last :: ”’a list ⇒ ’a” where ”last ([x]) = x” | ”last (x # xs) = last xs” from these definitions’ structures one can derive a new induction principle. structural induction on lists considers the base-case [] (the empty list) and step-case x # xs (a list with an element inserted at the front). in the case of sorted, cases are broken down into more detailed ones by including an additional base-case [x] (the singleton list) and restricting the step-case to lists with at least two elements x1 # x2 # xs. meanwhile last is not defined for the case [] and hence partially defined, meaning the induction scheme it gives rise to is to be completed with such a case. this, in fact, results in the same recursion induction scheme derived from sorted: sortedind p ([]) ∀x p ([x]) ∀x, y, xs (p (x # xs) =⇒ p (y # (x # xs))) ∀x p (x) lastind ∀x p ([x]) ∀x, y, xs (p (x # xs) =⇒ p (y # (x # xs))) p ([]) ∀x p (x) 5 / 15 volume 72 (2015) conditional lemma discovery and recursion induction in hipster induction determined by these schemata is called recursion induction or computation induction. they can isolate sub-units not represented in a datatype’s structure as being atomic, such as lists with at least two elements in the scheme for sorted. recursion induction hence provides an immediate and more specific structure for reasoning about other recursion patterns where a simple structural induction might fail to set appropriate base and step-cases for the induction to succeed. within isabelle/hol these schemata are automatically derived and proved as theorems from recursive functions’ termination order, and hence guaranteed to be sound [kst+11]. example 3: recursion induction in a proof. to exemplify the potential difference between recursion and structural induction, let us take the already introduced conditional lemma sorted xs =⇒ sorted (insert x xs). applying structural induction on the list xs would produce the subgoals: 1. sorted [] =⇒ sorted (insert x []) 2. ∧ y ys. (sorted ys =⇒ sorted (insert x ys)) =⇒ sorted (y # ys) =⇒ sorted (insert x (y # ys)) whilst sorted’s recursion induction scheme would yield: 1. sorted [] =⇒ sorted (insert x []) 2. ∧ y. sorted [y] =⇒ sorted (insert x [y]) 3. ∧ y1 y2 ys. (sorted (y2 # ys) =⇒ sorted (insert x (y2 # ys))) =⇒ sorted (y1 # y2 # ys) =⇒ sorted (insert x (y1 # y2 # ys)) the latter set of subgoals leads to an immediate proof of the main lemma thanks to its steps mirroring the actual predicate definition, hence having a correspondence with its simplification rules. in contrast, the former, even though it intuitively looks immediate to prove, is not sufficiently generalised nor does it specify any intermediate result on inserting an element on a concrete non-empty list (in our case, the singleton list) which would enable to prove the second subgoal for any arbitrary list. structural induction is in some way a weaker scheme and additional case-splits or lemmas would be required to close the proof employing it in our example. a new induction tactic for hipster we have implemented a new automated tactic, called hipster induct schemes, for induction in isabelle. this tactic searches not only for proofs by structural induction, but may also employ recursion induction when appropriate. it is designed for hipster to use as its ”difficult reasoning” component, but human users may of course also employ this tactic in interactive proofs. the tactic first tries structural induction using the induction scheme associated with the datatype(s) of variables in the problem. if this fails, the tactic then tries recursion induction, using the induction schemes associated with the functions occurring in the problem. when instantiating recursion induction schemes with variables of the problem, more complete instantiations are considered first. this leaves less specific partial instantiations to be tried later. for each attempted induction, the tactic will apply isabelle’s simplifier followed by (if necessary) first-order reasoning using isabelle’s built in first-order prover metis. figure 2 shows an overview of the tactic. the user can configure the tactic to specify how to select facts to be passed to the simplifier and to metis. the default is the simplification rules from the relevant function definitions, the datatype proc. avocs 2015 6 / 15 eceasst try structural induction schemes(s) then simplification + f.o. reasoning otherwise try recursion induction sceheme(s) then simplification + f.o. reasoning figure 2: overview of hipsters new tactic. case distinction rules which are automatically derived by isabelle, and the lemmas discovered by theory exploration so far. however, if we pass too many facts to metis, it becomes slower. therefore, the user can configure hipster to include fewer of the discovered lemmas if needed. hipster also impose a timeout on simplification and first-order reasoning, which can be set by the user. the default timeout is 1 second for each proof attempt. as further work, we plan to experiment with using sledgehammer instead [pb10], which calls powerful external first-order provers and reports back exactly which facts were needed in the proof. metis can then reconstruct the proof quickly inside isabelle’s trusted kernel. example 4: simultaneous induction. a notable gain of the new tactic with recursion induction is that of having the capability of performing simultaneous induction, whereas previously only structural inductions on a single variable were performed by hipster. simultaneous induction schemata are those inducting over more than one variable at a time, whether those variables are of the same type or not. such is the case for the list function zip’s recursion induction scheme, which corresponds to parallel induction on two lists: fun zip :: ”’a list ⇒ ’b list ⇒ (’a × ’b) list” where ”zip [] y = []” | ”zip (x # xs) [] = []” | ”zip (x # xs) (y # ys) = (x, y) # (zip xs ys)” zipind ∀ys p ([], ys) ∀x, xs p (x # xs,[]) ∀x, y, xs, ys (p (xs, ys) =⇒ p (x # xs, y # ys)) ∀xs, ys p (xs, ys) this scheme, along with some initial theory exploration, allows theorems like the following to be proven automatically: zip (xs @ ys) zs = (zip xs (take (len xs) zs)) @ (zip ys (drop (len xs) zs)) or even the alternative related conditional lemma to be proven without prior exploration: len xs = len ys =⇒ (zip xs ys) @ (zip zs ws) = zip (xs @ zs) (ys @ ws) neither of these lemmas were provable before, even having done exploration for all the occurring functions in them. hipster’s prior structural induction approach could not capture in a scheme the relation between two variables. in these two cases, zip traverses its arguments taking steps on 7 / 15 volume 72 (2015) conditional lemma discovery and recursion induction in hipster both at the same time, a pattern we can only capture with some form of simultaneous induction. instead of synthesising a series of possible simultaneous structural induction schemata, recursion induction gives us an immediate choice which is also closer to the problem at hand. 3.3 interactive case study: insertion sort we here showcase hipster’s handling of conditional lemmas via the proof of correctness for the theorem sorted (isort ts). for it, we assume the less-or-equal operator ≤ for naturals (and no prior, additional lemmas), and the function definitions: fun sorted :: ”nat list ⇒ bool” where ”sorted [] = true” | ”sorted ([x]) = true” | ”sorted (x1 # (x2 # xs)) = (x1 ≤ x2 & sorted (x2 # xs))” fun insert :: ”nat ⇒ nat list ⇒ nat list” where ”insert x [] = [x]” | ”insert x1 (x2 # xs) = (if (x1 ≤ x2) then x1 # (x2 # xs) else x2 # (insert x1 xs))” fun isort :: ”nat list ⇒ nat list” where ”isort [] = []” | ”isort (x # xs) = insert x (isort xs)” running exploration from the simpler components is the first step, considering both equational and conditional lemmas, since we have two predicates involved in the definiens of functions in the final theorem. the following command invokes conditional exploration for ≤: hipster cond ≤ which, along with conditional exploration for its negation, results in 10 discovered and proven lemmas, 6 of which are conditionals (we present the vital lemmas towards the final proof) and all require recursion induction: lemma lemma ac [thy expl]: ”x ≤ y =⇒ x ≤ (s y) = true” by (hipster induct schemes ≤.simps nat.exhaust) lemma lemma ad [thy expl]: ”y ≤ x =⇒ (s x) ≤ y = false” by (hipster induct schemes ≤.simps nat.exhaust) (...) lemma lemma ai [thy expl]: ”(¬ (x ≤ y)) =⇒ x ≤ z = false” by (hipster induct schemes ≤.simps nat.exhaust) (...) hipster automatically generates this output. for each case, the lemma command makes the statement to be proven and is followed by a tactic application via the by command, here using hipster’s recursion induction tactic hipster induct schemes, which employs recursion induction where necessary. to enable the completion of the proof, exploration provides it with the proc. avocs 2015 8 / 15 eceasst automatically generated isabelle rules for simplification of function definitions, such as ≤.simps, and datatype case distinction rules, such as nat.exhaust. with a new exploration considering the functions about sorting itself and (potentially) taking sorted as a side-condition for which to discover lemmas, hipster discovers and proves both the conditional auxiliary lemma required and the goal theorem. note that the exploration command takes as its first argument the predicate with which to construct side-conditions: hispter cond sorted isort insert (...) lemma isortinvariant [thy expl]: ”sorted ys =⇒ sorted (insert x ys) = true” by (hipster induct schemes sorted.simps isort.simps insert.simps) (...) theorem isortsorts [thy expl]: ”sorted (isort x) = true” by (hipster induct schemes sorted.simps isort.simps insert.simps) during this last exploration, other interesting lemmas are discovered, all of which can be now proven automatically by using the sub-lemma about insert’s invariant isortinvariant: lemma isortfixes [thy expl]: ”sorted x =⇒ isort x = x” by (hipster induct schemes sorted.simps isort.simps insert.simps) lemma insertcomm [thy expl]: ”insert x (insert y z) = insert y (insert x z)” by (hipster induct schemes insert.simps) invoking the recursion induction tactic hipster induct schemes once proves all of the statements above, simplifying the interaction with the proof assistant. particularly, the crucial lemma isortinvariant is proven applying sorted’s associated recursion induction scheme, highlighting once again the need for support of conditional lemmas in automated inductive proving and the possibilities recursion induction brings towards proof automation. 4 evaluation in this section we present an evaluation of hipster’s automated tactics, an analysis which had not been performed for hipster to the same extent priorly. keeping in mind evaluation of automated tools for interactive theorem proving necessarily has to consider some degree of interaction, two forms of evaluation have been carried out1: • case studies on algebraic data types and operations on them; in particular focusing on inductive theories for natural numbers and lists • evaluation on problems from tip (tons of inductive problems) [cjrs15], a set of benchmarks and challenge problems for inductive theorem provers. from tip, we evaluate hipster over two sets of problems employed in previous works on inductive theorem proving: johansson, dixon and bundy’s work on case-analysis for rippling 1 source code for hipster, examples presented and benchmarks are available online: https://github.com/moajohansson/ isahipster 9 / 15 volume 72 (2015) https://github.com/moajohansson/isahipster https://github.com/moajohansson/isahipster conditional lemma discovery and recursion induction in hipster [jdb10] (we denote it case-analysis 2), and prior work by ireland and bundy on employing proof failure to guide lemma discovery and patch inductive proofs [ib96] (we denote it prod-failure 3). we now present these results and compare them with other tools’ reported results. 4.1 method to evaluate performance on tip, each problem is analysed individually, in isolation from others, to assess how far hipster can go from bare definitions. theory explorations were only run whenever the problem was not provable by the induction tactic directly, i.e. when the problem was missing helping lemmas. explorations were first performed on the individual functions appearing in the problem definition, jointly with their auxiliary functions. these were followed by explorations on groups of said functions if required, leaving conditional exploration as the last exploration to be run before defining the problem as non-provable by hipster. as already specified, conditional lemma discovery is limited to explore a single predicate at a time to define side-conditions. for the present evaluation this has sufficed. additionally, to test hipster’s capacity when working on strictly newly defined theories, no assumptions nor properties from theories in isabelle/hol were considered during proof search. as an example, natural numbers are not isabelle/hol’s, but redefined. hence, predefined notions of orderings and other properties do not play a part in proofs obscuring the results of hipster’s actual work. in this way, we only consider as the base starting point a set of definitional statements, aligning with the purpose of proving based on structure and construction of programs. 4.2 results the following set of tables summarises statistics on the two sets of the benchmarks, with respect to the number of problems solved. columns eq and cond do so for problems defined by an equational and a conditional theorem respectively. case-analysis prod-failure total eq cond eq cond total number of benchmarks 71 14 38 12 135 number of problems solved 71 13 35 12 131 table 1: total number of problems solved. automation table 2 shows the number of problems with automated solutions out of those which were solved. full automation is understood as solving a problem only with discovered lemmas about the function symbols involved in the target theorem and hipster’s automated recursion induction. partially automated problems are those for which additional related functions of a datatype’s theory were provided to exploration for completion. 2 case-analysis problems: https://github.com/tip-org/benchmarks/tree/master/benchmarks/isaplanner 3 prod-failure problems: https://github.com/tip-org/benchmarks/tree/master/benchmarks/prod proc. avocs 2015 10 / 15 https://github.com/tip-org/benchmarks/tree/master/benchmarks/isaplanner https://github.com/tip-org/benchmarks/tree/master/benchmarks/prod eceasst case-analysis prod-failure total eq cond eq cond fully automated 67 13 29 12 121 partially automated 4 0 6 0 10 table 2: automation of problems solved. case-analysis prod-failure eq cond eq cond no additional lemmas 38 10 1 8 only equational lemmas 27 2 32 1 equational and conditional lemmas 6 1 2 3 table 3: number of problems requiring discovery of auxiliary lemmas. overall, the rate of fully automated provability on the benchmark set is 90% ; considering partially automated problems as well, the overall rate is 97%. a number of theorems (problems 52, 53, 72, 74 from case-analysis; and 2, 4, 5, 20, 22, 23 from prod-failure) required one of the following two similar lemmas: len (x @ y) = len (y @ x) count z (x @ y) = count z (y @ x) these two lemmas are not automatically proven in a first instance (neither by structural nor recursion induction). each of them in turn needs an auxiliary lemma which is not discovered. nonetheless, their proof can be partially automated. in both cases, one can observe that the outermost function applied, len and count respectively, acts as a relator function between two datatypes. furthermore, these will in fact act as relator functions between list concatenation @ and addition for natural numbers plus. since plus does not occur in the problems to be proven, it is not added to the exploration directly. adding plus interactively, hipster discovers and proves automatically the lemmas: len (x @ y) = plus (len x) (len y) count z (x @ y) = plus (count z x) (count z y) along with the commutative law for plus, also discovered and proven automatically, they enable the automation of the two pending proofs without further intervention. and so, the corresponding tip problems are solved as well. these two cases seem to indicate that recursion induction may not suffice when a noncommutative operation nested within another has commuting arguments on both sides of an equality. at least not in the absence of smaller related lemmas corresponding to subgoals. this seems reasonable: the structure of the terms at each side of the equality will differ upon induction. theory exploration just over half of the problems required prior lemma discovery, showcasing the benefit of theory exploration. in table 3 we show the number of solved problems which required prior theory exploration and specify how many required further conditional lemmas. 11 / 15 volume 72 (2015) conditional lemma discovery and recursion induction in hipster a smaller subset of problems were provable with the aid of conditional exploration, namely those involving functions defined in terms of some predicate. recursion induction whereas recursion induction was not necessary as often as theory exploration (whether for the main theorem or auxiliary lemmas), its impact is still notable. some problems would not be provable employing only hipster’s prior structural induction approach. in table 4, problems solved by structural induction are those for which both the main theorem and any required auxiliary lemma only needed structural induction. those solved by recursion induction required it for the main theorem’s proof or any of its helping lemmas. case-analysis prod-failure eq cond eq cond structural induction 38 7 30 11 recursion induction 33 6 5 1 table 4: number of problems solved with both kinds of induction. overall, there seems to be a trade-off between using weaker induction schemes (structural induction) and reducing the number and complexity of needed auxiliary lemmas. structural induction was always attempted first by the tactic, meaning theorems solved via recursion induction (around a third of the benchmarks) would have not been solved otherwise, at least not with the degree of exploration carried out. the results suggest recursion induction can save on exploration time. it provides appropriate induction patterns that avoid the need for sub-lemmas about specific constructor combinations. 4.3 comparison other inductive provers have also been evaluated on these test suites, serving as a good point of comparison. the following table collects the number of problems solved by some of them in comparison with hipster; note that we compare on problems for which other provers have available data. plain figures correspond to fully automated solutions and those in parentheses (x) indicate number of successful proofs after some adaptation of settings. in total, case-analysis has 85 problems whilst prod-failure has 50. hipster hipspec zeno isaplanner cvc4 pirate case-analysis 80 (84) 80 82 47 80 85 prod-failure 41 (47) 44 (47) 21 40 (47) the already mentioned hipspec uses theory exploration, structural induction and external firstorder provers to prove properties about functional programs [cjrs13]. zeno is a tool for proving equational inductive properties of haskell programs [sde12]. cvc4’s approach to inductive proving is built on smt solving whilst pirate is built on first-order prover spass, both with a top-down approach in conjecture generation [rk15, ww]. isaplanner is a proof planning tool for isabelle based on rippling [dj07, jdb10]. proc. avocs 2015 12 / 15 eceasst in comparison to other (automated) inductive provers, the new hipster is the only one (to the best of our knowledge) to employ recursion induction. as results show, its performance is on par to other state-of-the-art tools’. additionally, unlike these tools, hipster produces formal, certified proofs. to be noted is that the failing problems for hipster in the benchmark set prod-failure (problems 33-35) differ from those hipspec and pirate fail at (with the exception of 33 in pirate’s case). these three problems involve definitions for multiplication, factorial and exponentiation operations for peano numerals with accumulator arguments. particularly, hipspec employed adjusted settings for lemma discovery in these three cases: the generators for random values of datatypes are manually defined. as already pointed out in §3.1, hipster derives generators automatically, which means the simplicity of these could lead to inefficiencies when it comes to generating values of larger sizes. hipster has not been evaluated with adjusted settings at the hipspec/quickspec level and hence the exploration phase was not feasible to perform for these problems due to memory usage during testing in quickspec. with similar settings to hipspec’s, problems 33-35 are likely to be solvable in hipster too. 5 related work the work on lemma discovery for inductive proofs has mainly focused on equational lemmas, for instance in the theory exploration systems isascheme and isacosy [mmdb12, jdb11], which also work on isabelle/hol theories. isascheme requires the user to provide term schemas, which are then automatically filled in with available symbols. isacosy only generates irreducible terms, and uses an internal constraint language to avoid generating anything that could be reduced by a known equation. these systems focused more on automation, while hipster is designed to be useable in an interactive theory development. hipster is faster, and now also supports conditional theory exploration where the user specifies an interesting condition. conditional lemma discovery has also been missing from the isaplanner system, which uses proof critics to deduce lemmas from failed proof attempts [dj07, jdb10]. theory exploration systems rely on having an automated prover at hand to prove generated conjectures. in the context of inductive theories, most other automated provers supporting induction such as isaplanner, zeno, hipspec, dafny and cvc4 [dj07, sde12, cjrs13, lei12, rk15] only support structural induction. hipster now also provides an automated tactic for recursion induction by exploiting isabelle’s automated derivation of such induction schemata. it can both be used in theory exploration and as a stand-alone automated tactic. the use of recursion induction and the fact that hipster produces lcf-style re-checkable proofs is also the main difference between hipster and its sister system hipspec [cjrs13], with which hipster shares its conjecture generation component. hipspec does instead rely on external first-order provers to solve the proof obligations arising in the stepand base-cases for inductive proofs, and does not produce checkable proofs. 13 / 15 volume 72 (2015) conditional lemma discovery and recursion induction in hipster 6 conclusion and further work generation of conditional lemmas in theory exploration is a challenging problem, not least as it is difficult for a tool to automatically assess which side conditions are interesting. hipster is an interactive theory exploration system, and gets around this obstacle by relying on the user to decide which predicates are deemed interesting as conditions. in this paper we have also presented a new automated tactic for recursion induction, which improves the level of proof automation of discovered conjectures in hipster. it can also be used as a powerful stand-alone induction tactic in isabelle. further work on the proving side includes experimenting with different heuristics for choosing which function’s recursion induction scheme is most likely to produce a proof, as well as extending hipster with tactics that can handle mutualand co-induction automatically. hipster has various configuration options for adjusting which of the discovered lemmas are passed to its tactics in subsequent proofs. for example, in larger theories, with many explorations, we may not want to pass all discovered lemmas to isabelle’s metis tactic, as too many lemmas might slow down the proof process. we plan to experiment with combining hipster’s tactics with the relevance filtering ideas used in sledgehammer [kbku13]. another item of further work is to extend hipster to produce structured proofs in isabelle’s isar language, instead of just a one-line application of hipster’s custom tactics. this will be easier to read for a human user, and can be more streamlined, not needing to repeat the search done in the automatic proof found by hipster’s powerful tactics. bibliography [ch00] k. claessen, j. hughes. quickcheck: a lightweight tool for random testing of haskell programs. in proceedings of icfp. pp. 268–279. 2000. [cjrs13] k. claessen, m. johansson, d. rosén, n. smallbone. automating inductive proofs using theory exploration. in proceedings of the conference on automated deduction (cade). lncs 7898, pp. 392–406. springer, 2013. [cjrs15] k. claessen, m. johansson, d. rosén, n. smallbone. tip: tons of inductive problems. in proceedings of the conference on intelligent computer mathematics (cicm). lncs 9150. springer, 2015. [csh10] k. claessen, n. smallbone, j. hughes. quickspec: guessing formal specifications using testing. in proceedings of tap. pp. 6–21. 2010. [dj07] l. dixon, m. johansson. isaplanner 2: a proof planner in isabelle. 2007. dream technical report (system description). http://dream.inf.ed.ac.uk/projects/isaplanner/docs/isaplanner-v2-07.pdf [hn10] f. haftmann, t. nipkow. code generation via higher-order rewrite systems. in blume et al. (eds.), functional and logic programming. lncs 6009, pp. 103–117. springer, 2010. proc. avocs 2015 14 / 15 http://dream.inf.ed.ac.uk/projects/isaplanner/docs/isaplanner-v2-07.pdf eceasst [ib96] a. ireland, a. bundy. productive use of failure in inductive proof. journal of automated reasoning 16:79–111, 1996. [jdb10] m. johansson, l. dixon, a. bundy. case-analysis for rippling and inductive proof. in proceedings of itp. pp. 291–306. 2010. [jdb11] m. johansson, l. dixon, a. bundy. conjecture synthesis for inductive theories. journal of automated reasoning 47(3):251–289, 2011. [jrsc14] m. johansson, d. rosén, n. smallbone, k. claessen. hipster: integrating theory exploration in a proof assistant. in proceedings of the conference on intelligent computer mathematics (cicm). lncs 8543, pp. 108–122. springer, 2014. [kbku13] d. kuhlwein, j. c. blanchette, c. kaliszyk, j. urban. mash: machine learning for sledgehammer. in interactive theorem proving. lncs 7998, pp. 35–50. springer, 2013. [kst+11] a. krauss, c. sternagel, r. thiemann, c. fuhs, j. giesl. termination of isabelle functions via termination of rewriting. in eekelen et al. (eds.), interactive theorem proving. lecture notes in computer science 6898, pp. 152–167. springer berlin heidelberg, 2011. [lei12] k. r. leino. automating induction with an smt solver. in proceedings of vmcai. springer, 2012. [mba07] r. mccasland, a. bundy, s. autexier. automated discovery of inductive theorems. in matuszewski and rudnicki (eds.), from insight to proof: festschrift in honor of a. trybulec. 2007. [mmdb12] o. montano-rivas, r. mccasland, l. dixon, a. bundy. scheme-based theorem discovery and concept invention. expert systems with applications 39(2):1637–1646, 2012. [npw02] t. nipkow, l. c. paulson, m. wenzel. isabelle/hol — a proof assistant for higherorder logic. lncs 2283. springer, 2002. latest online version 25 may 2015. http://isabelle.in.tum.de/doc/tutorial.pdf [pb10] l. c. paulson, j. c. blanchette. three years of experience with sledgehammer, a practical link between automatic and interactive theorem provers. iwil-2010, 2010. [rk15] a. reynolds, v. kuncak. induction for smt solvers. in proceedings of vmcai. 2015. [sde12] w. sonnex, s. drossopoulou, s. eisenbach. zeno: an automated prover for properties of recursive datatypes. in proceedings of tacas. pp. 407–421. springer, 2012. [ww] d. wand, c. weidenbach. automatic induction inside superposition. https://people. mpi-inf.mpg.de/∼dwand/datasup/draft.pdf. 15 / 15 volume 72 (2015) http://isabelle.in.tum.de/doc/tutorial.pdf https://people.mpi-inf.mpg.de/~dwand/datasup/draft.pdf https://people.mpi-inf.mpg.de/~dwand/datasup/draft.pdf introduction hipster theory exploration in hipster conjecture generation in quickspec conditional lemmas and recursion induction generating conditional conjectures automating recursion induction interactive case study: insertion sort evaluation method results comparison related work conclusion and further work threadsafe: static analysis for java concurrency electronic communications of the easst volume 72 (2015) proceedings of the 15th international workshop on automated verification of critical systems (avocs 2015) threadsafe: static analysis for java concurrency robert atkey and donald sannella 15 pages guest editors: gudmund grov, andrew ireland eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst threadsafe: static analysis for java concurrency robert atkey1,3 and donald sannella2,3 1 robert.atkey@strath.ac.uk, http://bentnib.org/ computer & information sciences, university of strathclyde glasgow, united kingdom 2 dts@inf.ed.ac.uk, http://homepages.inf.ed.ac.uk/dts/ school of informatics, university of edinburgh edinburgh, united kingdom 3 http://www.contemplateltd.com/ contemplate ltd edinburgh, united kingdom abstract: threadsafe is a commercial static analysis tool that focuses on detection of java concurrency defects. threadsafe’s bug-finding capabilities and its look and feel are presented through examples of bugs found in the codebases of two widely-used open source projects. keywords: static analysis, concurrency, java 1 introduction it is widely acknowledged that developing reliable concurrent software is very difficult [goe06, lee06, hs12]. concurrency-related defects like data races and deadlocks can be subtle and hard to understand. the fact that concurrent software is inherently non-deterministic means that reliance on testing for software quality assurance is inappropriate and dangerous; intermittent bugs that show up once in a million runs are not uncommon. at the same time, requirements for improved performance force the increased use of application-level concurrency in order to exploit multicore hardware. use of static analysis to discover and diagnose concurrency defects during software development is a cost-effective solution to this problem. static analysis can take all possible execution paths into consideration, not just the ones that are explored by specific test data for a specific scheduling of threads. static analysis can be applied to a codebase while it is still under construction, long before testing is possible. bugs that are detected early in the development process are easier and cheaper to fix. threadsafe is a commercial static analysis tool that focuses on detection of java concurrency defects. by focusing on concurrency bugs, threadsafe can find bugs that other static analysis tools, both commercial and freely available, miss or are not designed to look for. threadsafe is fully automatic. it requires no annotations to be added to codebases, but if @guardedby annotations [goe06] are present then they are checked for accuracy. findings are generated from information produced by a class-by-class flow-sensitive, path-sensitive, 1 / 15 volume 72 (2015) mailto:robert.atkey@strath.ac.uk http://bentnib.org/ mailto:dts@inf.ed.ac.uk http://homepages.inf.ed.ac.uk/dts/ http://www.contemplateltd.com/ threadsafe: static analysis for java concurrency context-sensitive points-to and lock analysis. heuristics are used to tune the analysis to avoid false positives. knowledge of concurrency aspects of the java library and of some aspects of frameworks including android has been built in. threadsafe has been successfully applied to codebases of more than a million lines. it is being used in a companies across a wide range of industry sectors and in university teaching, for example [ses14]. in the following sections we give an outline of how threadsafe works and present some examples of bugs that it finds in two widely-used open source projects. we then present some of our experiences with developing, using, and observing industrial developers using threadsafe, discuss how its performance can be measured, and give a comparison of its performance against findbugs on a java concurrency benchmark. 2 how threadsafe operates threadsafe analyses .class files containing jvm bytecode for evidence of concurrency errors, which are reported back to the user. in theory, because threadsafe analyses jvm bytecode, it could be used to analyse the output of any compiler that targets the jvm (for example, the scala compiler). however, due to the need for heuristics that encode assumptions about the nature of the code being analysed, and built-in knowledge about the compilers, idioms, libraries and frameworks that are common when programming with java as the source language, threadsafe only really operates effectively on the output of a java compiler. the operation of threadsafe can be thought of as a kind of compiler for jvm bytecode, only instead of producing optimised machine code threadsafe produces static analysis warnings. the front-end is the same: jvm .class files are read in; translated into a simpler form for analysis; and analysed. instead of using the results of analysis to generate and optimise bytecode, checkers are run over the analysis results to find evidence of problematic code. below, we explain in more detail the three main stages of threadsafe’s operation, and discuss the trade-offs that have been made. although threadsafe is primarily targeted at finding concurrency flaws, there is nothing concurrency focused about the analysis it performs (other than explicit tracking of lock acquisitions) before the checkers are run. it would be relatively straightforward to write new checkers that discover defects such as resource handling errors or information leakage errors on top of the existing analysis. 2.1 class loading and preprocessing in order to be scalable to very large code bases, threadsafe analyses .class files on a bypackage basis, only referring to .class files in other packages as needed to fill in details in the inheritance hierarchy, or to analyse inherited method bodies. class files are parsed lazily, as required by the analysis described below, and the bytecode in the body of each method is preprocessed in two phases: 1. the jvm’s native stack-based bytecode is translated into a register-based format. this translation enables the interprocedural dataflow analysis to be more efficient, as the local variables in each method can be modelled as a flat array of abstract values, rather than a proc. avocs 2015 2 / 15 eceasst stack that changes size dynamically. the translation also makes all control flow in each method body explicit, turning the flat arrays of bytecode instructions into an explicit control flow graph, with one instruction per node. intra-method subroutines, implemented by the jvm’s jsr and ret instructions, are inlined. the jsr and ret instructions complicate analysis, since they result in the same segment of bytecode being invoked in several contexts within the same method. since recursive subroutines via jsr and ret are not possible, it is safe to inline bytecode subroutines into their callsites. 2. accessor methods are inlined into their callers. accessors methods are methods generated by the java compiler that allow inner (or outer) classes to access private fields and methods in their outer (resp. inner) classes. the access control enforced by the jvm does not allow any class’s methods to access another class’s private fields or methods (the jvm knows nothing about inner or outer classes), so the java compiler generates a public “synthetic method” that provides direct access to the underlying private field or method. this results in the actual field or method access appearing in a different bytecode method to the one that appears in the source code. if a static analysis author is not careful, the presence of synthetic compiler-generated methods can lead to confusing reports from a static analyser that naively reports which method a problematic field access appears in. moreover, while it would be possible to just let the normal interprocedural analysis handle accessor methods, it is more efficient to preemptively inline the methods into their callers. therefore, threadsafe inlines all accessor methods into their callers. this process is not entirely straightforward, because the compilation scheme for accessor methods is not specified by the java language specification, and different compilers have slightly different strategies. threadsafe handles the schemes used by the standard openjdk javac compiler, and eclipse’s ecj compiler. 2.2 interprocedural per-class analysis after translation, a context-sensitive interprocedural points-to and lock analysis is run on each public entry point of each instantiable class. the idea is that each class is a self-contained entity whose instances can be invoked arbitrarily via its public entry points. a method is deemed to be a “public entry point” if it is either declared as public, or has been specially marked for the analysis because it will be called by some framework or library method. an example of a non-public “public entry point” is the protected doinbackground method from the android asynctask class, which is invoked by the framework code as an asynchronous background task (see section 3.3 for an example of misuse of this api that threadsafe catches). this analysis is interprocedural, but to keep the overall analysis scalable, only calls to private and protected methods on the same class are followed. all other calls are treated symbolically. this again follows the idea that each public method of a class is special in that it represents how classes are used by other classes, while private and protected methods are part of a class’s implementation. a disadvantage of this approach is that it relies heavily on the programmer having good taste when it comes to assigning access qualifiers to methods. a class whose methods are all public when they could be declared as private or protected will act the same during 3 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency execution, but will produce very different analysis results. a possible mitigation, which we have not yet attempted, is to run a whole-program analysis to infer a tightest possible access qualifier for each method, ignoring the programmer’s annotations. given a set of entry points, the interprocedural analysis computes, for each object-manipulating instruction, an approximation of the objects that will be accessed and the set of locks that are held when that instruction is executed. objects (whether locks or directly manipulated) are represented as paths relative to one of: the symbolic this reference pointing to the instance of the class currently under analysis; the parameters of the public entry point method being analysed; or to references acquired from globally accessible static fields and methods. the abstract lock sets computed for each instruction consist of representations of monitors, i.e., the jvm’s intrinsic lock associated with every object, and first-class locks as represented by implementations of the java.util.concurrent.locks.lock interface. monitors are slightly simpler to model because they must strictly nest with respect to method calls, so it is easier to accurately track whether or not they are held through recursive method calls. the points-to and lock analysis are intentionally unsound in their handling of aliasing and mutation. for example, the sequence of code this.f.lock(); ... this.f.unlock(); is assumed to lock and unlock the same object, even if the field f could be updated in between. a later checker checks to see whether fields containing locks are ever mutated while the corresponding lock is held, because this is often a mistake by the programmer. once the analysis of each public entry point of each instantiable class is complete, some general summary information is collected: fields that contain collection objects are identified, and classified according to whether the collection object is known to be suitable for concurrent use or not. for example, the concurrent collections from the java.util.concurrent package are known to be safe for concurrent use (though see the description of the get-check-put checker below), while instances of a collection class like java.util.arraylist are definitely known to be unsafe if accessed concurrently without synchronisation. 2.3 checkers finally, to produce the analysis findings that are presented to the user, checkers are run over the information gathered by the interprocedural analysis. threadsafe contains over a dozen checkers that look for specific instances of wrong or problematic code that uses concurrency. two of the checkers are the inconsistent synchronisation checker and the get-check-put checker. inconsistent synchronisation the inconsistent synchronisation checker examines, for each field, all the accesses to that field, and determines if they all share a common lock. a warning is produced if either: all accesses are locked, but do not share a common lock; or some accesses are locked and some are not, with the ratio of locked/total being over some threshold. the inconsistent synchronisation analysis was originally based on the inconsistent synchronisation analysis in the findbugs tool, but has been extended to have a more precise determination of which locks are held, and to handle accesses to thread unsafe collections stored in fields, as well as direct accesses to fields. in particular, threadsafe’s analysis can handle cases where all accesses are locked, but with different locks. proc. avocs 2015 4 / 15 eceasst get-check-put the get-check-put checker looks for sequences of method calls on an instance of a concurrent collection, e.g., a concurrenthashmap, that check the status of the collection, and then mutate the collection based on the result of the check. if other threads are not prevented from mutating the collection between the check and the mutation, then there is a potential race condition, because the information gathered from the check will have become outdated. the name “get-check-put” comes from the common pattern where a .get(key) method is invoked to find out whether key is in the map, the result is compared to null (the “check”), and if so, the .put method is used to associate key with a new value in the map. such a pattern is often used to maintain caches of objects that are expensive to create. instances of get-check-put are detected by performing another interprocedural analysis over the results of the first interprocedural analysis, tracking the evolution of method invocations on each collection object using a typestate-style analysis. 3 examples the following sections give a few examples of bugs and potential bugs that threadsafe finds in open source codebases. as far as we are aware, none of these bugs is found by any other static analysis tool. our aim in presenting these examples is to give an impression of the kinds of bugs that threadsafe is able to find as well as a feeling for the overall look and feel of the tool. 3.1 example: incorrect synchronization of collection accesses java provides a rich assortment of collection classes, each with its own requirements on whether or not synchronization is required for concurrent access to the collection. inconsistent synchronization on collections can be particularly harmful to program behaviour. while incorrectly synchronizing accesses to a field may “only” result in missed updates or stale information, incorrectly synchronizing accesses to collections that have not been designed for concurrent use can lead to violations of the collections’ internal invariants. this may not immediately cause visible effects, but may cause odd behaviour, including infinite loops or corrupted data, at a later point in the program’s execution. see [tym09, ora12] for an example. an example of inconsistent use of synchronization when accessing a shared collection is present in version 2.10 of apache jmeter, an open source tool for testing application performance under load. threadsafe produces 44 findings for jmeter, including the one shown below in threadsafe’s eclipse plug-in. 5 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency this finding suggests a possible mistake in synchronizing accesses to the collection stored in the field resptimegraphvisualizer.internallist. we can click on the “problem location” to display the declaration of this field or on any of the locations in the list of accesses. we have clicked above on the location of the unsynchronized read access and the relevant line of code is highlighted. detailed documentation is available under “rule description” on the finding, the risks that it raises and their potential severity, and remediation options. we can ask threadsafe to show us the accesses to this field along with the locks that are held, by clicking on “accesses and locks”: there are three methods that access the collection stored in the field internallist. one of these methods is actionperformed, which will be invoked by the swing gui framework on the ui thread. another method that accesses the collection stored in internallist is add(). by investigating the possible callers of this method in the call hierarchy, we can see that it is indeed called from the run() method of a thread that is not the main ui thread of the application, indicating that synchronization ought to have been used. proc. avocs 2015 6 / 15 eceasst 3.2 example: potential deadlock k9mail is an android email client, written in java, that uses concurrent threads to prevent the user interface from becoming unresponsive during communication with the network. k9mail consists of over 1000 classes. it would be impractical to go through all of them to look for potential deadlock cycles. running the threadsafe eclipse plug-in on k9mail1 produces the following deadlock warning: we can see from this report that the intrinsic locks associated with objects of the preferences and account classes form a circular relationship. investigating this finding in the call hierarchy, we learn that the circularity can arise from the following pieces of code: 1. in the synchronized method preferences.getavailableaccounts(), there is a call on line 95 to the synchronized method account.isenabled(). 2. in the synchronized method account.save(preferences), there is a call on line 679 to the synchronized method preferences.getaccounts(). if two threads invoke the preferences.getavailableaccounts() method and the account.save() method concurrently, then there is a real chance that a deadlock will result. it will take further investigation to determine whether or not this scenario is actually possible, but threadsafe has successfully narrowed down the number of lines of code that we have to examine. 1 the version used here and in subsection 3.3 was taken from github.com/k9mail/k-9 and has commit sha1 id cc8353d25572b5f1c19047c0c093371f5ac721b4. 7 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency 3.3 example: missing synchronization in android the concurrent environment in which an application may run is almost never under the complete control of the application developer. frameworks invoke various parts of applications in response to user, network, or other external events, and often have implicit requirements about which methods may be invoked on which threads. an example of the incorrect use of frameworks is visible in k9mail. running threadsafe on k9mail yields the following finding, indicating that the field mdraftid appears to be accessed from an android background thread, and another thread, with no synchronization. clicking on “accesses and locks”, we can see that the field mdraftid has been accessed from a doinbackground method. the doinbackground method is part of the android framework’s asynctask facilities for running time-consuming tasks in the background separately from the main ui thread. correct use of asynctask.doinbackground(..) can ensure that android applications remain responsive to user input, but care must be taken to ensure that interaction between the background thread and the main ui thread is correctly synchronized. investigating further in eclipse’s call hierarchy, we discover that the ondiscard() method, which also accesses the mdraftid field, is called by the onbackpressed() method. this method is in turn always invoked by the android framework on the main ui thread, not the background thread for running asynctasks, indicating the presence of a potential concurrency defect. proc. avocs 2015 8 / 15 eceasst 4 experiences with developing and using threadsafe 4.1 what is a concurrency bug, and how do developers react to static analysis? in our experience, the primary difficulty in developing an unsound and incomplete static analysis like threadsafe is in determining exactly what ought to be reported, given the information that we are able to compute statically from the program. for example, the inconsistent synchronisation checker described above only discovers places where the programmer has been inconsistent in their use of locks when accessing a given field. it does not attempt to determine whether or not the two accesses can actually happen in parallel. nor does it attempt to determine any particular severity of the potential race condition. there may be several reasons why accesses to a field use locks inconsistently: 1. the programmer has genuinely made a mistake, and the unlocked, or incorrectly locked accesses, represent real data races that will manifest at runtime; 2. the programmer has genuinely made a mistake, but the unlocked, or incorrectly locked accesses are extremely rare at runtime, and are very unlikely to produce a real reliability issue when the code is in production; 3. the field is only ever accessed by one thread, but other fields are accessed by multiple threads, and the locks used to protect them are irrelevantly being held while accessing the single-thread field, giving the impression that it has been locked unnecessarily; 4. the methods that access the fields without locking are never executed in parallel with those that do. for example, initialisation methods or shutdown methods may be guaranteed to have exclusive access to their object; 5. the programmer has been able to determine via other means that this particular field access is safe, perhaps by reasoning based on the java memory model. in case 2, some developers that we observed using threadsafe were loath to change their code to assuage the analysis. changing code has costs of its own, and if the bug is extremely unlikely (at least in the developer’s estimation), then why bother? other users of threadsafe take a longer view, reasoning that a bug that is unlikely now may become more likely in the future after the structure and intended purpose of the code evolves. leaving concurrency bugs hidden inside code that is believed to be reliable creates technical debt that obstructs future development. in cases 3 and 5, it could be argued that the design of the code is at fault. if the locking strategy used by the code is sufficiently complex that subtle reasoning is required to determine when access to a field requires a lock or not, then the code ought to be changed. in our experience, different developers differ widely in their responses to static analysis results. some actively dislike false positives, treating them as noise; some appreciate the warnings, but require a reliable way of turning them off for code they have hand-checked to be safe; while others will rewrite code to remove static analysis warnings, perhaps in the belief that code that confuses a static analyser will confuse a human too. in general, however, we observed a lean towards conservatism in most developers: unless a particular static analysis warning obviously indicates a real, costly, bug, then it was usually felt better to leave apparently working and tested 9 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency code alone until time could be set aside to work on it properly. performing quick fixes piecemeal on complex code, following the warnings produced by a static analyser, may actually decrease reliability. accidental reduction in reliability is especially likely if it is done without an understanding of what the code is meant to do, a common situation when the original developers of some code have left the organisation. at a level above data races on individual fields, it can be very difficult to determine the difference between intended behaviour and unintended behaviour. in some cases, non-deterministic behaviour may be exactly what the programmer intends to happen in a concurrency environment, but in other cases it may be a mistake. threadsafe is designed to operate on code that has no formal specification describing what the intended behaviour is (and, often, no written informal specification either). arguably, the most damaging concurrency errors are not the low-level data races and race conditions on concurrent collections, but high-level design errors that lead to user-visible data inconsistencies. however, in the absence of information about what ought to happen, these bugs are the most difficult for a static analysis tool to find. 4.2 testing a static analyser beyond the basic question of what kinds of errors threadsafe ought to catch, or not catch, a practical aspect of the development of any heuristic static analysis is how to maintain and track the behaviour of the analysis as it is added to and bugs in the analyser are fixed. it is all too easy to fix a bug that removes some false positive, while at the same time also removing many more valuable true positives. whether or not such trade-offs are a net positive is one of the hardest parts of developing a static analyser intended for a mass audience. in order to be able to make judgements about such tradeoffs, we need to have visibility into the effect of changes we make during development of threadsafe. to ensure that development of threadsafe does not accidentally introduce new false positives, or remove valuable true positives, we maintain a large test suite of over 500 tests identifying specific true/false positives and negatives, and for each one whether it is expected to be reported or not. since most of the checkers in threadsafe rely to a greater or lesser extent on heuristics, the only practical way to document and maintain the desirable behaviour of checkers is via testing — there is no formal specification of how threadsafe ought to behave. since the individual test cases are not representative of real java projects, threadsafe is also systematically tested against a portfolio of open source and proprietary java code bases. threadsafe is tested against the benchmark suite before and after each change to determine the impact of altered heuristics, or new checkers. we do not attempt to classify all the findings of threadsafe on the benchmark suite — this turns out to be too difficult and time consuming for a small company, see what is a concurrency bug? above — but we do inspect the difference in the analysis results before and after the change, to determine whether the proposed change has an overall positive impact. the benchmark suite is also used to track the runtime of threadsafe on realistic code bases for regressions. proc. avocs 2015 10 / 15 eceasst 5 performance 5.1 measuring static analysis tool performance the performance of a static analysis tool like threadsafe can be measured in a number of ways. for example: • what percentage of the real bugs in a given codebase does the tool find (true positives) and miss (false negatives)? • what is the potential impact of the bugs found versus the ones missed? • how difficult would it be to find the same bugs by other means? • what percentage of findings produced are false alarms (false positives)? • how easy is it to investigate a finding to discover if it is a genuine bug and how to fix it? • how much time and space does the tool consume? • how well does it scale for use with very large codebases? • how smoothly is the tool integrated with a software developer’s existing workflow? many of these are difficult to measure. for instance: in any very large real-world codebase, computing the percentage of real bugs found assumes knowledge of all of the real bugs that it contains. determining whether or not a finding reports a real bug or is a false alarm, and the potential impact of a real bug, often relies on expert knowledge of the architecture of the system and/or the way that it is intended to be used. some aspects are somewhat subjective and a matter of taste or attitude. finally, measurements of performance on small or textbook examples may bear little relation to performance on large real-world codebases. for a commercial static analysis tool like threadsafe, intended for use in industrial software development, the tradeoffs between these factors are different from what an academic researcher might expect. when working with a million-line codebase, developers are primarily interested in finding and fixing very high-impact bugs quickly. their experience tells them that some bugs will remain, no matter what they do. and since fixing a bug might introduce a new bug, as explained earlier, it might well be prudent to leave a low-impact bug unfixed. a tool that reports large numbers of low-impact bugs alongside a few high-impact bugs may therefore be less useful than one that reports only the high-impact bugs. but filtering lists of findings according to type will sometimes allow likely high-impact bugs to be separated from low-impact bugs. any static analysis tool needs to balance false positives against false negatives: higher sensitivity will tend to reveal more potential bugs, but some of those found will turn out to be false alarms. if the aim is to eliminate high-impact bugs quickly, then a high level of false positives is more damaging than a moderate level of false negatives. developers will quickly lose patience with a tool if they need to investigate a long list of false alarms before encountering a genuine high-impact bug. 11 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency more interesting than the absolute performance of a tool is its performance in comparison to competing tools. unfortunately, a head-to-head comparison between threadsafe and competing commercial tools is not possible because their licenses forbid their use for this purpose. our experiments suggest that threadsafe outperforms other commercial static analysis tools with respect to detection of java concurrency defects, and users of other tools are invited to undertake a comparison themselves. we provide a comparison below between threadsafe and the non-commercial findbugs tool [fin15], the most widely-used static analysis tool for java. 5.2 performance on ibm benchmark the ibm concurrency bugs benchmark [ehsu07, etu08, ibm15] is a collection of java programs with concurrency bugs, meant for use in comparing bug-finding tools; it is the most widely-used benchmark for this purpose. most are small programs produced by students at the university of haifa as coursework for a software testing course and the bugs vary in complexity. see table 1 for a list. an example of a simple bug is a use of the double-checked locking anti-pattern [pug00] in the program dcl. an example of a bug that seems well beyond the reach of an automatic static-analysis tool is an atomicity bug in the program lottery that leads to violation to an implicit logical invariant. an earlier version of the benchmark was annotated with information about the location(s) and type(s) of bugs in each program, but this information and some of the programs in the benchmark have been lost [tb15]. threadsafe and findbugs (with only “multithreaded correctness” checkers enabled) were both applied to each program, first with default settings (first column) and then with sensitivity increased (second column). for threadsafe, sensitivity was increased by reducing the threshold for inconsistent synchronisation from 70% to 50%. for findbugs, sensitivity was increased by setting analysis effort to “maximal” and setting the minimum rank to report so that all findings would be reported. findings produced by both tools were manually classified as true positives or false positives and the number of concurrency bugs in each program was counted. the tp/fp classification and count of bugs are admittedly somewhat subjective. the results produced by threadsafe and findbugs on this benchmark are given in table 1. there are many bugs in the benchmark, including the atomicity bug in lottery mentioned above, that neither tool is able to detect. threadsafe finds about twice as many bugs as findbugs does, with about the same false positive rate. there are a few bugs that findbugs detects which threadsafe, despite its more sophisticated analysis, misses. an example is the bug in piper, where there is an invocation of the method java.lang.object.wait() that is not in a loop. findbugs contains a checker for exactly this bug pattern. in developing threadsafe, we decided not to duplicate checkers in findbugs unless we were able to significantly improve on its results, since it is easy to use findbugs alongside threadsafe. this is just such a case. spathoulas [spa14] did a similar comparison of threadsafe with findbugs and chord [cho12] on the ibm benchmark and on concurrency examples from the cert oracle secure coding standard for java [lms+11], comparing threadsafe with default sensitivity against findbugs with increased sensitivity and the datarace and deadlock analyses in chord. his comparison did not classify findings into true and false positives. he found that chord was the most accurate of the three tools but that it was not scalable for use with codebases of even modest proc. avocs 2015 12 / 15 eceasst threadsafe threadsafe findbugs findbugs default sensitive default sensitive # bugs tp / fp tp / fp tp / fp tp / fp a b push pop 0 / 0 0 / 0 0 / 0 0 / 0 1 account 0 / 0 1 / 0 0 / 0 1 / 0 1 airlines tickets 0 / 0 0 / 0 0 / 1 0 / 1 1 allocationvector 0 / 0 0 / 0 0 / 0 0 / 0 1 boundedbuffer 0 / 3 0 / 4 0 / 0 0 / 3 1 bubblesort 1 / 0 1 / 0 1 / 0 1 / 0 1 bubblesort2 1 / 0 1 / 0 0 / 0 0 / 0 1 bufwriter 1 / 3 2 / 5 0 / 0 0 / 0 2 critical 0 / 0 0 / 0 1 / 0 1 / 0 1 dcl 1 / 0 1 / 0 1 / 0 1 / 0 1 deadlock 0 / 0 0 / 0 0 / 1 0 / 1 1 deadlockexception 0 / 0 0 / 0 0 / 0 0 / 0 1 fiforeadwritelock 0 / 0 0 / 0 0 / 0 0 / 0 1 filewriter 0 / 0 2 / 0 1 / 0 1 / 0 2 garagemanager 1 / 0 1 / 2 0 / 0 0 / 0 1 hierarchy example 0 / 1 0 / 1 0 / 0 0 / 1 1 linkedlist 0 / 0 0 / 0 0 / 0 0 / 0 1 liveness 0 / 0 0 / 0 0 / 0 0 / 0 1 lottery 1 / 0 2 / 0 0 / 0 0 / 0 2 manager 2 / 3 2 / 3 0 / 0 0 / 0 2 mergesort 0 / 1 0 / 1 0 / 0 0 / 0 1 mergesortbug 0 / 0 0 / 0 0 / 0 0 / 0 1 pingpong 0 / 0 0 / 0 0 / 0 0 / 0 1 piper 0 / 0 0 / 0 1 / 0 1 / 0 1 producerconsumer 0 / 0 0 / 0 0 / 0 0 / 0 1 shop 1 / 0 1 / 0 0 / 0 1 / 0 2 suns account 0 / 0 0 / 0 0 / 0 0 / 0 1 xtangoanimation 5 / 0 5 / 0 2 / 3 2 / 5 5 total 14 / 11 19 / 16 7 / 5 9 / 11 37 false positive rate: fp / (fp+tp) 44% 46% 42% 55% percentage of bugs found 38% 51% 19% 24% table 1: threadsafe and findbugs applied to the ibm benchmark 13 / 15 volume 72 (2015) threadsafe: static analysis for java concurrency size. he also found that threadsafe was much better than findbugs at resisting attempts to hide bugs using very simple obfuscation methods. this is to be expected because of the comparatively simple bug-pattern detection methods used by findbugs [hp04], but it suggests that the performance gap between threadsafe and findbugs will widen for codebases containing more complex and hidden bugs. although comparison of performance on small examples like the ones in the ibm benchmark provides a useful data point, is not a reliable guide to performance on large real-world examples. at least, that is our experience with deadlock detection. the heuristics that threadsafe uses to achieve a good rate of deadlock detection in reasonable time sometimes yield poor performance on small “textbook-style” examples. for real industrial-scale examples, its results tend to be much better. threadsafe’s heuristics could be adjusted to perform better on the small examples, but this would increase the false positive rate on larger examples. we have not yet found a way to achieve good results in both cases. 6 packaging threadsafe is tightly integrated with eclipse, as shown above, and with the sonarqube quality platform. it can also be used from the command line to generate an html report containing information similar to what is available in eclipse. contemplate’s website http://www.contemplateltd.com/threadsafe provides more information about threadsafe and access to free two-week trials. bibliography [cho12] chord 2.1. 2012. http://www.cc.gatech.edu/∼naik/chord.html [ehsu07] y. eytani, k. havelund, s. d. stoller, s. ur. towards a framework and a benchmark for testing tools for multi-threaded programs. concurrency and computation: practice and experience 19(3):267–279, 2007. http://dx.doi.org/10.1002/cpe.1068 [etu08] y. eytani, r. tzoref, s. ur. experience with a concurrency bugs benchmark. in first international conference on software testing verification and validation, icst 2008, lillehammer, norway, april 9-11, 2008, workshops proceedings. pp. 379– 384. 2008. http://dx.doi.org/10.1109/icstw.2008.17 [fin15] findbugs 3.0.1. 2015. http://findbugs.sourceforge.net [goe06] b. goetz. java concurrency in practice. addison-wesley, 2006. proc. avocs 2015 14 / 15 http://www.contemplateltd.com/threadsafe http://www.cc.gatech.edu/~naik/chord.html http://dx.doi.org/10.1002/cpe.1068 http://dx.doi.org/10.1109/icstw.2008.17 http://findbugs.sourceforge.net eceasst [hp04] d. hovemeyer, w. pugh. finding bugs is easy. sigplan notices 39(12):92–106, dec. 2004. http://doi.acm.org/10.1145/1052883.1052895 [hs12] m. herlihy, n. shavit. the art of multiprocessor programming. morgan kaufmann, 2012. [ibm15] ibm. concurrency benchmark. accessed aug 2015. http://researcher.watson.ibm.com/researcher/view person subpage.php?id=5722 [lee06] e. a. lee. the problem with threads. ieee computer 39(5):33–42, 2006. http://dx.doi.org/10.1109/mc.2006.180 [lms+11] f. long, d. mohindra, r. c. seacord, d. f. sutherland, d. svoboda. the cert oracle secure coding standard for java. addison wesley, 2011. [ora12] jdk-7027300 : unsynchronized hashmap access causes endless loop. 2012. oracle bug report. http://bugs.java.com/view bug.do?bug id=7027300 [pug00] w. pugh. the ”double-checked locking is broken” declaration. 2000. http://www.cs.umd.edu/users/pugh/java/memorymodel/doublecheckedlocking. html [ses14] p. sestoft. practical concurrent and parallel programming. 2014. course at it university of copenhagen. http://www.itu.dk/people/sestoft/itu/pcpp/e2014/ [spa14] a. spathoulas. assessing tools for finding bugs in concurrent java. master’s thesis, 2014. school of informatics, university of edinburgh. http://homepages.inf.ed.ac.uk/dts/students/spathoulas/spathoulas.pdf [tym09] p. tyma. a beautiful race condition. 2009. mailinator blog post. http://mailinator.blogspot.co.uk/2009/06/beautiful-race-condition.html [tb15] r. tzoref-brill. private communication. august 2015. 15 / 15 volume 72 (2015) http://doi.acm.org/10.1145/1052883.1052895 http://researcher.watson.ibm.com/researcher/view_person_subpage.php?id=5722 http://dx.doi.org/10.1109/mc.2006.180 http://bugs.java.com/view_bug.do?bug_id=7027300 http://www.cs.umd.edu/users/pugh/java/memorymodel/doublecheckedlocking.html http://www.cs.umd.edu/users/pugh/java/memorymodel/doublecheckedlocking.html http://www.itu.dk/people/sestoft/itu/pcpp/e2014/ http://homepages.inf.ed.ac.uk/dts/students/spathoulas/spathoulas.pdf http://mailinator.blogspot.co.uk/2009/06/beautiful-race-condition.html introduction how threadsafe operates class loading and preprocessing interprocedural per-class analysis checkers examples example: incorrect synchronization of collection accesses example: potential deadlock example: missing synchronization in android experiences with developing and using threadsafe what is a concurrency bug, and how do developers react to static analysis? testing a static analyser performance measuring static analysis tool performance performance on ibm benchmark packaging electronic communications of the easst volume 079 (2020) second interactive workshop on the industrial application of verification and testing, etaps 2020 workshop (interavt 2020) preface 2 pages guest editors: stylianos basagiannis, goetz botterweck, anila mjeda eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst preface this volume presents the proceedings of the second interactive workshop on the industrial application of verification and testing (interavt 2020), a workshop co-hosted with etaps 2020. interavt aims to remove barriers that hinder the application of modern verification and testing techniques in industrial practice. modern verification and testing techniques are highly relevant for industrial softwareintensive systems. recent technological trends (e.g., the new dominating role of software in traditional domains like automotive and aerospace, the arrival of autonomous and “smart” systems in everyday life) only increase the need for industrial-scale robust approaches. however, there are still many barriers and challenges that impede the application of modern verification and testing techniques in industrial practice. the industrial scale and complexity of “real” systems (transfer and scale-up of research prototypes); usability or feasibility of formal techniques in practice (packaging novel approaches in usable tools); new system paradigms, which make systems harder to model, analyse, verify and test (e.g., autonomous systems, machine learning); domain-specific challenges and constraints, e.g., in safety-critical systems like automotive, aerospace, medical systems; and, research approaches targeting problems that lead to publishable results, but are disconnected from industrial practice are but a few examples of the current and pressing challenges for industry. arguably these challenges can only be tackled with increased exchange and collaboration between academic researchers and industrial practitioners. hence, the interavt workshop focuses on fostering communication between people working on similar problems and establishing new links and opportunities for collaboration between participants from different backgrounds. initially, it was planned to hold an in-person workshop that maximises the interaction between participants and the resulting progress – with a program that includes short presentations, a “speed-dating” session, in which participants engage in a time-limited discussion with each other, and a collaboration session that focuses on summarising the identified key pain points facing the application of modern verification and testing techniques in industrial practice. however, due to the continued effects of the ongoing covid 19 pandemic, interavt 2020 focussed on reviewing and selecting articles for a special issue on the topic. we sincerely hope to return to interactive in-person events very soon. we thank the authors for their contributions and also express our deep gratitude to all the members of the program committee for their insightful and rigorous work, which was essential to ensure high-quality content. stylianos basagiannis, anila mjeda, and goetz botterweck 1 / 2 volume 079 (2020) preface interavt workshop organization organizing committee: stylianos basagiannis raytheon technologies research centre goetz botterweck lero the irish software research centre and university of limerick anila mjeda lero the irish software research centre and university of limerick program committee: alessandra bagnato, softeam, fr stylianos basagiannis, raytheon technologies research centre, ie cinzia bernardeschi, university of pisa, it armin biere,johannes kepler university linz, at goetz botterweck, lero and university of limerick, ie jorg brauer, verified systems, de alessandro fantechi, university of florence, it alberto griggio, fondazione bruno kessler, it patrick heymans, university of namur, be christian koenig, twt, de tiziana margaria, lero and university of limerick, ie anastasia mavridou, nasa ames research centre, us anila mjeda, lero and university of limerick, ie david parker, university of birmingham, uk tomáš vojnar, brno university of technology, cz interavt 2020 2 / 2 implementing petri net transformations using graph transformation tools electronic communications of the easst volume 14 (2008) proceedings of the third workshop on petri nets and graph transformations (pngt 2008) implementing petri net transformations using graph transformation tools enrico biermann, claudia ermel, tony modica and peggy sylopp 14 pages guest editors: paolo baldan, barbara könig managing editors: tiziana margaria, julia padberg, gabriele taentzer eceasst home page: http://www.easst.org/eceasst/ issn 1863-2122 http://www.easst.org/eceasst/ eceasst implementing petri net transformations using graph transformation tools enrico biermann, claudia ermel, tony modica and peggy sylopp institut für softwaretechnik und theoretische informatik technische universität berlin, germany formalnet@cs.tu-berlin.de abstract: petri net transformations have been defined formally in the abstract framework of adhesive hlr categories, which allows rule-based rewriting of graphlike structures, similar to graph transformation. in this paper we discuss differences between petri net rewriting and graph rewriting which makes it necessary to add checks and conditions when implementing petri net transformations using an existing graph transformation tool like agg. the extensions concern the preservation of petri net transition firing behavior and the mapping of markings. as a running example, we present the ron environment, a visual editor, simulator and net transformation tool for reconfigurable petri nets which has been developed as a plug-in for eclipse based on the graph transformation engine agg. keywords: petri nets, net transformation, graph transformation, visual editor, reconfigurable petri nets, petri net tool 1 introduction modeling the adaption of a system to a changing environment becomes more and more important. application areas cover e.g. computer-supported cooperative work, multi agent systems, dynamic process mining or mobile networks. one approach to combine formal modeling of dynamic systems and controlled model adaption are reconfigurable petri nets [hep07, ehp+07]. the main idea is the stepwise reconfiguration of place/transition nets by given net transformation rules [ehp+08, ehp06]. think of these rules as replacement systems where the left-hand side is replaced by the right-hand side while preserving a context. this approach increases the expressiveness of petri nets and allows in addition to the well known token game a formal description of structural changes. since petri nets can be considered as bipartite graphs the concept of graph transformations [eept06] can be applied in principle to define also transformations of petri nets [ep04]. unfortunately, there are some differences between graph morphisms and our notion of petri net morphisms which lead to different transformation behaviors. the aim of this paper is to identify the differences in both approaches and to offer solutions that allow a simulation of petri net transformations by graph transformation. note that other graph transformation-based simulation tools also deal with structural transformations of petri nets (e.g. [lva04]), but they do not guarantee the preservation of firing behavior in petri net transformations. similarly, in the approach of llorens and oliver [lo04], rewriting of petri nets by graph transformation rules is used for 1 / 14 volume 14 (2008) mailto:formalnet@cs.tu-berlin.de implementing petri net transformations the reconfiguration of nets, but does not preserve the firing behavior. instead, reconfiguration focuses on the modification of the flow relation, i.e. only arcs are added / deleted or reconnected. our paper is structured as follows: in section 2 we review the formal definitions of graph and net morphisms, as well as of graph and net transformations. section 3 outlines the main differences between both transformation formalisms, dealing in particular with markings and the preservation of transition-firing behavior. we present solutions that allow a simulation of petri net transformations by graph transformation. finally, in section 4, we sketch a prototypical implementation of a visual petri net simulation and transformation tool, which converts petri nets and net transformation rules to graphs and graph transformation rules and uses the graph transformation engine agg [agg] to compute net transformations. 2 graph and petri net transformation 2.1 graph transformation the research area of graph transformation [eept06, roz97] dates back to the early seventies. methods, techniques, and results from the area of graph transformation have already been studied and applied in many fields of computer science such as formal language theory, pattern recognition and generation, software engineering, concurrent and distributed system modeling, model transformation, and visual language design. for our aim to represent petri net transformations by graph transformation we need typed, attributed graphs and graph morphisms which are especially suited for visual language modeling: a visual language (vl) is modeled by an attributed type graph capturing the definition of the underlying visual alphabet, i.e. the symbols and relations which are available. sentences or diagrams of the vl are given by attributed graphs typed over (i.e. conforming to) the type graph. such a vl type graph corresponds closely to a meta model. the following definitions are taken from [eept06]. the key idea is to model an attributed graph with node and edge attribution, where the underlying graph g is a new kind of graph, called an e-graph, which allows attribution edges not only from graph nodes to attribute nodes but also from graph edges to attribute nodes. this new kind of attributed graph, combined with the concept of typing, leads to a category agraphsatg of attributed graphs typed over an attributed type graph at g. this category proved to be an adequate formal model not only for various applications in software engineering and visual languages but also for the internal representation of attributed graphs in our graph transformation tool agg [ter99, agg]. definition 1 (e-graph and e-graph morphism) an e-graph g with g = (vg,vd, eg, ena, eea, (source j,target j) j∈{g,na,ea}) consists of the sets vg and vd, called the graph and data nodes (or vertices), respectively; proc. pngt 2008 2 / 14 eceasst eg, ena, and eea called the graph edges, node attribute edges, and edge attribute edges; srcg : eg →vg, targ : eg →vg, source and target functions for graph edges; srcna : ena → vg, tarna : ena → vd, source and target functions for node attribute edges; srcea : eea → eg, tarea : eea → vd, source and target functions for edge attribute edges. consider the e-graphs g1 and g2 with gk = (v kg,v k d, e k g, e k na, e k ea, (source k j,target k j ) j∈{g,na,ea}) for k = 1, 2. an e-graph morphism f : g1 → g2 is a tuple ( fvg , fvd , feg , fena , feea ) with fvi : v 1i → v 2 i and fe j : e 1 j → e 2 j for i ∈{g, d}, j ∈{g, na, ea} such that f commutes with all source and target functions, for example fvg ◦source 1 g = source 2 g ◦ feg . an attributed graph is an e-graph combined with an algebra over a data signature dsig. in the signature, we distinguish a set of attribute value sorts. the corresponding carrier sets in the algebra can be used for the attribution. definition 2 (attributed graph and attributed graph morphism) let dsig = (sd, opd) be a data signature with attribute value sorts s′d ⊆ sd. an attributed graph ag = (g, d) consists of an e-graph g together with a dsig-algebra d such that � ∪s∈s′d ds = vd. for two attributed graphs ag1 = (g1, d1) and ag2 = (g2, d2), an attributed graph morphism f : ag1 → ag2 is a pair f = ( fg, fd) with an e-graph morphism fg : g1 → g2 and an algebra homomorphism fd : d1 → d2 such that (1) commutes for all s ∈ s′d, where the vertical arrows are inclusions: for the typing of attributed graphs, we use a distinguished graph attributed over the final dsig-algebra z (see [eept06]). this graph defines the set of all possible types. definition 3 (typed attributed graph and typed attributed graph morphism) given a data signature dsig, an attributed type graph is an attributed graph at g = (t g, z), where z is the final dsig-algebra. a typed attributed graph (ag,t) over at g consists of an attributed graph ag together with an attributed graph morphism t : ag → at g. a typed attributed graph morphism f : (ag1,t 1)→(ag2,t 2) is an attributed graph morphism f : ag1 → ag2 such that t 2 ◦ f = t 1: as an example for typed, attributed graphs we refer to def. 11 in section 3. the main idea of graph transformation is the rule-based modification of graphs where each application of a graph transformation rule leads to a graph transformation step. the core of a graph transformation rule p = (l l← k r→ r) is a pair of graphs (l, r), called left-hand side (lhs) and right-hand side (rhs), an interface k = l∩r and an two injective graph morphisms l l← k and k r→ r embedding the interface in the leftand right-hand sides. applying the rule p means to find a match of l in the source graph g and to replace this matched part by r, thus 3 / 14 volume 14 (2008) implementing petri net transformations transforming the source graph into the target graph of the graph transformation. intuitively, the application of rule p = (l l← k r→ r) to graph g via a match m from l to g deletes the image m(l) from g and replaces it by a copy of the right-hand side m∗(r). note that a rule may only be applied if the so-called gluing condition is satisfied, i.e. the deletion step must not leave dangling edges, and for two objects which are identified by the match, the rule must not preserve one of them and delete the other one. definition 4 (typed attributed graph rule) given an attributed type graph at g with a data signature dsig, a typed attributed graph rule, or “rule” for short, p = (l l←k r→r) consists of typed attributed graphs l, k, and r with a common dsig-algebra tdsig(x ), the dsig-termalgebra with variables x , and injective typed attributed graph morphisms l : k → l, r : k → r, where the dsig-part of l and r is the identity on tdsig(x ). for the definition of graph transformations we need pushouts in the category agraphsatg, the existence of which is shown in [eept06]. definition 5 (typed attributed graph transformation) given a rule p = (l l← k r→ r) as defined above and a typed attributed graph g with a typed attributed graph morphism m : l → g, called match, a direct typed attributed graph transformation, or “direct graph transformation” for short, g p,m =⇒ h from g to a typed attributed graph h is given by the following double pushout (dpo) diagram in the category agraphsatg, where (1) and (2) are pushouts. informally, a pushout in a category cat is a gluing construction of two objects over a specific interface. the rule r may be extended by a set of negative application conditions (nacs) [hht96, eept06]. a match l m−→ g satisfies a nac with the injective nac morphism n : l → nac, if there is no injective graph morphism nac q −→ g with q◦n = m. nac q | dd dd !!d dd d l m �� noo (1) k k �� r //loo (2) r m∗ �� g d //oo h a sequence g0 ⇒ g1 ⇒ ... ⇒ gn of graph transformation steps is called graph transformation and denoted as g0 ∗⇒ gn. note that for flexible rule application, variables for attributes can be used in rules, which are instantiated by concrete values by the rule match. moreover, boolean attribute conditions (expressions over attributes) may be combined with a rule to control rule application. 2.2 petri net transformation reconfigurable place/transition systems are petri nets with initial markings and a set of rules which allow the modification of the net during runtime in order to adapt the net to new requirements of the environment. thus, not only the successor marking can be computed but also the structure can be changed by rule application to obtain a new p/t-system that is more appropriate with respect to some requirements of the environment. moreover these activities can be interleaved [ehp+07]. for rule-based transformations of p/t-systems we use the framework of net transformations from [ehp+08, ehp+07]. proc. pngt 2008 4 / 14 eceasst definition 6 (p/t net, p/t system) a p/t-net is given by pn = (p, t, pre, post) with sets of places p and transitions t , and preand post domain functions pre, post : t → p⊕, where p⊕ is the free commutative monoid over the set p of places with binary operation ⊕ (e.g. the monoid notation m = 2p1 ⊕3p2 means that we have two tokens on place p1 and three tokens on p2). a p/t-system is given by (pn, m) with marking m ∈ p⊕, note that m can also be considered as function m : p → n where only for a finite set p′ ⊆ p we have m(p) ≥ 1 with p ∈ p′. we can switch between these notations by defining ∑p∈p m(p)·p = m ∈p⊕. moreover, for m1, m2 ∈p⊕ we have m1 ≤ m2 if m1(p) ≤ m2(p) for all p ∈ p. a transition t ∈ t is m-enabled for a marking m ∈ p⊕ if we have pre(t) ≤ m, and in this case the successor marking m′ is given by m′ = m pre(t)⊕ post(t) and (pn, m) t−→ (pn, m′) is called firing step. note that the inverse of ⊕ is only defined in m1 m2 if we have m2 ≤ m1. in order to define rules and transformations of p/t-systems we introduce p/t-morphisms which preserve firing steps by condition (1) below. additionally they require that the initial marking at corresponding places is increasing (condition (2)) or even stronger (condition (3)). definition 7 (p/t-morphisms) given p/t-systems pni = (pni, mi) with pni = (pi, ti, prei, posti) for i = 1, 2, a p/t-morphism f : (pn1, m1) → (pn2, m2) is given by f = ( fp, ft ) with functions fp : p1 → p2 and ft : t1 → t2 satisfying (1) f⊕p ◦ pre1 = pre2 ◦ ft and f ⊕ p ◦ post1 = post2 ◦ ft and (2) m1(p) ≤ m2( fp(p)) for all p ∈ p1. note that the extension f⊕p : p ⊕ 1 �