Bio-based and Applied Economics 6(1): 57-79, 2017 ISSN 2280-6180 (print) © Firenze University Press ISSN 2280-6172 (online) www.fupress.com/bae Full Research Article DOI: 10.13128/BAE-18535 A stakeholder engagement approach for identifying future research directions in the evaluation of current and emerging applications of GMOs DaviDe Menozzi1*, Kaloyan Kostov2, Giovanni soGari1, salvatore arpaia3, Daniela MoyanKova2, Cristina Mora1 1 Department of Food and Drug, University of Parma, Parma 43125, Italy 2 ABI – Agrobioinstitute, Sofia 1164, Bulgaria 3 ENEA – National Agency for New Technologies, Energy and Sustainable Economic Development, Rotondella (MT) 75026, Italy Date of submission: 2016 30th, June; accepted 2017 3rd, February Abstract. The yield of several commodity crops is provided in large part by genetically modified crops in North and South America. However, reservations exist in Europe due to possible negative effects on human health or environment. This paper aims to ana- lyse the current research priorities identified in EU countries and to engage European stakeholders into the formulation of future common research needs regarding the effects of the possible adoption of commercially available and forthcoming genetically modi- fied organisms (GMOs) in the areas of socio-economics, human and animal health, and environment. Additionally, it aims to identify the requirements for sharing avail- able research capacities and existing infrastructures. First a mapping exercise of existing research activities in Europe was performed. A questionnaire was developed on a web- based platform and submitted to national focal points to collect information from EU Member States. Information was collected from 320 research projects conducted in the last 10 years in Europe. To refine results of the surveys, twenty invited experts and stake- holders from the public funding agencies of different EU Member States participated in an international workshop. This paper reports the main findings of these activities. Keywords. Genetically modified organisms (GMOs), socio-economic, human and animal health, environment, workshop JEL codes. Q16, Q55, O30 1. Introduction Scientific and technological progress in agriculture has resulted in innovations that have contributed to increase production and productivity. Genetically modified (GM) *Corresponding author: davide.menozzi@unipr.it 58 D. Menozzi crops have shown an extremely rapid adoption rate in many areas of the world. About 12 percent (179.7 million of 1.5 billion hectares) of global cropland was invested with GM crops in 2015 (James, 2015). Maize area summed up to 53.7 million hectares in 2015 and GM soybean was cultivated over 92 million hectares during the same cropping season. Herbicide tolerance (HT) crops occupy 100 million ha, insect resistant (IR) 26 million ha and crops expressing stacked HT and IR traits were planted on 45 million ha. However at the same time, in different world areas genetically modified organisms (GMOs) have experienced a transnational opposition from different interests groups (Herring, 2008). Opposition to transgenic crops has often argued the lack of sufficient scientific data dem- onstrating that GM crops are harmless to humans and to the environment (Rausser et al., 2015; Yang and Chen, 2016). Although these uncertainties about food and feed products derived from plant breeding is not confined to transgenic plants (Herring, 2008) and that, beyond transgenic plants, alternative methods are being applied to obtain new crop varie- ties (Parisi et al., 2016), GMOs are often questioned in regards of the uncertainty of their possible risks. Schurman and Munro (2010) describe how these concerns gained consen- sus in a network of stakeholders, including consumer, environmental, and social-justice organizations. The European Union has endorsed the precautionary principle and there- fore in its risk assessment a central role is sought in addressing and dealing with these uncertainties. The EU regulations on GMOs constitute a salient issue of risk governance given the politically high visibility of the topic (Drott et al., 2013). The EU regulatory framework on GMOs includes rules on authorization conditions, traceability, labelling, segregation, co-existence, which are established by the European Commission based on the risk assessment procedures conducted by the European Food Safety Authority (EFSA) which provides independent scientific advice on this topic (Drott et al., 2013). According- ly, scientific research promoted by the European Commission so far has also been framed considering the (potential) positive and negative effects of GMOs. Despite the global opposition, transgenic crops have spread rapidly in the agribusi- ness, and the number of GM events at the commercial cultivation, precommercial or reg- ulatory stages has more than doubled between 2008 and 2014 (Parisi et al., 2016). The uneven adoption rate of GM crops is still evident. US continues to be the lead country with 70.9 million hectares (ca 40% of global) with about 90% adoption for the principal crops: maize, soybean and cotton (James, 2015). While GM plants currently available fea- ture a limited set of different traits, there are several crops with novel traits in the regu- latory pipeline and at late stages of research and development (R&D) (e.g., resistance to viruses and pests, tolerance to drought, modified chemical composition, enhanced nutri- tional content, etc.) (National Academy of Science, Engineering and Medicine, 2016). Cul- tivation in the EU has remained limited to Bt-maize that in 2013 has been cultivated in almost 150,000 hectares mainly in Spain (137,000 ha), followed by Portugal, the Czech Republic, and Romania and Slovakia (European Commission, 2015). However, five new GM events for cultivation are currently being examined by EU Commission for a possible imminent approval. At the same time, the number of experimental field release trials has seen a continuous decline over the last years (Gómez-Galera et al., 2012). It is notable that although a plethora of research projects have been conducted result- ing in scientific publications which examine the impacts of GM crops on the receiving environments, on animal and human health, and on the functioning of farms, markets and 59Stakeholder engagement: identifying future directions for GMO research rural communities, the technology is still controversial on a number of levels. There is a large body of scientific evidence suggesting that, although there are still reasons for concern and associated risks which must be carefully assessed (e.g., crop failures, price increases, seed market monopolisation and farmers’ dependency on a few technology providers, co- existence with non-GM crops, negative impacts on non-target organisms, and resistance development in target pest populations, etc.), when managed and used appropriately GM crops may provide notable benefits (e.g., reduced use of pesticides, implementation of no- till agriculture which sequesters carbon and builds up exhausted soils, increased harvests, revenues and profits for farmers, reduced mycotoxin content in harvested maize, etc.) (Bar- am and Bourrierm, 2011; Graef et al., 2012; Mora et al., 2012; Jacobsen et al., 2013; Devos et al., 2014; Mannion and Morse, 2012). At the same time, there is also a growing body of completed and on-going scientific programmes specifically dedicated to the assessment of the potential socio-economic, environmental and health effects of the use of GM crops within both Europe and globally (European Commission, 2010). Given the number of research initiatives, it is important to focus the available resourc- es for research on the most critical gaps in our knowledge, so that more informed regula- tory and policy decisions can be made in the future. This means, at the EU level, to sig- nificantly enhance the alignment of the research programmes of the individual Member States, identifying knowledge gaps and capacity building needs, in order to avoid duplica- tion of work in these areas, to leverage complementarities, and to enhance coordination between scientists from all over Europe. This should be done improving the engagement of stakeholders (e.g., industry, farming organisations, civil society organisations – CSO, non-governmental organisations – NGOs, EU and national competent authorities, fund- ing organisations, academia, etc.) in the shaping of future research agendas and pro- grammes, in order to make these research programmes more meaningful to the end-users of the scientific results, and to increase legitimisation of research trajectories and owner- ship (Ross, 2007; Noteborn and van Duijne, 2011; Graef et al., 2012). The involvement of stakeholders in the identification of risks and concerns is believed to have a key role in the process of technology evaluation. The use of GMOs is given as an example for con- tested innovation in the EU, which failed to take into consideration the ethical concerns, uncertainties and risks at an early stage of the technology development (van den Hoven, 2013). Although the use of GM crops in agriculture remains greatly questioned in the EU, new varieties have been developed around the world, which may find their way into the EU market in the near future (Parisi et al., 2016). Genomic technologies have substan- tially improved since the appearance of first cultivated GM crops, so that individual plant’s genome can be sequenced and analysed, genotyping methods have improved in through- put and cost efficiency; thus, it is likely that additional traits can be introduced into cul- tivated plants with increased efficiency and reduce costs associated with breeding. Such developments imply the need to steer the public research policy to invest its resources in better correspondence to the social concerns related to genetic technologies. This paper is part of the PreSto GMO ERA-Net project1 aiming at creating and suc- cessfully implementing an ERA-Net (European Research Area Network)2 that will coor- 1 PreSto GMO ERA-Net (Preparatory steps towards a GMO research ERA-Net), EU FP7, Grant agreement n. 612739. See the website: http://www.presto-gmo-era-net.eu. 2 The objective of the ERA-NET scheme is to step up the cooperation and coordination of research activities 60 D. Menozzi dinate research activities carried out at national or regional level in the Member States and the mutual opening of national and regional research programmes on the effects of GMOs in the areas of socio-economics, human and animal health, and the environment (Rauschen et al., 2015). In particular, this paper aims to identify knowledge gaps and future research needs on the effects of GMOs based on the analysis of the research priori- ties and a dialogue with the stakeholders. We present the results of a mapping exercise of existing research activities on the effects of GMOs in Europe, and the main outcomes of an international workshop with relevant experts and stakeholders, European institutions, and CSOs held in Milan in November 2014. 2. Material and methods 2.1 Mapping of existing research activities The first step was to provide an overview of existing research activities and knowl- edge regarding the socio-economic, health, and environmental effects of GMOs in Europe. This was performed by an up-to-date mapping of national research programmes, projects, infrastructures, activities, research groups and capacities in the EU and internationally. The following GMO assessment databases or datasets were used in mapping existing research activities: SCAR-Collaborative Working Group “GMO Risk Research” (SCAR-CWG, 2012), the BiosafeRes database3, and the European Commission’s compendium summarizing the results of 50 GMO research projects, co-funded by the EC and conducted in the period 2001-2010 (European Commission, 2010). The data were integrated and updated with a questionnaire developed on a web-based platform (the CADIMA4 database) and submit- ted to national focal points to collect information from Member States (Moyankova and Kostov, 2015). Through the questionnaire it was possible to collect details describing recent and ongoing projects examining the GMO effects, such as the thematic area of the research, the sources of funding and the type of organizations that carried out the research. The questionnaire was designed in a specialized section of the CADIMA database for col- lection of data about the recent and ongoing GMO assessment projects. The projects were characterized by one reviewer according to several categories including: a. type of funding (government, EU funding, industry, other); b. type of project leading organization (research/academy, individual, private company, other); c. regional level of the project consortium (national, international EU, international beyond EU); d. type of organizations in the project consortium (industry, research/academy, govern- carried out at national or regional level in the Member States and Associated States through the networking of research activities conducted at national or regional level, and the mutual opening of national and regional research programmes. 3 The BiosafeRes database is a worldwide, web-based, free and public-access database of past and current research projects in GMO Biosafety, is improving communication within the scientific community, and thus clearly facilitates development of more and better worldwide collaborative research ventures in this field by encouraging synergy. Available at: http://biosaferes.icgeb.org. 4 Central Access Database for Impact Assessment of Crop Genetic Improvement Technologies, see the website: http://www.cadima.info. 61Stakeholder engagement: identifying future directions for GMO research ment, other, or mix of them); e. type of GMO analysed in the project (GM plant, GM animal, GM micro-organisms); f. main topic of GMO impact assessment (environment, animal health, human health, technology/society, other); g. sub-topics of impact assessment (e.g., for environment: soil, water, air, biodiversity, plant pest and diseases, geochemical variables, landscape structure, target effects, non- target effects, other). 2.2 The stakeholder engagement approach Starting from the results of the mapping exercise, the objective of the international workshop held in Milan on November 24th, 2014, was to use a transparent and structured approach for recommending a list of transnational research needs regarding the effects of GMOs in the areas of socio-economics, human and animal health, and environment, as well as requirements for sharing of available research capacities and existing infrastruc- tures. The focus of the workshop was on GM crops or other applications (e.g., animals, micro-organisms, etc.) on the marketplace or near to be commercialized, not necessarily in the EU, but that may have effects in the EU. Applications intentionally released into the environment and/or used immediately in feed and food applications were considered. A Stakeholder Engagement Protocol was agreed with project partners to clarify workshop aims, activities, and research question development (Menozzi et al., 2014). The stakehold- er involvement process began with the generation of a “Potential Stakeholder Database”. The experts were specifically selected based on their career, successfully achievements and long-standing expertise in the field of GMOs related to scientific, economic, social and policies aspects. In addition, a broadest group of stakeholders in different fields were added in the database, including representative leaders of farmer’s organizations, public authorities and agencies, EU research Institutions, private companies and other relevant stakeholders. A preliminary list was sent to the project partners and integrated with their suggestions. Then, the experts and stakeholders were contacted following a step-by-step criteria in order to have a right balance between the three scientific areas, as well as a fair representation of the Member States. A registration before the deadline was required in order to define the number of participants, their role and activities, to guarantee the right balance of the attendees. Forty-five stakeholders were invited to represent a right balance of expertise between the three areas (i.e. socio-economics, human and animal health, and environment), organ- izational perspective (academic, Member State and EU agency, CSO communities) and geographical areas. The international workshop activities, conducted by three facilitators of the University of Parma, were divided into two sections. The morning session was dedi- cated to share and discuss preliminary results of the project with the participants, includ- ing the results of the mapping of existing research activities. In the afternoon session the participants were divided into three working groups, based on the area of expertise or interest, to identify the relevant transnational research needs. The three working groups used a structured multi-stage approach, consisting in six steps. Steps from 1 to 5 aimed at populating the list of transnational research needs (Figure 1), while Step 6 consisted in the identification of capacity/infrastructure needs to cover those research needs. 62 D. Menozzi Figure 1. Flow chart for identifying a list of future research needs by the stakeholders. Stage 1. A questionnaire was sent two weeks before the workshop to all the experts and stakeholders to identify the main research questions across GM species/traits and effects. Based on their replies to the questionnaire, a structured dataset of initial research questions was populated. Stage 2. Participants in each working group were encouraged to submit modifica- tions to existing research questions or add new ones, given the evidence provided by the mapping of existing research activities. The initial dataset of research questions was then established. Stage 3. Each working group reviewed the initial dataset of research questions on its area of expertise discussing whether an existing solution to the research need exists and was available. If this was the case, the use of research outcomes already undertaken was recommended. Stage 4. If the research need was not investigated so far, or if the results were not already available or applicable, the experts considered if it was on the European agenda or not. If yes, an EU funded project was more appropriate and therefore recommended. Stage 5. If the research need was not on the current EU agenda, the experts checked if it could be defined as a transnational interest. If yes, then a programme funded transna- tionally by the ERA-Net was a likely solution (transnational research need); otherwise, if it was only on a national agenda, then a national project or programme was suggested. Stage 6. Once the list of transnational research needs was populated, the moderator asked participants which capacity/infrastructure needs were available in order to cover those transnational research needs. A trained facilitator conducted the discussion in each working group. Potential disagree- ments were discussed, and eventually reported as such. All inputs from the working groups were presented in the plenary session, including the areas of persistent disagreements. 63Stakeholder engagement: identifying future directions for GMO research 3. Results: existing research activities on GMOs in Europe Information about 320 projects on existing research activities regarding the techno- and socio-economic, health, and environmental effects of GMOs in Europe were collected through the mapping exercise (Moyankova and Kostov, 2015). The data included the type of organizations leading the projects, the funding source, the topics of the GMO assess- ment projects (human health, animal health, environment, technological/social), and the studied GMO species. Unfortunately, it was not possible to collect consistent information about the projects’ budget and timing. This may have introduced a bias since it was not possible to evaluate the projects’ relative importance and achieved results. Nevertheless, the number and type of projects alone provides a valid mapping of current research activi- ties to be used as a starting point for the purposes of the study. The surveyed GMO projects in Europe have started between 1989 and 2010. Most of the projects (85%) were led by research or academy organizations such as universities, insti- tutes or research centers. A relatively small portion of the projects were led by government organization and private companies, accounting for only 9% and 5% respectively. Most of the projects were carried out at national level (198 projects). International collaboration was predominantly among European countries (80 projects), while only 27 projects included countries outside Europe. A number of 15 projects did not provide this information. The GMO projects were led by institutions in different European countries. When considering the number of GMO projects per million capita inhabitants, Austria is on leader position with 6.0 projects per million capita, followed by Denmark (3.1), Norway (2.8), Finland (2.3), Ireland (1.4), Hungary (1.2) and Belgium (1.1). The other countries, i.e. France, Germany, Greece, Italy, Netherlands, Spain, Sweden, Switzerland and United Kingdom, have less than 1 project per million capita. The EU is the only funding source for projects leaded by Greek and Irish organizations, while other organizations based in Hungary, Sweden and Switzerland had only projects funded by national sources (Fig- ure 2). Seven of the project leader countries are funded mainly by governments, namely UK, Spain, Norway, Italy, Germany, Finland and Austria. The EU was the major funding source for project leaders based in the Netherlands, Denmark, France and Belgium. Not surprisingly, projects were mostly focusing on GM plants (196 projects, 68% of the total, Figure 3). GM micro-organisms and GM animals were analysed only in 20 (6%) and 10 (3%) projects, respectively. A few projects (7) were dealing with GM plants and micro-organisms at the same time. Many projects did not specify the type of GMO that was analysed (n=65, 20%). The interaction of GMO with the environment was investigat- ed in more than a half of the projects (52%, Figure 3). One third of the projects (33%) were dealing with the developments of new methods, tools for detection in and analyses of food and feed, methods for risk assessments, new technique, etc. The effect of GMOs on human and animal health is a topic of interest in 10% and 4% of the projects, respec- tively. Many projects covered several topics at the same time. The main subjects (environ- ment, human health, animal health and technology/society) showed the same distribution when crossed by type of studied GMOs (Figure 3). Biodiversity preservation is the predominant sub-topic (58%) in the projects studying the interaction of GMOs with the environment (Table 1). The effect on non-target spe- cies was analysed in 25% of the projects. Other sub-categories were objects of only few 64 D. Menozzi projects. Among the different types of GMOs, the GM crops are the most investigated for their effect on the environment (Table 1). The technological and socio-economic aspects of the GMOs were analysed in 120 projects. Development of new methods for GMO detection and technological innovation were predominantly studied in 76% of these pro- jects. Among the socio-economic issues, the economic efficiency was studied the most, followed by consumer demand and food security (Table 1). Many projects covering this Figure 2. Number of GMO projects by funding source (EU, Government, Industry and other funding source) in Europe per country (n = 305). 0% 20% 40% 60% 80% 100% United Kingdom Switzerland Sweden Spain Norway Netherlands Italy Ireland Hungary Greece Germany France Finland Denmark Belgium Austria EU funding Government Industry Other Source: own elaboration. Figure 3. Type of analysed GMOs and main topics (n = 320*). 0 50 100 150 200 250 300 GM animals GM microorganisms GM crops GM crops and microorganisms Not specified Number of projects Technology / Society Environment Human health Animal health Source: own elaboration. * Note: Many projects covered more than one topic. 65Stakeholder engagement: identifying future directions for GMO research topic did not specify the type of GMO and this hampered the data analysis. Fewer pro- jects treated the effects of GMOs on human health (Table 1). Food safety and allergenicy were the most explored sub-topics in particular for GM crops. The effect of GMOs on ani- mal health was explored in 15 projects, where feed safety was the only subject analysed. GM crops interaction with human and animal health was the most studied topic, whilst GM animals impact was analysed in only one project. Considering the country of the leading organization, Switzerland and Spain had projects that only dealt with the effects of GMOs on the environment. This subject was the most studied in 7 project leader coun- tries, namely UK, Sweden, Ireland, France, Finland, Denmark and Austria. Organizations from Belgium, Italy and Norway were leading mainly projects about technology and social effect of GMOs. GMOs safety for human health is relatively more studied in 2 countries, i.e. Hungary and Greece. The less studied subject was the effect of GM feed on animal’s health. Only 5 countries were leading few projects about this topic, the most relevant of which was Hungary. Table 1. Health, environment and socio-economics subtopics of the GMO assessment projects in Europe (n=320*). GM crops Micro- organisms GM animals GM crops and micro- organisms Not specified Total Food safety 17 1 1 - 4 23 Feed safety 13 - - - 2 15 Allergenicity 5 - - - - 5 Toxicity 3 - - - - 3 Therapeutic use 1 1 - - - 2 Nutritional value 2 - - - - 2 Total Health 41 2 1 0 6 50 Biodiversity 67 11 2 4 14 98 Non-target effects 39 - - 2 1 42 Soil 8 2 - 1 1 12 Target effects 6 - 3 - - 9 Plant pest and diseases 6 - - - - 6 Other effects 2 - - - - 2 Total Environment 128 13 5 7 16 169 Economic efficiency 11 - - 1 2 14 Consumer demand 1 - - 1 9 11 Food security 2 - - - - 2 Other effects - - - - 2 2 Total Socio-Economics 14 0 0 2 13 29 Innovative technology 53 7 4 1 18 83 Technical application 5 - - - 3 8 Total Technology 56 7 4 1 21 91 Total 241 22 10 10 55 339 Source: own elaboration. * Note: Many projects covered more than one topic. 66 D. Menozzi 4. Results: stakeholders view on the future research needs 4.1 Workshop participants Of the total 45 experts contacted for participating in the international workshop, a final number of 20 individuals participated in the activities (Table 2); Powers et al. (2014) suggested that a number between 20 and 25 gives a right balance between the diversity of technical and sector perspectives and fluid working relationships. Table 2. Workshop contracted persons and participants per category. Category Contacted persons Participants Participation rate Expert Environment 11 7 64% Expert Socio-Economics 4 4 100% Expert Health 7 2 29% Stakeholders 14 2 14% European Institutions 4 1 25% Funding Bodies 5 4 80% Total 45 20 44% The experts studying the environmental effects of GMOs were the most represented, followed by the socio-economic experts. The participants were 44% of total the number of contacted persons. Although a fair gender balance was taken into consideration in contacting the experts, males were more represented than women in the final group of participants (75% males). The weakest participation was found among the stakeholders (14%), whilst the highest rate was found among the socio-economic experts (100%), then the environmental (64%) and finally among the health (29%). The participants originat- ed from ten European countries (Italy 3, Germany 3, The Netherlands 2, Austria 2, UK 2, Swiss 1, Spain 1, Denmark 1, Bulgaria 1, Romania 1); three stakeholders represented European organizations/institutions (i.e., COPA/Cogeca, the Public Research & Regula- tion Initiative – PRRI –, and the European Food Safety Authority – EFSA). The career stage of the participants was quite homogeneous, since most of them were in a senior position, leaders of research units or project leaders. As observed by Schneider and Gill (2016) having both young and senior researchers in the same group could discourage some participants to express their views, leading seniors to control the discussion with their point of view. 4.2 Initial dataset of research questions The initial dataset of research questions to consider in developing research needs, as resulted from stages 1) and 2) (see Section 2.2), have been categorised across the subject areas (i.e. socio-economics, human and animal health, and environment), and species/ traits (i.e. GM crops, insects, animals, micro-organisms and all GMOs). Thus, defined cat- egories of questions were aligned with the results from the mapping exercise to identify 67Stakeholder engagement: identifying future directions for GMO research potential gaps in the research that has been done so far, and the future research directions considered relevant by the stakeholders. Table 3 provides a synthetic representation of the initial research needs considered by the experts in their deeper analysis. A list of 48 research questions was developed by the experts in the socio-economic area. The identified projects in the mapping of current research activities (see Table 1, Sec- tion 3) are related mainly to the economic efficiency and consumer demand. Similarly, the stakeholders consider important the general economic effects, such as the costs and the profitability, but also more specific subjects like the economics of segregation/co-exist- ence, legislative framework, consumers perception and attitudes, macro-economic, yields, and other effects (Table 3). Most of them were considered for all GMOs and GM crops in general, or insect resistant (IR) crops and herbicide tolerant/herbicide resistant (HT/ HR) crops. For instance, the experts considered the necessity to include ‘non-pecuniary’ benefits in the analysis of costs savings related to HT and IR crops (e.g., off-farm income, management time saving, labour flexibility, equipment cost savings, better standability, etc.), to improve the methodology to analyse the segregation rules of GMOs (e.g., the wel- fare effects of labelling and segregation policy), and to evaluate the economic effects of relatively less widespread varieties, such as virus-resistant cassava and Golden rice. Only one research question was identified by the workshop participants for species other than crops: the assessment of the socio-economic impact of GM animals. The large portion of research questions in the initial dataset defined by the stakeholder are specific to differ- ent types of GM crops which already exist on the market or are under development. GM plants and crops are also subjects of many projects identified during the mapping exer- cise (Table 1). Since new GM crops and traits are under development outside the EU, and potentially access to the common market in the next future, the experts and stakeholders have suggested to concentrate research efforts on vegetable species. The European research carried out so far in the area of human and animal health is primary related to the safety of GM food and feed (Table 1). During stages 1) and 2) a number of 25 research needs were identified in the area of health (Table 3), divided into the main effects food and feed safety, nutritional value, allergenicity, toxicity and other. For instance, the experts considered the need to assess the positive health effects of the reduced fumonisin content in IR crops. It was argued that GM crop unintentionally pro- duce high amount of toxic substances, e.g. acetylated aspartic acid, and that animal will be the main users being exposed to such substances. Therefore, research is needed to focus on developing and validating a test protocol in livestock animals. Most of these research needs were considered for all GMOs, while none where identified for insects and other animals. This probably because it was stressed that the focus was on GMOs on the mar- ket or near to be commercialized. These results indicate no substantial differences in the research carried out so far and the future research needs in the area of human and animal health as the questions related to the toxicity, allerginicity and food safety are still central for the novel applications of GMOs (Parisi et al., 2016). A number of 47 research needs were identify by the experts to cover the area of envi- ronment (Table 3). Most of the projects carried out so far on the environmental impact of GMOs are dealing with biodiversity and non-target effects (Table 1). Questions related to the possible effects to biodiversity of the emerging crop varieties and traits are still of interest according the stakeholders for different kinds of GMOs. Quality of soil and water 68 D. Menozzi Ta b le 3 . I n it ia l r es ea rc h n ee d s in t h e th re e ar ea s: h ea lt h , e n vi ro n m en t an d s o ci o -e co n o m ic s. A ll G M O s G en er al cr op s IR c ro ps H T /H R cr op s G M c ro ps w ith st ac ke d tr ai ts R N A i- ba se d pl an ts O th er pl an ts In se ct s O th er an im al s M ic ro - or ga ni sm s To ta l Fo od s af et y 2 - 2 - - - - - - 2 6 N ut ri tio na l v al ue 1 1 - - - - 3 - - - 5 A lle rg in ic ity 3 - - - - - - - - 1 4 To xi ci ty 2 - 1 - - - - - - - 3 Fe ed s af et y 2 1 - - - - - - - - 3 O th er e ffe ct s - - - - - 4 - - - - 4 To ta l H ea lt h 10 2 3 0 0 4 3 0 0 3 25 Bi od iv er si ty 2 - 3 2 4 - 2 - - 1 14 So il 2 - - 1 - - 1 - - 2 6 W at er - - 2 - - - 1 - - 2 5 Pl an t p es t a nd d is ea se s 2 - 1 1 - - 1 - - - 5 A ir - 1 1 - - - - - - 1 3 Ec os ys te m s er vi ce s 2 - - - - - - - - - 2 C lim at e ch an ge 1 - - - - - - - - - 1 O th er e ffe ct s 2 - - - - 5 4 - - - 11 To ta l E nv ir on m en t 11 1 7 4 4 5 9 0 0 6 47 Ec on om ic e ffe ct s in g en er al 1 3 2 2 2 - 1 - - - 11 C os ts - 2 1 2 - - 1 - - - 6 Pr ofi ta bi lit y 1 1 2 - 1 - - - - - 5 Se gr eg at io n/ co ex is te nc e 2 - 2 - - - - - - - 4 Le gi sl at iv e fr am ew or k 2 2 - - - - - - - - 4 C on su m er s an d so ci et y 1 1 - - - - 1 - 1 - 4 M ac ro -e co no m ic e ffe ct s - 1 1 1 - - - - - - 3 Y ie ld s 1 - - - - - 1 - - - 2 O th er e ffe ct s 4 2 2 1 - - - - - - 9 To ta l S oc io -E co no m ic s 12 12 10 6 3 0 4 0 1 0 48 So u rc e: o w n e la b o ra ti o n . 69Stakeholder engagement: identifying future directions for GMO research are also pointed as important followed by the effects to plant pest and diseases, air, ecosys- tem services, climate change, and other effects. Most of the research needs were identified for all GMOs, IR crops or other plants. For instance, stakeholders claimed to develop field methods to monitor soil process intensity and changes, to implement an application of environmental DNA to detect changes in biodiversity, and to evaluate the potential effects on non-targeted organisms (NTOs) and the possible protein interactions or synergistic effects of stacked events expressing Cry and Vip proteins. No research need was identified for GM insects or other animals. The initial dataset of research questions was provided to the workshop participant as a starting point, to support thinking about potential research needs that were outside their specific expertise area. The lists supplemented explicit encouragement to participants to think broadly about potential research areas. The activities of the three working groups have followed the structured multi-stage approach described in Figure 1. Each working group was arranged of 5/6 experts, one moderator and one note-taker, chosen among the project partners. The participants in each working group reached a relatively small list of research needs (from 14 to 18), also by individual questions into groups of questions that were similar or closely related. Consolidating individual questions into broader research areas was encouraged when they had similar implications for an assessment or subsequent risk management decision. The complete list of research needs is reported in Menozzi et al. (2014). 4.3 Socio-economic research needs The “Socio-economic” working group identified several categories of research needs. According to the experts efforts are needed to develop a methodological framework for assessing the socio-economic effects of GMOs. This framework should be used to inform policy development. Socio-economic considerations are already included in the regulatory frameworks on GMOs of some countries (Binimelis and Myhr, 2016). However, it is nec- essary to work to develop a robust framework and methodology, including criteria, indi- cators, etc., capturing socio-economic considerations in biosafety decision-making. Gar- cia-Yì et al. (2014) identified six topics, i.e. the farm-level economic impacts, the econom- ics of co-existence, the economics of segregation at the supply chain level, the consum- ers’ acceptance of GMOs, the environmental economics impact and the impacts of GMOs on food security. Thus, although there is a high interest in the implementation of socio- economic considerations in biosafety regulations (Binimelis and Myhr, 2016), research is needed to establish a robust framework on the socio-economic impacts of GMOs, and methodology covering data gathering, assessment and decision making. From the supply chain point of view, Park et al. (2011) have estimated the revenue forgone by EU farmers due to on-going limited use of IR and HT crops. Similarly, the effects of the EU regulation on GMOs on EU competitiveness and on innovative research developed at the EU level should be assessed, as well as the welfare effects across different groups in society (e.g., farmers, consumers, etc.) in the context of different policy settings (e.g., labelling). For instance, the EU co-existence measures affect farmers differently across EU Member States according to the isolation distances (between GM and non-GM crops) required by different countries and for different species (Ramessar et al., 2010). An eco- 70 D. Menozzi nomic evaluation of the welfare distribution of flexible co-existence regulations may assist the adoption of proportionate measures (Devos et al., 2014). The stakeholders claimed that the effects of GMOs along the whole supply chain should be investigated further. This point goes beyond the common cost-benefit analysis; it considers how the structure of sup- ply chains is affected by innovation, how the efficiency is impacted, how the horizontal/ vertical relationships could change, what are the implication on labour market, etc. Up to now, research mostly concentrates on how costs and benefits are distributed along the sup- ply chain (Garcia-Yì et al., 2014). Few projects have already dealt with supply chain impact, on structure and performance, of co-existence and segregation measures (e.g., the EU FP6 projects SIGMEA and CO-EXTRA; see also Ghozzi et al., 2016); however, a more deep analysis of supply chain effects (e.g., on structure and relationships) is missing. Consumers’ attitude towards the use of new techniques in food production (e.g., new breeding techniques, nanotechnology – GMOs) needs to be investigated further. In fact, although most of what can be known by questioning on a hypothetical base has already been investigated (Dannenberg, 2009), research is missing on consumers’ acceptance stud- ies using real settings. Further research efforts are needed to explore on the economic evaluation of the effects (positive and negative) of GMOs on the environment, using a multidisciplinary approach (Garcia-Yì et al., 2014). In this context, a comparative analysis of GM, organic and conventional crops in terms of environmental, social and economic sustainability, should be elaborated. At farm level, research usually evaluated the main economic effects of GM crops, such as yield, costs, gross margin (European Commission, 2011). More research is needed to study the economic implications of more efficient GM varieties, like second generation GMOs (e.g., nitrogen-efficient GM wheat), e.g., assessing how these GMOs move the yield frontier, how improvements in yield efficiency affect the economic performance of farm- ers, etc. This research need is likely a transnational one, since not all the countries could have the same interest. Moreover, other socio-economic impacts are scarcely documented, such as the indirect effects arising from the GM crops management (e.g., how GM appli- cations affect farm management planning, cropping system, crop rotation, etc.). Research should also study the differences between intensive and extensive margin effects (Bennett et al., 2013), the stability of new GM crops yields (e.g. draught resistant) on a mid- and long- term basis, and the economic performance of HT crops (Areal et al., 2012). Finally, the socio-economic group noted the need to develop systematic reviews and meta-analyses to consolidate existing knowledge, and to improve the communication of available evidence. In terms of communication, research is also needed to better under- stand the key elements in stakeholders’ communication and interaction. 4.4 Human and animal health research needs The “Human and animal health” working group distinguished research needs across all types of GM species. Major consideration related to the GM crops intended to be used as food and feed is whether they are safe for consumption, which should be evalu- ated under the EU risk assessment frame. While there is substantial amount of experi- mental data (e.g., feeding studies with laboratory and livestock animals) for the varieties which already exist on the market (Flachowsky et al., 2012; Snell et al., 2012; Ricroch 71Stakeholder engagement: identifying future directions for GMO research et al., 2014), a specific food safety concerns could emerge with the development of new types of GM crops such as plants combining several modifications (stacks) and GM plants with deliberately modified nutritional properties (Halford et al., 2014; Ramon et al., 2014). Specific health related questions were pointed by the stakeholders; for instance, the group agreed that research is needed to explore toxicity effects of multiple Bt proteins in in-vitro systems, the potential hypo-allergenicity of GM crops, and the definition of the minimal required inclusion level of plant-expressed phytase for efficient phosphorus utilization of animals. Traceability and post-marker monitoring are also among the stockholders con- cerns; there is also a considerable lack of data on the traceability of specific GM crops, on verification of consumption and/or potential health impacts of GM food ingredients (e.g., GM crops with enhanced fatty acids), as well as on toxic substances produced by GM plants used in feed production (i.e. acetylated aspartic acid). Further, the group noted that little knowledge is available on the health effects of producing pharmaceuticals by the use of GM plants. While substantial progress has been made in the development of GM plants for molecular farming, still the scale remains rela- tively small, mainly performed in laboratory or contained conditions. The major challenge is the legislative frame and the adaptation of the risk assessment principles to this plant biotechnology applications (Sparrow et al., 2013). The group agreed that a common fea- ture of risk assessment of potential protein toxicity is needed (bioinformatics). A specific issue was raised for myco-toxins in Bt maize; research is needed to assess whether reduced fumonisin content can be found, as well as their potential and real benefits. Although there is a large body of knowledge available, there is a need to develop a systematic analy- sis of the data collected by the regulatory bodies. GM plants producing RNAi molecules represent a biotechnological development which seems close to the EU market; therefore, experts have raised concerns over the potential health implications due to the technological differences with the first generation of GM plants. Steps towards the identification and evaluation of the specific human and animal health risk that may come with the new generations of GMOs and the adaptation of the regulations and risk assessment guidelines have already been taken by the scien- tists, agencies and regulatory bodies (Petrick et al., 2013; Ramon et al., 2014). The “health” working group considered a number of research needs related to RNAi based plants, for instance more information is needed on survival and uptake in humans and animals, and post-market monitoring. Although it’s a corporate responsibility to deal with the commer- cial production and marketing, the EU authorities must provide the appropriate methodol- ogy and tools for their monitoring along the whole supply chain. An example for this is the database on food consumption of the EU Member States that is being collated by the EFSA. The same is needed for feed ingredients. Allergenicity is commonly considered for humans, whereas there is only limited knowledge available on the impact on farm animals. This type of investigation should also explore possible links with post-market monitoring. Finally, there is still uncertainty about the potential for horizontal gene transfer of genetically modified micro-organisms (GMMs) and viral DNA. Although this concern has been investigated and discussed in the past (Dröge et al., 1998; Keese, 2008), stakeholders pointed that better methods need to be developed to assess the presence and diffusion of recombinant DNA and cells. 72 D. Menozzi 4.5 Environmental research needs The “Environment” working group determined that the research needs can be priori- tised according to the criterion of “ecosystem services” provision (Tscharntke et al., 2005). Such an approach would involve the monitoring of cultivated land (on-crop area), but also of the space between crops in a landscape (off-crop area), and analyse how these two different kinds of areas influence each other in terms of ecological functionality. In this respect, the need for comparative study of different Integrated Pest Management (IPM) systems used in the EU Member States was highlighted, as well as the need to assess the role that GMOs (plants and insects) might play in such IPM systems (Hokkanen, 2015). Moreover, efforts should be directed to study the efficacy of GMOs in different GM events which hold promises of relevant economic and environmental benefits, such as blight resistance potato (Haverkort et al., 2016), and to a deeper understanding of the develop- ment dynamics of insecticidal protein resistance mechanisms in target insects (e.g., corn borers, etc.). Additionally, regulating and supporting services (pollination, pest control, soil fertility maintenance, etc.) were considered. The experts concluded that goals for the protection against undesirable effects have to be assessed at the landscape level. This type of moni- toring, however, requires instruments to study the possible effects of different stressors, including GMOs, on key species and ecosystem services (e.g., on bees and wild pollina- tors). System interfaces (i.e. land and water) were also defined as important points to explore further, as well as the change of dynamics in the system over time. More informa- tion on species assemblages before introduction of GM crops, to define appropriate base- line indicators, is needed for plants, arthropods and micro-organisms (e.g., soil indicators) (van Capelle et al., 2016). Further, the protection of cultural services was discussed (e.g., how people perceive agriculture, recreation, psychological benefits from contact with nature, etc.). Research is needed to study biodiversity in protected areas from different perspectives, in particular to qualify what type and level of biodiversity that society would like to maintain locally. Cropping practices (i.e. weed control efficiency) may have indirect effects on nearby val- ued areas. For instance, the elimination of weeds by farmers on their fields has also an impact on the trophic level in terms of available resources for sap feeders, pollinators, nat- ural enemies, etc., that may in turn affect the functional biodiversity of neighbouring areas and need to be investigated further (Bürger et al., 2015). In general, there is also a need to study the people’s perception of different agricultural systems, and this should ideally involve both the natural and the social sciences (multi-disciplinary research). Finally, the group included research needs not strictly related to ecological services. There is strong support for looking more into automated and harmonized methods for general surveillance, for studying non-target effects of new modes of action (e.g., RNAi) (Lundgren and Duan, 2013), and for new traits and breeding techniques on the envi- ronment. There are still knowledge gaps about the traceability and environmental fate of GMMs, and to develop bioinformatic tools for studying their evolution in the system. Finally, there is also limited knowledge about the effects of GM arthropods on the environ- ment, and this is also an area characterized by quickly progressing of genomic technologies for possible applications in agricultural as well as in human diseases prevention projects. 73Stakeholder engagement: identifying future directions for GMO research 5. Results: requirements for sharing capacities and infrastructures The main requirements for sharing existing (national) capacities and infrastructures were identified during the working groups’ activities. The “Human and animal health” working group found that in various countries there is a high level of expertise available for studying the hypo-allergenicity of GM crops which needs to be shared. Harmonization and joint initiatives are possible for sharing experiences on the traceability of specific GM crops. Since applications for RNAi-expressing crops have been mostly developed outside the EU, and limited expertise is available at the EU level, the group concluded that a transnational organization of capacities would put the EU in a posi- tion to overcome this deficit in the future. As a lot of research has been done on peptide (e.g., cytotoxic peptides, food peptides) and their physiological effects (e.g., dairy research, antibiot- ics), the group defined the need to integrate these research capacities available across certain EU countries and sectors, for the purpose of assessing potential protein toxicity as another important study area. For the assessment of allergenicity in farm animals there are probably only limited capacities available at the EU level, and future research would also benefit from transnational organization of resources. A high level of expertise for GMMs and viral DNA horizontal gene transfer was found in various EU countries; again, the relative research needs are invited to be organized transnationally to increase the overall efficiency. The “Environment” working group defined requirements for sharing controlled exper- imental field sites throughout Europe, to allow GMO field testing on representative envi- ronments of the various European settings. The fields could also be used to avoid several regulatory constrains which make it difficult for public research in Europe to study the effects of GMOs. The group discussed the necessity to have meso-cosm facilities for soil- based experiments. The group concluded that calls for multi-/inter-disciplinary actions and projects should invite applications that combine different technologies/methods of scientific enquiry. Finally, it was felt that the GM regulatory, testing and monitoring methods should be harmonised, as much as possible, with other, similar methods and approaches in related areas (e.g., pesticide registration, international work on the valuing and monitoring of ecosystem services, etc.). The “Socio-economic” working group discussed the need to develop protocols and guide- lines for conducting socio-economic impact assessments, which would ensure basic compat- ibility of results, without sacrificing the flexibility of approaches in the process. Similarly to the “Environment” group, they also highlighted the need to share field trials, and to develop more field studies for assessing yields, costs, and other economic aspects of the use of GMOs. In addition, the need to develop multidisciplinary tasks capable of taking qualitative research (e.g., socio-psychology, behavioural economics, etc.) into account, was included in the list of priorities. Finally, the group concluded that researchers’ capacities should be shared, via train- ing and staff exchange programs, thus developing ways to facilitate future collaboration among researchers from different countries (e.g., sharing capacities, PhD programmes, etc.). 6. Conclusions This paper aims to promote a critical debate among relevant stakeholders and policy makers by identifying future research directions in the evaluation of environmental, health 74 D. Menozzi and socio-economic effects of current and emerging applications of GMOs. The outcome of the international workshop, which was the follow up of a mapping exercise of existing research activities in Europe, is a list of perceived transnational research needs and con- sequent requirements for sharing existing capacities and infrastructures; it therefore rep- resents more than a generalized description of research trends. The list of research needs will allow those developing research plans to focus more deeply on one or a few themes that were deemed more relevant by the scientific community as well as the European risk managers’ network. This process may provide a basis to develop a prioritization from the perspective of the diverse group of stakeholders, which is to be developed in a next step of the project. The results of the PreSto GMO ERA-Net project formed the basis for a joint- ly prepared strategic plan and roadmap for the implementation of the ERA-Net that will coordinate transnational research on the effects of GMOs (Rauschen et al., 2015). As pointed by Twardowski and Małyska (2015), the slow progress in the EU decision making process forced several biotech companies to move R&D and applied activities to other regions, thus reducing personnel in the EU and transferring existing know-how out- side the EU. Moreover, this prevented the commercialization of innovative GMOs, possi- bly resulting in competitive disadvantages for European farmers (Park et al., 2011). Future directions of socio-economic research on the effects of GMOs should primarily con- sider the development of a methodological framework for analysing the socio-economic effects of GMOs. An assessment of the welfare effects across different groups in society may assist the definition of more informed policies. The research questions related to the effects of newly emerging GMOs to human and animal health raised by the stakehold- ers during the workshop were general in many occasions or rather case specific in others. While some of them might be found relevant for the risk assessment to different extend, others are related to traceability and post-market monitoring, and some could appear to be within the scope of the risk assessment frameworks to be developed for the new GMOs, such as RNAi plants. The environmental research needs identified were prioritised according to the criterion of “ecosystem services”, involving the monitoring of on-crop and off-crop land area, and how these two different kinds of areas influence each other in terms of ecological functionality. The emerging applications, as well as the tendency of GM developers to combine different traits to produce new commercial varieties (the so-called “commercial stacks”) (Parisi et al., 2016), pose relevant questions to researchers, risk assessors and policy makers on how to adapt the EU regulatory framework consider- ing the environmental and health-related issues, as well as the socio-economic dimensions (e.g., international trade) when making decisions on GMOs. A key factor in implementing this process will be the confidence that those planning and funding research have in the method used (Schneider and Gill, 2016). The activi- ties described in this paper has explicitly taken into account the wider views of a diver- sity of stakeholders and end-users (i.e. industry, farming organisations, CSOs, NGOs, EU and national competent authorities, funding organisations, academia). This intended to encourage participation of different scientific communities (scientists from all over Europe) in the future joint transnational calls managed through the ERA-Net, to enhance collaboration between actors (to leverage complementarities) and to increase the account- ability of research trajectories and outcomes (create an internationally recognizable criti- cal mass). The decision to consider a single group of 20 participants allowed balancing 75Stakeholder engagement: identifying future directions for GMO research the greater diversity of technical and sector perspectives, and the facilitation of more fluid working involvement from all participants. Breakout groups allowed the experts to elab- orate on a greater number of details surrounding the interfaces of their disciplines with others, and this can thoroughly inform the scientific community in planning and promot- ing interdisciplinary and transdisciplinary research (Powers, 2014). Viewed from the per- spective of group dynamics, the entire process aimed to achieve a common understanding of the research gaps and needs in the evaluation of current and emerging applications of GMOs able to design future research directions. 7. Funding This work was supported by the European Commission contract 612739, PreSto GMO ERA-Net (Preparatory steps towards a GMO research ERA-Net), funded by the EU FP7 programme. The authors declare no competing financial interests. 8. Acknowledgements The authors wish to thank the PreSto GMO ERA-Net project partners for their inputs into the research, in particular the project coordinator Dr. Stefan Rauschen. The authors gratefully acknowledge the experts and stakeholders who participated in the international workshop for their active contribution to the identification of future research directions in the evaluation of emerging applications of GMOs. The authors thank also the anonymous reviewers for their constructive comments. The views expressed are purely those of the authors and may not under any circumstances be regarded as stating an official position of the European Commission. References Areal, F.J., Riesgo, L. and Rodriguez-Cerezo, E. (2012). Economic and Agronomic Impact of Commercialized Gm Crops: A Meta-Analysis. Journal of Agricultural Science 151(1): 7-33. Baram, M. and Bourrierm, M. (2011). Governing Risk in GM Agriculture. Cambridge, UK: Cambridge University Press. Bennett, A.B., Chi-Ham, C., Barrows, G., Sexton, S. and Zilberman, D. (2013). Agricultur- al Biotechnology: Economics, Environment, Ethics, and the Future. Annual Review Environmental Research 38: 247-279. Binimelis, R. and Myhr, A.I. (2016). Inclusion and Implementation of Socio-Economic Considerations in GMO Regulations: Needs and Recommendations. Sustainability 8: 62. Bürger, J., Darmency, H., Granger, S., Guyot, S.H.M., Messéan, A. and Colbach, N. (2015). Simulation Study of the Impact of Changed Cropping Practices in Conventional and Gm Maize on Weeds and Associated Biodiversity. Agricultural Systems 137: 51–63. Dannenberg, A. (2009). The Dispersion and Development of Consumer Preferences for Genetically Modified Food — A Meta-Analysis. Ecological Economics 68: 2182-2192. 76 D. Menozzi Devos, Y., Dillen, K. and Demont, M. (2014). How Can Flexibility Be Integrated into Coexistence Regulations? A Review. Journal of the Science of Food and Agriculture 94: 381-387. Dröge, M., Pühler, A. and Selbitschka, W. (1998). Horizontal Gene Transfer as a Biosafety Issue: A Natural Phenomenon of Public Concern. Journal of Biotechnology 64(1): 75-90. Drott, L., Jochum, L., Lange, F., Skierka, I., Vach J. and van Asselt, M.B.A. (2013). Accountability and Risk Governance: A Scenario-Informed Reflection on European Regulation of GMOs. Journal of Risk Research 16(9): 1123-1140. European Commission (2010). A Decade of EU-Funded GMO Research (2001-2010). Directorate-General for Research and Innovation, Luxembourg: Publications Office of the European Union. European Commission (2011). Report from the Commission to the European Parliament and the Council on Socio-Economic Implications of GMO Cultivation on the Basis of Member States Contributions, as requested by the Conclusions of the Environ- ment Council of December 2008. SANCO/10715/2011. European Commission (2015). Fact Sheet: Questions and Answers on EU’s policies on GMOs. Brussels. http://europa.eu/rapid/press-release_MEMO-15-4778_en.htm. Accessed 25 May 2016. Flachowsky, G., Schafft, H. and Meyer, U. (2012). Animal Feeding Studies for Nutritional and Safety Assessments of Feeds from Genetically Modified Plants: A Review. Jour- nal für Verbraucherschutz und Lebensmittelsicherheit 7(3): 179-194. Garcia-Yi, J., Lapikanonth, T., Vionita, H., Vu, H., Yang, S., Zhong, Y., Li, Y., Nagelsch- neider, V., Schlindwein, B. and Wesseler, J. (2014). What Are the Socio-Economic Impacts of Genetically Modified Crops Worldwide? A Systematic Map Protocol. Environmental Evidence 3: 24. Ghozzi, H., Soregaroli, C., Boccaletti, S. and Sauvée, L. (2016). Impacts of Non-GMO Standards on Poultry Supply Chain Governance: Transaction Cost Approach vs. Resource Based View. Supply Chain Management: an International Journal 21(6): 743-758. Gómez-Galera, S., Twyman, R. M., Sparrow, P. A., Van Droogenbroeck, B., Custers, R., Capell, T. and Christou, P. (2012). Field Trials and Tribulations – Making Sense of the Regulations for Experimental Field Trials of Transgenic Crops in Europe. Plant Biotechnology Journal 10(5): 511-523. Graef, F., Römbke, J., Binimelis, R., Myhr, A.I., Hilbeck, A., Breckling, B., Dalgaard, T., Stachow, U., Catacora-Vargas, G., Bøhn, T., Quist, D., Darvas, B., Dudel, G., Oehen, B., Meyer, H., Henle, K., Wynne, B., Metzger, M.J., Knäbe, S., Settele, J., Székács, A., Wurbs, A., Bernard, J., Murphy-Bokern, D., Buiatti, M., Giovannetti, M., Debeljak, M., Andersen, E., Paetz, A., Dzeroski, St, Tappeser, B., van Gestel, C.A.M., Wosniok, W., Séralini, G.-E., Aslaksen, I., Pesch, R., Maly, S. and Werner, A. (2012). A Frame- work for a European Network for a Systematic Environmental Impact Assessment of Genetically Modified Organisms (GMO). BioRisk 7: 73-97. Halford, N.G., Hudson, E., Gimson, A., Weightman, R., Shewry, P.R. and Tompkins, S. (2014). Safety Assessment of Genetically Modified Plants with Deliberately Altered Composition. Plant Biotechnology Journal 12(6): 651-654. 77Stakeholder engagement: identifying future directions for GMO research Haverkort, A.J., Boonekamp, B.M., Hutten, R., Jacobsen, E., Lotz, L.A.P., Kessel, G.J.T., Vossen, J.H. and Visser, R.G.F. (2016). Durable Late Blight Resistance in Potato Through Dynamic Varieties Obtained by Cisgenesis: Scientific and Societal Advanc- es in the DuRPh Project. Potato Research 59(1): 35-66. Herring, R.J. (2008). Opposition to Transgenic Technologies: Ideology, Interests and col- lective Action Frames. Nature Reviews Genetics 9: 458-463. Hokkanen, H. (2015). Integrated Pest Management at the Crossroads: Science, Politics, or Business (as usual)? Arthropod-Plant Interactions 9(6): 543-545. Jacobsen, S.-E., Sørensen, M., Pedersen, S. M. and Weiner, J. (2013). Feeding the World: Genetically Modified Crops versus Agricultural Biodiversity. Agronomy for Sustain- able Development 33(4): 651-662. James, C. (2015). 20th Anniversary (1996 to 2015) of the Global Commercialization of Biotech Crops and Biotech Crop Highlights in 2015. ISAAA Brief No. 51. Ithaca, N.Y.: ISAAA. Keese, P. (2008). Risks from GMOs due to Horizontal Gene Transfer. Environmental Biosafety Research 7(3): 123-149. Lundgren, J.G. and Duan, J. (2013). RNAi-Based Insecticidal Crops: Potential Effects on Nontarget Species. BioScience 63(8): 657-665. Mannion, A.M. and Morse, S. (2012). Biotechnology in Agriculture: Agronomic and Envi- ronmental Considerations and Reflections Based on 15 Years of GM crops. Progress in Physical Geography 36(6): 747-763. Menozzi, D., Mora, C. and Sogari, G. (2014). Report on the workshop. PreSto GMO ERA- Net, University of Parma, Italy. http://www.presto-gmo-era-net.eu. Accessed 24 May 2016. Mora, C., Menozzi, D., Kleter, G., Aramyan, L.H., Valeeva, N.I., Zimmermann, K.L. and Pakki Reddy, G. (2012). Factors Affecting the Adoption of Genetically Modified Animals in the Food and Pharmaceutical Chains. Bio-based and Applied Economics 1(3): 313-329. Moyankova, D. and Kostov, K. (2015). Report on the Analyzed Database. PreSto GMO ERA-Net, AgroBioInstitute, Sofia, Bulgaria. http://www.presto-gmo-era-net.eu. Accessed 24 May 2016. National Academies of Sciences, Engineering, and Medicine (2016). Genetically Engi- neered Crops: Experiences and Prospects. Washington, DC: The National Acad- emies Press. Noteborn, H.P.J.M. and van Duijne, F.H. (2011). The Dutch Approach to Safety Govern- ance of GM Agriculture. In Baram, M. and Bourrierm, M. (eds), Governing Risk in GM Agriculture, Cambridge, UK: Cambridge University Press. Parisi, C., Tillie, P. and Rodríguez-Cerezo, E. (2016). The Global Pipeline of GM Crops out to 2020. Nature Biotechnology 34(1): 31-36. Park, J., McFarlane, I., Phipps, R. and Ceddia, G. (2011). The Impact of the EU Regulatory Constraint of Transgenic Crops on Farm Income. New Biotechnology 28(4): 396–406. Petrick, J.S., Brower-Toland, B., Jackson, A.L. and Kier, L.D. (2013). Safety Assessment of Food and Feed from Biotechnology-Derived Crops Employing Rna-Mediated Gene Regulation to Achieve Desired Traits: A Scientific Review. Regulatory Toxicology and Pharmacology 66(2): 167-176. 78 D. Menozzi Powers, C., Hendren, C., Wang, A. and Davis, M. (2014). Transparent Stakeholder Engagement in Practice: Lessons Learned from Applying Comprehensive Environ- mental Assessment to Research Planning for Nanomaterials. U.S. Environmental Protection Agency Papers, Paper 230. Ramessar, K., Capell, T., Twyman, R.M. and Christou, P. (2010). Going to Ridiculous Lengths —European Coexistence Regulations for GM Crops. Nature Biotechnology 28(2): 133-136. Ramon, M., Devos, Y., Lanzoni, A., Liu, Y., Gomes, A., Gennaro, A. and Waigmann, E. (2014). RNAi‐Based GM Plants: Food for Thought for Risk Assessors. Plant Biotech- nology Journal 12(9): 1271-1273. Rauschen, S., Krieger, K., Racovita, M., Valeeva, N.I., Aramyan, L.H., Zimmermann, K., Ovesná, J., Pla, M., Ogras, T., Fuhrmann, E., Karner, S., Kerins, G., Romeis, J. and Spök, A. (2015). Strategic Implementation Plan for an ERA-Net on GMO Impact Research. PreSto GMO ERA-Net. http://www.presto-gmo-era-net.eu. Accessed 19 October 2016. Rausser, G., Zilberman, D. and Kahn, G. (2015). An Alternative Paradigm for Food Production,Distribution, and Consumption: A Noneconomist’s Perspective. Annual Review of Resource Economics 7: 309-331. Ricroch, A.E., Boisron, A. and Kuntz, M. (2014). Looking Back at Safety Assessment of GM Food/Feed: An Exhaustive Review of 90-Day Animal Feeding Studies. Interna- tional Journal of Biotechnology 13(4): 230-256. Ross, K. (2007). Providing “Thoughtful Feedback”: Public Participation in the Regulation of Australia’s first Genetically Modified Food Crop. Science and Public Policy 34(3): 213-225. SCAR-CWG (2012). Risk Research on Genetically Modified Organisms – Summary Report 2009 –2011. Standing Committee on Agricultural Research Collabora- tive Working Group on GMOs. http://bmg.gv.at/home/Schwerpunkte/Gentech- nik/Fachinformation_Allgemeines/SCAR_Collaborative_Working_Group_Risk_ Research_on_GMOs_. Accessed 25 May 2016. Schneider, M. and Gill, B. (2016). Biotechnology versus Agroecology: Entrenchments and Surprise at a 2030 Forecast Scenario Workshop. Science and Public Policy 43(1): 74-84. Schurman, R. and Munro, W.A. (2010). Fighting for the Future of Food: Activists versus Agribusiness in the Struggle over Biotechnology. Minneapolis, US: University of Minnesota Press. Snell, C., Bernheim, A., Bergé, J.-B., Kuntz, M., Pascal, G., Paris, A. and Ricroch, A.E. (2012). Assessment of the Health Impact of GM Plant Diets in Long-Term and Mul- tigenerational Animal Feeding Trials: A Literature Review. Food and Chemical Toxi- cology 50(3-4): 1134-1148. Sparrow, P., Broer, I., E Hood, E., Eversole, K., Hartung, F. and Schiemann, J. (2013). Risk Assessment and Regulation of Molecular Farming - A Comparison between Europe and US. Current Pharmaceutical Design 19(31): 5513-5530. Tscharntke, T., Klein, A.M., Kruess, A., Steffan-Dewenter, I. and Thies, C. (2005). Land- scape Perspectives on Agricultural Intensification and Biodiversity – Ecosystem Ser- vice Management. Ecology Letters 8: 857-874. 79Stakeholder engagement: identifying future directions for GMO research Twardowski, T. and Małyska, A. (2015). Uninformed and Disinformed Society and the GMO Market. Trends in Biotechnology 33(1): 1-3. van Capelle, C., Schrader, S. and Arpaia, S. (2016). Selection of Focal Earthworm Species as Non-Target Soil Organisms for Environmental Risk Assessment of Genetically Modified Plants. Science of the Total Environment 548-549: 360-369. van den Hoven, J. (2013). Options for Strengthening Responsible Research and Inno- vation: Report of the Expert Group on the State of Art in Europe on Responsible Research and Innovation. Publications Office of the European Union. Yang, Y.T. and Chen, B. (2016). Governing GMOs in the USA: Science, Law and Public Health. Journal of the Science of Food and Agriculture 96: 1851-1855. A systematic approach to understanding and quantifying the EU’s bioeconomy Tévécia Ronzon1*, Stephan Piotrowski2, Robert M’Barek1, Michael Carus2 Cost function and positive mathematical programming Quirino Paris What if meat consumption would decrease more than expected in the high-income countries? Fabien Santini*, Tevecia Ronzon, Ignacio Perez Dominguez, Sergio Rene Araujo Enciso, Ilaria Proietti A stakeholder engagement approach for identifying future research directions in the evaluation of current and emerging applications of GMOs Davide Menozzi1*, Kaloyan Kostov2, Giovanni Sogari1, Salvatore Arpaia3, Daniela Moyankova2, Cristina Mora1 A spatial analysis of terrain features and farming styles in a disadvantaged area of Tuscany (Mugello): implications for the evaluation and the design of CAP payments Laura Fastelli1*, Chiara Landi2, Massimo Rovai1, Maria Andreoli1