5_Rotigliano_et al.indd 361Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.DOI: 10.15201/hungeobull.67.4.5 Hungarian Geographical Bulletin 67 2018 (4) 361–373. Introduction Landslide susceptibility is defined as the probability of the occurrence of landslides in a given area according to local geo-environ- mental variables (Brabb, E.E. 1984; Carrara, A. et al. 1995; Guzzetti, F. et al. 1999). For defining landslide susceptibility, both direct and indirect methods can be used. Direct methods are based on analyses per- formed by expert geomorphologists who divide the territory into areas with different susceptibility, defining the latter in quali- tative terms (Verstappen, H.T. 1983; Van Westen, C.J. et al. 2003). By using indirect, deterministic or statistical methods, it is in- stead possible to obtain a quantitative clas- sification of landslide susceptibility (Chung, C.J.F. et al. 1995; Ohlmacher, G.C. and Davis, J.C. 2003; Conoscenti, C. et al. 2008). Landslide susceptibility assessment, de- veloped in recent years, has seen an increas- ingly frequent use of the statistical approach. This method is based on the assumption 1 Dipartimento di Scienze della Terra e del Mare, Univestità degli Studi di Palermo, Via Archirafi 22. 90123 Palermo, Italy. Corresponding author’s e-mail: christian.conoscenti@unipa.it Evaluation of debris flow susceptibility in El Salvador (CA): a comparison between Multivariate Adaptive Regression Splines (MARS) and Binary Logistic Regression (BLR) Edoardo ROTIGLIANO , Chiara MARTINELLO Valerio AGNESI and Christian CONOSCENTI1 Abstract In the studies of landslide susceptibility assessment, which have been developed in recent years, statistical methods have increasingly been applied. Among all, the BLR (Binary Logistic Regression) certainly finds a more extensive application while MARS (Multivariate Adaptive Regression Splines), despite the good performance and the inno- vation of the strategies of analysis, only recently began to be employed as a statistical tool for predicting landslide occurrence. The purpose of this research was to evaluate the predictive performance and identify possible drawbacks of the two statistical techniques mentioned above, focusing in particular on the prediction of debris flows. To this aim, an inventory of debris flows triggered by the passage of the hurricane IDA and the low-pressure system associated with it 96E, on 7th and 8th November 2009, in an area of about 26 km2 close to the Caldera Ilopango, El Salvador (CA), was employed. Two validation strategies have been applied to both statistical techniques, thus obtaining four models – BLR (I), MARS (I), BLR (II) and MARS (II) – to be compared in pairs. Model performance was assessed in terms of AUC (area under the receiver operating characteristic (ROC) curve), Sensitivity, Specificity, Positive Prediction Value and Negative Prediction Value. Moreover, to evaluate the robustness of the modelling procedure, 50 replicates were created for each model and standard deviation was calculated for each of them. The results show that both techniques allow for obtaining good or excellent performances so that it is not possible to define one of the two techniques as absolutely better. However, the validation procedure reveals slightly better performance of the MARS models, with greater sensitivity and greater discrimination among True Negatives (TNs). Keywords: landslide susceptibility, debris flows, Multivariate Adaptive Regression Splines (MARS), Binary Logistic Regression (BLR), hurricane Ida, El Salvador Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.362 that new landslides occur in areas with geo- environmental conditions that have caused landslides in the past (Carrara, A. et al. 1995; Guzzetti, F. et al. 1999; Van Westen, C.J. et al. 2003, 2008). In light of this, geo-environmen- tal variables can be considered as predictors or independent variables while the past/pre- sent distribution of landslides as the depend- ent variable. The covariates are selected to re- flect the variability of geo-environmental fac- tors that are considered related to the activa- tion of landslides and, moreover, the choice is also a function of the quality and resolution of available data. The landslide archive can be generated through field mapping or de- tection from high resolution remotely-sensed images. In this study, we employed the land- slide archive produced by Rotigliano, E. et al. (2018), which was obtained through re- mote mapping of the Google EarthTM image dated 11/21/2009 (DigitalGlobe Catalog ID: 101001000AA5D801). The statistical method is therefore aimed at determining relationships existing between the covariates and the dependent variable. For the determination of relationships, statis- tical analysis of bivariate (logistic regression, binary logistic regression [BLR]) or mul- tivariate type (e.g., Multivariate Adaptive Regression Splines [MARS], cluster analy- sis, discriminant analysis) can be used. In the literature, a number of examples of landslide susceptibility studies performed using bivar- iate (e.g., Guzzetti, F. et al. 1999; Rotigliano, E. et al. 2012; Costanzo, D. et al. 2014) or mul- tivariate analysis (e.g., Vorpahl, P. et al. 2012; Felicísimo, Á.M. et al. 2013; Conoscenti, C. et al. 2015, 2016) can be found. Comparative studies of the two methods are however rare (e.g. Conoscenti, C. et al. 2015) and, as far as we know, no comparison has ever been made in the case of evaluation of suscepti- bility from debris flow. The main objective of the study is to show the difference in terms of predictive per- formance of the two methods, i.e. BLR and MARS, by using two different validation schemes (Chung, C.J.F. and Fabbri, A.G. 2003). Firstly, we created 50 datasets consist- ing of balanced samples of event and non- event pixels. In the first validation scheme, each dataset was exploited for both calibra- tion and validation of the models. In the sec- ond scheme, each archive was split in two: the first, containing 75 per cent of the positive and negative cases, was used for the calibra- tion of the models; the second, containing the remaining 25 per cent of the balanced archive, was used for validation. The perfor- mance of the models was assessed through the analysis of AUC (area under the receiver operating characteristic (ROC) curve) values and the confusion matrices. The two validation schemes have been developed for both BLR and MARS models. The creation of 50 models for each procedure also allowed us to evaluate the robustness of the analysis (Costanzo, D. et al. 2014). Materials and methods Study area The study area is a small drainage basin of about 26 km2, located along the slopes of the Caldera Ilopango, El Salvador, CA (Figure 1). The slopes of the area are covered by levels of tefra and ignimbrite, derived from the most recent Quaternary eruptions of the Caldera (Figure 2). In the area are found deep V-shaped valleys (Figure 3), whose formation is linked to intense weathering and mass movements that affected the volcanic bedrock. The latter, characterized by poor mechanical properties, can be in fact easily eroded by water espe- cially during extreme meteorological events (cyclones and hurricanes), which occur very frequently in the region. The climate regime of El Salvador is tropi- cal-humid, with average annual rainfall above 1,500 mm and average annual temperature be- tween 20 °C and 30 °C. Under these conditions, physical degradation is favoured, with further deterioration of the geotechnical properties of the pyroclastics. The meteorological phenom- ena are then responsible for the saturation of the degraded material and thus for a signifi- 363Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. cant decrease of cohesion. Due to these envi- ronmental conditions, the study area is particu- larly prone to landsliding, especially to debris flow type landslides. Unregulated deforesta- tion and intensive cultivation of the area, even in very steep slopes, further promotes soil ero- sion as well as landslide processes. Landslide inventory Between 7th and 8th November 2009, El Sal- vador was affected by the simultaneous pas- sage of Hurricane Ida and the 96/E low-pres- sure system. The greatest damage occurred in an area of about 400 km2 around the Cal- dera Ilopango, where more than 300 mm of rainfall in 12 hours were recorded at the Ilopango rain gauge station. Such an intense rainfall event triggered more than two thou- sand debris flows and caused flooding of the valleys, causing approximately 200 fatalities and immense economic loss with destruction of houses, roads and crops (MARN 2010). The landslide archive used in this study is a database of landslide phenomena that occurred in the catchment due to the concomitant pas- sage of Ida and 96/E. The archive has already been used in Rotigliano, E. et al. (2018). The recognition of the landslides and their mapping has been carried out remotely, using a high-res- olution satellite image available on the Google Earth software, which is dated 11/21/2009 (DigitalGlobe Catalog ID: 101001000AA5D801). This image, acquired only 2 weeks after the passage of Ida-96/E, allowed the identification and mapping of 2231 debris flows triggered by the aforementioned rainfall event. Each failure has been mapped by using a landslide identification point (LIP), located at the point of origin of the movement. In the case of evaluation of susceptibility to debris flow, according to Rotigliano, E. et al. (2011), LIPs allow us to obtain the most reliable landslide prediction as their environ- mental characteristics are those that best rep- resent pre-failure conditions and thus can be considered the best diagnostic areas for cali- brating (and validating) landslide predictive models (Rotigliano, E. et al. 2011, Lombardo, L. et al. 2014; Cama, M. et al. 2015). For this reason, it was decided to use the archive without making any changes with respect to the initial characteristics. Statistical modelling In the last 20 years, many studies have dealt with landslide susceptibility modelling and a number of them have used a statistical mod- elling approach. Binary logistic regression (BLR) is among the most frequently used sta- tistical techniques in geomorphology, and in particular in the field of landslide susceptibil- ity assessment (e.g. Bai, S.B. et al. 2010; Atkin- son, P.M. and Massari, R. 2011; Costanzo, D. et al. 2014). On the other hand, Multivariate Adaptive Regression Splines (MARS) (Fried- man, J.H. 1991), has been employed only a few times in geomorphology (e.g., Gómez-Gutiér- rez, Á. et al. 2009, 2015; Conoscenti, C. et al. 2016, 2018; Garosi, Y. et al. 2018). Both BLR and MARS make it possible to identify relationships between a set of inde- pendent variables (predictors), both continu- ous or categorical, and a dependent dicho- tomic variable, which is usually coded as 0 (non-event) or 1 (event). The aim of BLR is to describe the linear relationship between the logit (or log odds) of the dependent variable and the set of n independent variables (Hosmer, D.W. and Fig. 1. Location of the study area Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.364 Fig. 2. Outcropping lithology in the study area (from Weber, H.S. et al. 1978) Fig. 3. Slope gradient map of the study area 365Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. Lemeshow, S. 2000). This is described by the following equation: where π (x) is the conditional mean of the response given specific values of x, α is the constant or intercept, xi (i = 1, 2, ... n) is the nth independent variable and βi (i = 1, 2, ... n) is the nth coefficient of the independent variable. To optimize the values of y having certain independent variables, or rather to identify the value of α and βi, the maximum likeli- hood technique is used, actually the log- likelihood (LL) function (Menard, S. 1995). MARS is a non-parametric regression tech- nique capable of identifying non-linear ad- aptation relationships between independent variables and dependent variables. MARS divides the range of the predictor values into regions and generates a linear regres- sion equation for each region. The “nodes” are the extreme values of each region while each distinct interval is called “basis func- tion” (BF). The latter can take the form: max (0, x – c) o max (0, c – x), where x is an independent variable and c is a constant corresponding to a knot. The general expression of MARS can be written as follows: where y is the dependent variable, α is the constant, N is the number of terms, βn is the coefficient of the nth term and hn (x) is a single basic fiction or a product of two or more BFs. MARS builds the model in two phases. In the first step (forward pass) a complex model is produced in which basis functions generated for each variable are added. In the second phase (backward pass) a leaner model is established through generalized cross validation (GCV) (Craven, P. and Wahba, G. 1979). Basically, MARS removes the least influential pair in the creation of the best model and the GCV allows the identification, among the models gener- ated, of the one offering the best compromise between adaptation (low RSS) and complexity/ completeness of the model (Briand, L.C. et al. 2004; Gómez-Gutiérrez, Á. et al. 2009). Both regression techniques and related anal- ysis have been implemented using the software R (R CoreTeam, 2017). For the BLR analysis the “stats” package has been used, for MARS analysis the “earth” package (Milborrow, S. et al. 2011; Milborrow, S. 2015). Predictors The predictor variables were chosen accord- ing to their expected influence on slope in- stability and to their control on slope failure mechanisms (e.g., Conoscenti, C. et al. 2015; Cama, M. et al. 2016; Prokos, H. et al. 2016). A set of ten geo-environmental variables was employed to predict debris flow sus- ceptibility in the Caldera Ilopango. This set includes lithology (LIT) and land use (USE), in addition to the following eight terrain at- tributes: landform classification (LCL), el- evation (ELE), slope steepness (STP), slope aspect (ASP), plan curvature (PLN), profile curvature (PRF), topographic wetness index (TWI) and terrain ruggedness index (TRI). The outcropping lithology was obtained through the acquisition and processing of a 1:100,000 geological map (Weber, H.S. et al. 1978), which was derived from field survey 1: 25,000 scale. An up-to-date land use map was created on the basis of field surveys and analysis of ASTER and Google Earth im- ages. The terrain attributes were extracted from a digital elevation model (DEM) with a ground resolution of 10-m, by using the soft- ware SAGA-GIS. All terrain attributes were extracted as continuous variables with the exception of LCL, which was classified into 10 classes, namely: streams, mid-slope drain- ages, upland drainages, valleys, plains, open slopes, local ridges, mid-slope ridges, high ridges. The 10-m cells of the DEM were em- ployed as mapping units of the debris flow susceptibility in the studied area. , ), Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.366 In order to detect collinearity among the chosen covariates, we calculated the Variance Inflation Factor (VIF) by using the “usdm” package (Naimi, B. 2015) implemented in the software R. A VIF value equal or higher than 10 indicates collinearity among the selected covariates (Heckmann, T. et al. 2014; Jebur, M.N. et al. 2014; Bui, D.T. et al. 2016). As VIF values calculated for our variables were be- low this threshold, all were included in both BLR and MARS models. Model building and validation strategy Landslide susceptibility assessment requires a validation procedure in order to evaluate the accuracy of the predictive models. This is generally performed in two steps: i) calibra- tion of the models and ii) validation of the models (Chung, C.J.F. and Fabbri, A.G. 2003). In this study, we evaluated adaptation, ac- curacy and robustness of the models gener- ated with BLR and with MARS. To this aim, two validation strategies were developed, applying a random partition to the same landslide archive. First, the study area was divided into 249,994 10-m grid cells corresponding to the pixels of the employed DEM. This data set includes 2,231 “event” or “positive” cells (i.e. cells hosting at least one LIP) and 247,763 “non-event” or “negative” cells (i.e. cells not intersecting any LIP). Through random se- lection, 50 balanced data sets were created, each of them containing all event cells and an equal number of randomly selected negative cells (Conoscenti, C. et al. 2016), thus includ- ing in total 4,462 cells. The first validation strategy involved the calibration and validation of one mod- el for each of the 50 data sets. Therefore, each data set was exploited as both learn- ing and validation data set. In the second validation scheme, each of the 50 data sets was randomly divided into two bal- anced subsets: a training set, including 75 per cent of the cases, and a test set, includ- ing the remaining 25 per cent of the cases. For both the validation schemes, it was possible to obtain a pair of models, one gen- erated with the BLR and one with MARS, for each balanced data set (Figure 4 and 5). This allowed us to analyse the difference in terms of performance and robustness between the two employed statistical techniques. As train- ing and test datasets were the same, these differences were assumed as due only to the different characteristics of the two statistical techniques. Statistical analyses were carried out to evaluate and quantify the goodness of fit, the prediction skill and the robustness of the models. By comparing the prediction image of each model with the spatial occurrence of the event cells, the confusion matrix and thus the number of true positive, true negative, false positive and false negative cases (TP, TN, FP and FN, respectively) for each model, applying a Youden index optimized cut-off (Youden, W.J. 1950). To evaluate the goodness of fit and pre- diction skill of the susceptibility models the AUC (area under the receiver operating char- acteristic [ROC] curve) (Goodenough, D.J. et al. 1974; Hanley, J.A. and McNeil, B.J. 1982; Lasko, T.A. et al. 2005) was used. A ROC curve plots the true positive rate (sen- sitivity) against the false negative rate (1 – specificity), at any given cut-off value. For the AUC values, Hosmer, D.W. and Lemeshow, S. (2000) identify the threshold values of 0.7, 0.8 and 0.9 corresponding to acceptable, ex- cellent and outstanding predictions respec- tively. Finally, to evaluate the robustness of the models, the validation procedures have been applied to all the model runs (50 for BLR and 50 for MARS, for each validation strategy) in order to analyse the accuracy and reliabil- ity of the models through the study of the average and standard deviation of the AUC values. These validation tools have already been successfully used in previous studies with the aim of comparing different methods and models (e.g., Von Ruette, J. et al. 2011; Conoscenti, C. et al. 2015, 2016; Cama, M. et al. 2017). 367Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. Fig. 4. Graphical summary scheme of the first adopt- ed validation strategy Fig. 5. Graphical summary scheme of the second adopted validation strategy Results For the description of the developed mod- els and the relative results, a subscript (I) is adopted for those generated through the first validation strategy, while subscript (II) is used for those created with the second vali- dation strategy. The mean AUC values of the BLR (I), MARS (I), BLR (II) and MARS (II) models are 0.796, 0.821, 0.789 and 0.811, respectively. According to the classification proposed by Hosmer, D.W. and Lemeshow, S. (2000), these values indicate excellent (> 0.8) and accept- able (> 0.7) performance of the models. As shown by the AUC standard deviation values (Table 1), the performance of both modelling techniques is quite stable. The boxplots of Figure 6 show a low degree of dispersion in the AUC values, which, as expected, appears slightly higher for the second validation strat- egy. Figure 7 shows the ROC curves obtained from the replicates of each model (grey) while the average ROC curves are plotted in red. Table 2 shows the cumulative confusion matrices extracted by applying the models to the 50 validation data sets of both validation strategies. Table 3 shows the average values Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.368 the first validation strategy the training and the test data sets coincide whereas in the sec- ond strategy, learning and validation sets do not share any pixels and they are randomly extracted from the training/test data sets em- ployed in the first procedure. Models validation using the first validation strategy MARS (I) demonstrate slightly better perfor- mance than BLR (I). It should be noted that the difference in terms of AUC is very small, being between 0.02 and 0.05. The accuracy of MARS (I) is only 0.02 higher than the accu- racy of BLR (I), whereas the difference of av- erage AUCs is only 0.03. Regarding the ability to predict event cells, a greater difference is recorded: the average sensitivity of MARS (I) is indeed 0.82 whereas that of BLR (I) is 0.77. However, PPV values reveal the same ability for both BLR (I) and MARS (I). On the other hand, although the specificity values suggest similar abilities of BLR (I) and MARS (I) to predict the non-event cells, NPV values demonstrate better performance of MARS (I) (0.78) compared to that of BLR (I) (0.74). Models validation using the second validation strategy Also the second validation strategy reveals a slightly better performance of MARS compared to that of BLR, although the observed differ- ences are once again weak. The difference of both accuracy and AUC values are indeed ap- proximately 0.02. Again, the difference in terms of sensitivity between MARS (II) (0.81) and BLR Table 1. Characteristics of the AUC values for the four susceptibility models Models Accuracy AUC-mean AUC-min AUC-max AUC SD* BLR (I) MARS (I) BLR (II) MARS (II) 0.720 0.744 0.716 0.736 0.796 0.822 0.789 0.811 0.783 0.805 0.754 0.779 0.806 0.833 0.815 0.836 0.005 0.006 0.012 0.012 *SD = Standard deviation. Fig. 6. AUC boxplots for the four models of sensitivity, specificity, positive prediction value (PPV) and negative prediction value (NPV) and the relative Youden index cut-off. The accuracy of the models can be con- sidered good, with values between 0.71 and 0.74. Sensitivity values between 0.76 and 0.82, attest to a good predictive power of positive cases while slightly lower is the ability to discriminate the true negatives (specificity in the range 0.66–0.67). On the other hand, it is noteworthy that the NPV values, which are between 0.74 and 0.78, reveal acceptable predictions of the TNs whereas the PPV val- ues, which are approximately 0.7, attest to a slightly worse ability to predict the TPs. Discussion For the discussion of the results of model val- idations we have to take into account that in 369Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. Fig. 7. ROC-plots for the four models Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.370 (II) (0.76) does not result in a greater discrimi- natory power of TP (the difference of PPV is 0.01). Finally, the two techniques show the same specificity (0.66), but the discriminatory ability of TN is higher for MARS (II), with NPV values equal to 0.77 versus 0.74 of BLR (II). Concluding remarks The use of statistical methods in landslide susceptibility assessment raises the prob- lem of the type of analysis to perform and which one is the best modelling approach and technique. BLR has been proven a useful technique for achieving reliable assessment of landslide susceptibility. In recent years, however, several other statistical techniques have also demonstrated equally good, and sometimes even better, performance. MARS, which is a relatively new technique, has been employed in few cases for assessing land- slide susceptibility but it has already been demonstrated to provide very good accuracy in predicting the occurrence of slope failures. However, as far as we know, MARS has nev- er been employed to predict debris flows. The aim of this study was to highlight the differences in terms of predictive perfor- mance between BLR and MARS and, thus, identify the best method for the assessment of debris flow susceptibility in the area of Ilopango Caldera. The obtained results show that both meth- ods achieve good to excellent predictive per- formances. Although MARS demonstrated slightly better performance, the difference is too small to be able to define this technique as clearly better than BLR. Rotigliano, E. et al. (2018) hypothesize that in the 2009 dataset there is a problem related to a secondary triggering of a num- ber of phenomena due to incision or lateral erosion produced by debris flows activated directly by the storm event. In fact, even in this study, the models obtained are af- fected by this problem, as shown by the low specificity values. In light of this, however, the performance in terms of NPV is higher than expected. MARS, in fact, with the same dataset, is able to discriminate TN with bet- ter ability than BLR. This is probably due to the ability of MARS of identifying different relationships between the dependent and the independent variables, for different regions of the predictors’ ranges. This allows MARS to overcome, even if only slightly, the prob- lem of secondary triggering of landslides, Table 2. Confusion matrices of the four susceptibility models BLR (I) Reference MARS (I) Reference Prediction 0 1 Prediction 0 1 0 74,786 25,594 0 74,558 20,004 1 36,764 85,956 1 36,992 91,546 BLR (II) Reference MARS (II) Reference Prediction 0 1 Prediction 0 1 0 18,569 6,490 0 18,453 5,234 1 9,331 21,410 1 9,447 22,666 Table 3. Summary of the validation metrics for the four susceptibility models Models Youden index cut-off Sensitivity Specificity Positive Negative prediction value BLR (I) MARS (I) BLR (II) MARS (II) 0.48 0.46 0.48 0.46 0.77 0.82 0.76 0.81 0.67 0.66 0.66 0.66 0.70 0.71 0.69 0.70 0.74 0.78 0.74 0.77 371Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. certainly with a better distinction of cases with respect to BLR. Furthermore, both vali- dation strategies, albeit with subtle results, show a greater ability of MARS to identify positive cases compared to BLR. In light of this, although the differences are not marked and certainly the results do not allow the definition of a modelling technique as absolutely better than the other, it is pos- sible to identify more merits in the MARS technique than in the BLR. Acknowledgments: The present research was support- ed by a project funded by the Ministry of the Foreign Affairs of the Italian Government and carried out by the University of Palermo (resp. Prof. G. Giunta). All the authors equally contributed to the research. REFERENCES Atkinson, P.M. and Massari, R. 2011. Autologistic modelling of susceptibility to land sliding in the Central Apennines, Italy. Geomorphology 130. 55–64. Bai, S.B., Wang, J., Lü, G.N., Zhou, P.G., Hou, S.S. and Xu, S.N. 2010. GIS-based logistic regres- sion for landslide susceptibility mapping of the Zhongxian segment in the Three Gorges area, China. Geomorphology 115. 23–31. Brabb, E.E. 1984. Innovative approaches to land- slide hazard and risk mapping. In Proceedings 4th International Symposium on Landslides 1. Toronto, Canadian Geotechnical Society, 307–324. Briand, L.C., Freimut, B. and Vollei, F. 2004. Using multiple adaptive regression splines to support decision making in code inspections. Journal of Systems and Software 73. (2): 205–217. Bui, D.T., Tuan, T.A., Klempe, H., Pradhan, B. and Revhaug, I. 2016. Spatial prediction models for shal- low landslide hazards: a comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides 13. 361–378. https:// doi.org/10.1007/s10346-015-0557-6 Cama, M., Lombardo, L., Conoscenti, C., Agnesi, V. and Rotigliano, E. 2015. Predicting storm- triggered debris flow events: Application to the 2009 Ionian Peloritan disaster (Sicily, Italy). Natural Hazards and Earth System Sciences 15. (8): 1785–1806. https://doi.org/10.5194/nhess-15-1785-2015 Cama, M., Conoscenti, C., Lombardo, L. and Rotigliano, E. 2016. Exploring relationships be- tween grid cell size and accuracy for debris-flow susceptibility models: a test in the Giampilieri catchment (Sicily, Italy). Environmental Earth Sciences 75. (238): 1–21. https://doi.org/10.1007/ s12665-015-5047-6 Cama, M., Lombardo, L., Conoscenti, C. and Rotigliano, E. 2017. Improving transferability strategies for debris flow susceptibility assessment. Application to the Saponara and Itala catchments (Messina, Italy). Geomorphology 288. 52–65. https:// doi.org/10.1016/j.geomorph.2017.03.025 Carrara, A., Cardinali, M., Guzzetti, F. and Reichenbach, P. 1995. GIS technology in map- ping landslide hazard. In Geographical Information Systems in Assessing Natural Hazards. Eds.: Carrara, A. and Guzzetti, F., Dordrecht, Kluwer, 135–175. Chung, C.J.F., Fabbri, A.G. and Van Westen, C.J. 1995. Multivariate regression analysis for landslide hazard zonation. In Geographical Information Systems in Assessing Natural Hazards. Eds.: Carrara, A. and Guzzetti, F., Dordrecht, Kluwer, 107–133. https:// doi.org/10.1007/978-94-015-8404-3_7 Chung, C.J.F. and Fabbri, A.G. 2003. Validation of spatial prediction models for landslide hazard map- ping. Natural Hazards 30. (3): 451–472. https://doi. org/10.1023/B:NHAZ.0000007172.62651.2b Conoscenti, C., Di Maggio, C. and Rotigliano, E. 2008. GIS analysis to assess landslide susceptibility in a fluvial basin of NW Sicily (Italy). Geomorphology 94. 325–339. Doi:10.1016/j.geomorph.2006.10.039 Conoscenti, C., Ciaccio, M., Caraballo-Arias, N.A., Gómez-Gutiérrez, Á., Rotigliano, E. and Agnesi, V. 2015. Assessment of susceptibility to earth-flow landslide using logistic regression and multivariate adaptive regression splines: a case of the Belice River basin (western Sicily, Italy). Geomorphology 242. 49–64. https://doi.org/10.1016/j. geomorph.2014.09.020 Conoscenti, C., Rotigliano, E., Cama, M., Caraballo- Arias, N.A., Lombardo, L. and Agnesi, V. 2016. Exploring the effect of absence selection on land- slide susceptibility models: A case study in Sicily, Italy. Geomorphology 261. 222–235. https://doi. org/10.1016/j.geomorph.2016.03.006 Conoscenti, C., Agnesi, V., Cama, M., Caraballo-Arias, N.A. and Rotigliano, E. 2018. Assessment of Gully Erosion Susceptibility Using Multivariate Adaptive Regression Splines and Accounting for Terrain Connectivity. Land Degradation and Development 29. 724–736. https://doi.org/10.1002/ldr.2772 Costanzo, D., Chacón, J., Conoscenti, C., Irigaray, C. and Rotigliano, E. 2014. Forward logistic regression for earth-flow landslide susceptibility assessment in the Platani river basin (southern Sicily, Italy). Landslides 11. (4): 639–653. https://doi. org/10.1007/s10346-013-0415-3 Craven, P. and Wahba, G. 1979. Smoothing noisy data with spline functions. Numerische Mathematik 31. (4): 377–403. https://doi.org/10.1007/BF01404567 Felicísimo, Á.M., Cuartero, A., Remondo, J. and Quirós, E. 2013. Mapping landslide susceptibility Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.372 with logistic regression, multiple adaptive regres- sion splines, classification and regression trees, and maximum entropy methods: a comparative study. Landslides 10. 175–189. Friedman, J.H. 1991. Multivariate adaptive regression splines. The Annals of Statistics 19. (1): 1–141. http:// www.jstor.org/stable/2241837 Garosi, Y., Sheklabadi, M., Porghasemi, H.R., Besalatpour, A.A., Conoscenti, C. and Van Oost, K. 2018. Comparison of differences in resolution and sources of controlling factors for gully ero- sion susceptibility mapping. Geoderma 330. 65–78. https://doi.org/10.1016/j.geoderma.2018.05.027 Gómez-Gutiérrez, Á., Schnabel, S. and Lavado Contador, F. 2009. Using and comparing two nonparametric methods (CART and MARS) to model the potential distribution of gullies. Ecological Modelling 220. (24): 3630–3637. https:// doi.org/10.1016/j.ecolmodel.2009.06.020 Gómez-Gutiérrez, Á., Conoscenti, C., Angileri, S.E., Rotigliano, E. and Schnabel, S. 2015. Using topo- graphical attributes to evaluate gully erosion prone- ness (susceptibility) in two Mediterranean basins: advantages and limitations. Natural Hazards 79. (1): 291–314. https://doi.org/10.1007/s11069-015-1703-0 Goodenough, D.J., Rossmann, K. and Lusted, L.B. 1974. Radiographic applications of receiver oper- ating characteristic (ROC) curves. Radiology 110. 89–95. Guzzetti, F., Carrara, A., Cardinali, M. and Reichenbach, P. 1999. Landslide hazard evaluation: a review of current techniques and their application in a multi-scale study, Central Italy. Geomorphology 31. 181–216. http://dx.doi.org/10.1016/S0169- 555X(99) 00078-1 Heckmann, T., Gegg, K., Gegg, A. and Becht, M. 2014. Sample size matters: investigating the effect of sam- ple size on a logistic regression susceptibility model for debris flows. Natural Hazards and Earth System Sciences 14. 259–278. http://dx.doi.org/10.5194/ nhess-14-259-2014 Hanley, J.A. and McNeil, B.J. 1982. The meaning and use of the area under a receiver operating charac- teristic (ROC) curve. Radiology 143. 29–36. Hosmer, D.W. and Lemeshow, S. 2000. Applied logistic regression. Wiley Series in Probability and Statistics, Wiley. http://dx.doi.org/10.1198/tech.2002.s650 Jebur, M.N., Pradhan, B. and Tehrany, M.S. 2014. Optimization of landslide conditioning factors using very high-resolution airborne laser scan- ning (LiDAR) data at catchment scale. Remote Sensing of Environment 152. 150–165. http://dx.doi. org/10.1016/j.rse.2014.05.013 Lasko, T.A., Bhagwat, J.G., Zou, K.H. and Ohno- Machado, L. 2005. The use of receiver operating characteristic curves in biomedical informatics. Journal of Biomedical Informatics 38. 404–415. http:// dx.doi.org/10.1016/j.jbi.2005.02.008 Lombardo, L., Cama, M., Maerker, M. and Rotigliano, E. 2014. A test of transferability for landslides sus- ceptibility models under extreme climatic events: Application to the Messina 2009 disaster. Natural Hazards 74. (3): 1951–1989. https://doi.org/10.1007/ s11069-014-1285-2 MARN 2010. Síntesis de los informes de evaluación técnica de las lluvias del 7 y 8 de noviembre 2009 en El Salvador: Análisis del impacto físico natural y vul- nerabilidad socio ambiental (Technical report of the tropical storm event of 7–8 November 2009 in El Salvador: Physical impact and social vulnerabil- ity analysis). San Salvador, Ministerio de Medio Ambiente y Recursos Naturales de Salvador. http://www.oas.org/summit/sisca/Download. aspx?type=C&lang=es&id=728 Menard, S. 1995. Applied Logistic Regression Analysis. London, Sage Publications. Milborrow, S. 2015. Notes on the earth package. An online www document. http://www.milbo.org/ doc/earth-notes.pdf Milborrow, S., Hastie, T. and Tibshirani, R. 2011. Earth: Multivariate Adaptive Regression Spline Models. R Software Package. Naimi, B. 2015. Uncertainty Analysis for Species Distribution Models. R Software Package. Doi: 10.1111/j.1365-2699.2011.02523.x Ohlmacher, G.C. and Davis, J.C. 2003. Using mul- tiple logistic regression and GIS technology to predict landslide hazard in northeast Kansas, USA. Engineering Geology 69. 331–343. Prokos, H., Baba, H., Lóczy, D. and El Kharim, Y. 2016. Geomorphological hazards in a Mediterranean mountain environment – Example of Tétouan, Morocco. Hungarian Geographical Bulletin 65. (3): 283–295. https://doi.org/10.15201/hungeobull.65.3.6 R Core Team 2017. R: A language and environment for statistical computing. Vienna, R Foundation for Statistical Computing. https://www.R-project.org/ Rotigliano, E., Agnesi, V., Cappadonia, C. and Conoscenti, C. 2011. The role of the diagnostic areas in the assessment of landslide susceptibil- ity models: a test in the Sicilian chain. Natural Hazards 58. (3): 981–999. https://doi.org/10.1007/ s11069-010-9708-1 Rotigliano, E., Cappadonia, C., Conoscenti, C., Costanzo, D. and Agnesi, V. 2012. Slope units- based flow susceptibility model: Using validation tests to select controlling factors. Natural Hazards 61. (1): 143–153. https://doi.org/10.1007/s11069- 011-9846-0 Rotigliano, E., Martinello, C., Hernandéz, M.A., Agnesi, V. and Conoscenti, C. 2018. Predicting the landslides triggered by the 2009 96E/Ida tropical storms in the Ilopango caldera area (El Salvador, C.A.): optimizing MARS-based model building and validation strategies. Environmental Earth Sciences (in press) 373Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373. Van Westen, C.J., Rengers, N. and Soeters, R. 2003. Use of geomorphological information in indirect landslide susceptibility assessment. Natural Hazards 30. (1): 399–419. Van Westen, C.J., Castellanos, E. and Kuriakose, S.L. 2008. Spatial data for landslide susceptibility, hazard, and vulnerability assessment: an overview. Engineering Geology 102. 112–131. Verstappen, H.T. 1983. Geomorphology of the Agri valley, Southern Italy. ITC Journal 4. 291–301. Von Ruette, J., Papritz, A., Lehmann, P., Rickli, C. and Or, D. 2011. Spatial statistical modeling of shallow landslides-validating predictions for dif- ferent landslide inventories and rainfall events. Geomorphology 133. 11–22. http://dx.doi.org/10.1016/j. geomorph.2011.06.010 Vorpahl, P., Elsenbeer, H., Märker, M. and Schröder, B. 2012. How can statistical models help to determine driving factors of landslides? Ecological Modelling 239. 27–39. Weber, H.S., Wiesemann, G., Lorenz, W. and Schmidt-Thome, M. 1978. Mapa geologico de la Republica de El Salvador/America Central, 1:100,000 (Geological map of the El Salvador Republic/Central America, 1:100,000). Hannover, Bundesanstalt für Geowissenschaften und Rohstoffe. Youden, W.J. 1950. Index for rating diagnostic tests. Cancer 3. (1): 32–35. Doi: 10.1002/1097-0142(1950)3:1<32::AID- CNCR2820030106>3.0.CO;2-3 Rotigliano, E. et al. Hungarian Geographical Bulletin 67 (2018) (4) 361–373.374