Plane Thermoelastic Waves in Infinite Half-Space Caused Decision Making: Applications in Management and Engineering Vol. 4, Issue 1, 2021, pp. 127-152. ISSN: 2560-6018 eISSN: 2620-0104 DOI: https://doi.org/10.31181/dmame2104127b * Corresponding author. E-mail addresses: mahmut.bakir@samsun.edu.tr (M. Bakır), oatalik@eskisehir.edu.tr (Ö. Atalık). APPLICATION OF FUZZY AHP AND FUZZY MARCOS APPROACH FOR THE EVALUATION OF E-SERVICE QUALITY IN THE AIRLINE INDUSTRY Mahmut Bakır 1* and Özlem Atalık 2 1 Department of Aviation Management, School of Civil Aviation, Samsun University, Samsun, Turkey 2 Department of Aviation Management, Faculty of Aeronautics and Astronautics, Eskişehir Technical University, Eskişehir, Turkey Received: 28 December 2020; Accepted: 29 January 2021; Available online: 16 February 2021. Original scientific paper Abstract: Airlines today use e-services extensively for marketing activities and the distribution of services. Monitoring and evaluating e-service quality are essential for customers’ satisfaction and thus the success of airlines. This study aims to evaluate e-service quality in the airline industry from the point of view of the consumers. To achieve this, an integrated Fuzzy Analytical Hierarchy Process (F-AHP) and Fuzzy Measurement Alternatives and Ranking according to Compromise Solution (F-MARCOS) approach was proposed to handle the uncertain and imprecise nature of e-service evaluation. In the first stage, e- service quality criteria were prioritized using the F-AHP method. Then, a real- world case study was carried out on scheduled airlines to demonstrate the applicability of the proposed approach using the F-MARCOS method, utilizing a total sample of 395 airline passengers in Turkey. As a result, the top three e- service criteria were found as reliability, understandability, and security. A three-stage sensitivity analysis was also conducted to examine the credibility and stability of the results. This study is the first study to integrate F-AHP and F-MARCOS methods for the first time in literature. Key words: E-service quality; Airlines; Fuzzy AHP; Fuzzy MARCOS; Fuzzy sets theory. 1. Introduction The spread of Internet and information technologies (IT) has profoundly affected many industries. With the growth of the Internet, firms started to set up their websites and have offered their marketing and distribution activities through this channel (Cheng, 2011). One of the industries most affected by this development is the airline file:///C:/Users/Green/AppData/Local/AppData/Local/Temp/mahmut.bakir@samsun.edu.tr file:///C:/Users/Green/AppData/Local/AppData/Local/Temp/oatalik@eskisehir.edu.tr Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 128 industry. Until the 1990s, airlines had distributed their services through travel agencies, call centers, and global distribution systems (GDS) while later adopting the Internet as an important outlet for the distribution of e-services. Airlines, especially low-cost carriers, have gained a significant presence and power owing to online channels and geographic barriers losing their importance (Díaz & Martín-Consuegra, 2016). Thus, airlines have bypassed intermediaries and have the chance to reach consumers directly and at lower prices. The elimination of intermediaries has also provided opportunities such as reducing costs, reducing uncertainty in the use of e- services, sharing detailed product information, and continuing marketing activities (Tsai et al., 2011). Today, airlines offer many services to current and potential consumers over the Internet. An airline website is a pivotal channel where these e-services are offered, and it includes many functions such as providing flight routes, price information, an interactive communication channel, online booking, ticket purchase, and online check- in (Díaz & Martín-Consuegra, 2016). Moreover, in addition to providing core services, complementary services, such as hotel booking and car rental, are available on these websites (Harison & Boonstra, 2008). Therefore, it is necessary to create cost- effective, content-rich, and attractive websites to increase e-service use. By doing so, the online presence and effectiveness of airlines will increase, thus increasing business performance, customer satisfaction, and loyalty (Díaz & Martín-Consuegra, 2016; Shankar & Datta, 2020). In the existing literature, many studies have addressed the evaluation of e-service quality in the airline industry (Elkhani et al., 2014; Güreş et al., 2015; Tarkang et al., 2020). Due to the multi-dimensional nature of e-services, this situation can also be considered as a Multiple Criteria Decision-Making (MCDM) problem (Büyüközkan et al., 2020; Tsai et al., 2011). MCDM methods are applied in complex problems involving conflicting criteria to assist decision-making processes. In this regard, service quality, measured by quantitative and qualitative criteria, can be handled with MCDM methods (Mardani et al., 2015). However, human judgments are often uncertain and imprecise in service quality evaluation (Hu & Liao, 2011). This study aims to evaluate e-service quality in the airline industry considering the uncertain and imprecise environment. In doing so, we propose an integrated Fuzzy Analytical Hierarchy Process (F-AHP) and Fuzzy Measurement Alternatives and Ranking according to Compromise Solution (F- MARCOS) approach. This proposed approach provides a systematic and well-defined solution for e-service quality evaluation in the airline industry. The contribution of this study to literature is twofold. First, this study employs an integrated F-AHP and F- MARCOS methods for the first time in literature. Second, the e-service performance of scheduled airlines in Turkey has been successfully evaluated utilizing a comprehensive framework. The rest of this paper proceeds as follows: Section 2 reviews the concept of e- service quality and the existing literature within the airline industry context. Section 3 explains the detailed algorithms for the F-AHP and F-MARCOS methods. Section 4 presents the application of the real-world case study, followed by a three-stage sensitivity analysis. Next, Section 5 discusses the results. This study ends with a discussion of the managerial and theoretical implications, presenting research limitations and avenues for future research. Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 129 2. Literature review Fassnacht and Koese (2006) defined e-services as “services delivered via information and communication technology where the customer interacts solely with an appropriate user interface (e.g. automated teller machine or Web site) in order to retrieve desired benefits”. In literature, this concept is also discussed under different labels such as website quality and online service quality. However, it should be noted that the main focus is the needs of consumers in the virtual environment and what they expect from online services. Researchers have discussed that e-service quality has implications for many key marketing concepts such as customer satisfaction, loyalty, and purchase intention, and it consequently affects financial outcomes positively. Starting from this point of view, the exploration of e-service quality and the question of which critical factors e-services cover have attracted researchers. E- services provided in a virtual environment have distinctive characteristics, and since service quality scales are incapable of measuring e-services, e-service-specific scales have been proposed in literature (Shankar & Datta, 2020). Considering the main scales, Yoo and Donthu (2001) developed the SiteQual scale to evaluate e-service quality of online shopping websites. Wolfinbarger and Gilly (2003) used the eTailQ scale, consisting of fulfillment/reliability, privacy/security, website design, and customer service dimensions for retail e-services. Parasuraman et al. (2005) applied the E-S-QUAL scale to evaluate e-commerce websites. Based on the multi-dimensional nature of e-services, different second-order measurement models with various sub-dimensions have also been proposed (Blut, 2016). Reviewing the related studies, e-service quality is generally discussed in different contexts such as e-services, e-retailing, e-banking, and website-based services (Shankar & Datta, 2020). In addition, it is observed that proposed models are predominantly based on the Technology Acceptance Model (TAM) or the Theory of Reasoned Action (TRA). Thus, the e-service quality scales are generally a combination of usefulness and ease of use concepts, as well as other related beliefs such as entertainment (Cheng, 2011; Loiacono et al., 2002). In the context of the airline industry, many studies have addressed e-service quality. Generally speaking, the methodology used in these studies can be grouped as conventional statistical methods, usability testing, content analysis, and MCDM methods (Chong & Law, 2019). In the first group, Elkhani et al. (2014) examined the effect of e-service quality on e-satisfaction and e-loyalty through e-SERVQUAL, e- marketing, and Expectancy Disconfirmation Theory (EDT) frameworks. Llach et al. (2013) analyzed the impact of e-service quality on perceived value and loyalty in Spanish airline services. Lee and Wu (2011) added hedonics to the E-S-QUAL model and examined the relationships between website quality, perceived value, and satisfaction for 30 different airlines. Vuthisopon and Srinuan (2017) reported the positive impact of e-service quality on customer satisfaction in low-cost airlines in Thailand. Güreş et al. (2015) studied the relationship between e-service quality, passenger satisfaction, and passenger loyalty of domestic and international passengers in Turkey. In a recent study, Tarkang et al. (2020) examined the associations between airline website quality, electronic word of mouth, and purchase intention in Turkey. In the second group, Economides and Apostolou (2009) introduced a holistic airline site evaluation framework (ASEF) model and analyzed the websites of 30 major airlines using multiple quality criteria from the perspectives of the consumers. Díaz and Martín-Consuegra (2016) conducted a content analysis using 240 airline websites through six dimensions, including informativeness, usability, involvement, Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 130 inspiration, credibility, and reciprocity. Ceballos Hernandez et al. (2020) evaluated the web-based e-service quality of 25 Chinese airlines with content analysis. There are also some studies examining airline e-services using usability testing. Ekşioğlu et al. (2013) analyzed the website quality of airlines in Turkey through usability testing and heuristic evaluation. Murillo et al. (2017) evaluated LATAM airlines’ critical website functions through usability testing and heuristic evaluation. Although these studies provide powerful tools that incorporate qualitative and quantitative methods, they only aimed to identify usability problems on the websites (Chong & Law, 2019). Finally, limited studies in the extant literature have evaluated e-service quality of airlines using MCDM methods. In fact, the MCDM approach is frequently preferred in the airline industry. Badi and Abdulshaded (2019) analyzed the overall performance of Libyan airlines using AHP and the Full Consistency Method (FUCOM). On the other hand, e-services are also very suitable to be evaluated using MCDM methods. It also enables the imprecise information that may arise in quality evaluations to be easily overcome using fuzzy logic (Pamucar & Ecer, 2020). For example, Pamucar et al. (2018) applied AHP and MABAC (Multi-Attributive Border Approximation area Comparison) methods using interval rough numbers (IRN) and evaluated faculty web pages. In the airline industry, Tsai et al. (2011) examined e-marketing and e-service performance of airlines in Taiwan using the Decision Making Trial and Evaluation Laboratory (DEMATEL), ANP, and VIKOR methods. Çelik and Gök Kısa (2017) presented an e-service quality evaluation in the Turkish civil aviation industry by employing AHP and Promethee methods in a fuzzy environment. Abbasi et al. (2018) evaluated website quality of 5 airlines in Iran using the methods of F-AHP and Fuzzy Technique for Order Preference by Similarity to Ideal Solutions (F-TOPSIS). Similarly, Bakır and Atalık (2019) prioritized the factors affecting website quality of airline firms. More recently, Büyüközkan et al. (2020) developed a digital service quality model for airlines using the interval-valued intuitionistic fuzzy AHP (IVIF-AHP) method. Apart from these, some other studies have analyzed the website performance of airlines using ordinary or fuzzy MCDM methods (Dominic & Khan, 2014; Vatansever & Akgül, 2018). However, it should be noted that these studies use many criteria such as website traffic and broken link based on web diagnostic tools. Therefore, these studies focusing on technical issues do not reflect the consumer perspective. Built on the consumer perspective, the present study contributes to the evaluation of e-service quality in airlines using a larger sample size. 3. Research methodology This section covers the computation steps of the proposed F-AHP and F-MARCOS methods. The F-AHP method was used since it is the most frequently and successfully used methodology in service quality evaluation (Mardani et al., 2015). The F-MARCOS is also a recent and reasonable method that combines the rate and the reference point approaches, thus providing more robust results under uncertainty (Stanković et al., 2020). 3.1. Preliminaries for triangular fuzzy sets In many real-world problems, human judgments and perceptions are not certain or precise. The Fuzzy sets theory was proposed by Zadeh (1965) to handle the uncertainty of judgments. Fuzzy sets are sets with membership degrees defined as real Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 131 numbers in the interval [0;1] (Lin, 2010). A triangular fuzzy number (TFN) can be defined as ( , , )A l m u and membership functions are found in Eq. (1) as follows: 1 , ( ) , 0, x m l u x u mA l x m x m x u otherwise              (1) Where l and u stand for the lower and upper bounds of the fuzzy number A , and m is the mid-value of A . If it is assumed that 1 1 1 1 ( , , )A l m u and 2 2 2 2 ( , , )A l m u are two TFNs, the basic arithmetic operations for these two sets can be shown in Eq. (2)- (6) (Lin, 2010; Stanković et al., 2020): 1 2 1 1 1 2 2 2 1 2 1 2 1 2 ( , , ) ( , , ) ( , , )A A l m u l m u l l m m u u       (2) 1 2 1 1 1 2 2 2 1 2 1 2 1 2 ( , , ) ( , , ) ( , , )A A l m u l m u l l m m u u       (3) 1 2 1 1 1 2 2 2 1 2 1 2 1 2 ( , , ) ( , , ) ( , , )A A l m u l m u l u m m u l       (4) 1 1 1 1 1 1 1 2 2 2 2 2 22 ( , , ) , , ( , , ) A l m u l m u l m u l m uA         (5) 1 1 1 1 1 1 1 1 1 1 1 1 ( , , ) , ,A l m u u m l           (6) 3.2. The F-AHP method The AHP method is used to calculate criteria weights based on decision-maker (DM) judgments (Saaty, 1980). The AHP method uses pairwise comparisons and allows both qualitative and quantitative criteria to be evaluated. To handle uncertainty, the ordinary AHP method was extended to the fuzzy sets theory, and F- AHP was introduced. In literature, the F-AHP method has been successfully applied in many studies, including in-flight service quality evaluation (Li et al., 2017), the classification of container terminals (Adenso-Díaz et al., 2019), and the prioritization of traffic accessibility criteria (Stanković et al., 2019). In this study, we adopted Buckley’s (1985) approach, which received the least criticism in the existing literature (Kahraman et al., 2018). The application steps can be summarized as follows (Havle & Kılıç, 2019; Singh & Prasher, 2019): Step 1. Construct a fuzzy pairwise comparison matrix. In this step, DMs construct a pairwise comparison matrix of criteria using linguistic terms. In doing so, we adopted the nine-point conversion scale of Anagnostopoulos et al. (2007) to convert responses into fuzzy numbers (See Table 1). The resulting comparison matrix is given in Eq. (7). 12 1 21 2 1 2 1 1 1 n n m m a a a a A a a             (7) Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 132 Table 1. The nine-point fuzzy conversion scale (Anagnostopoulos et al., 2007) Linguistic Terms Crisp Scale TFS Scale Reciprocal TFN Scale Equally Preferred 1 (1,1,1) (1/1,1/1,1/1) Equally to moderately preferred 2 (1,2,3) (1/3,1/2/,1/1) Moderately preferred 3 (2,3,4) (1/4,1/3,1/2) Moderately to strongly preferred 4 (3,4,5) (1/5,1/4,1/3) Strongly preferred 5 (4,5,6) (1/6,1/5,1/4) Strongly to very strongly preferred 6 (5,6,7) (1/7,1/6,1/5) Very strongly preferred 7 (6,7,8) (1/8,1/7,1/6) Very strongly to extremely preferred 8 (7,8,9) (1/9,1/8,1/7) Extremely preferred 9 (8,9,9) (1/9,1/9,1/8) Step 2. Aggregate the fuzzy pairwise comparison matrix. In case of group decision- making, the judgments of the DMs are aggregated using Eq. (8). 1/ 1 K K ij ijk k l l          , 1/ 1 K K ij ijk k m m          , 1/ 1 K K ij ijk k u u          (8) Where ( , , ) ij ij ij A l m u and K denotes the number of DMs. Step 3. Calculate the fuzzy weights matrix. In this step, the fuzzy comparison values are first calculated using Eq. (9), as Buckley (1985) suggested. 1/ 1 n n i ij j r a          , 1, 2,...,i n (9) Then, fuzzy weights i w of criteria are calculated using Eq. (10). 1 1 2 ( ... ) i i n w r r r r       (10) Where i r represents the geometric mean of the fuzzy comparison values, while i w represents criteria weights. Step 4. Defuzzy fuzzy weights i w . Since i w is a fuzzy number, it is defuzzified with the center of area (COA) method using Eq. (11). ( ) / 3 i i i i w lw mw uw   (11) Step 5. Normalize the crisp weights. The calculated crisp values are normalized using Eq. (12) and crisp criteria weights are obtained. 1 i r n i i w w w    (12) 3.3. The F-MARCOS method The MARCOS method has been proposed by Stević et el. (2020) more recently. The basic principle of the MARCOS method is to find a solution based on the relationship between alternatives and reference values. Accordingly, the utility functions of the alternatives are calculated based on the ideal and anti-ideal solutions indicating best Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 133 and worst values according to the criteria. Similar to the TOPSIS method, the best alternative is located closest to the ideal solution and farthest from the anti-ideal solution (Puška et al., 2020). In literature, the MARCOS method has successfully been applied in sustainable supplier selection (Stević et al., 2020), the evaluation of human resource (Stević & Brković, 2020), and the selection of software (Puška et al., 2020). It also has been applied in fuzzy numbers (Stanković et al., 2020), D numbers (Chakraborty et al., 2020), and grey numbers (Badi & Pamucar, 2020). The steps of the F-MARCOS method can be summarized as follows (Stanković et al., 2020): Step 1. Formulate a fuzzy aggregated initial matrix. First, a decision matrix, which includes m alternatives and n criteria, is established. In this step, the fuzzy anti-ideal ( )A AI and the fuzzy ideal ( )A ID solutions are also determined using Eq. (13). 1 2 11 12 1 1 2 1 2 ai ai ain n m m mn id id idn x x x x x x X x x x x x x                 (13) The fuzzy ideal ( )A ID solution marks the desirable alternative, while the fuzzy anti-ideal ( )A AI solution shows the undesirable alternative. Considering the criterion type, solutions ( )A ID and ( )A AI are defined using Eq. (14) and (15). ( ) min i îj A AI x if j B and maxi ijx if j C (14) ( ) max i îj A ID x if j B and min i ijx if j C (15) Where B denotes the benefit type criteria, and C denotes the cost type criteria. Step 2. Create the fuzzy normalized decision matrix. In this step, the fuzzy decision matrix including ( )A ID and ( )A AI solutions is normalized using Eq. (16) and (17). ( , , ) , , l l l l m u id id id ij ij ij ij u m l ij ij ij x x x n n n n x x x           if j C (16) ( , , ) , , l m u ij ij ijl m u ij ij ij ij u u u id id id x x x n n n n x x x           if j B (17) Where the elements , , l m u ij ij ij x x x and ,l u id id x x are subtracted from the fuzzy decision matrix X . Step 3. Construct the fuzzy weighted-normalized decision matrix. The elements of the fuzzy normalized matrix are multiplied by weight coefficients using Eq. (18). ( , , ) l l m m u u ij ij j ij j ij j v n w n w n w    (18) Step 4. Calculate the fuzzy summation matrix ( ) i S . In this step, the fuzzy weighted- normalized matrix elements are summed using Eq. (19). Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 134 1 n i ij i S v    (19) For , , l m u ij ij ij n n n elements, the calculation is repeated and the column values are summed. Step 5. Determine the utility degree for each alternative. Two different matrices ( i K  and i K  ) are constructed, taking into account the ideal and anti-ideal solutions. This procedure is performed using Eq. (20) and (21). , , l m u i i i i i u m l ai ai aiai S s s s K s s sS          (20) , , l m u i i i i i u m l id id idid S s s s K s s sS          (21) Step 6. Construct the fuzzy combined matrix  iT . The utility degree scores of the alternatives are summed using Eq. (22). ( , , ) l l m m u u i i i i i i i i i T K K k k k k k k               (22) In this step, i T elements also need to be converted to a new fuzzy number  D using Eq. (23). Note that in doing so, the maximum values of the columns are used. ( , , ) max l m u i ij D d d d t  (23) In this step, the fuzzy number D is finally defuzzified by applying the formula 4 6 l m u crisp df   . By doing so, a crisp number is obtained. Step 7. Determine the utility functions of alternatives. Based on the formula crisp df , the utility functions are calculated according to the ideal ( ) i f K  and anti-ideal ( ) i f K  solutions. In doing so, Eq. (24) and (25) are applied. ( ) , , u m l i i i i i crisp crisp crisp crisp K k k k f K df df df df                (24) ( ) , , l m u i i i i i crisp crisp crisp crisp K k k k f K df df df df                (25) Step 7 is finalized with the defuzzification of i K  , i K  , ( ) i f K  and ( ) i f K  values through the same defuzzification formula. Step 8. Calculate the defuzzified utility function ( ) i f K . Using Eq. (26), the final utility function score for each alternative is calculated. ( ) 1 ( ) 1 ( ) 1 ( ) ( ) i i i i i i i K K f K f K f K f K f K             (26) Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 135 Step 9. Alternatives are ranked according to the decreasing values of utility function ( ) i f K . 4. Empirical case study and findings This section presents an empirical real-world case study in the Turkish airline industry employing the proposed approach. In this approach, we aimed to demonstrate the applicability of the integrated F-AHP and F-MARCOS methods in e- service quality evaluation. To achieve this, we focused on the web-based e-services provided by scheduled airlines in Turkey. A three-stage approach was adopted to address this problem, and the research framework of this study is given in Figure 1. 4.1. The proposed hierarchical model The existing literature proposes many hierarchical models for measuring e-service quality from the perspective of the consumers. One of these models is DeLone and McLean’s (2003) updated Information Systems (IS) success model. According to this model, information systems consist of three quality dimensions, namely: information quality, system quality, and service quality. In this study, this three-dimensional hierarchical model based on the IS success model was used, which has since been successfully applied in many studies (Chou & Cheng, 2012; Ecer, 2014; Nilashi et al., 2012; Tsai et al., 2011). The definitions of the hierarchical model elements are presented in Table 2. Define research objective Literature review Extract evaluation criteria and alternatives Determine the decision-makers (DMs) Create a decision hierarchy Decision- makers (DMs) Construct pairwise comparison matrix Get DMs judgments through the Fuzzy AHP method Obtain criteria weights Conduct a survey on the criteria by passenger Aggregate and establish a fuzzy decision matrix Evaluate alternatives through the Fuzzy MARCOS method Rank alternatives Stage 1 Stage 2 Stage 3 Conduct a sensitivity analysis Figure 1. Framework for research methodology Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 136 Table 2. E-service quality criteria and definitions Criteria Definition References Informatio n Quality (C1) It refers to the appropriateness of the information provided by the website. Relevancy (C11) It means that the information on the website complies with customer needs and expectations. (Chou & Cheng, 2012; Tsai et al., 2011; Ustasüleyman, 2013) Understan dability (C12) It expresses that the information on the website is clear and easy to understand. (Blut, 2016; Ecer, 2014; Kim & Stoel, 2004; Y. Lee & Kozar, 2006) Currency (C13) It means that the information on the website is current and timely. (Ecer, 2014; Lin, 2010; Nilashi et al., 2012; Tsai et al., 2011) Richness (C14) It means how detailed the information is about the service provided by the website. (Chou & Cheng, 2012; Ecer, 2014; Ustasüleyman, 2013) System Quality (C2) It refers to the technological equipment and infrastructural competence of the website. Security (C21) It means the level of confidentiality and protection of customer information on the website. (Blut, 2016; Hu & Liao, 2011; Tsai et al., 2011) Response time (C22) It expresses how quickly the website loads. (Alptekin et al., 2015; Lin, 2010; Nilashi et al., 2012) Personaliza tion (C23) It refers to the level of the website's ability to be personalized for the users. (Blut, 2016; Ecer, 2014; Hu & Liao, 2011) Navigabilit y (C24) It means how easy it is to navigate around the website. (Ecer, 2014; Lin, 2010; Tsai et al., 2011) Accessibilit y (C25) It refers to how easily the website can be accessed. (Alptekin et al., 2015; Chou & Cheng, 2012; Lin, 2010) Service Quality (C3) It refers to the overall support to the users offered by the website. Empathy (C31) It refers to how caring the information is and the attention paid to the users. (Ecer, 2014; Nilashi et al., 2012; Tsai et al., 2011) Responsive ness (C32) It expresses the level of willingness to provide service promptly and helpfully to online customers. (Chou & Cheng, 2012; Hu & Liao, 2011; Tsai et al., 2011) Reliability (C33) It expresses how accurate the services offered by the website are and the fulfillment of the promised service. (Chou & Cheng, 2012; Fassnacht & Koese, 2006; Hu & Liao, 2011) Trust (C34) It means how well the reputation of the e-services is perceived and that using the website is reassuring and relaxing. (Chou & Cheng, 2012; Ecer, 2014; Lin, 2010; Nilashi et al., 2012) 4.2. Data collection process The questionnaire technique was employed as a primary data source in both the weighting and evaluation stages. In the first questionnaire, DMs made pairwise Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 137 comparisons for dimensions and criteria, while in the second questionnaire, passengers evaluated airlines based on the same criteria. Questionnaires for the F- AHP application were collected from 11 DMs in the June-July period in 2017. The DMs encompassed airline managers, website designers, and aviation academics. In the second step, the questionnaire was distributed to passengers to evaluate their perceptions about airline e-services. The responses were collected in the Antalya Airport, Turkey during the period 6-10 November 2017. Since five domestic airlines operate at the airport, the weighted stratified sampling approach was adopted based on the number of passengers they carried. A total of 395 valid questionnaires were obtained. Demographic characteristics and the number of airline-based observations for the sample are given in Table 3. Table 3. Characteristics of sample and strata Variables Alternatives Percentage (%) Gender Male 65 Female 35 Age 18-25 27 26-35 41 36-45 24 46-55 6 56+ 2 Educational status High School 22 University 60 Postgraduate 18 Travel purpose Business 35 Leisure 28 Education 13 VFR (Visiting friends and relatives) 18 Other 6 Airline Passengers Carried Size of Strata (n) X1 2,682,682 153 X2 1,915,506 110 X3 1,117,509 67 X4 564,745 32 X5 589,551 33 Total 6,869,993 395 4.3. Application of the F-AHP method In this section, pairwise comparison matrices for dimensions and criteria are constructed using Eq. (7). Then, using Eq. (8), the responses from DMs are aggregated through Anagnostopoulos et al.’s (2007) scale into fuzzy numbers. The comparison results and local weights of e-service quality dimensions obtained using Eq. (9)-(12) are given in Table 4. Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 138 Table 4. Fuzzy comparison matrix of dimensions C1 C2 C3 Weight C1 (1.00,1.00,1.00) (0.87,1.28,1.85) (0.68,0.91,1.25) 0.348 C2 (0.54,0.78,1.15) (1.00,1.00,1.00) (0.51,0.72,1.08) 0.276 C3 (0.80,1.09,1.47) (0.93,1.39,1.98) (1.00,1.00,1.00) 0.386 Table 4 shows that the most important dimension is service quality 3 ( 0.386) C w  , followed by information quality 1 ( 0.348) C w  and system quality 2 ( 0.276) C w  . After the analysis of the dimensions, the evaluation criteria were subjected to pairwise comparison. The aggregated comparison matrices expressing local criteria weights are given in Tables 5-7. Table 5. Fuzzy comparison matrix for information quality criteria C11 C12 C13 C14 Weight C11 (1.00,1.00,1.00) (0.57,0.72,0.96) (0.56,0.76,1.05) (1.22,1.75,2.36) 0.234 C12 (1.04,1.38,1.76) (1.00,1.00,1.00) (1.13,1.38,1.75) (1.86,2.66,3.59) 0.350 C13 (0.95,1.32,1.77) (0.57,0.72,0.88) (1.00,1.00,1.00) (1.72,2.33,2.86) 0.281 C14 (0.42,0.57,0.70) (0.28,0.38,0.52) (0.36,0.44,0.78) (1.00,1.00,1.00) 0.135 Table 6. Fuzzy comparison matrix for system quality criteria C21 C22 … C25 Weight C21 (1.00,1.00,1.00) (3.50,4.39,5.22) … (1.18,1.63,2.09) 0.431 C22 (0.19,0.23,0.29) (1.00,1.00,1.00) … (0.54,0.72,0.99) 0.147 C23 (0.16,0.20,0.25) (0.31,0.37,0.45) … (0.26,0.36,0.55) 0.072 C24 (0.24,0.28,0.34) (0.65,0.83,1.12) … (0.46,0.58,0.79) 0.130 C25 (0.48,0.61,0.85) (1.01,1.40,1.86) … (1.00,1.00,1.00) 0.221 Table 7. Fuzzy comparison matrix for service quality criteria C31 C32 C33 C34 Weight C31 (1.00,1.00,1.00) (0.75,0.97,1.21) (0.29,0.39,0.54) (0.40,0.53,0.71) 0.155 C32 (0.83,1.03,1.34) (1.00,1.00,1.00) (0.31,0.38,0.53) (0.46,0.70,1.11) 0.172 C33 (1.84,2.59,3.42) (1.90,2.61,3.27) (1.00,1.00,1.00) (0.94,1.21,1.58) 0.388 C34 (1.41,1.88,2.49) (0.90,1.42,2.18) (0.63,0.83,1.07) (1.00,1.00,1.00) 0.285 The judgments of the DMs are consistent since the consistency ratio of the comparison matrices presented above is below 0.10 (Ecer, 2014). Lastly, the synthesizing procedure is carried out to find the global weight of each criterion. The local and global weights of the criteria are given in Table 8. As can be seen in Table 8, the most important criterion is found to be reliability 33 ( 0.146) C w  . Following this criterion, understandability 12 ( 0.122) C w  and security 21 ( 0.119) C w  rank second and third, respectively. Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 139 Table 8. Global weights of the criteria Dimension Weight Criteria Local Weight Global Weight C1 0.348 C11 0.234 0.081 C12 0.350 0.122 C13 0.281 0.098 C14 0.135 0.047 C2 0.276 C21 0.431 0.119 C22 0.147 0.040 C23 0.072 0.020 C24 0.130 0.036 C25 0.221 0.061 C3 0.386 C31 0.155 0.058 C32 0.172 0.065 C33 0.388 0.146 C34 0.285 0.107 4.4. Application of the F-MARCOS method In this subsection, a real-world case study on the Turkish airline industry is presented to evaluate e-service quality of airlines. Air traffic has steadily increased in Turkey and a total of 11 airlines are already based in the country (SHGM, 2020). These airlines are composed of different carriers, such as scheduled airlines, cargo carriers and charter airlines. As we focused on the consumer market, cargo carriers and charter airlines that usually do not sell directly and trade their seats with tour operators were excluded from this study (Williams, 2011). Therefore, the remaining five scheduled airlines (X1-X5) were the subject of the study. Using a five-point scale (1=“strongly disagree” to 5=“strongly agree”), a total of 395 passengers were surveyed to evaluate the e-services offered by scheduled airlines. The triangular fuzzy numbers (TFNs) corresponding to these values are presented in Table 9 (Pandey & Shukla, 2019). After collecting the questionnaire forms, they were aggregated via the arithmetic mean and fuzzy aggregated decision matrix including the fuzzy anti-ideal ( )A AI and the fuzzy ideal ( )A ID solutions are given in Table 10. Table 9. Five-point fuzzy rating scale Range Linguistic Terms TFNs 1 SD Strongly Disagree (0.0,1.0,2.0) 2 D Disagree (1.0,2.0,3.0) 3 N Neutral (2.0,3.0,4.0) 4 A Agree (3.0,4.0,5.0) 5 SA Strongly Agree (4.5,5.0,5.0) Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 140 Table 10. Average performance ratings and reference values X1 … X5 AI ID C11 (3.67,4.39,4.82) … (3.42,4.25,4.91) (3.17,4.00,4.67 (3.67,4.39,4.91) C12 (3.62,4.35,4.82) … (3.53,4.28,4.78) (3.37,4.17,4.77) (3.62,4.35,4.88) C13 (3.83,4.51,4.87) … (3.50,4.28,4.84) (3.50,4.28,4.76) (3.83,4.51,4.88) C14 (3.34,4.13,4.71) … (3.53,4.31,4.88) (3.00,3.88,4.65) (3.53,4.31,4.88 C21 (2.87,3.69,4.33) … (2.61,3.50,4.28) (2.29,3.18,3.97) (2.87,3.69,4.33) C22 (3.25,4.05,4.65) … (3.11,3.97,4.69) (2.85,3.73,4.48) (3.25,4.05,4.70) C23 (2.49,3.39,4.21) … (2.53,3.44,4.25) (1.98,2.94,3.84) (2.82,3.68,4.41) C24 (3.40,4.16,4.67) … (3.81,4.53,4.97) (3.17,4.00,4.67) (3.81,4.53,4.97) C25 (3.55,4.27,4.71) … (3.73,4.44,4.84) (3.42,4.19,4.71) (3.73,4.44,4.85) C31 (3.00,3.84,4.53) … (2.66,3.63,4.56) (2.66,3.63,4.48) (3.18,4.01,4.66) C32 (2.47,3.39,4.23) … (2.89,3.75,4.47) (2.15,3.09,3.97) (2.89,3.75,4.47) C33 (3.32,4.12,4.72) … (3.61,4.38,4.91) (2.84,3.72,4.47) (3.61,4.38,4.91) C34 (3.15,3.96,4.59) … (3.34,4.16,4.78) (2.88,3.76,4.52) (3.34,4.16,4.78) The next step was the normalization process to eliminate the anomalies in the decision matrix. Since all criteria are of the benefit type, the normalization procedure of alternatives and reference values was completed using Eq. (17). The fuzzy normalized decision matrix is given in Table 11. Table 11. Fuzzy normalized decision matrix X1 … X5 AI ID C11 (0.75,0.89,0.98) … (0.70,0.87,1.00 (0.65,0.82,0.95) (0.75,0.89,1.00) C12 (0.74,0.89,0.99) … (0.72,0.88,0.98) (0.69,0.86,0.98) (0.74,0.89,1.00) C13 (0.78,0.92,1.00) … (0.72,0.88,0.99) (0.72,0.88,0.97) (0.78,0.92,1.00) C14 (0.69,0.85,0.97) … (0.72,0.88,1.00) (0.62,0.80,0.95) (0.72,0.88,1.00) C21 (0.66,0.85,1.00) … (0.60,0.81,0.99) (0.53,0.73,0.92) (0.66,0.85,1.00) C22 (0.69,0.86,0.99) … (0.66,0.84,1.00) (0.61,0.79,0.95) (0.69,0.86,1.00) C23 (0.56,0.77,0.95) … (0.57,0.78,0.96) (0.45,0.67,0.87) (0.64,0.83,1.00) C24 (0.68,0.84,0.94) … (0.77,0.91,1.00) (0.64,0.81,0.94) (0.77,0.91,1.00) C25 (0.73,0.88,0.97) … (0.77,0.92,1.00) (0.71,0.86,0.97) (0.77,0.92,1.00) C31 (0.64,0.82,0.97) … (0.57,0.78,0.98) (0.57,0.78,0.96) (0.68,0.86,1.00) C32 (0.55,0.76,0.95) … (0.65,0.84,1.00) (0.48,0.69,0.89) (0.65,0.84,1.00) C33 (0.68,0.84,0.96) … (0.74,0.89,1.00) (0.58,0.76,0.91) (0.74,0.89,1.00) C34 (0.66,0.83,0.96) … (0.70,0.87,1.00) (0.60,0.79,0.94) (0.70,0.87,1.00) After normalization, the fuzzy weighted-normalized matrix was constructed. In doing so, the fuzzy normalized matrix elements were multiplied by criteria weights using Eq. (18). The fuzzy weighted-normalized matrix is given in Table 12. Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 141 Table 12. Fuzzy weighted-normalized decision matrix X1 … X5 AI ID C11 (0.06,0.07,0.08) … (0.06,0.07,0.08) (0.05,0.07,0.08) (0.06,0.07,0.08) C12 (0.09,0.11,0.12) … (0.09,0.11,0.12) (0.08,0.10,0.12) (0.09,0.11,0.12) C13 (0.08,0.09,0.10) … (0.07,0.09,0.10) (0.07,0.09,0.10) (0.08,0.09,0.10) C14 (0.03,0.04,0.04) … (0.03,0.04,0.04) (0.03,0.04,0.04) (0.03,0.04,0.04) C21 (0.08,0.10,0.12) … (0.07,0.10,0.12) (0.06,0.09,0.11) (0.08,0.10,0.12) C22 (0.03,0.03,0.04) … (0.03,0.03,0.04) (0.02,0.03,0.04) (0.03,0.03,0.04) C23 (0.01,0.01,0.02) … (0.01,0.01,0.02) (0.01,0.01,0.02) (0.01,0.02,0.02) C24 (0.02,0.03,0.03) … (0.03,0.03,0.03) (0.02,0.03,0.03) (0.03,0.03,0.03) C25 (0.04,0.05,0.06) … (0.05,0.05,0.06) (0.04,0.05,0.06) (0.05,0.05,0.06) C31 (0.04,0.05,0.06) … (0.03,0.05,0.06) (0.03,0.05,0.06) (0.04,0.05,0.06) C32 (0.04,0.05,0.06) … (0.04,0.05,0.06) (0.03,0.04,0.06) (0.04,0.05,0.06) C33 (0.07,0.09,0.10) … (0.08,0.10,0.11) (0.06,0.08,0.10) (0.08,0.10,0.11) C34 (0.10,0.12,0.14) … (0.10,0.13,0.15) (0.09,0.12,0.14) (0.10,0.13,0.15) In the next step, the fuzzy summation matrix ( ) i S was calculated using Eq. (19). After this step, the utility degree of each alternative was determined based on the ideal and anti-ideal solutions. In doing so, fuzzy numbers i K  and i K  were calculated applying Eq. (20) and (21). Following this, the ideal and anti-ideal utility degrees of the alternatives were summed to construct the fuzzy combined matrix ( ) i T using Eq. (22). However, the i T elements needed to be converted to fuzzy number ( )D using Eq. (23) and subsequently defuzzified. The results obtained by applying Eq. (19)-(23) are given in Table 13. Table 13. Calculation of Steps 4-6 through the Fuzzy MARCOS Application i S iK  i K  i T D AI (0.61,0.79,0.94) X1 (0.69,0.85,0.97) (0.73,1.08,1.59) (0.69,0.97,1.35) (1.42,2.05,2.95) (1.42,2.06,3.01) 2.11 X2 (0.66,0.83,0.97) (0.70,1.05,1.59) (0.66,0.94,1.35) (1.35,1.99,2.94) X3 (0.66,0.84,0.97) (0.71,1.06,1.59) (0.66,0.95,1.35) (1.37,2.01,2.95) X4 (0.62,0.80,0.94) (0.66,1.01,1.55) (0.62,0.91,1.31) (1.28,1.92,2.86) X5 (0.69,0.86,0.99) (0.73,1.09,1.62) (0.69,0.98,1.38) (1.42,2.06,3.01) ID (0.72,0.88,1.00) As seen in Table 13, the defuzzification was carried out as the final step. The elements of the matrix i T were summed as follows using Eq. (22): 1 (0.73 0.69, 1.08 0.97, 1.59 1.35) (1.42, 2.05, 2.95) X t      Then, a new fuzzy number D shows the maximum values of the column iT . The maximum elements in this column were calculated as  , , (1.42, 2.06, 3.01)l m uD d d d  . In the defuzzification of the number D , the formula 4 6 l m u crisp df   was used and 1.42 4 2.06 3.01 6 2.11 crisp df     . The remaining Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 142 application steps were carried out based on the crisp df values. Firstly, the utility functions of alternatives related to the ideal  if K  and anti-ideal  if K  solutions were calculated using Eq. (24) and (25). The calculated i K  , i K  ,  if K  and  if K  values were also converted to crisp numbers with the defuzzification procedure above. In the last step, the utility functions  if K of the criteria were calculated using these crisp values applying Eq. (26). The results and ranking applied by Eq. (24)-(26) are depicted in Table 14. Table 14. Ranking of alternatives via utility functions  if K   if K  i K  i K   if K   if K   if K Rank X1 (0.33,0.46,0.64) (0.35,0.51,0.75) 1.107 0.987 0.467 0.524 0.686 2 X2 (0.31,0.45,0.64) (0.33,0.50,0.75) 1.081 0.964 0.456 0.511 0.649 4 X3 (0.31,0.45,0.64) (0.33,0.50,0.75) 1.088 0.969 0.459 0.514 0.658 3 X4 (0.29,0.43,0.62) (0.31,0.48,0.73) 1.040 0.927 0.438 0.492 0.593 5 X5 (0.33,0.46,0.65) (0.35,0.51,0.77) 1.118 0.996 0.471 0.529 0.702 1 Table 14 provides the ranking of airline alternatives. Based on the table, the best alternative is X5. On the other hand, X4 is the worst alternative. 4.5. Sensitivity analysis In this section, a sensitivity analysis was conducted to ensure the robustness of the application and validate the calculation. In doing so, our sensitivity analysis consisted of three parts to perform a rigorous analysis. In the first part, we tested the effect of the change in criteria weights on the calculation. Considering Keshavarz Ghorabaee et al.’s (2018) guidelines, 13 simulated scenarios (Set1-Set13) were employed to generate different criteria weights (See Figure 2). The ranking of the alternatives resulting from the scenarios is given in Figure 3. As Figure 3 shows, the ranking is largely stable except for a slight change in X2 and X3. Figure 2. Simulated weights for scenarios 0 0.03 0.06 0.09 0.12 0.15 Set1 Set2 Set3 Set4 Set5 Set6 Set7Set8 Set9 Set10 Set11 Set12 Set13 C11 C12 C13 C14 C21 C22 C23 C24 C25 C31 C32 C33 C34 Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 143 Figure 3. Ranking of airlines under different scenarios The rank reversal effect is considered to be one of the major shortcomings of MCDM methods. It means that adding or removing an alternative in the decision matrix affects the ranking (Stanković et al., 2020; Yazdani et al., 2019). A scenario- based solution on dynamic decision matrices is presented in the second part. As mentioned above, the ranking of airlines was obtained as X5>X1>X3>X2>X4. Accordingly, X4 is clearly the worst alternative. Therefore, if this alternative is eliminated, in a robust calculation, the ranking of the remaining alternatives is expected to remain the same. To ensure this, we created dynamic decision matrices based on eliminating the worst alternative in each round and progressing until the last alternative remained. The scenario-based rankings in this study are presented in Table 15. As shown in Table 15, the worst alternative (X4) was deleted first, and then the remaining four alternatives are ranked in Scenario 1. The alternative X2, which was the worst in the new ranking, was then deleted, and the application was finalized in Scenario 3. It is clear that the ranking remained unchanged. Table 15. Rank reversal effect in the application Alternative Initial Rank Scenario 1 Scenario 2 Scenario 3 X1 2 2 2 2 X2 4 4 ● ● X3 3 3 3 ● X4 5 ● ● ● X5 1 1 1 1 In the last part of the sensitivity analysis, the stability of the results was compared with the results of other alternative fuzzy MCDM techniques. In this context, some effective methods such as TOPSIS, MABAC, MOORA, WASPAS, and MAIRCA were employed under a fuzzy environment. Spearman’s rank-order correlation coefficients  sr were also employed for rankings in the analysis. As shown in the correlation heatmap in Figure 4, the proposed airline ranking is highly credible. 0 1 2 3 4 5 6 Set1 Set2 Set3 Set4 Set5 Set6 Set7 Set8 Set9 Set10 Set11 Set12 Set13 X1 X2 X3 X4 X5 Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 144 Figure 4. Correlation heatmap for results of alternative methods 5. Results and discussion In the application, the dimensions and criteria in the hierarchical model were first subjected to pairwise comparison. Fuzzy pairwise comparison matrices are presented in Tables 4-7. The results are consistent since the consistency ratios of the comparison matrices are below 0.10 (CR <0.10) (Ecer, 2014). From the fuzzy pairwise comparison matrix the most important dimension was service quality (C3), which expresses the overall support offered by the airline website. Service quality was followed by the information quality (C1) and system quality (C2) dimensions, respectively (See Table 8). Considering the existing literature, service quality comes to the fore in many studies (Alptekin et al., 2015; Ustasüleyman, 2013). With regards to information quality, Lionella et al. (2020) reported that this is the most determining factor on overall e-service quality from the meta-analytical perspective. Moreover, they argued that information quality is the factor most associated with perceived trust. In addition, similar to the findings of Chou and Cheng (2012), system quality was found to be the least important criteria. In light of these findings, considering the characteristics of the airline industry, it can be concluded that passengers prioritize; a) the delivery of services without any problems, b) smooth interactions, c) the acquisition of high-quality information about the service to be purchased. In terms of the global weight of the criteria, the most important criterion was reliability (C33), followed by the understandability (C12) and security (C21) criteria. The importance of these criteria has also been identified in previous studies (Chou & Cheng, 2012; Lee & Kozar, 2006). Additionally, it is seen that the reliability criterion, which considers the degree to which the delivered services were as promised (i.e., flight delay due to increased air traffic, flight cancellation, and overbooking), was the most important criterion for airline passengers. In fact, reliability was also found to be the most important criterion in previous studies (Lee & Lin, 2005; Shankar & Datta, Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 145 2020; Tsai et al., 2011). Following the deregulation of the US airline industry in 1978, both flight frequencies and air traffic have increased considerably, which has caused some confusion for passengers. Thus, in this context, the understandability criterion may represent the expectations of passengers that the information provided will be understandable and satisfactory. In addition, as seen in previous studies (Çelik & Gök Kısa, 2017; Hsu et al., 2012), in the airline industry where services are mainly distributed online, consumers are concerned about the confidentiality of their personal and financial data and, therefore, they pay particular attention to the security criterion. This issue has also been found in past studies as one of the main concerns of passengers during online booking (Bigné et al., 2010; Lee et al., 2019). In light of the above-mentioned findings, consumer expectations can also be associated with Hofstede’s Cultural Dimensions Theory (Hofstede & Bond, 1984). As has previously been established (Blut et al., 2015), cultural dimensions exert a moderating effect on people’s perceptions of the quality of e-services. Thus, it can be deduced that passengers care about receiving adequate general support and satisfactory information so as to minimize their risks in Turkey, a country in which the population is generally considered to be characterized by high levels of uncertainty avoidance. After weighting, the web-based e-service quality of airlines was evaluated using F- MARCOS under uncertainty. As a result, the best and worst alternatives were X5 and X4, respectively. Finally, a three-stage sensitivity analysis was conducted to check the robustness of the calculation. With regards to the rank reversal effect, it was seen that there is no rank reversal effect. In addition, Figures 3 and 4 indicate that the weight changes do not affect ranking, thus indicating that the application presented is credible. 5. Conclusions Being aware of consumer expectations helps to ensure firms’ survival. Yet, it is evident that firms cannot fully understand consumer expectations (Kurtulmuşoğlu et al., 2016). From a different view, it is also not always possible to meet all consumer expectations in practice. Therefore, the most sensible approach is to recognize those criteria which have been prioritized. As the importance of providing satisfactory e- services has been established in literature, the importance levels of the constituent criteria should be ascertained in order to satisfy consumers. In this study, the criteria affecting e-service quality in airlines have been prioritized, and a real-world case study based on passenger evaluations of scheduled airlines is presented. It should be noted, however, that service quality evaluations often suffer from imprecise judgments. Therefore, we employed the proposed approach in a fuzzy environment in order to handle the subjective and imprecise judgments of people. Theoretically, the key contributions of this study are twofold. This study is the first to integrate F-AHP and F-MARCOS methods in a case study. This approach, which was applied successfully in the area of airline e-service quality, can be used in different domains. Additionally, e-service quality of scheduled airlines in Turkey has been analyzed for the first time in literature. In this regard, the proposed approach is expected to contribute to the existing literature. The present study also has numerous implications for Turkish airline managers. From a managerial perspective, the findings will assist airlines in providing more satisfactory online services by becoming more aware of the priorities of customers in terms of the e-service quality elements. Moreover, since a hierarchical model with 13 criteria has been developed in this study, Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 146 airlines can use these criteria to monitor their e-service processes. The success of airlines depends on their compatibility with the voice of the customer. Therefore, this study, in which passengers evaluated the e-service quality of scheduled airlines, provides vital understanding for airline managers. In terms of the findings, reliability, understandability, and security criteria were the most important criteria. Therefore, airlines should consider these in their web-based marketing strategies and strengthen the perception of reliability in the minds of the consumers. Several recommendations for further research can be made based on the findings of this study. First, since a comprehensive e-service quality evaluation model is still required for the airline industry (Chong & Law, 2019), we suggest that the hierarchical model could be enriched by using focus groups, in-depth interviews, etc. Prior studies concerning e-service quality have considered matters such as website quality and e- service quality, and they have predominantly focused on services delivered through a personal computer (PC). However, it has been shown that the use of mobile devices in relation to the airline industry is increasing dramatically. According to SITA (2019), mobile applications for passenger services are one of the investment priorities of airlines. Mobile devices are frequently used in many services such as booking, bag tracking, self-boarding, etc. In addition, due to the ubiquity and localization characteristics of mobile devices, consumer expectations can be shaped accordingly (Lionello et al., 2020). Therefore, future studies should address airline e-service quality from the perspective of mobile devices. Second, when the literature regarding airline e-service quality is considered, it quickly becomes apparent that the issue of business model segmentation has previously been ignored. Thus, future studies can provide deeper insights by focusing on different airlines such as full-service carriers (FSCs) and low-cost carriers (LCCs). It is important to recognize that this study has a few limitations. First, the survey data were only collected from a limited number of passengers using scheduled airlines in Turkey. Therefore, it is assumed that the sample represents the population. Is it also worth mentioning that demographics (age/culture etc) of passengers could also affect which criteria are more important. In addition, the ratings reflect the period in which the data were collected. As airline websites are very dynamic, there is a possibility that different results would be obtained in future studies. Second, the research data were collected at only a single airport due to procedural difficulties and time limitations. Third, the proposed hierarchical model may have excluded some factors that affect e- service quality. Therefore, the evaluation performed in this study reflects the investigated airlines’ performances according to only certain criteria. Finally, the results should not be generalized because they were derived from data concerning just five scheduled airlines. Acknowledgement: This paper was derived from Mahmut Bakır’s master’s thesis entitled “An Integrated Approach to the Evaluation of E-Service Quality in Airline Companies” conducted at Anadolu University, Eskişehir. Author Contributions: Each author has participated and contributed sufficiently to take public responsibility for appropriate portions of the content. Funding: This research received no external funding. Conflicts of Interest: The authors declare that there is no conflict of interest. Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 147 References Abbasi, R., Rezaei, N., Esmaili, S., & Abbasi, Z. (2018). Website quality and evaluation: a perspective of Iranian airline industry. International Journal of Electronic Business, 14(2), 103–127. https://doi.org/10.1504/ijeb.2018.10016233 Adenso-Díaz, B., Álvarez, N. G., & Alba, J. A. L. (2019). A fuzzy AHP classification of container terminals. Maritime Economics and Logistics, 22, 218–238. https://doi.org/10.1057/s41278-019-00144-4 Alptekin, N., Hall, E. E., & Sevim, N. (2015). Evaluation of Websites Quality Using Fuzzy TOPSIS Method. International Journal of Academic Research in Business and Social Sciences, 5(8), 221–242. https://doi.org/10.6007/ijarbss/v5-i8/1778 Anagnostopoulos, K. P., Gratziou, M., & Vavatsikos, A. P. (2007). Using the fuzzy Analytic Hierarchy Process for selecting wastewater facilities at prefecture level. European Water, 19(20), 15–24. Badi, I., & Abdulshahed, A. (2019). Ranking the Libyan airlines by using full consistency method (FUCOM) and analytical hierarchy process (AHP). Operational Research in Engineering Sciences: Theory and Applications, 2(1), 1–14. https://doi.org/10.31181/oresta1901001b Badi, I., & Pamucar, D. (2020). Supplier selection for steelmaking company by using combined Grey-Marcos methods. Decision Making: Applications in Management and Engineering, 3(2), 37–48. https://doi.org/10.31181/dmame2003037b Bakır, M., & Atalık, Ö. (2019). Havayolu İşletmelerinde E-hizmet Kalitesi Boyutlarının Önem Düzeylerinin Belirlenmesi. Eskişehir Osmangazi Üniversitesi İktisadi ve İdari Bilimler Dergisi, 14(1), 149–168. https://doi.org/10.17153/oguiibf.453820 Bigné, E., Sanz, S., Ruiz, C., & Aldás, J. (2010). Why Some Internet Users Don’t Buy Air Tickets Online. In U. Gretzel, R. Law, & M. Fuchs (Eds.), Information and Communication Technologies in Tourism 2010 (pp. 209–221). Springer-Verlag. https://doi.org/10.1007/978-3-211-99407-8_18 Blut, M. (2016). E-Service Quality: Development of a Hierarchical Model. Journal of Retailing, 92(4), 500–517. https://doi.org/10.1016/j.jretai.2016.09.002 Blut, M., Chowdhry, N., Mittal, V., & Brock, C. (2015). E-Service Quality: A Meta-Analytic Review. Journal of Retailing, 91(4), 679–700. https://doi.org/10.1016/j.jretai.2015.05.004 Buckley, J. J. (1985). Fuzzy hierarchical analysis. Fuzzy Sets and Systems, 17(3), 233– 247. https://doi.org/10.1016/0165-0114(85)90090-9 Büyüközkan, G., Havle, C. A., & Feyzioğlu, O. (2020). A new digital service quality model and its strategic analysis in aviation industry using interval-valued intuitionistic fuzzy AHP. Journal of Air Transport Management, 86. https://doi.org/10.1016/j.jairtraman.2020.101817 Ceballos Hernandez, C., Paios-Sánchez, P., & Rios, M. A. (2020). Website quality assessment: A case study of chinese airlines. Indian Journal of Marketing, 50(1), 42– 64. https://doi.org/10.17010/ijom/2020/v50/i1/149773 Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 148 Çelik, P., & Gök Kısa, A. C. (2017). Fuzzy AHP-Fuzzy Promethee Approach in Evaluation of E-Service Quality: Case of Airline Web Sites. The Journal of International Social Research, 10(52), 1003–1013. https://doi.org/10.17719/jisr.2017.1954 Chakraborty, S., Chattopadhyay, R., & Chakraborty, S. (2020). An integrated D-MARCOS method for supplier selection in an iron and steel industry. Decision Making: Applications in Management and Engineering, 3(2), 49–69. https://doi.org/10.31181/dmame2003049c Cheng, Y. H. (2011). Evaluating web site service quality in public transport: Evidence from Taiwan High Speed Rail. Transportation Research Part C: Emerging Technologies, 19(6), 957–974. https://doi.org/10.1016/j.trc.2011.04.003 Chong, S., & Law, R. (2019). Review of studies on airline website evaluation. Journal of Travel and Tourism Marketing, 36(1), 60–75. https://doi.org/10.1080/10548408.2018.1494084 Chou, W. C., & Cheng, Y. P. (2012). A hybrid fuzzy MCDM approach for evaluating website quality of professional accounting firms. Expert Systems with Applications, 39(3), 2783–2793. https://doi.org/10.1016/j.eswa.2011.08.138 DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30. https://doi.org/10.1080/07421222.2003.11045748 Díaz, E., & Martín-Consuegra, D. (2016). A latent class segmentation analysis of airlines based on website evaluation. Journal of Air Transport Management, 55, 20–40. https://doi.org/10.1016/j.jairtraman.2016.04.007 Dominic, P. D. D., & Khan, H. (2014). Performance measure of airline websites using analytical hierarchy process & fuzzy analytical hierarchy process. Proceedings - 4th IEEE International Conference on Control System, Computing and Engineering, ICCSCE 2014, 530–535. https://doi.org/10.1109/ICCSCE.2014.7072775 Ecer, F. (2014). A hybrid banking websites quality evaluation model using AHP and COPRAS-G: A turkey case. Technological and Economic Development of Economy, 20(4), 758–782. https://doi.org/10.3846/20294913.2014.915596 Economides, A. A., & Apostolou, G. (2009). User-centred evaluation of airlines’ websites. International Journal of Digital Culture and Electronic Tourism, 1(4), 280– 301. https://doi.org/10.1504/IJDCET.2009.025355 Ekşioǧlu, M., Kiriş, E., Çakir, T., Güvendik, M., Koyutürk, E. D., & Yilmaz, M. (2013). A user experience study of airline websites. Lecture Notes in Computer Science, 8015 LNCS(PART 4), 173–182. https://doi.org/10.1007/978-3-642-39253-5_19 Elkhani, N., Soltani, S., & Jamshidi, M. H. M. (2014). Examining a hybrid model for e- satisfaction and e-loyalty to e-ticketing on airline websites. Journal of Air Transport Management, 37, 36–44. https://doi.org/10.1016/j.jairtraman.2014.01.006 Fassnacht, M., & Koese, I. (2006). Quality of electronic services: Conceptualizing and testing a hierarchical model. Journal of Service Research, 9(1), 19–37. https://doi.org/10.1177/1094670506289531 Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 149 Güreş, N., Arslan, S., & Yılmaz, H. (2015). E-Service Quality, Passenger Satisfaction and Passenger Loyalty Relationship in Airline Industry. Journal of Management, Marketing and Logistics, 2(1), 44–56. https://doi.org/10.17261/pressacademia.2015111604 Harison, E., & Boonstra, A. (2008). Reaching new altitudes in e-commerce: Assessing the performance of airline websites. Journal of Air Transport Management, 14(2), 92– 98. https://doi.org/10.1016/j.jairtraman.2008.02.002 Havle, C. A., & Kılıç, B. (2019). A hybrid approach based on the fuzzy AHP and HFACS framework for identifying and analyzing gross navigation errors during transatlantic flights. Journal of Air Transport Management, 76, 21–30. https://doi.org/10.1016/j.jairtraman.2019.02.005 Hofstede, G., & Bond, M. H. (1984). Hofstede’s culture dimensions: An Independent Validation Using Rokeach’s Value Survey. Journal of Cross-Cultural Psychology, 15(4), 417–433. https://doi.org/10.1177/0022002184015004003 Hsu, T. H., Hung, L. C., & Tang, J. W. (2012). A hybrid ANP evaluation model for electronic service quality. Applied Soft Computing Journal, 12(1), 72–81. https://doi.org/10.1016/j.asoc.2011.09.008 Hu, Y. C., & Liao, P. C. (2011). Finding critical criteria of evaluating electronic service quality of Internet banking using fuzzy multiple-criteria decision making. Applied Soft Computing Journal, 11(4), 3764–3770. https://doi.org/10.1016/j.asoc.2011.02.008 Kahraman, C., Onar, S. Ç., & Öztayşi, B. (2018). B2C Marketplace Prioritization Using Hesitant Fuzzy Linguistic AHP. International Journal of Fuzzy Systems, 20(7), 2202– 2215. https://doi.org/10.1007/s40815-017-0429-4 Keshavarz Ghorabaee, M., Amiri, M., Zavadskas, E. K., & Antucheviciene, J. (2018). A new hybrid fuzzy MCDM approach for evaluation of construction equipment with sustainability considerations. Archives of Civil and Mechanical Engineering, 18(1), 32– 49. https://doi.org/10.1016/j.acme.2017.04.011 Kim, S., & Stoel, L. (2004). Dimensional hierarchy of retail website quality. Information and Management, 41(5), 619–633. https://doi.org/10.1016/j.im.2003.07.002 Kurtulmuşoğlu, F. B., Can, G. F., & Tolon, M. (2016). A voice in the skies: Listening to airline passenger preferences. Journal of Air Transport Management, 57, 130–137. https://doi.org/10.1016/j.jairtraman.2016.07.017 Lee, F. H., & Wu, W. Y. (2011). Moderating effects of technology acceptance perspectives on e-service quality formation: Evidence from airline websites in Taiwan. Expert Systems with Applications, 38(6), 7766–7773. https://doi.org/10.1016/j.eswa.2010.12.131 Lee, G. G., & Lin, H. F. (2005). Customer perceptions of e-service quality in online shopping. International Journal of Retail and Distribution Management, 33(2), 161– 176. https://doi.org/10.1108/09590550510581485 Lee, K. F., Haque, A., Maulan, S., & Abdullah, K. (2019). Determining intention to buy air e-tickets in Malaysia. Management Science Letters, 9(6), 933–944. https://doi.org/10.5267/j.msl.2019.2.009 Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 150 Lee, Y., & Kozar, K. A. (2006). Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach. Decision Support Systems, 42(3), 1383–1401. https://doi.org/10.1016/j.dss.2005.11.005 Li, W., Yu, S., Pei, H., Zhao, C., & Tian, B. (2017). A hybrid approach based on fuzzy AHP and 2-tuple fuzzy linguistic method for evaluation in-flight service quality. Journal of Air Transport Management, 60, 49–64. https://doi.org/10.1016/j.jairtraman.2017.01.006 Lin, H. F. (2010). An application of fuzzy AHP for evaluating course website quality. Computers and Education, 54(4), 877–888. https://doi.org/10.1016/j.compedu.2009.09.017 Lionello, R. L., Slongo, L. A., & Matos, C. A. de. (2020). Electronic service quality: a meta- analysis. Marketing Intelligence and Planning. https://doi.org/10.1108/MIP-06-2019- 0340 Llach, J., Marimon, F., Alonso-Almeida, M. del M., & Bernardo, M. (2013). Determinants of online booking loyalties for the purchasing of airline tickets. Tourism Management, 35, 23–31. https://doi.org/10.1016/j.tourman.2012.05.006 Loiacono, E. T., Watson, R. T., & Goodhue, D. L. (2002). WebQual: A measure of website quality. Marketing Theory and Applications, 13(3), 432–438. Mardani, A., Jusoh, A., Zavadskas, E. K., Khalifah, Z., & Nor, K. M. D. (2015). Application of multiple-criteria decision-making techniques and approaches to evaluating of service quality: a systematic review of the literature. Journal of Business Economics and Management, 16(5), 1034–1068. https://doi.org/10.3846/16111699.2015.1095233 Murillo, B., Vargas, S., Moquillaza, A., Fernández, L., & Paz, F. (2017). Usability testing as a complement of heuristic evaluation: A case study. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10288 LNCS, 434–444. https://doi.org/10.1007/978-3-319-58634- 2_32 Nilashi, M., Bagherifard, K., Ibrahim, O., Janahmadi, N., & Ebrahimi, L. (2012). Ranking parameters on quality of online shopping websites using multi-criteria method. Research Journal of Applied Sciences, Engineering and Technology, 4(21), 4380–4396. Pamucar, D., & Ecer, F. (2020). Prioritizing the weights of the evaluation criteria under fuzziness: The fuzzy full consistency method – fucom-f. Facta Universitatis, Series: Mechanical Engineering, 18(3), 419–437. https://doi.org/10.22190/FUME200602034P Pamucar, D., Stević, Ž., & Zavadskas, E. K. (2018). Integration of interval rough AHP and interval rough MABAC methods for evaluating university web pages. Applied Soft Computing Journal, 67, 141–163. https://doi.org/10.1016/j.asoc.2018.02.057 Pandey, M. M., & Shukla, D. (2019). Evaluating the human performance factors of air traffic control in Thailand using Fuzzy Multi Criteria Decision Making method. Journal of Air Transport Management, 81. https://doi.org/10.1016/j.jairtraman.2019.101708 Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). E-S-QUAL a multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(3), 213–233. https://doi.org/10.1177/1094670504271156 Application of Fuzzy AHP and Fuzzy MARCOS Approach for the Evaluation of E-Service… 151 Puška, A., Stojanović, I., Maksimović, A., & Osmanović, N. (2020). Evaluation software of project management used measurement of alternatives and ranking according to compromise solution (MARCOS) method. Operational Research in Engineering Sciences: Theory and Applications, 3(1), 89–102. https://doi.org/10.31181/oresta2001089p Saaty, T. L. (1980). The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. MacGraw-Hill. Shankar, A., & Datta, B. (2020). Measuring e-service quality : a review of literature. International Journal of Services Technology and Management, 26(1), 77–100. https://doi.org/10.1504/IJSTM.2020.10027043 SHGM. (2020). Sivil Havacılık Genel Müdürlüğü Faaliyet Raporu. http://web.shgm.gov.tr/documents/sivilhavacilik/files/pdf/kurumsal/faaliyet/201 9.pdf Singh, A., & Prasher, A. (2019). Measuring healthcare service quality from patients’ perspective: using Fuzzy AHP application. Total Quality Management and Business Excellence, 30(3–4), 284–300. https://doi.org/10.1080/14783363.2017.1302794 SITA. (2019). Air Transport IT Insights 2019. Surveys & Reports. https://www.sita.aero/resources/type/surveys-reports/air-transport-it-insights- 2019 Stanković, Milan, Gladović, P., & Popović, V. (2019). Determining the importance of the criteria of traffic accessibility using fuzzy AHP and rough AHP method. Decision Making: Applications in Management and Engineering, 2(1), 86–104. https://doi.org/10.31181/dmame1901086s Stanković, Miomir, Stević, Ž., Das, D. K., Subotić, M., & Pamucar, D. (2020). A New Fuzzy MARCOS Method for Road Traffic Risk Analysis. Mathematics, 8(3), 1–18. https://doi.org/10.3390/math8030457 Stević, Ž., & Brković, N. (2020). A Novel Integrated FUCOM MARCOS Model for Evaluation of Human Resources in a Transport Company. Logistics, 4(1), 1–14. https://doi.org/10.3390/logistics4010004 Stević, Ž., Pamucar, D., Puška, A., & Chatterjee, P. (2020). Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to COmpromise solution (MARCOS). Computers and Industrial Engineering, 140. https://doi.org/10.1016/j.cie.2019.106231 Tarkang, M. M. E. M., Alola, U. V., Nange, R. Y., & Ozturen, A. (2020). Investigating the factors that trigger airline industry purchase intention. Current Psychology. https://doi.org/10.1007/s12144-020-00815-z Tsai, W. H., Chou, W. C., & Leu, J. Der. (2011). An effectiveness evaluation model for the web-based marketing of the airline industry. Expert Systems with Applications, 38(12), 15499–15516. https://doi.org/10.1016/j.eswa.2011.06.009 Ustasüleyman, T. (2013). Bankacılık Sektöründe İnternet Sitesi Kalitesi Boyutlarının (Kriterlerinin) Önem Derecesinin Belirlenmesi. Manisa Celal Bayar Üniversitesi Sosyal Bilimler Dergisi, 11(1), 146–162. Bakır and Atalık/Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 127-152 152 Vatansever, K., & Akgül, Y. (2018). Performance evaluation of websites using entropy and grey relational analysis methods: The case of airline companies. Decision Science Letters, 7(2), 119–130. https://doi.org/10.5267/j.dsl.2017.6.005 Vuthisopon, S., & Srinuan, C. (2017). Low-cost carrier passenger repurchase intention: A structural equation model analysis. Asia-Pacific Social Science Review, 17(2), 249– 266. Williams, G. (2011). Comparing the Economic and Operating Characteristics of Charter and Low-cost Scheduled Airlines. In J. F. O’Connell & G. Williams (Eds.), Air Transport in the 21st Century (pp. 185–195). Ashgate Publishing. Wolfinbarger, M., & Gilly, M. C. (2003). eTailQ: Dimensionalizing, measuring and predicting etail quality. Journal of Retailing, 79(3), 183–198. https://doi.org/10.1016/S0022-4359(03)00034-4 Yazdani, M., Chatterjee, P., Pamucar, D., & Abad, M. D. (2019). A risk-based integrated decision-making model for green supplier selection. Kybernetes, 49(4), 1229–1252. https://doi.org/10.1108/K-09-2018-0509 Yoo, B., & Donthu, N. (2001). Developing a scale to measure the perceived quality of an Internet shopping site (SITEQUAL). Quarterly Journal of Electronic Commerce, 2(1), 31–45. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. https://doi.org/10.1016/S0019-9958(65)90241-X © 2018 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).