Decision Making: Applications in Management and Engineering Vol. 4, Issue 2, 2021, pp. 106-125. ISSN: 2560-6018 eISSN: 2620-0104 DOI: https://doi.org/10.31181/dmame210402106a * Corresponding author. E-mail address: taliparsu@aksaray.edu.tr (T. Arsu) INVESTIGATION INTO THE EFFICIENCIES OF EUROPEAN FOOTBALL CLUBS WITH BI-OBJECTIVE MULTI-CRITERIA DATA ENVELOPMENT ANALYSIS Talip Arsu 1* 1 Vocational School of Social Sciences, University of Aksaray, Turkey Received: 15 March 2021; Accepted: 24 May 2021; Available online: 12 June 2021. Original scientific paper Abstract: A financially successful football club can achieve sporting achievements as well as become financially stable. In other words, the success of football clubs depends on both financial and sportive success. Contrary to the studies in the literature that focus on financial and sportive success separately, the present study aimed to examine the 5-season activities of 10 football clubs in the big-five league, which are the top leagues of Europe, by using financial and sports criteria. Bi-objective multi criteria data envelopment analysis (BiO-MCDEA) was used for the efficiency analysis. In the study, the number of social media followers, the average number of viewers and total market value were used as input, and the UEFA club score and total revenues were used as output. As a result, Arsenal, Paris Saint-Germain, and Juventus were determined as efficient in the 2015-2016 season, Paris Saint- Germain and Liverpool in the 2016-2017 season, Manchester United, Paris Saint-Germain and Chelsea in the 2016-2017 season, Manchester United, Real Madrid, Bayern Munich and Arsenal in the 2018-2019 season, Manchester United, Paris Saint-Germain and Chelsea. The reasons why PSG was the most successful club in the efficiency analysis (efficient in four out of five seasons) were examined. In addition, in the sensitivity analysis conducted to determine the effect of inputs and outputs on the model, it was concluded that efficiency was highly related to financial data. Keywords: European football clubs, efficiency, multi-criteria data envelopment analysis, bi-objective multi-criteria data envelopment analysis mailto:taliparsu@aksaray.edu.tr Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 107 1. Introduction Football is the most popular sport in the world. Although there are many factors that underlie this popularity, the simplicity of the rules and the low cost can be considered as the most important factors (Galariotis et al., 2018). However, in professional football, which has undergone a great transformation since the early 1990s, footballer salaries have started to increase exponentially (Dobson & Goddard, 2011). The Bosman ruling introduced by the European Court of Justice in 1995 had a significant impact on the future of European football. The Bosman ruling was named after the Belgian midfielder Jean-Marc Bosman's lawsuit that was filed for blocking his transfer from Belgium to France at the end of his contract. The Bosman ruling included the liberalization of the immigration of professional athletes within the EU and the abolition of transfer fees after the expiry of contracts. In addition, restrictions on the number of EU players that clubs can have playing on the field were also considered illegal according to Bosman ruling (Marcén, 2019). After the Bosman ruling was recognized by UEFA in March 1996, the transfers of football players between teams began to be carried out at astronomical figures. In addition, the fact that broadcasting contracts yielded an unimaginable scale of revenue just a few years ago, the complete reconstruction of many football fields, and the immeasurable increase in the importance of commercial sponsorship and merchandising increased the importance of football's financial infrastructure (Dobson & Goddard, 2011). Football clubs are no longer organizations that only provide emotional and symbolic satisfaction to their supporters and focus on sporting success without profit. Instead, football clubs have become a complex system in which investors invest capital and expect financial returns (Miragaia et al., 2019). This development in professional football has turned football from being not only a sport branch in Europe but also an industry branch. The revenues and brand values of football clubs have become competitive with many industries and brands. Spain (La Liga), England (Premier League), Italy (Serie A), Germany (Bundesliga) and France (Ligue 1), which are called the “big five league”, constitute a large part of the world football industry. The big five league, which ha s gained great value in the last 20 years, increased its total value from EUR 2.95 billion in 1998 to EUR 26.8 billion in 2021 (transfermarkt.com, 2021). However, these financial values are not governed by all the clubs in the big five league, but only the top 10 clubs in Europe in terms of both sporting success and financial standing. Manchester United, Real Madrid, FC Barcelona, Bayern Munich, Manchester City, Arsenal, Paris Saint-Germain (PSG), Chelsea, Liverpool and Juventus have a value of EUR 7.96 billion, which is almost one third of all other clubs total value of the big five league (transfermarkt.com, 2021). The growth of the football industry at this scale in as little as 20 years has brought along both control and financial difficulties. Although football clubs have many financial resources, these resources are largely related to sporting success. In other words, football clubs must be continuously successful in order to avoid experiencing financial difficulties, which is not possible. As the financial difficulties experienced by football clubs are beginning to become continuous, UEFA has brought some restrictions on clubs with a regulation called Financial Fair Play (FFP). FFP, which entered into force in 2009 and is updated every three years, basically aims to improve the economic and financial capabilities of the clubs and increase both their transparency and reliability. At the same time, thanks to the FFP, which aims to bring more discipline and rationality to club football financing, the ratio of net debts of clubs to their income has decreased from 65% to 35% in a short period of time (UEFA, Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 108 2021). In order to achieve this financial success, UEFA has imposed many restrictions on clubs and imposed severe penalties such as the deletion of points, transfer restrictions and ban from tournaments, for those who do not comply with these restrictions. Football clubs, which are suppressed by UEFA, are also trying to meet the sportive success expectations of the stakeholders. It does not seem realistic to evaluate these two processes independently from each other in football clubs where financial success supports sportive success. Some studies in the literature have carried out financial evaluations by only considering the financial data of clubs (Pradhan et al., 2016; Chelmis et al., 2019), some have focused only on sportive success (Rossi et al., 2019; Salabun et al.2020) and others have tried to associate financial success with sportive success (Sakınç et al., 2017; Galariotis et al., 2018). However, the success of football clubs is possible with the realization of the financial and sportive success together in this cycle. The aim of this study, which was designed with the motivation of the idea of realizing financial and sportive success together and the lack in the literature, was to investigate the efficiency values for three seasons of 10 football clubs that are at the top in terms of both sport and finance in Europe. When conducting the effectiveness analysis, the bi-objective multiple criteria data envelopment analysis (BiO-MCDEA) method, recommended by Ghasemi (2014) to eliminate the low discrimination problem of the classical data envelopment analysis (DEA), was used. The paper begins with a detailed literature review in Section 2. In the Section 3, firstly, the classical DEA and multiple criteria data envelopment analysis (MCDEA) methods that form the basis of the BiO-MCDEA model are introduced and the BiO- MCDEA model is shown. In the data title at the end of the Section 3, how the criteria used in this study were determined, the source of the data used as criteria and the criteria values are shown. In Section 4, the findings of the study are presented and in Section 5, the findings are discussed. In Section 6, the sensitivity analysis is given to determine the contribution of each criterion to the model. In the last section, the conclusions, advantages and limitations of the study and managerial implications are given. 2. Literature Review The popularity of football around the world and the huge budgets managed by football clubs have made the football industry the subject of many academic studies conducted to examine the sportive or financial performances of national and international leagues, clubs and even players. In many of these studies, MCDM methods have been used for performance evaluation. Pradhan et al. (2016) investigated the financial performance of Italian clubs using gray relation analysis (GRA), Galariotis et al. (2018) determined the business, financial and sports performance of clubs in the French league using the PROMETHEE II method, Sakınç et al. (2017) studied the financial and sporting performance of 22 European clubs using the TOPSIS method, Chelmis et al. (2019) investigated the financial, commercial and sporting performance of clubs in the Greek league using PROMETHEE II and Salabun et al. (2020) determined the performance of football players using the characteristic objects method (COMET) and TOPSIS method. In addition to these methods used, the most used MCDM method is DEA which was developed by Charnes et al. (1978). In recent years DEA has been used in many decision problems such as the effectiveness of agricultural practices (Angulo-Meza et al., 2019), financial performance assessment (Anthony et al., 2019), hospital efficiency assessments (Kohl et al., 2019), sustainability assessment of the water sector (Lombardi et al., 2019), bank activities Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 109 assessments (Kamarudin et al., 2019), sustainable supplier selection (Rashidi & Cullinane, 2019), efficiency assessment of railway enterprises (Blagojević et al., 2020), assessment of medium and large-sized industries in the diversity sector (Hassanpour, 2020). DEA studies in the literature generally consist of efficiency analyses conducted for all teams in the league of a specific country. DEA was used to determine the efficiencies of the teams in England’s Premier league (Pestana Barros & Leach, 2006; Guzman & Morrow, 2007; Haas, 2003a; Kern et al., 2012), Germany’s Bundesliga (Haas et al., 2004), France’s Ligue 1 (Jardin, 2009), USA’s Major League Soccer (MLS) (Haas, 2003b), Italian Serie A (Rossi et al., 2019), and Brazil’s Serie A (Pestana Barros et al., 2010). In addition, the efficiencies of European clubs (Halkos & Tzeremes, 2013; Miragaia et al., 2019) and national teams participating in EURO 2012 (Rubem & Brandao, 2015) were determined using DEA. However, no study examining the 5- season efficiency values of 10 top European clubs which make up half of the total value of the big five league were found in the literature. Furthermore, classical DEA was used in almost all efficiency studies in the literature. Although classical DEA is a widely used nonparametric efficiency instrument, it has the disadvantage of low discrimination power. In order to avoid this disadvantage, the BiO-MCDEA model, which was developed by Ghasemi et al. (2014) and has been used in decision problems such as the equipment efficiency assessment for automotive industry (da Silva et al., 2017), port efficiency assessment (de Andrade et al., 2019), electrical distribution units efficiency assessment (Ghofran et al., 2021), was used in this study. 3. Material and Methods BiO-MCDEA is a goal programming based efficiency determination model developed by Ghasemi et al. (2014) in which the DEA model aims to improve the discrimination power. Bal et al. (2010) proposed the goal programming data envelopment analysis (GPDEA) model which is based on goal programming that would eliminate the problem of discrimination power and weight distribution of the DEA model. The GPDEA model is based on solving unwanted deviations using equal weight. BiO-MCDEA was used in the present study to exclude classical DEA and thus avoid the disadvantage of low discrimination power and because its solution steps are easier. 3.1. Multiple Criteria Data Envelopment Analysis (MCDEA) Classical DEA is a widely used non-parametric analysis for efficiency analysis, used especially in social sciences. The conversion of the classical DEA method into a linear programming form proposed by Charnes et al. (1978) is shown below. 𝑀𝑎𝑥 ℎ0 = ∑ 𝑢𝑟 𝑦𝑟𝑗 𝑠 𝑟=1 𝑠. 𝑡. ∑ 𝑣𝑖 𝑥𝑖𝑗 = 1 (1) 𝑚 𝑖=1 Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 110 ∑ 𝑢𝑟 𝑦𝑟𝑗 𝑠 𝑟=1 − ∑ 𝑣𝑖 𝑥𝑖𝑗 ≤ 0 𝑚 𝑖=1 , 𝑗 = 1, … , 𝑛 𝑢𝑟 ≥ 0 𝑣𝑖 ≥ 0 Where; j is the number of decision-making units (DMU), r is the number of outputs, i is the number of inputs, 𝑦𝑟𝑗 is the value of the rth output for the jth DMU, 𝑥𝑖𝑗 is the value of the ith input for the jth DMU, 𝑢𝑟 is the weight of the rth output, 𝑣𝑖 is the weight of the ith input and ℎ0 refers to relative efficiency. In this model, any DMU must be ℎ0 = 1 in order to be effective (Charnes et al., 1978; Despic et al., 2019) . Although classical DEA is an efficiency measurement method, Li & Reeves’ (1999) MCDEA model is based on ineffectiveness. 𝑑0, which is limited to the [0, 1] range can be considered a measurement of “ineffectiveness” and is defined as ℎ0 = 1 − 𝑑0. Therefore the smaller the 𝑑0 value, the less ineffective (and therefore more effective) DMU is. In the method of Li & Reeves (1999), besides the minimization of d0, which is the measure of ineffectiveness, there are two independent objective functions, namely, minimizing maximum deviation and minimizing the sum of deviations. Their model is as follows: 𝑀𝑖𝑛 𝑑0 (𝑜𝑟 max ℎ0 = ∑ 𝑢𝑟 𝑦𝑟𝑗0) 𝑠 𝑟=1 𝑀𝑖𝑛 𝑀 𝑀𝑖𝑛 ∑ 𝑑𝑗 (2) 𝑛 𝑗=1 𝑠. 𝑡. ∑ 𝑣𝑖 𝑥𝑖𝑗0 = 1 𝑚 𝑖=1 ∑ 𝑢𝑟 𝑦𝑟𝑗 − ∑ 𝑣𝑖 𝑥𝑖𝑗 + 𝑑𝑗 = 0 𝑚 𝑖=1 𝑠 𝑟=1 𝑀 − 𝑑𝑗 ≥ 0 𝑗 = 1, … . . , 𝑛 𝑢𝑟 , 𝑣𝑖 , 𝑑𝑗 ≥ 0 The MCDEA model was proposed primarily as a tool for improving the discrimination power of the classical DEA model. In the solution procedure, MCDEA was proposed as an interactive approach to solve three objectives. The first objective accommodates the classical RIA solution within a set of MCDEA solutions. The other two objectives, Minimax and Minsum, provide more restrictive or lax efficiency solutions, respectively. This model proves that a wider solution is possible to achieve more reasonable input and output weights (Ghasemi et al., 2014). 3.2. A Bi-Objective Multiple Criteria Data Envelopment Analysis (BiO-MCDEA) The MCDEA model consists of three independent objective functions: 𝑀𝑖𝑛 𝑑0 , 𝑀𝑖𝑛 𝑀 and 𝑀𝑖𝑛 ∑ 𝑑𝑗𝑗 as defined in Model 2. In a weighted model, these three independent objective functions can be weighted as 𝑤1𝑑0 + 𝑤2𝑀 + 𝑤3 ∑ 𝑑𝑗𝑗 into a single-objective problem. Different efficiency scores can be achieved by changing the Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 111 weights 𝑤𝑖 (𝑖 = 1,2,3). However, since the first objective function (𝑀𝑖𝑛 𝑑0 ) has the same meaning as the classical DEA model, it can be removed from the MCDEA model as the discrimination power of the second (𝑀𝑖𝑛 𝑀) and third (𝑀𝑖𝑛 ∑ 𝑑𝑗𝑗 ) objective functions has been proved to be higher than the 𝑀𝑖𝑛 𝑑0 objective (Li & Reeves, 1999; San Cristobal, 2011; Hatami-Marbini & Toloo, 2017). Therefore, only the 𝑀𝑖𝑛 𝑀 and 𝑀𝑖𝑛 ∑ 𝑑𝑗𝑗 objectives are weighted in the BiO-MCDEA model, which is shown below: 𝑀𝑖𝑛 ℎ = (𝑤2𝑀 + 𝑤3 ∑ 𝑑𝑗 𝑗 ) 𝑠. 𝑡. ∑ 𝑣𝑖 𝑥𝑖𝑗0 = 1 𝑚 𝑖=1 (3) ∑ 𝑢𝑟 𝑦𝑟𝑗 − ∑ 𝑣𝑖 𝑥𝑖𝑗 + 𝑑𝑗 = 0 𝑚 𝑖=1 𝑠 𝑟=1 𝑀 − 𝑑𝑗 ≥ 0 𝑗 = 1, … . . , 𝑛 𝑢𝑟 ≥ 𝜀, 𝑟 = 1,2, … , 𝑠 𝑣𝑖 ≥ 𝜀, 𝑖 = 1,2, … , 𝑚 𝑑𝑗 ≥ 0 𝑗 = 1,2, … , 𝑛 The constraints of the BiO-MCDEA model consist of the same constraints as the MCDEA model of Li & Reeves (1999). Only the 𝑢𝑟 and 𝑣𝑖 variables are constrained by the constant 𝜀. Although Ghasemi et al. (2014) used 𝜀 = 0,0001 in the samples they solved, they did not suggest an approach to find a suitable value for the constant 𝜀. In addition, the BiO-MCDEA model is still robust if 𝜀 = 0 in the sample solved using a different data set. In this study, 𝜀 = 0 was used as in the original model. 3.3. Data The data of this study was obtained from the 2020, 2019, 2018, 2017 and 2016 Deloitte Football Money League reports and transfermrkt.com, which regularly collects data on the European football industry every year. In the study, the number of social media followers (v1), average number of viewers (v2) and total market value (v3) were used as input, while the UEFA club score (u1) and total revenue (u2) variables were used as output. In their studies Aichner (2018), Alaminos et al. (2020) and Weimar et al. (2021) used number of social media followers, Haas (2003a), Pestana Barros et al. (2010), Kern et al. (2012), Alaminos et al. (2020) used average number of viewers, Kulikova & Goshunova (2014) and Rubem & Brandao (2015) used total market value, Rubem & Brandao (2015) used UEFA club score, Halkos & Tzeremes (2013), Kulikova & Goshunova (2014), Jardin (2009), Guzman & Morrow (2007), Pestana Barros et al. (2010), Kern et al. (2012), Chelmis et al. (2019) and Miragaia et al. (2019) used the total revenue of the club as input or output variable. The definitions of the input and output variables are shown in Table 1. Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 112 Table 1. BiO-MCDEA model input and output variable definitions Variables Definition The number of social media followers (𝑣1) The number of people following the clubs on facebook, instagram and twitter (*106) Average number of viewers (𝑣2) The average number of people who came to the stadium as a spectator in matches hosted by clubs Total market value(𝑣3) The sum of the market values of the club's footballers (*106 €) UEFA club score (𝑢1) The total points the club has obtained from all matches during a season Total revenue (𝑢2) The sum of club's matchday revenues, broadcasting revenues and commercial revenues (*106 €) Pearson correlation coefficients are widely used when choosing input and output in DEA (Lewin et al., 1982; Thanassoulis et al., 1987; Golany & Roll, 1989; Friedman & Sinuany-Stern, 1998; Dyson et al., 2001). Lewin et al. (1982) argued that inputs should not be highly correlated with other inputs and outputs should not be highly correlated with other outputs. They also stated that if the inputs and outputs are negatively correlated with each other, these variables may be excluded from the model since the increase in inputs will affect the output negatively. The Pearson correlation coefficients of the data used in the present study are shown in Table 2. Table 2. Pearson correlation coefficients for BiO-MCDEA model inputs and outputs v1 v2 v3 u1 u2 v1 1 v2 0.578 1 v3 0.448 0.136 1 u1 0.179 0.209 0.219 1 u2 0.781 0.712 0.534 0.205 1 According to the results given in Table 2, none of the Pearson correlation coefficients had a very high, very low or negative value. Therefore, no input or output variable was excluded from the model. In this study, an analysis of the efficiency of five seasons of 10 top European football clubs was performed. The values of input and output variables selected to determine the effectiveness of the 2015-2016, 2016-2017, 2017-2018, 2018-2019 and 2019- 2020 seasons are shown in Table 3. Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 113 Table 3. Input and output values of the BiO-MCDEA model 2019-2020 Season Football Clubs v1 v2 v3 u1 u2 Manchester United 127.2 74698 670.45 22000 711.5 Real Madrid 226.7 61040 913.75 17000 757.3 FC Barcelona 216.5 76104 930.93 24000 840.8 Bayern Munich 74.4 75865 777.33 36000 660.1 Manchester City 62.9 54130 1048.6 25000 610.6 Arsenal 69.7 59897 607.65 10000 445.6 PSG 73.7 46911 874.15 31000 635.9 Chelsea 82.2 40445 705.85 17000 513.1 Liverpool 71.9 53053 1002.7 18000 604.7 Juventus 83.4 39101 661.88 22000 459.7 2018-2019 Season Manchester United 117.2 75102 797.6 19000 666 Real Madrid 207.8 66337 1033.1 19000 750.9 FC Barcelona 195.5 70872 1201.4 30000 690.4 Bayern Munich 68.9 75354 784.88 20000 629.2 Manchester City 53.1 54054 1203.4 25000 568.4 Arsenal 64.7 59323 659.05 26000 439.2 PSG 60.4 46929 1009.9 19000 541.7 Chelsea 74.4 41281 1166.6 30000 505.7 Liverpool 54.8 52958 1172.4 29000 513.7 Juventus 63.1 36510 871.05 21000 394.9 2017-2018 Season Manchester United 110.2 75305 645.10 28985 676.3 Real Madrid 189.7 69426 716.2 37028 674.6 FC Barcelona 184.3 78678 772.5 27028 648.3 Bayern Munich 59.5 75000 610.25 24914 587.8 Manchester City 41 54019 616.35 20985 527.7 Arsenal 61.2 59957 633.90 21985 487.6 PSG 49.9 45160 581.10 22883 486.2 Chelsea 69.9 41532 642.15 2985 428 Liverpool 45.3 53094 495 2985 424.2 Juventus 45.2 37195 540.53 35850 405.7 2016-2017 Season Manchester United 97.4 75327 533.25 15850 689 Real Madrid 158.4 71280 743.1 37785 620.1 FC Barcelona 159.1 79724 787.2 30785 620.2 Bayern Munich 52.3 75017 595.4 32285 592 Manchester City 30.7 54013 621.4 28850 524.9 Arsenal 55 59980 522.75 17850 468.5 PSG 37.5 46160 502.05 26216 520.9 Chelsea 63.3 41500 603.3 20850 447.4 Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 114 Liverpool 39.8 44108 394.15 24850 403.8 Juventus 34.6 39106 463.78 20300 341.1 2015-2016 Season Manchester United 83.1 75335 374.15 2714 519.5 Real Madrid 128.9 72969 700.75 33042 577 FC Barcelona 132.8 77632 618.5 38042 560.8 Bayern Munich 41.5 72882 608.5 31171 474 Manchester City 25.3 45345 452.75 17714 463.5 Arsenal 46.4 59992 408.6 22714 435.5 PSG 28.9 45789 433.3 23183 480.8 Chelsea 56.1 41546 579.8 23714 420 Liverpool 34.5 44675 325 12714 391.8 Juventus 26.3 36292 394.33 32800 323.9 The reason why the 10 clubs were included in the efficiency evaluation is that these 10 clubs were ranked in the top 10 for five seasons in the Deloitte Football Money League report, which was the main data source of this study. The Deloitte Football Money League report publishes data for the 20 top financially successful clubs each season. However, 10 clubs other than the top 10 change at a certain rate each year. As data of some of the clubs other than the top 10 clubs from different sources could harm the homogeneity of the data, the clubs not included in the top 10 clubs were excluded from the scope of the study. 4. Results The MCDEA and BiO-MCDEA efficiency scores of the football clubs were calculated using LINDO w32 software. The first three columns in Table 4 are the efficiency results of the MCDEA model solution. The fourth column consists of efficiency values obtained as a result of the BiO-MCDEA model solution. The last column refers to the ranking of the football clubs according to the results of the efficiency values obtained with the BiO-MCDEA model solution. The efficient football clubs (eff. 1) were ranked first, while the other clubs were ranked in order after that. Table 4. BiO-MCDEA model efficiency scores. Football Clubs Classical DEA/Min d0 Min M Min ∑d BiO- MCDEA Rank 2 0 1 9 -2 0 2 0 S e a so n Manchester United 1 1 1 1 1 Real Madrid 1 0.902 0.852 0.890 5 FC Barcelona 1 0.911 0.886 0.917 2 Bayern Munich 0.988 0.890 0.834 0.834 7 Manchester City 0.996 0.878 0.859 0.859 6 Arsenal 0.826 0.843 0.800 0.800 8 PSG 1 1 1 1 1 Chelsea 1 1 1 1 1 Liverpool 0.924 0.929 0.908 0.908 3 Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 115 Juventus 0.928 0.852 0.891 0.891 4 2 0 1 8 -2 0 1 9 S e a so n Manchester United 1 1 1 1 1 Real Madrid 1 1 1 1 1 FC Barcelona 0.965 0.940 0.931 0.941 5 Bayern Munich 1 1 1 1 1 Manchester City 1 0.943 0.922 0.922 7 Arsenal 1 0.996 0.994 1 1 PSG 1 0.977 0.995 0.981 3 Chelsea 1 1 0.986 0.986 2 Liverpool 0.996 0.931 0.934 0.934 6 Juventus 0.953 0.943 0.953 0.946 4 2 0 1 7 -2 0 1 8 S e a so n Manchester United 1 1 1 1 1 Real Madrid 1 0.965 0.972 0.965 4 FC Barcelona 0.866 0.863 0.866 0.863 6 Bayern Munich 1 0.883 0.896 0.896 5 Manchester City 1 0.979 0.990 0.990 2 Arsenal 0.849 0.835 0.827 0.827 8 PSG 1 1 1 1 1 Chelsea 1 0.864 1 1 1 Liverpool 1 0.979 0.969 0.969 3 Juventus 1 0.946 0.828 0.828 7 2 0 1 6 -2 0 1 7 S e a so n Manchester United 1 0.810 0.992 0.992 2 Real Madrid 0.944 0.844 0.844 0.844 4 FC Barcelona 0.731 0.731 0.731 0.731 9 Bayern Munich 1 0.922 0.922 0.922 3 Manchester City 1 0.837 0.837 0.837 5 Arsenal 0.825 0.783 0.783 0.783 6 PSG 1 1 1 1 1 Chelsea 0.955 0.744 0.744 0.744 7 Liverpool 1 1 1 1 1 Juventus 0.913 0.738 0.738 0.738 8 2 0 1 5 -2 0 1 6 S e a so n Manchester United 1 0.671 0.562 0.662 8 Real Madrid 0.798 0.791 0.798 0.798 5 FC Barcelona 0.936 0.903 0.936 0.936 3 Bayern Munich 0.789 0.766 0.788 0.788 6 Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 116 Manchester City 1 0.871 0.852 0.852 4 Arsenal 1 0.909 1 1 1 PSG 1 1 1 1 1 Chelsea 1 0.726 0.673 0.673 7 Liverpool 1 0.934 0.983 0.983 2 Juventus 1 1 1 1 1 It can be seen from Table 4 that PSG is the only club that was efficient for all three seasons. However, the ranking of other clubs according to the BiO-MCDEA model differed for each season. For instance, Manchester United ranked first in the 2017- 2018 season, second in the 2016-2017 season and eighth in the 2015-2016 season. This shows that the financial and sporting success of the clubs affects their rankings in different seasons. Spearman rank correlation was commonly used in the literature to test the relationship between DMU rankings (Haas et al., 2004; Bal et al., 2010; Örkcü & Bal, 2011). In this study, the relationship between the ranks determined for three different seasons as a result of the BiO-MCDEA model was tested with spearman rank correlation. When the results of the Spearman rank correlation were examined, no statistically significant relationship was found between the rankings for the three seasons. This result supports the idea that the financial and sporting achievements of the clubs affect their rankings in different seasons. In other words, the clubs achieved a ranking according to how successful they were in sports or financial terms. Although the model was created for three consecutive seasons, the rankings differed greatly. The Spearman correlation values are shown in Table 5. Table 5. BiO- MCDEA efficiency ranking spearman rank correlations values Seasons 2015-2016 2016-2017 2017-2018 2018-2019 2019-2020 2015-2016 1.000 2016-2017 0.529 1.000 2017-2018 -0.503 -0.080 1.000 2018-2019 -0.354 0.132 -0.076 1.000 2019-2020 -0.242 0.160 0.714 -0.146 1.000 As can be seen from the figure, FC Barcelona ranked third in 2015-2016, ninth in 2016-2017 and sixth in 2017-2018. FC Barcelona was the second club with the highest total market value among the clubs included in the analysis of the 2015-2016 season. It was also the second club with the highest total revenue in the same season. This was reflected in their sporting success as they reached the highest UEFA score among the clubs involved in the analysis. Total revenue, total market value and UEFA club points placed FC Barcelona in third place in the BiO-MCDEA model. However, although FC Barcelona seemed to be the most valuable club in terms of total market value in the 2016-2017 and 2017-2018 seasons, it was observed that the quality of the footballers was not sufficient to increase their UEFA club points and total revenue. Due to this result, FC Barcelona ranked lower in the 2016-2017 and 2017-2018 seasons according to the BiO-MCDEA model. Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 117 5. Sensitivity Analysis These remarkable results raise the question of how much input and output variables contribute to the model when determining the BiO-MCDEA model ranking. Therefore, a sensitivity analysis was performed to determine which input or output contributed to the model. To determine the contribution of each input and output variable, BiO-MCDEA efficiency values including all the variables and BiO-MCDEA efficiency values calculated by excluding each input and output variable were examined. In addition, Pearson correlation coefficients were examined to determine the relationship between the efficiency values of the model including all input and output variables and the efficiency values when each variable was excluded from the model. The sensitivity analysis results and Pearson correlation coefficients are shown in Table 6. Table 6. Sensitivity analysis results for BiO-MCDEA model variables Football Clubs BiO- MCDEA Without v1 (r1= 0.943*) Without v2 (r2= 0.334) Without v3 (r3= 0.302) Without u1 (r4= 0.760*) Without u2 (r5= 0.220) 2 0 1 9 -2 0 2 0 S e a so n Manchester Uni 1 1 1 0.795 1 0.660 Real Madrid 0.890 1 0.687 0.880 0.864 0.476 FC Barcelona 0.917 1 0.757 0.821 0.996 0.600 Bayern Munich 0.834 0.893 1 0.621 0.877 1 Manchester City 0.859 0.804 0.758 0.931 0.818 0.675 Arsenal 0.800 0.725 0.822 0.702 0.750 0.350 PSG 1 0.990 0.883 1 1 0.992 Chelsea 1 0.954 0.811 1 0.965 0.663 Liverpool 0.908 0.809 0.758 0.998 0.836 0.504 Juventus 0.891 0.918 0.757 0.809 0.908 0.902 2 0 1 8 -2 0 1 9 S e a so n Manchester Uni 1 0.968 0.839 0.774 1 0.589 Real Madrid 1 1 0.454 0.839 1 0.361 FC Barcelona 0.941 0.965 0.511 0.730 0.854 0.780 Bayern Munich 1 0.945 1 0.723 1 0.620 Manchester City 0.922 0.888 0.654 0.887 0.913 0.750 Arsenal 1 1 1 0.616 0.855 0.988 PSG 0.981 0.925 0.684 1 1 0.665 Chelsea 0.986 1 0.654 0.997 0.895 0.990 Liverpool 0.934 0.893 0.653 0.794 0.842 0.870 Juventus 0.946 0.937 0.663 0.887 0.867 0.882 2 0 1 7 -2 0 1 8 Manchester Uni 1 1 0.996 0.849 1 0.522 Real Madrid 0.965 0.972 0.781 0.861 0.998 0.259 FC Barcelona 0.863 0.866 0.742 0.779 0.867 0.189 Bayern Munich 0.896 0.896 0.990 0.741 0.891 0.349 Manchester City 0.990 0.990 0.925 0.913 0.961 0.504 Arsenal 0.827 0.827 0.801 0.751 0.829 0.528 PSG 1 1 0.866 1 1 0.595 Chelsea 1 1 0.763 1 0.870 0.069 Liverpool 0.969 0.969 1 0.775 0.860 0.090 Juventus 0.828 0.828 0.687 0.959 0.952 1 2 0 1 6 -2 0 1 7 Manchester Uni 0.992 0.993 1 0.810 1 0.380 Real Madrid 0.844 0.844 0.719 0.771 0.789 0.680 FC Barcelona 0.731 0.731 0.664 0.678 0.726 0.677 Bayern Munich 0.922 0.923 0.940 0.699 0.819 0.775 Manchester City 0.837 0.838 0.782 0.861 0.835 0.750 Arsenal 0.783 0.784 0.764 0.692 0.776 0.536 PSG 1 1 0.940 1 1 0.996 Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 118 Chelsea 0.744 0.744 0.672 0.955 0.808 0.375 Liverpool 1 1 0.992 0.811 0.898 0.994 Juventus 0.738 0.739 0.704 0.773 0.737 0.711 2 0 1 5 -2 0 1 6 Manchester Uni 0.662 0.662 0.895 0.639 0.657 0.041 Real Madrid 0.798 0.798 0.803 0.763 0.745 0.628 FC Barcelona 0.936 0.936 0.937 0.705 0.775 0.761 Bayern Munich 0.788 0.788 0.790 0.631 0.676 0.623 Manchester City 0.852 0.852 0.859 0.961 0.937 0.478 Arsenal 1 1 0.995 0.693 0.864 0.704 PSG 1 1 1 0.992 1 0.649 Chelsea 0.673 0.673 0.682 0.957 0.719 0.427 Liverpool 0.983 0.983 0.969 0.822 1 0.470 Juventus 1 1 0.996 0.889 0.769 0.984 When the results of the sensitivity analysis were examined, a significant correlation was observed between the BiO-MCDEA efficiency scores, which included all inputs and outputs, and the BiO-MCDEA efficiency scores, where two inputs and one output were excluded from the model. In particular, when the number of social media followers (v1) input variable was excluded from the model, an excellent correlation (r1=0.943) was observed between the obtained efficiency values and the activity values in which all variables were included in the model. That is to say, this variable did not contribute to the model. In the same way, a statistically significant and strong relationship (r4=0.760) was observed between the efficiency values obtained by excluding the UEFA club score output (u1), and the BiO-MCDEA model in which all variables were included. It was found that these variables did not contribute to the model. The biggest contribution to the model was average number of viewers (v2) and the total market value (v3) inputs and the total revenue (u2) output. In other words, mainly the financial variables influenced the ranking of the BiO-MCDEA model of the clubs. 6. Discussion In the analysis made using the data of the 2015-2016, 2016-2017, 2017-2018, 2018-2019 and 2019-2020 seasons, a comprehensive assessment was made by using the number of social media followers, average number of viewers, total market values, UEFA club scores and total revenues. When the MCDEA model Min d0, which gives the same results with classical output-oriented DEA efficiency values, and BiO-MCDEA model efficiency values were compared, it was concluded that the BiO-MCDEA model improved the discrimination power. This was because while seven clubs in the 2015- 2016 season, five clubs in the 2016-2017 season, eight clubs in the 2017-2018 season, seven clubs in the 2018-2019 season and five clubs in the 2019-2020 season were efficient according to the classical DEA model, only three clubs in the 2015-2016 season, two clubs in the 2016-2017 season, three clubs in the 2017-2018 season, four clubs in the 2018-2019 season and three clubs in the 2019-2020 season were efficient according to the BiO-MCDEA model. According to the results, Arsenal, PSG and Juventus emerged as the efficient clubs in the 2015-2016 season, PSG and Liverpool emerged as the efficient clubs in the 2016-2017 season, Manchester United, PSG and Chelsea emerged as the efficient clubs in the 2017-2018 season, Manchester United, Real Madrid, Bayern Munich and Arsenal emerged as the efficient clubs in the 2018- 2019 season and Manchester United, PSG and Chelsea emerged as the efficient clubs in the 2019-2020 season. PSG was determined as an efficient club in four out of five seasons included in the analysis. In other words, PSG was the most successful club among the analyzed clubs. Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 119 This may be attributed to the sale of the club to a Qatari fund group in 2011. Correspondingly, the market value of the club increased with the large expenditures made for transfers immediately after the sale. With this acceleration, the club which had only won two championships in the France Ligue 1 since 1970, became the champion seven times in eight seasons after the 2012 season. The club, which became the champion almost every season after the 2012 season, increased its UEFA club points by joining the Champions League every year and achieved a financially stable structure. In other words, the club, which was financially supported after the 2011 season, increased its sporting achievements, which in turn stabilized its financial support. Among the 10 clubs included in the analysis, Manchester City, which was purchased by a funding organization like in the case of PSG, was not as efficient as PSG according to the BiO-MCDEA model. In the 2008 season when Manchester City was acquired, it was financially supported, similar to PSG. However, Manchester City has not been as successful as PSG. The reason for this is that Manchester City cannot dominate the Premier League as PSG dominates Ligue 1, as 5 of the top 10 clubs at the top of the Big-Five League compete in the Premier League. This suggests that financial support alone does not have an impact on success. In this study, a sensitivity analysis was performed to measure the sensitivity of efficiency measurement results according to different input/output combinations. Each input and output was removed from the model, which was then resolved and the behavior of the model against the extracted variable was monitored. While a perfect correlation was found between the model created by subtracting the “the number of social media followers” input and the original BiO-MCDEA model, statistically significant correlations were observed between the model created by subtracting the “UEFA club scores” output from the model and the BiO-MCDEA model. This means that while the “the number of social media followers” input makes almost no contribution to the model, the “UEFA club scores” output provide relatively less contribution to the model than other inputs and outputs. No relationship was found between the original model and the BiO-MCDEA model created by excluding the “average number of viewers” and the “total market values” inputs and the “total revenues” output from the model. These variables were determined as the main determinants of the model. This suggests that the variables that contribute to the model are mainly financial variables. However, especially considering the use of social media in the 21st century, it is noteworthy that the “the number of social media followers” variable is not determinative in terms of the model. 7. Conclusion The purpose of this study was to determine the efficiencies of the top 10 clubs in the Big-Five League, which make up the largest share of the world football industry. The analysis of efficiency for only 10 clubs can be counted among the limitations of this study. The reason for the inclusion of these 10 clubs in the efficiency review was that although the rankings of the clubs have changed, they are still in the top 10. The Deloitte Football Money League report, from which most of the data in this study was obtained, publishes data on the top 20 clubs in terms of finance every year. While the clubs in the top 10 almost never change, the clubs in the last 10 can enter and exit the list. Only 10 clubs were included in the analysis to obtain consistent data over the entire five years of the analysis. Another limitation of this study was that the analysis was carried out with only quantitative data. However, this analysis could be supported Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 120 by qualitative data obtained from football professionals including club managers, sponsors, etc. In future studies, the number of football clubs included in the analysis can be increased by using more resources and time, and the obtained quantitative findings can be supported by qualitative findings. The BiO-MCDEA model, which is an efficiency determination method based on linear programming, was used in the efficiency analysis. It can be said that using this model was the most obvious advantage of the study. The reason for selecting the BiO- MCDEA model was to prevent the low discrimination problem of classical DEA. The findings of the study also included the results obtained with classical DEA. When the classical DEA findings were examined, it was concluded that a very high number of clubs were efficient. In this case, it will be difficult to distinguish between clubs. Moreover, useful information for decision-makers cannot be obtained. "Super efficiency" models can be used to determine which of the efficient clubs are more efficient. In this case, it will produce more complex results for both decision makers and analysts. In addition, the BiO-MCDEA model has easier solution steps compared to other methods such as MCDEA and GPDEA that aim to eliminate the low discrimination power problem of classical DEA. Another advantage of this study is that sports and financial data were used together. This is because financial success is to be used as leverage for sportive success. In this respect, instead of evaluating and associating financial and sports data separately, this study included both in the same model. As financial and sportive success can only be achieved through successful management practices, some managerial implications were made in line with the findings of the study. In order to examine the contribution of the criteria to the model, a sensitivity analysis was conducted in which each criterion was removed from the model, which was then solved again. According to the results of this analysis, the criteria of average audience number, total market value and total income were determined as the criteria that made the greatest contribution to the model. Although the criteria for total market value and total income are direct financial criteria, the average number of viewers seems to be a non-financial criterion. This is because the matchday revenues are at the lower ranks among the revenue items of football clubs. However, bringing fans to the stadium does not only contribute to the clubs as ticket revenue but also to the sales in commercial products and to sponsors spending more on stadium advertisements. In addition, the fact that football clubs achieve more sportive success in the home field can be explained with the support of the fans. From this point of view, club management can implement various practices to make stadiums more attractive to the fans. Among the practices that increase the attractiveness of stadiums are the club management selling tickets at lower prices, facilitating access to the stadium, and creating areas where families can spend time in the stadium. Although the 10 clubs analyzed are not in the same league, they are constantly in competition as they participate in international tournaments every year. In order to keep competition alive, the financial resource must be sustainable. In order for the financial resource to be sustainable, clubs want to continuously participate in international tournaments, which are one of the most revenue generating elements of the industry. The financial benefits of a successful season will only benefit the club in the next season. Although Real Madrid was champion in the champions league in the 2017-2018 season, according to the analysis conducted in this study, it was found to be an efficient club in the 2018-2019 season, not the 2017-2018 season. Similarly, Chelsea, which was champion in the European League in 2018-2019, was only found to be an efficient club in the 2019-2020 season. As these examples show, clubs can Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 121 only provide sustainable financial resources with sustainable sportive success. Moreover, they can transfer talented players who can participate in international tournaments every year to make financial resources sustainable, or they can invest in their academies to produce their own qualified football players. Clubs that make their financial resources sustainable are referred to as "big clubs". It can be said that achieving sportive success is easier for these clubs compared to other clubs, as big clubs are more advantageous in terms of attracting talented and qualified players. However, financial sustainability depends on a number of factors that are not constantly under control. For example, some penalties imposed by UEFA in accordance with FFP policies harm the financial sustainability of clubs. In addition to the clubs' efforts to cope with the FFP limitations, the covid-19 pandemic, which emerged in the province of Wuhan in China in December 2019 and spread all over the world in a short time, led to huge decreases in the revenues of the clubs. Due to Covid- 19, some countries have suspended their leagues for a long period of time, broadcasting agreements were interrupted and stadium revenues were not obtained. To avoid the effects of factors such as these that could harm financial sustainability, clubs sometimes turn to finding new sources of funding. For example, clubs may try to provide additional financing with initiatives such as the "European Super League", which was established on April 19, 2021 and was dissolved after only 48 hours. However, for a sport whose rules and organizations are deeply rooted, such initiatives may cause clubs to disconnected from other clubs. Therefore, in order to make the financial resource sustainable, clubs should make their sportive success sustainable in every platform. Funding: This research received no external funding. Conflicts of Interest: The author declare no conflicts of interest. References Andrade, R. M. D., Lee, S., Lee, P. T. W., Kwon, O. K., & Chung, H. M. (2019). Port efficiency incorporating service measurement variables by the BiO-MCDEA: Brazilian case. Sustainability, 11(16), 4340. https://doi.org/10.3390/su11164340 Angulo-Meza, L., González-Araya, M., Iriarte, A., Rebolledo-Leiva, R., & de Mello, J. C. S. (2019). A multiobjective DEA model to assess the eco-efficiency of agricultural practices within the CF+ DEA method. Computers and Electronics in Agriculture, 161, 151-161. Anthony, P., Behnoee, B., Hassanpour, M., & Pamucar, D. (2019). Financial performance evaluation of seven Indian chemical companies. Decision Making: Applications in Management and Engineering, 2(2), 81-99. https://doi.org/10.31181/dmame1902021a Bal, H., Örkcü, H. H., & Çelebioğlu, S. (2010). Improving the discrimination power and weights dispersion in the data envelopment analysis. Computers & Operations Research, 37(1), 99-107. https://doi.org/10.1016/j.cor.2009.03.028 Blagojević, A., Vesković, S., Kasalica, S., Gojić, A., & Allamani, A. (2020). The application of the fuzzy AHP and DEA for measuring the efficiency of freight transport railway Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 122 undertakings. Operational Research in Engineering Sciences: Theory and Applications, 3(2), 1-23. https://doi.org/10.31181/oresta2003001b Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429-444. https://doi.org/10.1016/0377-2217(78)90138-8 Chelmis, E., Niklis, D., Baourakis, G., & Zopounidis, C. (2019). Multiciteria evaluation of football clubs: the Greek Superleague. Operational Research, 19(2), 585-614. https://doi.org/10.1007/s12351-017-0300-2 da Silva, A. F., Marins, F. A. S., Tamura, P. M., & Dias, E. X. (2017). Bi-Objective Multiple criteria data envelopment analysis combined with the overall equipment effectiveness: An application in an automotive company. Journal of cleaner production, 157, 278-288. https://doi.org/10.1016/j.jclepro.2017.04.147 Deloitte. Football money league report (2016). https://www2.deloitte.com/me/en/pages/consumer-business/articles/deloitte- football-money-league1.html Accessed 13 March 2020 Deloitte. Football money league report (2017). https://www2.deloitte.com/tr/en/pages/consumer-industrial- products/articles/deloitte-football-money-league.html Accessed 13 March 2020 Deloitte. Football money league report (2018). https://www2.deloitte.com/mk/en/pages/consumer-business/articles/deloitte- football-money-league2.html Accessed 13 March 2020 Deloitte. Football money league report (2019). https://www2.deloitte.com/global/en/pages/consumer-business/articles/deloitte- football-money-league.html 1 May 2021 Deloitte. Football money league report (2020). https://www2.deloitte.com/bg/en/pages/finance/articles/football-money-league- 2020.html Accessed 1 May 2021 Despic, D., Bojovic, N., Kilibarda, M. & Kapetanovic, M. (2019). Assessment of efficiency of military transport units using the DEA and SFA methods. Military Technical Courier, 67(1), 68–92. https://doi.org/10.5937/vojtehg67-18508. Dobson, S. & Goddard, J. (2011). The economics of football (second edition). New York: Cambridge University Press. Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sarrico, C. S., & Shale, E. A. (2001). Pitfalls and protocols in DEA. European Journal of operational research, 132(2), 245-259. https://doi.org/10.1016/S0377-2217(00)00149-1 Friedman, L., & Sinuany-Stern, Z. (1998). Combining ranking scales and selecting variables in the DEA context: The case of industrial branches. Computers & Operations Research, 25(9), 781-791. https://doi.org/10.1016/S0305-0548(97)00102-0 Galariotis, E., Germain, C., & Zopounidis, C. (2018). A combined methodology for the concurrent evaluation of the business, financial and sports performance of football clubs: the case of France. Annals of Operations Research, 266(1), 589-612. https://doi.org/10.1007/s10479-017-2631-z Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 123 Ghasemi, M. R., Ignatius, J., & Emrouznejad, A. (2014). A bi-objective weighted model for improving the discrimination power in MCDEA. European Journal of Operational Research, 233(3), 640-650. https://doi.org/10.1016/j.ejor.2013.08.041 Ghofran, A., Sanei, M., Tohidi, G., & Bevrani, H. (2021). Applying MCDEA models to rank decision making units with stochastic data. International Journal of Industrial Mathematics, 13(2), 101-111. Golany, B., & Roll, Y. (1989). An application procedure for DEA. Omega, 17(3), 237-250. https://doi.org/10.1016/0305-0483(89)90029-7 Guzmán, I., & Morrow, S. (2007). Measuring efficiency and productivity in professional football teams: evidence from the English Premier League. Central European Journal of Operations Research, 15(4), 309-328. https://doi.org/10.1007/s10100-007-0034- y Haas, D. J. (2003a). Productive efficiency of English football teams—a data envelopment analysis approach. Managerial and Decision Economics, 24(5), 403-410. https://doi.org/10.1002/mde.1105 Haas, D. J. (2003b). Technical efficiency in the major league soccer. Journal of Sports Economics, 4(3), 203-215. https://doi.org/10.1177/1527002503252144 Haas, D., Kocher, M. G., & Sutter, M. (2004). Measuring efficiency of German football teams by data envelopment analysis. Central European Journal of Operations Research, 12(3), 251-268. Halkos, G. E., & Tzeremes, N. G. (2013). A Two‐Stage double bootstrap DEA: The case of the top 25 European football clubs' efficiency levels. Managerial and Decision Economics, 34(2), 108-115. https://doi.org/10.1002/mde.2597 Hassanpour, M. (2020). Evaluation of Iranian small and medium-sized industries using the DEA based on additive ratio model–a review. Facta Universitatis, Series: Mechanical Engineering, 18(3), 491-511. https://doi.org/10.22190/FUME200426030H Hatami-Marbini, A., & Toloo, M. (2017). An extended multiple criteria data envelopment analysis model. Expert Systems with Applications, 73, 201-219. https://doi.org/10.1016/j.eswa.2016.12.030 Jardin, M. (2009). Efficiency of French football clubs and its dynamics. Munich Personal RePEc Archive. https://mpra.ub.uni-muenchen.de/19828/ Accessed 18 June 2020. https://mpra.ub.uni-muenchen.de/19828/1/MPRA_paper_19828.pdf Kamarudin, F., Sufian, F., Nassir, A. M., Anwar, N. A. M., & Hussain, H. I. (2019). Bank efficiency in Malaysia a DEA approach. Journal of Central Banking Theory and Practice, 8(1), 133-162. https://doi.org/10.2478/jcbtp-2019-0007 Kern, A., Schwarzmann, M., & Wiedenegger, A. (2012). Measuring the efficiency of English Premier League football. Sport, Business and Management: an International Journal, 2(3), 177-195. https://doi.org/10.1108/20426781211261502 Kohl, S., Schoenfelder, J., Fügener, A., & Brunner, J. O. (2019). The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals. Health care management science, 22(2), 245-286. https://doi.org/10.1007/s10729-018-9436-8 https://mpra.ub.uni-muenchen.de/19828/ Arsu/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 106-125 124 Kulikova, L. I., & Goshunova, A. V. (2014). Efficiency measurement of professional football clubs: a non-parametric approach. Life Science Journal, 11(11), 117-122. Lewin, A. Y., Morey, R. C., & Cook, T. J. (1982). Evaluating the administrative efficiency of courts. Omega, 10(4), 401-411. https://doi.org/10.1016/0305-0483(82)90019-6 Li, X. B., & Reeves, G. R. (1999). A multiple criteria approach to data envelopment analysis. European Journal of Operational Research, 115(3), 507-517. https://doi.org/10.1016/S0377-2217(98)00130-1 Lombardi, G. V., Stefani, G., Paci, A., Becagli, C., Miliacca, M., Gastaldi, M., Giannetti, B. F., & Almeida, C. M. V. B. (2019). The sustainability of the Italian water sector: An empirical analysis by DEA. Journal of Cleaner Production, 227, 1035-1043. https://doi.org/10.1016/j.jclepro.2019.04.283 Marcén M. (2019) Bosman Ruling. In: Marciano A., Ramello G.B. (eds) Encyclopedia of Law and Economics. New York: Springer. Miragaia, D., Ferreira, J., Carvalho, A., & Ratten, V. (2019). Interactions between financial efficiency and sports performance. Journal of Entrepreneurship and Public Policy, 8(1), 84-102. https://doi.org/10.1108/JEPP-D-18-00060 Örkcü, H. H., & Bal, H. (2011). Goal programming approaches for data envelopment analysis cross efficiency evaluation. Applied Mathematics and Computation, 218(2), 346-356. https://doi.org/10.1016/j.amc.2011.05.070 Pestana Barros, C. P., & Leach, S. (2006). Performance evaluation of the English Premier Football League with data envelopment analysis. Applied Economics, 38(12), 1449-1458. https://doi.org/10.1080/00036840500396574 Pestana Barros, C., Assaf, A., & Sá-Earp, F. (2010). Brazilian football league technical efficiency: a Simar and Wilson approach. Journal of Sports Economics, 11(6), 641-651. https://doi.org/10.1177/1527002509357530 Pradhan, S., Boyukaslan, A., & Ecer, F. (2017). Applying grey relational analysis to italian football clubs: a measurement of the financial performance of Serie A teams. International review of economics and management, 4(4), 1-19. https://doi.org/10.18825/iremjournal.290668 Rashidi, K., & Cullinane, K. (2019). A comparison of fuzzy DEA and fuzzy TOPSIS in sustainable supplier selection: Implications for sourcing strategy. Expert Systems with Applications, 121, 266-281. https://doi.org/10.1016/j.eswa.2018.12.025 Rossi, G., Goossens, D., Di Tanna, G. L., & Addesa, F. (2019). Football team performance efficiency and effectiveness in a corruptive context: the Calciopoli case. European Sport Management Quarterly, 19(5), 583-604. https://doi.org/10.1080/16184742.2018.1553056 Rubem, A. P. S., & Brandão, L. C. (2015). Multiple Criteria Data Envelopment Analysis– An Application to UEFA EURO 2012. Procedia Computer Science, 55, 186-195. https://doi.org/10.1016/j.procs.2015.07.031 Sakınç, İ., Açıkalın, S., & Soygüden, A. (2017). Evaluation of the relationship between financial performance and sport success in European football. Journal of Physical Education and Sport, 17(1), 16-22. https://doi.org/10.7752/jpes.2017.s1003 Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 125 Sałabun, W., Shekhovtsov, A., Pamučar, D., Wątróbski, J., Kizielewicz, B., Więckowski, J., Bozanic D., Urbaniak, K., & Nyczaj, B. (2020). A Fuzzy Inference System for Players Evaluation in Multi-Player Sports: The Football Study Case. Symmetry, 12(12), 2029. https://doi.org/10.3390/sym12122029 San Cristóbal, J. R. (2011). A multi criteria data envelopment analysis model to evaluate the efficiency of the Renewable Energy technologies. Renewable Energy, 36(10), 2742-2746. https://doi.org/10.1016/j.renene.2011.03.008 Thanassoulis, E., Dyson, R. G., & Foster, M. J. (1987). Relative efficiency assessments using data envelopment analysis: an application to data on rates departments. Journal of the Operational Research Society, 38(5), 397-411. https://doi.org/10.1057/jors.1987.68 Transfermrkt.com. (2019). https://www.transfermarkt.com/spieler- statistik/wertvollstemannschaften/marktwertetop Accessed 26 December 2019 UEFA (2021). Financial Fair Play. https://www.uefa.com/insideuefa/protecting-the- game/financial-fair-play/ Accessed 2 May 2021. Appendix 1. Example of LINDO Codes for BiO-MCDEA Model (Manchester United 2019-2020 Season) Min 0.5M+0.5d1+0.5d2+0.5d3+0.5d4+0.5d5+0.5d6+0.5d7+0.5d8+0.5d9+0.5d10 Subject to 127.2x1+74698x2+670.45x3=1 22000y1+711.5y2-127.2x1-74698x2-670.45x3+d1=0 17000y1+757.3y2-226.7x1-61040x2-913.75x3+d2=0 24000y1+840.8y2-216.5x1-76104x2-930.93x3+d3=0 36000y1+660.1y2-74.4x1-75865x2-777.33x3+d4=0 25000y1+610.6y2-62.9x1-54130x2-1048.6x3+d5=0 10000y1+445.6y2-69.7x1-59897x2-607.65x3+d6=0 31000y1+635.9y2-73.7x1-46911x2-874.15x3+d7=0 17000y1+513.1y2-82.2x1-40445x2-705.85x3+d8=0 18000y1+604.7y2-71.9x1-53053x2-1002.7x3+d9=0 22000y1+459.7y2-83.4x1-39101x2-661.88x3+d10=0 M-d1>=0 M-d2>=0 M-d3>=0 M-d4>=0 M-d5>=0 M-d6>=0 M-d7>=0 M-d8>=0 M-d9>=0 M-d10>=0 End © 2018 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).