Operational Research in Engineering Sciences: Theory and Applications Vol. 3, Issue 1, 2020, pp. 89-102 ISSN: 2620-1607 eISSN: 2620-1747 DOI:https://doi.org/10.31181/oresta2001089p * Corresponding author adispuska@yahoo.com (A. Puška), ilija.stojanovic@teol.net (I. Stojanović), a.maksimovic22@gmail.com (A. Maksimović), nasihao@yahoo.com (N. Osmanović) PROJECT MANAGEMENT SOFTWARE EVALUATION BY USING THE MEASUREMENT OF ALTERNATIVES AND RANKING ACCORDING TO COMPROMISE SOLUTION (MARCOS) METHOD Adis Puška 1*, Ilija Stojanović 2, Aleksandar Maksimović 1, Nasiha Osmanović 2 1 Institute for Scientific Research and Development, Brčko District of Bosnia and Herzegovina 2 Al Ghurair University Dubai, United Arab Emirates Received: 27 February 2020 Accepted: 28 March 2020 First online: 07 April 2020 Original scientific paper Abstract: Every organization needs to invest in order for it to grow, and investments are made through projects. Thus, investment management is performed by applying project management techniques. Different project management software programs are used to manage multiple projects. There is a lot of project management software on the market, and four pieces of the software were selected and analyzed. In this paper, the best management software rated by the beneficiaries of these projects in the United Arab Emirates are explored. The research required for this study was conducted in the United Arab Emirates. The MARCOS method was used to evaluate the program. The results showed that Smartsheet had been rated the best by users. This paper provides an overview of how multicriteria analysis methods can be used when ranking project management programs. Keywords: project management, software, United Arab Emirates, MARCOS method 1. Introduction Every organization is required to invest. In order to implement their project investments, it is necessary for such organizations to apply the project management techniques that enable the smooth implementation of project investments. A project is a very complex undertaking, especially when organizational constraints and elements used, resources and costs involved, a large number of people working on it, as well as the other elements that further complicate the project, are concerned (Puška, 2013). Due to project complexity and its importance for any organization, a Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 90 project requires special IT support for its implementation. Market demand in this field has led to the development of a broad range of IT software solutions supportive of the creation, monitoring and implementation of projects in order to meet set investment goals. The task of project management software is to facilitate the business operations of the company pertaining to project management. These pieces of software are used to plan, monitor and control projects (Sajad et al., 2016). However, project management software often does not efficiently facilitate project work and failures in project management occur. The failure status of project management software programs has not changed much today, some projects still being delayed, overbilling or not meeting customer expectations (Hassan et al., 2018). All these software solutions are different, so it is necessary to choose the software that best suits the appropriate project in a company. On the market, there are several tools for automated project program management, and these projects are steadily increasing in number. Given the ongoing process of change, it is not possible to provide a list of all project management programs (Kostalova and Tetrevova, 2014). Each tool specializes in the different fields of project management, so some programs are used in specific project lifecycles, whereas others are used throughout the project. The purpose of this study is to rank different software solutions intended for the project management implementation based on the evaluations provided by project managers in the United Arab Emirates (UAE). This will provide information on which software features are best-suited for the project manager in the UAE. This allows software companies to find out the gaps within their software solutions which need to be addressed so as to make them more user-friendly. For this study, a comparison was made between the four pieces of software that are very much appreciated globally among practitioners, namely Smartsheet, Asana, Microsoft Project and Basecamp, based on a total of six criteria. Since software evaluation was based on multiple criteria, the logical choice is to use multicriteria analysis methods. The contribution of this study reflects in the application of the Measurement of Alternatives and Ranking According to Compromise Solution (MARCOS) method in project management software ranking. This multicriteria analysis method is a new method developed by Stević et al. (2020) and has only just begun to be put into practice. This method has shown excellent results in the sustainable selection of suppliers (Stević et al., 2020) and has shown a certain advantage over other multicriteria analysis methods, which is why this method was chosen. This paper is divided into six sections, excluding the literature. The introductory section sets out the purpose and contribution of this paper. The second section is dedicated to the application of multicriteria methods in project management software ranking, and the theoretical foundations of the MARCOS method are presented. In the third section, the model is presented and the research methodology is explained. The fourth section is focused on processing the results of the survey. In the fifth section, the results of the sensitivity analysis performed are shown, and the obtained results are confirmed. In the sixth selection, the most important results, shortcomings and recommendations for future research are presented. Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 91 2. The Application of Multicriteria Analysis Methods in Project Management Multicriteria analysis methods are concerned with decision-making taking into consideration multiple criteria. These criteria may be different. Some criteria are numerical, some are quantitative, some have units of measurement, and so on. Certain criteria can solely be obtained through a subjective attitude, whereas other criteria can be measured and determined (Erdogan et al., 2019). The basic features of the application of multicriteria analysis are: (Rifle, 2013) - It often happens that one alternative is better than another in one criterion, while the other is better than the first in the second criterion. - It is not always the case that one alternative is better than another in all criteria, so there is no optimal solution in a strict mathematical sense in that case. - A solution to the problem implies finding a compromise solution. Different methods have been used in different studies on project management. Alencar and Almeida (2010) used the PROMETHEE VI method in the selection of project management members. Zavadskas et al. (2012) applied the AHP and ARAS methods in the evaluation of project managers. Chang et al. (2012) applied a fuzzy approach in order to evaluate which criterion is the most important in improving the project team performance by using the DEMATEL, ANP, and VIKOR methods. Górecka (2013) applied the ELECTRA and PROMETHEE methods when choosing the best alternative for road construction. Wang et al. (2014) applied a hybrid model by using the DEMATEL, ANP and VIKOR methods in the estimation and improvement of the Six Sigma projects so as to reduce the performance differences in each criterion. Jafarnejad Chaghooshi et al. (2016) made a choice of the project manager by using the fuzzy DEMATEL method and the Fuzzy VIKOR method. Puška et al. (2017) used the TOPSIS method to examine the impact of subjective judgments on project management decision-making. Khoshnava et al. (2018) used the DEMATEL and fuzzy ANP methods to improve green project management. Erdogan et al. (2019) used the AHP method when designing a sustainable construction in project management. Piengang et al. (2019) selected project management programs by using the AHP and VIKOR methods. Based on this brief overview of the research studies carried out so far, it can be seen that different multicriteria analysis methods have been used for project management purposes. In the following section, the MARCOS method used in the paper in order to rank software intended for project management is explained. 2.1. The Measurement of Alternatives and Ranking According to Compromise Solution (MARCOS) method The MARCOS method is based on defining the relationship between alternatives and reference values (ideal and anti-ideal alternatives) (Stević et al., 2020). Decision- making preferences are defined based on utility functions. A utility function is the position of an alternative in relation to the ideal and anti-ideal solutions (Stanković Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 92 et al., 2020). The best alternative is that closest to the ideal point and farthest from the anti-ideal point. The MARCOS method is implemented through the following steps (Stević, et al, 2020): Step 1. The formation of the initial decision matrix. Step 2. The formation of an extended initial matrix. This step defines the ideal and anti-ideal solutions. The ideal solution is an alternative with the best alternative for certain criteria, whereas the anti-ideal solution is the worst alternative for certain criteria. This is based on the following equations: min maxij ij j j AAI x if j B and x if j C=   (1) max minij ij jj AI x if j B and x if j C=   (2) where B stands for the criteria to be maximized, and C stands for the criteria to be minimized. Step 3. The normalization of the extended initial matrix. Normalization is performed by using the following equations: ai ij ij x n if j C x =  (3) ij ij ai x n if j B x =  (4) where the elements ijx and aix represent the elements from the initial decision matrix. Step 4. The determination of a weighted matrix. Aggravation is performed by multiplying normalized matrix values by corresponding weights. Step 5. The calculation of the utility degree of the alternatives Ki. The utility degree is determined by applying the following equations: i i aai S K S − = (5) i i ai S K S + = (6) where iS (i=1,2,..,m) represents the sum of the elements of a difficult matrix: 1 n i ij i S v = =  (7) Step 6. The formation of the utility function of the alternatives f(Ki). The utility function is calculated by using the following equation: Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 93 ( ) ( ) ( ) ( ) ( ) ; 1 1 1 i i i i i i i K K f K f K f K f K f K + − + − + − + = − − + + (8) where f(K-i) is the utility function versus the anti-ideal solution, while f(K+i) is the utility function versus the ideal solution. The utility functions are calculated by using the following equations: ( ) ii i i K f K K K + − + − = + (9) ( ) ii i i K f K K K − + + − = + (10) Step 7. Ranking the alternatives. A rank is formed based on the final value of the utility function. It is desirable that the alternative should have the greatest value of the utility function (Stević and Brković, 2020). 3. Model and Methodology The methodology used in this research study consists of the following stages: • Phase 1. Defining the research problem and goal • Phase 2. Defining the criteria and alternatives, and the creation of the models • Phase 3. Drafting a survey questionnaire and data collection • Phase 4. The selection of multicriteria analysis methods • Phase 5. Data processing and preparation for analysis • Phase 6. The analysis of the collected data • Phase 7. Conducting a sensitivity analysis The initial stage in this study is focused on the definition of the research problem. The problem of this research is how to choose the software for project management needs that best meets the needs of managers in the UAE. Based on this problem, the aim of the research is presented, which is highlighted in the introductory part of the paper. The most important project management software ranking criteria are related to human, technical and managerial factors (Chatzoglou et al., 2007). Many authors have used these software rankings. Gharaibeh (2014) used the following criteria in his ranking of project management software: Accuracy, Affordability, Ease of Use and Ability to Handle Complexity. Ahmad and Laplante (2006) used the following criteria: Task Scheduling, Resource Management, Collaboration, Time Tracking, Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 94 Estimating, Risk Assessment, Change Management, Reporting/Charts, File Attachment, E-mail Notification, Process/Methodology and Portfolio Management. Rouhani and Zare Ravasan (2016) identified a total of the 48 criteria that may be used to evaluate project management software. Due to a large number of different criteria, these criteria were systematized in this paper into six criteria, each including several sub-criteria. In order to rank project management implementation software, it is necessary to, first, define the criteria and determine the alternatives that will be ranked. In this study, those six criteria were defined as follows: • The Tasks (C1) criterion evaluated how tasks can be created and assigned with software, what is done from identified activities, and how to set agreed dates in the project, how to prioritize the project, and how to determine the project to-do lists. • The Collaboration (C2) criterion assessed how collaborative joint project planning is, how comments can be made, how documents can be uploaded and downloaded. • The Projects criterion (C3) evaluated support for individual projects in terms of the project map design, respectively the Gantt map, what the calendar view of the planned activities is, what support for the activity display is, how the project is budgeted for individual activities and for the entire project, and how the implementation of certain activities is monitored. • The Portfolio Management (C4) criterion assessed the coordination of the projects that a particular company owns, how those projects are budgeted, how the What-If Scenarios are developed, the workflow for individual activities, how project request management is monitored, how support for cost tracking and Return on Investment (ROI) calculations is implemented, various project projections, and how project risk analysis is performed through software. • The Resource Management (C5) criterion evaluated the bases used to store the data, how resources could be allocated, how the workload of the project workers is monitored, and the time spent in the project execution. • The Platform (C6) criterion evaluated support for the project alerts, mobile access to these programs, user management, roles and access, the integration of the Application Programming Interface, reporting on and the monitoring of these programs, how the program is adjusted to users, how reliable the software is and how fast it runs. After the criteria for the evaluation of the project management software had been defined, the software that will be evaluated was determined. In this study, globally recognized software solutions (i.e. Smartsheet (A1), Asana (A2), Microsoft Project (A3) and Basecamp (A4)) and their applicability in the UAE were evaluated. Smartsheet is a cloud-based platform, which allows organizations of all sizes to plan, capture, manage, automate and report on work across the business, empowering you to move faster, drive innovation, and achieve more. Asana is a web and mobile application, designed to help teams organize, track and manage their work. Asana is the work management platform that teams use to stay focused on the goals, projects Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 95 and daily tasks that grow business. Microsoft Project is a project management software product developed and sold by Microsoft. It is so designed to assist a project manager in developing a schedule, assigning resources to tasks, tracking progress, managing the budget and analyzing workloads. Basecamp is focused on the development of a simple interface, in which users can create projects, document progress and manage tasks. It is a web-based piece of software, allowing users to sign in anywhere, anytime, either through a web browser or through applications compatible with many mobile devices. Based on the defined criteria and the alternative in the study, the research model was created, which is shown in Figure 1. In order to choose the best characteristics of project management software, it is necessary the weight of the criteria should, first, be determined, after which the value of the alternatives should be determined according to all the criteria. Based on this, two questionnaires were formed. The first survey questionnaire refers to the determination of the weight of the criteria representing a certain feature of the software indicated as important for practitioners in the UAE. In this questionnaire, the experts were asked to anonymously rate the importance of individual criteria, which ranged from 1 to 5, the questionnaire resulting in the experts’ ratings (Table 1). For the second questionnaire, the secondary data drawn from the g2.com user community were used. This web portal monitors how different software solutions are accepted by users. On the platform, they ask users daily to evaluate the software they use. These data represent the evaluations of different project management software solutions from global users and practitioners. They rated the selected programs with the grades ranging from 1 to 10. Based on their grades, a mean value was formed for each criterion for the individual alternatives (Table 2). In this way, the data needed to conduct the research were collected. Figure 1. The project management software selection model Once the data were collected, it was necessary to select a multicriteria analysis method. Since the focus of this research is on the MARCOS method, the weights of the criteria were calculated by applying a simple sum of ratings from the experts and converting those ratings into the percentage that, in fact, is the weight of each criterion. The MARCOS method was selected for ranking the alternatives. After the data had been collected, they were processed and prepared for the research. Based on the data collected, the initial decision-making tables were The selection of programs for the project management Tasks (C1) Collaboration (C2) Projects (C3) Portfolio Management (C4) Resource Management (C5) Platform (C6) Microsoft Project (A3) Smartsheet (A1) Asana (A2) Basecamp (A4) Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 96 formed; the first table was for the determination of the value of the criteria weights (Table 1), and the second was for the determination of the ranking of the alternatives (Table 2). Once the decision tables were formed, the defined analysis were carried out and the research results were obtained. In order to confirm the results obtained, a sensitivity analysis was also conducted. The sensitivity analysis was aimed at examining the dependence of the results on the change in the weights of the criteria. The details on stages 6 and 7 are given in Chapter 4. 4. Results This chapter provides a more detailed elaboration on the manner in which the research results were generated. First, the weights of the criteria used in the decision model were calculated and presented. The ratings given by the experts on an anonymous basis are presented in Table 1. They determined the rating based on the subjective assessment of the importance of certain criteria, where the score 1 represents a value of little or no importance, whereas the score 5 represents a value of a very important criterion. After the criteria had been collected by the experts, all the individual criteria were summed up. Based on the total value of the criteria, a percentage of the importance of the individual criteria expressed in decimals was formed. These values represent the weight of each criterion. Table 1. The experts’ evaluation of the importance of the criteria C1 C2 C3 C4 C5 C6 Expert 1 5 4 5 5 4 2 Expert 2 5 4 5 4 3 2 Expert 3 4 4 5 5 4 4 Expert 4 5 4 5 5 4 3 Expert 5 5 4 5 5 4 3 Sum 24 20 25 24 19 14 Weight 0.190 0.159 0.198 0.190 0.151 0.111 Based on the results obtained, it can be observed that the criterion C3 – Projects (w = 0.198) has the highest importance according to the experts, whereas the criterion C6 – Platform (w = 0.111) has the lowest importance. Based on all the other values, it can be concluded that the criteria C1 and C4 were assigned high values, whereas the criteria C2 and C5 were assigned mean values. This means that the criteria C1, C3 and C4 are the most important for the evaluation of individual software, the criteria C2 and C5 are of medium importance, while the criterion C6 is the least important. Once the weights were established for the criteria, it was necessary to determine the rankings for the selected alternatives. Based on the data collected, an initial decision matrix for the observed alternatives was formed (Table 2). Forming an initial decision matrix is the first step in all multicriteria analysis methods. In order to determine the ranking of the alternatives, it was necessary to normalize the data so as to make them be uniform. Simple linear normalization (Equation 3) was applied to the MARCOS method. It was necessary to determine the maximum value of Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 97 the criteria, since it is desirable for all criteria that the values of the alternatives should be maximized. The normalization of the initial decision matrix is Step 3 of the MARCOS method. Table 2. The initial decision matrix C1 C2 C3 C4 C5 C6 Smartsheet 8.675 8.433 8.000 7.800 8.025 8.043 Asana 8.825 8.600 7.420 7.463 7.825 8.229 Microsoft Project 8.325 7.600 8.040 7.700 7.925 7.600 Basecamp 8.525 8.667 7.180 7.375 7.750 8.071 Max 8.825 8.667 8.040 7.800 8.025 8.229 The results of the normalized decision matrix (Table 3) show that the alternatives A1 and A2 have two maximum values of the criteria each, while the alternatives A3 and A4 have one maximum value of the criteria. Based on this, it was necessary that a compromise solution should be made, since there was no optimal solution, speaking in a strictly mathematical sense. In these cases, multicriteria analysis methods were applied, because one alternative had all the best indicators and there was no need to rank them, because the best alternative was known. Table 3. The normalized decision matrix C1 C2 C3 C4 C5 C6 Smartsheet 0.983 0.973 0.995 1.000 1.000 0.977 Asana 1.000 0.992 0.923 0.957 0.975 1.000 Microsoft Project 0.943 0.877 1.000 0.987 0.988 0.924 Basecamp 0.966 1.000 0.893 0.946 0.966 0.981 Weight 0.190 0.159 0.198 0.190 0.151 0.111 After the normalization of the data in the decision matrix, these values needed to be aggravated by the weighting coefficients. This was the fourth step in the MARCOS method. The fifth step in the MARCOS method was to calculate the utility degree. In order to perform this step, it was first necessary to determine the ideal and anti-ideal solutions. The ideal solution represents the maximum value of a certain criterion, whereas anti-ideal values represent the minimum value of a specific criterion. Then, the values for the individual alternatives and for the ideal and anti-ideal solutions were summed up and the utility degrees were calculated (Equations 5 and 6). Table 4. The aggravated normalized decision matrix C1 C2 C3 C4 C5 C6 Sum Smartsheet 0.187 0.154 0.197 0.190 0.151 0.109 0.989 Asana 0.190 0.158 0.183 0.182 0.147 0.111 0.971 Microsoft Project 0.180 0.139 0.198 0.188 0.149 0.103 0.957 Basecamp 0.184 0.159 0.177 0.180 0.146 0.109 0.955 Ideal 0.190 0.159 0.198 0.190 0.151 0.111 1.000 Anti-ideal 0.180 0.139 0.177 0.180 0.146 0.103 0.924 Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 98 The sixth step of the MARCOS method was to form the utility function of the alternatives. The utility function was calculated by using Equation 8. In order to calculate the utility function of the alternatives, it was necessary to calculate the utility function in relation to the ideal and anti-ideal solutions. These functions are the same for all the values and their values: f (Ki-) = 0.520 and f (Ki +) = 0.480. The inclusion of these values generated the final value for the alternatives (Table 5) and determined the ranking of the alternatives. Table 5. The ranking of the alternatives. Ki- Ki+ f(Ki) Rank Smartsheet 1.070 0.989 0.685 1 Asana 1.051 0.971 0.673 2 Microsoft Project 1.035 0.957 0.663 3 Basecamp 1.033 0.955 0.661 4 The survey results show that Smartsheet received the highest ratings, while the Basecamp ranked the worst among the selected pieces of software. According to these results, there was a very small difference between the final values for the alternatives based on the MARCOS method calculation. Microsoft Project received slightly over 0.002 more value than Basecamp, while Asana was better than Microsoft Project by 0.01, and Smartsheet was better than Asana by 0.012. Based on all the foregoing, the difference between the first-ranked and the last-ranked software is 0.024, which leads to the conclusion that neither software deviates from the others according to the estimates made by the experts through their anonymous responses. The original normalized decision matrix also shows that all the selected pieces of software have the best grades for certain criteria. In order to confirm these results, a sensitivity analysis was carried out. 5. Sensitivity Analysis A sensitivity analysis was performed in such a manner that the weights of the criteria varied, and it was observed that the change made in the weights influenced the ranking of the alternatives (Puška et al., 2018). Seven scenarios were used for this purpose. In the first six scenarios, an individual criterion was assumed to be five times as significant as the other criteria, and it was assigned a weight of 0.5, while the other criteria were assigned a weight of 0.1 (Table 6). The seventh scenario assumed that the validity of all the criteria was the same, and the assigned weight was 0.167. The aim of the sensitivity analysis was to avoid the subjective evaluation of the criteria by the experts. Using these criteria, the results of the study were tested so as to understand whether they were sensitive to changing weights. This enabled us to confirm or deny the results. In addition, this analysis shows the advantages and disadvantages of different software solutions according to certain criteria. Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 99 Table 6. The scenarios for sensitivity analysis C1 C2 C3 C4 C5 C6 Scenario 1 0.5 0.1 0.1 0.1 0.1 0.1 Scenario 2 0.1 0.5 0.1 0.1 0.1 0.1 Scenario 3 0.1 0.1 0.5 0.1 0.1 0.1 Scenario 4 0.1 0.1 0.1 0.5 0.1 0.1 Scenario 5 0.1 0.1 0.1 0.1 0.5 0.1 Scenario 6 0.1 0.1 0.1 0.1 0.1 0.5 Scenario 7 0.167 0.167 0.167 0.167 0.167 0.167 The research study provided an opportunity to explore the opinion of the users of these programs. They provided the grades from 1 to 10, and this interval was the basis for the evaluation of these software solutions. These ratings were, then, ranked by using the defined scenarios. The results of the survey show that Smartsheet achieved the best results in six scenarios, whereas only in Scenario 6 it ranked the second. This shows that Asana has better C6 – Platform criterion than Smartsheet, according to the users’ ratings, so it did not rank the first in this scenario. The same situation is with Asana, which was the third in Scenario S3, in which Microsoft Project achieved better results than this software solution. Microsoft Project achieved the worst results in four scenarios, whereas the Basecamp achieved the worst results in three scenarios. Based on the results of the sensitivity analysis, a conclusion can be drawn that Smartsheet achieved the best results in the user-made evaluation, whereas Microsoft Project and Basecamp showed the worst results. Table 7. The ranking of the project management software by scenarios Rank (S1) Rank (S2) Rank (S3) Rank (S4) Rank (S5) Rank (S6) Rank (S7) Smartsheet 1 1 1 1 1 2 1 Asana 2 2 3 2 2 1 2 Microsoft Project 4 4 2 3 3 4 4 Basecamp 3 3 4 4 4 3 3 6. Conclusion Project management is very important for any organization. An organization may deal with multiple and very complex projects, so they need to be managed appropriately. In such cases, project management is performed with the support of IT software. There are many software solutions on the market. In this paper, four pieces of software were selected. The selection was made through the evaluation provided by the g2.com user community for different software features and whether these features were relevant in the United Arab Emirates or not was evaluated. These findings led to the results implicative of user preferences in the UAE for using a certain software solution. Only an overview of whether the software features for the selected software solutions were relevant for users in the UAE or not was presented. The weights of the criteria were determined in collaboration with the experts. Using the MARCOS method, the selected project management software was ranked. The Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 100 research results show that the Smartsheet features are the most relevant in the experts’ opinions. Still, this does not reflect the fact that this solution is the most used solution, but rather that its futures are most apricated by the experts. The disadvantage of this research study is that only a limited number of software solutions were included in the evaluation. Another disadvantage of this research study is the use of the six criteria according to which the pieces of software were evaluated. This number of the criteria were used due to the availability of the secondary data provided by the g2.com user community about different software solutions in project management. However, the respondents were instructed to perceive the sub-criteria through the basic criteria and provide a cumulative rating for the software they are using. In this manner, more criteria were considered than the stated. Future research may be focused on presenting the ratings of the sub- criteria by going deeper into the main criteria. It is also necessary to conduct a research study on the importance that criteria have for users in order to enable them to obtain the necessary information about the criteria which they should pay attention to when ranking different software options. The purpose of each evaluation is to see how a particular solution suits the user, so it is necessary to obtain all the necessary information from users. This study has presented a new approach to how to evaluate project management software by using the newly-introduced MACROS method. In future research, this model and this methodology should be applied, because they are very simple and can be applied to other decision problems as well. References Ahmad, N., & Laplante, P. (2006). Software Project Management Tools: Making a Practical Decision Using AHP. 2006 30th Annual IEEE/NASA Software Engineering Workshop. doi:10.1109/sew.2006.30 Alencar, L. H., & Almeida, A. T. de. (2010). A model for selecting project team members using multicriteria group decision making. Pesquisa Operacional, 30(1), 221–236. doi:10.1590/s0101-74382010000100011 Chang, Y. F., Watada, J., Ishii, H. (2012). A fuzzy MCDM approach to building a model of high performance project team – a case study. International Journal of Innovative Computing, Information and Control, 8(10 B), 7393–7404. Chatzoglou, P. D., Theriou, N. G., Dimitriadis, E., & Aggelides, V. (2007). Software project management and planning: the case of the Greek IT sector. International Journal of Applied Systemic Studies, 1(3), 305-317. doi: 10.1504/ijass.2007.017713 Erdogan, S. A., Šaparauskas, J., & Turskis, Z. (2019). A Multi-Criteria Decision-Making Model to Choose the Best Option for Sustainable Construction Management. Sustainability, 11(8), 2239. doi:10.3390/su11082239 Gharaibeh, H. M . (2014). Developing a Scoring Model to Evaluate Project Management Software Packages Based on ISO/IEC Software Evaluation Criterion. ournal of Software Engineering and Applications 7(1), 27-41. doi: 10.4236/jsea.2014.71004 Project management software evaluation by using the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method 101 Górecka, D. (2013). Multi-Criteria Decision Aiding in Project Management - Outranking Approach and Verbal Decision Analysis. Studia Ekonomiczne, 137, 11-38. Hassan, I. ul, Ahmad, N., Zuhaira, B. (2018). Calculating completeness of software project scope definition. Information and Software Technology, 94, 208-233. doi:10.1016/j.infsof.2017.10.010 Jafarnejad Chaghooshi, A., Arab, A., & Hosseini Dehshiri, S. J. (2016). A fuzzy hybrid approach for project manager selection. Decision Science Letters, 447–460. doi:10.5267/j.dsl.2016.1.001 Khoshnava, S. M., Rostami, R., Valipour, A., Ismail, M., & Rahmat, A. R. (2018). Rank of green building material criteria based on the three pillars of sustainability using the hybrid multi criteria decision making method. Journal of Cleaner Production, 173, 82–99. doi:10.1016/j.jclepro.2016.10.066 Kostalova, J., Tetrevova, L. (2014). Project Management and its Tools in Practice in the Czech Republic. Procedia - Social and Behavioral Sciences, 150, 678-689. doi:10.1016/j.sbspro.2014.09.087 Piengang, F.C.N., Beauregard, Y., Kenné, J-P. (2019). An APS software selection methodology integrating experts and decisions-maker’s opinions on selection criteria: A case study. Cogent Engineering (2019), 6: 1594509. doi:10.1080/23311916.2019.1594509 Puška, A. (2013). Comparative analysis of MCDM methods in investment decision. Berlin: LAP LAMBERT Academic Publishing Puška, A., Beganović I. A., Šadić, S. (2017). Uticaj subjektivnih procjena donosioca odluka na investicionu odluku. Serbian Journal of Engineering Managment, 2(2), 1- 13. doi:10.5937/SJEM1702001P Puška, A., Kozarević, S., Stević, Ž., Stovrag J. (2018). A New Way of Applying Interval Fuzzy Logic in Group Decision Making For Supplier Selection. Economic Computation and Economic Cybernetics Studies and Research, 52(2), 217-234, doi: 10.24818/18423264/52.2.18.13 Rouhani, S., & Zare Ravasan, A. (2016). Fuzzy TOPSIS evaluation approach for business process management software acquisition. Intelligent Automation & Soft Computing, 23(3), 459–468. doi: 10.1080/10798587.2016.1251017 Sajad, M., Sadiq, M., Naveed, K., Shahid Iqbal, M. (2016). Software Project Management: Tools assessment, Comparison and suggestions for future development. IJCSNS - International Journal of Computer Science and Network Security, 16(1), 31-42. Stanković, M.; Stević, Ž.; Das, D.K.; Subotić, M.; Pamučar, D. (2020). A New Fuzzy MARCOS Method for Road Traffic Risk Analysis. Mathematics 8, 457. doi: 10.3390/math8030457 Stević, Ž., & Brković, N. (2020). A Novel Integrated FUCOM-MARCOS Model for Evaluation of Human Resources in a Transport Company. Logistics, 4(1), 4. doi: 10.3390/logistics4010004 Puška et al./Oper. Res. Eng. Sci. Theor. Appl. 3 (1) (2020) 89-102 102 Stević, Ž., Pamučar, D., Puška, A., Chatterjee, P. (2020). Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement Alternatives and Ranking according to COmpromise Solution (MARCOS). Computers & Industrial Engineering, 140, article 106231. doi: 10.1016/j.cie.2019.106231 Wang, F.-K., Hsu, C.-H., & Tzeng, G.-H. (2014). Applying a Hybrid MCDM Model for Six Sigma Project Selection. Mathematical Problems in Engineering, 2014, 1-13. doi:10.1155/2014/730934 Zavadskas, E. K., Vainiūnas, P., Turskis, Z., Tamošaitienė, J. (2012). Multiple criteria decision support system for assessment of projects managers in construction. International Journal of Information Technology & Decision Making, 11(02), 501– 520. doi:10.1142/s0219622012400135 © 2020 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Project Management Software Evaluation by Using the Measurement of Alternatives and Ranking According to Compromise Solution (MARCOS) Method Adis Puška 1*, Ilija Stojanović 2, Aleksandar Maksimović 1, Nasiha Osmanović 2 1. Introduction 2. The Application of Multicriteria Analysis Methods in Project Management 2.1. The Measurement of Alternatives and Ranking According to Compromise Solution (MARCOS) method 3. Model and Methodology 4. Results 5. Sensitivity Analysis 6. Conclusion References