International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 17, 2021 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application Services: Multi-Criteria Decision-Making Approach https://doi.org/10.3991/ijim.v15i17.22509 Nasa Zata Dina1(*), Ria Triwastuti1, Mega Silfiani2 1Universitas Airlangga, Surabaya, Indonesia 2University of Gdańsk, Gdańsk, Poland nasazatadina@vokasi.unair.ac.id Abstract—In recent years, the use of ride hailing mobile application services is increasing exponentially. Customers’ expectation of these phone services var- ies and change dynamically as the needs of each individual also vary. Customer reviews about mobile application are honest, voluntary opinions; and these could become essential input for mobile application providers to measure satisfaction. However, managing a large number of reviews into actionable plans could be challenging. This study combines the Term Frequency-Inverse Document Fre- quency (TF-IDF) and Multiple-Criteria Decision-Making (MCDM)-VIKOR approach to process 600 reviews into a meaningful insight to enhance ride hail- ing mobile application services. The four-phase method analysis concluded that application ease of use and affordability are the most important aspects that most contribute to customers’ satisfaction in ride hailing mobile application services. Keywords—customer reviews, information and communication technology skills, multiple-criteria decision-making, phone service, term frequency-inverse document frequency, VIKOR 1 Introduction Mobile applications have become a primary tool used on a daily basis in this era of rapid technological advances. The estimated number of unique mobile subscribers was about 5 billion by 2020 globally [1]. In transportation sector, mobile applications were initially used mostly for navigation and location-based services. Recently, the use of transportation mobile applications varies from education, data collection, travel infor- mation, travel safety, route planning, and ride hailing. Ride hailing could be defined as a service that arranges a one-way trip in a short notice. The existence of this technology contributes to the changes of people’s life- style into more digital. The increasing demand for a ride hailing application includes taxis to commute to workplaces, and home shopping, food and goods delivery [2]. The most popular transportation mobile services in the United States (US) and 104 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i17.22509 mailto:nasazatadina@vokasi.unair.ac.id Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Europe are Uber, Lyft, Sidecar, and Carpool [3]. Meanwhile in Asia, three ride hailing applications—Didichuxing, Grab, and Go-Jek—have been listed in the top 10 Asia unicorns with the highest value and remarkable influence to people’s lives. These three-transportation applications value US$62 billion, USUS$14.3 billion, and $10 billion respectively [4]. The number of users of transportation mobile services grew exponentially in the last five years, but then the COVID-19 outbreak started in late 2019. This phenomenon has changed people’s life in almost every sector, including transportation. The enforce- ment of movement control order worldwide ceased most people’s mobility. In the first quarter of 2020, Uber and Lyft ridership in the US dropped between 70% and 80% [5]. The same trend happened in Indonesia, although the downturn was not as severe as the US, even the biggest mobile transportation applications—Grab and Go-Jek—reported a drop between 14% and 16% by March 2020 [6]. As the pandemic persist to date, the number of ride hailing customers has not rebound to the point it was before the pandemic. But as the lockdown restrictions in many countries have now been eased gradually, so the ridership is picking up, particularly for food delivery services within the same mobile application services [5][6]. As people’s habits and lifestyles now cannot be separated from digital platforms, and because market competition is becoming fiercer, customer expectations and satisfac- tion levels are also higher. People will have certain perception on some of ride hailing applications feature or attributes and expect more or less of a certain service availability within the applications. One of the best ways to find out about customers opinion is through the app reviews. Since mobile services’ users generally write their impres- sion by using their mobile devices rather than desktop computer, customer reviews in mobile applications tend to be concise and straightforward. It is because writing from a mobile device would be less convenient than using a computer keyboard. Customer reviews of mobile services application prefer to be clearer and explicit [7]. Customer reviews represent customers’ voice and so these are essential for mobile service providers in order to monitor and improve customer satisfaction. Customer reviews provide insights such as practical characteristics, superiorities, and flaws of certain mobile applications. From customers’ points of view, mobile application reviews will help them gain information of whether a certain mobile application can meet their needs, thus can help them decide to choose certain applications instead of the others. However, monitoring a big number of customer reviews could be a problem for both mobile application providers and customers themselves. For mobile service providers, analyzing millions of reviews into actionable plans will need a reliable meth- odology and much efforts. Meanwhile, for customers who aim to gather information about the applications, analyzing a large number of reviews may seem too daunting. As a result, people only examine global ratings when they first download a mobile applica- tion although these ratings are less helpful to understand customer satisfaction. Average ratings typically have bimodal distribution, where it is either extremely superior or extremely inferior, thus it might not be advantageous for the purpose of controlling and improving customer satisfaction [7]. iJIM ‒ Vol. 15, No. 17, 2021 105 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… This study presents customer satisfaction measurement based on term frequency– inverse document frequency (TF-IDF), then the computation of the value rank using a multicriteria decision making (MCDM), namely VIKOR. TF-IDF is used to weight factors from the consumer reviews. The weighting is essential to gain an accurate idea of the salient words from the frequency of appearance in the consumer reviews [8]. Meanwhile, VIKOR is a multi-criteria optimization that compromise ranking and solution from the initial weights [9][10]. Tang et al. [11], Gao et al. [12] and Zhang et al. [13] implemented MCDM and natural language processing (NLP) to mine public interests for each topic. However, the common natural language processing in that research only used term frequency (TF) to weight factor which will be used to create the MCDM matrix. Since the weight factor is important, then we consider using TF-IDF in NLP to gain accuracy of the weight factor. This study employed the integration of TF-IDF, MCDM and VIKOR approach on the application reviews to measure customers’ satisfaction in ride hailing mobile services prior and during the COVID-19 pandemic. MCDM is recognized as a highly reliable methodology for ranking multiple alterna- tives [9]. While these methods have been successfully applied in different areas of eco- nomics, energy management, transportation, human resources management and other domains, its integration with TF-IDF is relatively new, particularly in the field of busi- ness. The practical implication of the current study is for mobile application providers to gain insight of customer satisfactions and expectations of an ideal ride hailing mobile services. For customers, they can save time and efforts to understand other customers’ opinions about specific transportation mobile application services. 2 Methods Data collection and analysis on this study is classified into four steps. The first one is data collection and then it is followed by data processing to prepare the data before text mining will be done. Last, ranking model using VIKOR to determine the order of preference. The proposed system in this study is described by Figure 1. 2.1 Data collection Data collected in this study were obtained from Google Play Store website, accessed from https://play.google.com/. The interface of the website can be seen in Figure 2. The selected ride hailing mobile applications are Go-Jek, Grab and Didichuxing. The user interfaces from three selected ride hailing mobile applications can be seen in Figure 3. The online reviews from each mobile application were crawled and the specific variables were taken such as username, date, rating, and comments. The example of the data is described in Table 1. From four variables taken, we decided to use comments and ratings. The number of reviews from each application are 200, so in total there are 600 online reviews in this study. 106 http://www.i-jim.org Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Google Play Store Tokenize Data collection Data preprocessing Crawling Online Review for Ride Sharing Mobile Application Online Review Database Transform Cases Filter Stopwords Stemming Tag Words Type using Stanford POS Tagger Compile Dictionary of Features and Sentiment Words Text Mining VIKOR Calculating TF- IDF feature weight Determining weight Matrix Normalization of feature vectors Determining criteria weight Determining worst and best values f+ and f- Calculating separation measure S+ amd S- Calculating Q values Conducting sensitive analysis Fig. 1. Proposed system Fig. 2. Google play store review Fig. 3. User interface of ride hailing mobile application, (a) Go-Jek; (b) Grab; (c) Didichuxing iJIM ‒ Vol. 15, No. 17, 2021 107 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Table 1. Customer reviews data Application Comment Rating Didichuxing “Easy to use. Only negative is that I can’t cancel a ride when the driver decides he doesn’t want to take you any further and drops you off randomly before the destination and the app still shows that the the ride is in progress.” 5 Go-Jek “The app is great, but recently, I can’t access the Gogames menu, it just shows the promo picture and nothing else, I tried clearing the cache but nothing happened. Please fix this, everything else works fine except this one issue.” 4 Grab “Grab Transport matching seems inaccurate. I booked a car, ETA 3 mins, and it arrived 10 minutes later. That’s off by 3x. If I knew it will arrive 10 minutes later, I would’ve made other travel arrangements. This happened to me several times, and now I would not rely on Grab for my ad-hoc transport.” 3 2.2 Data pre-processing Before the online review data are processed in text mining, the data undergo a pre-processing for the removal of any irrelevant data text [14] and for data normaliza- tion [15]. There are four steps on the data pre-processing: 1) tokenization; 2) case trans- formation; 3) stop word filtration; 4) stemming. Tokenization separates the collection of information into individual words [16]. Verma et al. [16] use RapidMiner to process raw text into separated words. Tokenization step aims to cut the input string based on each constituent word. RapidMiner is used in this study for the tokenization. An exam- ple of this step can be seen in Table 2. Table 2. Example of input text Input Text Output of Tokenization ENGLISH customer service and presence in Over 400 Cities across ASIA ENGLISH customer service and presence in Over 400 Cities across ASIA After that, the individual words will be transformed into lower case so there is no uppercase in the data [17]. Case transformation is necessary to avoid confusion between similar words using uppercase and lowercase [18]. Mohadab et al. [17] and Gupta et al. [18] applied case transformation before doing filtering and stemming. Hence, the order of data pre-processing after tokenization is case transformation, filtration and stemming. The example of case transformation cases from Table 2 are “ENGLISH” to “english”; “Over” to “over”; “Cities” to “cities”; “ASIA” to “asia”. 108 http://www.i-jim.org Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Stop word filtration aims to remive stop words from the information. Since the lan- guage used on online reviews is English then we erase the English stop words from the review, the example of English stop words is ‘a/an’, ‘the’, ‘am/is/are’, ‘in’, ‘to’, ‘about’, et cetera. From the example in Table 2, the English stop words found include “and” and “in”. Both stop words were deleted. The last step of data pre-processing is stemming to reduce words to their stem words by erasing suffixes. Jivani [19] divided stemming algorithms into three types: truncating, statistical, and mixed. In this study, truncating method was applied to remove suffixes in plurals and then convert them into singular form. The exam- ple of stemming from the above sentence is “cities” into “city”. If the suffix in each word is reduced, then tagging the words into their part-of-speech type will be easier. Part-of-speech tags help to differ word types, and this can be seen in Table 3. 2.3 Text mining The most frequent words appear in a text could be identified through text mining [20]. The process starts with each word from the online reviews restored to its base form. Then, part-of-speech tagging is assigned (see Table 3). In this study, we classi- fied the stem words into two categories. The first category is attribute dictionary, and the second category is sentiment dictionary. The attribute dictionary consists of nouns while sentiment dictionary consists of other than nouns. In Table 3, verbs, adjectives, and adverbs are included in sentiment dictionary. The dictionaries are divided automat- ically based on part-of-speech tag using Stanford taggers [21]. Each category is called a dictionary because it consists of a lot of words like a dictionary. From the attribute dictionary, then the most frequent attributes are selected to be aspects (see Table 2). Each aspect will be used as measurement of customer satisfaction and these aspects will be named as A1, A2, A3, A4, A5, A6, A7, A8. Table 3. Part-of-speech tag Word Types POS Tag Dictionary Nouns NN (noun singular) Attribute Dictionary NNS (noun plural) NNP (noun proper) NNPS (noun proper plural) Verbs VB (verb base) Sentiment Dictionary VBD (verb past tense) VBG (verb present participle) VBN (verb past participle) Adjectives JJ (adjective) Sentiment DictionaryJJR (adjective comparative) JJS (adjective superlative) Adverbs RB (adverb) Sentiment DictionaryRBR (adverb comparative) RBS (adverb superlative) iJIM ‒ Vol. 15, No. 17, 2021 109 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… The simultaneous occurrence of attribute and sentiment word is calculated using TF-IDF to measure how relevant a given word is to a document in a collection of docu- ments [22][23][24]. The formula of the TF-IDF is shown in eq. 3, which is constructed from eq. 1 and 2. In this study term of presence is also calculated to count the number of aspect words that appear in each mobile application review. tfi,j = count (tj, dj) (1) log |{ : }|i i j D idf j t d = ∈ (2) where: ti : i-th word, dj: j-th documents containing ti, |D| : total number of documents. TF-IDF = tfi,j × idfi (3) Table 4. Aspects Aspects Description Application (A1) Ease of utilization Cancelation (A2) The experience of unilateral cancelation Payment (A3) Payment method options Cost (A4) Fare of the ride-hailing application Services (A5) Any facility offered by the application Complaint (A6) Unsatisfactory statement from user Message (A7) Communication between user and driver Route (A8) A way or course taken from a starting point to a destination. 2.4 VIKOR method VIKOR is multicriteria decision making (MCDM) because VIKOR helps to solve a problem by considering multicriteria [22]. It uses multicriteria ranking index to com- pare the proximity of each criterion so that the ideal alternative is obtained. Ranking index is derived by calculating the maximum group utility (Sj) and minimum individual regret (Rj) [23]. There are several steps to be completed using VIKOR, as follows: Step 1: Create the decision matrix. The decision matrix was formed from TF-IDF value for each aspect since in this study we used TF-IDF value. The TF-IDF value is calculated using eq. 1–3. Also, the word’s frequency of appearance is counted for each aspect. The formation of decision matrix is as follows: 110 http://www.i-jim.org Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… X A A A SN SN SN SN SN SN SN SN SN i m j n i ij in m mj mn =           1 11 1 1 1 1 where: Ai : i-th alternative, SNij : the value of j-th aspect for i-th alternative. Step 2: Normalized the decision matrix. The TF-IDF from each aspect from step 1 is normalized to a number between 0 and 1. The formulation for normalization is as shown in eq. 4. The word presence from step 1 will be normalized using eq. 5. The weighting scheme is implemented to measure the weighted aspects. x x x x x * min( ) max( ) ( ) = − − min (4) where: x* : normalization result, x : TF-IDF value. 1 ( ) ( ) k k m kk n d w n d = = ∑ (5) where: n dk �( ) : number of times k-th aspect occurs in document d, wk : weight of k-th aspect. Step 3: Determine the ideal negative and ideal positive solutions. The two referential sequences of positive and negative ideal solution are obtained using eq. 6 and 7. fj* = maxi fi, j (6) fj – = mini fi, j (7) where: fj*: positive ideal solution, fj –: negative ideal solution, fi, j: the value of normalized score of j-th aspects. iJIM ‒ Vol. 15, No. 17, 2021 111 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Step 4: Calculate new decision matrix with weight. The formula to assign weight for new decision matrix is written in eq. 8. w f f f f j j ij j ij ( )* * − − − (8) where: wj : weight of j-th aspect. Step 5: Count the utility measure and regret measure. Counting separation measure, as shown in eq. 9 and 10. S f f f f i j n j ij j ij = − −= −∑ 1 wj ( ) * * . (9) * * w ( ) max j j iji j ij f f R f f −  −  = −   (10) where: Si : utility value, Ri : maximum value of utilities. Step 6: Calculate the VIKOR index * *(1 ) * * i i i S S R R Q v v S S R R− − − −    = + −   − −    (11) where: Qi : VIKOR index, v : VIKOR index weight value, S* : maximum value of utilities, S– : minimum value of utilities, R* : maximum value of regret, R– : minimum value of regret. Step 7: List the order preference. The alternative solution with the least VIKOR index value is the perfect solution. 112 http://www.i-jim.org Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… 3 Results and discussion The first phase of VIKOR analysis is establishing decision matrix with TF-IDF value. From the calculated score using TF-IDF, a matrix of weighting score for each aspect is presented in Table 4. Also, there are word presence in the last row of the table. Word presence shows the frequency of each aspect’s appearance in the consumer reviews. In the second step, the scores from Table 4 are normalized between 0–1 then new decision matrix is developed as shown in Table 4. The weight from each aspect is obtained by using eq. 4. Step three, progression of determining ideal positive (fj *) and negative (fj –) solution is shown in Table 5. Each ideal solution is obtained by looking for the maximum and minimum number of normalized scores from Table 4. In step four, based on eq. 8, a new decision matrix is arranged to consider the weight of each aspect. The result is presented in Table 6. After obtaining weighted decision matrix, then step 5 is computed based on eq. 9 and 10 to calculate utility measure and regret measure. From this pro- cess, the value of Si, Ri and Qi could be obtained for each aspect, as shown in Table 7. In step 6, the Q-value is calculated using 10 different v-value. The number used as v-values are between 0–1. The v-value is a weighting factor to examined the result. The decision is made by comparing the results. Table 5. Decision matrix with TF-IDF value A1 A2 A3 A4 A5 A6 A7 A8 Didichuxing 11.922 4.429 7.169 16.207 16.368 8.195 5.639 5.978 Go-Jek 18.052 0.666 10.429 3.817 10.390 13.207 6.946 3.354 Grab 14.561 0.726 5.284 0.495 0.000 7.429 1.671 0.000 Word presence 464 19 104 68 117 103 52 32 Table 6. Normalized scores A1 A2 A3 A4 A5 A6 A7 A8 Didichuxing 0.660 0.245 0.397 0.898 0.907 0.454 0.312 0.331 Go-Jek 1.000 0.037 0.578 0.211 0.576 0.732 0.385 0.186 Grab 0.807 0.040 0.293 0.027 0.000 0.412 0.093 0.000 Weight 0.484 0.020 0.108 0.071 0.122 0.107 0.054 0.033 Table 7. Positive ideal solution A1 A2 A3 A4 A5 A6 A7 A8 fj* 1.000 0.245 0.578 0.898 0.907 0.732 0.385 0.331 fj – 0.660 0.037 0.293 0.027 0.000 0.412 0.093 0.000 iJIM ‒ Vol. 15, No. 17, 2021 113 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Table 8. Decision matrix with weight A1 A2 A3 A4 A5 A6 A7 A8 Didichuxing 0.484 0.000 0.069 0.000 0.000 0.093 0.013 0.000 Go-Jek 0.000 0.020 0.000 0.056 0.045 0.000 0.000 0.015 Grab 0.276 0.019 0.108 0.071 0.122 0.107 0.054 0.033 Table 9. Q values Si Ri Qi v=0 v=0.1 v=0.2 v=0.3 v=0.4 v=0.5 v=0.6 v=0.7 v=0.8 v=0.9 v=1 Didichuxing 0.659 0.484 1.000 0.980 0.960 0.940 0.919 0.899 0.879 0.859 0.839 0.819 0.799 Go-Jek 0.135 0.056 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Grab 0.791 0.276 0.513 0.562 0.611 0.659 0.708 0.757 0.805 0.854 0.903 0.951 1.000 From the value of Ri depicted in Table 8, the maximum value of utility shows the important aspects of ride hailing applications, where A1 (Application) and A4 (Cost) are the top among those listed aspects. From this result, it could be inferred that in order to improve the customer satisfaction, each ride hailing mobile application should take notice on the ease of utilization and the fare. Meanwhile, the result of the ranking model is greatly influenced by v-value as weighting factor, if the decision maker sets v-value as v=0 until v=0.7 then the rank of each ride hailing mobile application is sorted as Go-Jek > Grab > Didichuxing. The rank is sorted based on Q-values as shown in Table 9, where the smallest number means the best solution. However, if the v-value is set to 0.8–1 then the rank is changed into Go-Jek > Didichuxing > Grab. It is worth noting that in all v-value scenario, Go-Jek is in the top position among the three ride hailing applications. Therefore, Go-Jek could be considered as the ride-hailing mobile appli- cation benchmark because the rank of Go-Jek remains at first position for 10 possible v-value. The impact of v-value scenario toward VIKOR rank is known as sensitivity. Sensitivity analysis is conducted to rank the impact level of v-value as shown in Figure 4. Sensitivity analysis determines how different value of an independent vari- able affects dependent variable [27]. The ranking of Go-Jek was not altered by v-value. It means that the Go-Jek has higher customer satisfaction in terms of both S* and R–. On the contrary, the ranking of Grab was improving when v-value is decreased. It reveals that Grab has higher customer satisfaction when it focuses on R-, while Didichuxing is the opposite. The ranking of Didichuxing increases when the importance of S* was increased. The study can convert text-based customer opinions into a matrix and then it is ranked based on its criteria. The result is understandable, not time consuming and can provide an insight about the ride hailing mobile application. The study uses ride hailing mobile application as a case study to show the effectiveness of the proposed framework. The similar research by Liang et al. [28] uses hotel booking as a case study to explain the practicability and effectiveness of the method and a model of integrated MCDM. 114 http://www.i-jim.org Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… Distribution linguistic VIKOR was developed to rank travel hotels based on online reviews [28]. The similarity from the proposed method in this study is that informa- tion loss can be avoided with the integration of MCDM-VIKOR method because all aspects written in online reviews are used as attributes in the ranking process. Liang et al. [28] use common term-frequency (TF); while this study uses TF-IDF to measure a word’s relevance in a document in a collection of documents. TF-IDF can identify which words carry more information as opposed to those frequent words with lesser information. The limitation of this research is that the number of mobile ride hailing applica- tion services is limited to the three most well-known mobile ride hailing application services; online reviews of each mobile application is also limited to 200 reviews and MCDM method used in this study is only one i.e., VIKOR. The example of another MCDM methods is AHP, TOPSIS, ELECTRE, WSM, WPM [29]. In addi- tion, each country has different types of ride hailing mobile application and the language used in online reviews may vary. It is important that further studies to be carried out using bigger size of data and analyze the online reviews in different languages other than English. 1 2 3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 R an k v-Value Didichuxing Gojek Grab Fig. 4. Sensitivity analysis 4 Conclusions This study proposes multicriteria decision matrix to measure customer satisfaction of ride hailing mobile application. The technique used in MCDM is VIKOR combined with TF-IDF value as decision matrix. Based on the results, we can gain an insight about the customer opinions as expressed in the online review data in this study Ease of use and fare are the most demanded utility of the ride hailing applications. It means that ride hailing mobile application should highly consider these aspects to enhance their services as well as gaining customer satisfaction. Besides, it is also concluded that Go-Jek is ranked at first position among other ride hailing applications. The result shows that the proposed ranking model is effective to measure customer satisfaction iJIM ‒ Vol. 15, No. 17, 2021 115 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… without conducting a direct survey so that time and effort could be minimized. For future research, the proposed ranking model needs to be validated by comparing it with another MCDM method and using more extensive data so that its reliability could be tested. From those limitations of this study, then future study should be conducted to obtain better results and a better ranking model. 5 References [1] S. Siuhi, and J. Mwakalonge, “Opportunities and challenges of smart mobile applications in transportation,” Journal of Traffic and Transportation Engineering, vol. 3, no. 6, pp. 582–592, 2016. doi: https://doi.org/10.1016/j.jtte.2016.11.001 [2] A. Singh, R. Gupta, and A. Kumar, “A strategic focus on adoption of mobile taxi applica- tion-a move towards digital India,” International Journal of Advanced Science and Technol- ogy, vol. 29, no. 5 (Special Issue), pp. 352–364, 2020. [3] M. A. Javid, et al., “Travelers’ Attitudes Toward Mobile Application–Based Public Transport Services in Lahore,” SAGE Open, vol. 11, no. 1, 2021. doi: https://doi.org/ 10.1177/2158244020988709 [4] AFM Editorial Office, “Top 10 Asia unicorns disrupting markets at high-speed Top 10 Asia Unicorns by Valuation,” October 22, 2020. [Online], Available: https://www.asiafundman- agers.com/int/asia-unicorns-a-list-to-watch/. [Accessed January 20, 2021]. [5] M. Goldstein, “What Is The Future For Uber And Lyft After The Pandemic?” Forbes, pp. 4–13, 2020. [Online], Available: https://www.forbes.com/sites/michaelgold- stein/2020/07/27/what-is-the-future-for-uber-and-lyft--after-the-pandemic/?sh=56802be- a3bc8. [Accessed January 25, 2021]. [6] E. A. Eloksari, “Ride-hailing apps rely on deliveries during pandemic,” The Jakarta Post, pp. 1–7, 2020. [Online], Available: https://www.thejakartapost.com/news/2020/04/03/ride- hailing-apps-rely-on-deliveries-during-pandemic.html. [Accessed January 19, 2021]. [7] D. Kang, and Y. Park, “Review-based measurement of customer satisfaction in mobile ser- vice: Sentiment analysis and VIKOR approach,” Expert Systems with Applications, vol. 41, no. 4 (Part 1), pp. 1041–1050, 2014. doi: https://doi.org/10.1016/j.eswa.2013.07.101 [8] A. A. Jalal, and B. H. Ali, “Text documents clustering using data mining techniques,” International Journal of Electrical and Computer Engineering, vol. 11, no. 1, pp. 664–670, 2021. doi: https://doi.org/10.11591/ijece.v11i1.pp664-670 [9] S. A. A. Alrababah, K. H. Gan, and T. P. Tan, “Comparative analysis of MCDM methods for product aspect ranking: TOPSIS and VIKOR,” In Proc. 8th International Conference on Information and Communication Systems, 2017, pp. 76–81. doi: https://doi.org/10.1109/ IACS.2017.7921949 [10] S. Çalı, and Ş. Y. Balaman, “Improved decisions for marketing, supply and purchasing: Mining big data through an integration of sentiment analysis and intuitionistic fuzzy multi criteria assessment,” Computers and Industrial Engineering, vol. 129, pp. 315–332, 2019. doi: https://doi.org/10.1016/j.cie.2019.01.051 [11] M. Tang, and H. Liao, “Multi-attribute large-scale group decision making with data min- ing and subgroup leaders: An application to the development of the circular economy,” Technological Forecasting and Social Change, vol. 167, pp. 120719, 2021. https://doi. org/10.1016/j.techfore.2021.120719 [12] K. Gao, L. Sun, Y. Yang, F. Meng, and X. Qu, “Cumulative prospect theory coupled with multi-attribute decision making for modeling travel behavior,” Transportation Research Part A: Policy and Practice, vol. 148, pp. 1–21, 2021. https://doi.org/10.1016/j.tra.2021.03.025 116 http://www.i-jim.org https://doi.org/10.1016/j.jtte.2016.11.001 https://doi.org/10.1177/2158244020988709 https://doi.org/10.1177/2158244020988709 https://doi.org/10.1016/j.eswa.2013.07.101 https://doi.org/10.11591/ijece.v11i1.pp664-670 https://doi.org/10.1109/IACS.2017.7921949 https://doi.org/10.1109/IACS.2017.7921949 https://doi.org/10.1016/j.cie.2019.01.051 https://doi.org/10.1016/j.techfore.2021.120719 https://doi.org/10.1016/j.techfore.2021.120719 https://doi.org/10.1016/j.tra.2021.03.025 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… [13] C. Zhang, M. Zhao, M. Cai, and Q. Xiao, “Multi-stage multi-attribute decision making method based on online reviews for hotel selection considering the aspirations with different development speeds,” Computers & Industrial Engineering, vol. 143, pp. 106421, 2020. https://doi.org/10.1016/j.cie.2020.106421 [14] M. AbdelFattah, D. Galal, N. Hassan, D. S. Elzanfaly, and G. Tallent, “A Sentiment Analysis Tool for Determining the Promotional Success of Fashion Images on Instagram,” Interna- tional Journal of Interactive Mobile Technologies, vol. 11, no. 2, pp. 67–73, 2017. https:// doi.org/10.3991/ijim.v11i2.6563 [15] A. R. Prananda, and I. Thalib, “Sentiment Analysis for Customer Review: Case Study of GO-JEK Expansion,” Journal of Information Systems Engineering and Business Intelligence, vol. 6, no. 1, pp. 1–8, 2020. https://doi.org/10.20473/jisebi.6.1.1-8 [16] T. Verma, Renu, D. Gaur, “Tokenization and Filtering Process in RapidMiner,” Interna- tional Journal of Applied Information Systems, vol. 7, no. 2, pp. 16–18, 2017. https://doi. org/10.5120/ijais14-451139 [17] M. E. Mohadab, B. Bouikhalene, and S. Safi, “Automatic CV processing for scientific research using data mining algorithm,” Journal of King Saud University - Computer and Information Sciences, vol. 32, no. 5, pp. 561–567, 2020. https://doi.org/10.1016/j. jksuci.2018.07.002 [18] G. Gupta, and S. Malhotra, “Text Document Tokenization for Word Frequency Count using Rapid Miner (Taking Resume as an Example),” In. Proc. International Conference on Advancements in Engineering and Technology, 2015, pp. 24–26. [19] A. G. Jivani, “A Comparative Study of Stemming Algorithms,” Int. J. Comp. Tech. Appl., vol. 2, no. 6, pp. 1930–1938, 2011. [20] N. Z. Dina, “Tourist sentiment analysis on TripAdvisor using text mining: A case study using hotels in Ubud, Bali,” African Journal of Hospitality, Tourism and Leisure, vol. 9, no. 2, pp. 1–10, 2020. [21] K. Toutanova, D. Klein, C. Manning, and Y. Singer, “Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network,” In: Proceedings of HLT-NAACL, Canada, 2003. Edmonton: ACM, pp. 252–259, 2003. https://doi.org/10.3115/1073445.1073478 [22] M. Kumari, A. Jain, and A. Bhatia, “Synonyms Based Term Weighting Scheme: An Extension to TF-IDF,” Procedia Computer Science, vol. 89, pp. 555–561, 2020. https://doi.org/10.1016/j.procs.2016.06.093 [23] S. Qaiser, N. Yusoff, F. K, Ahmad, and R. Ali, “Sentiment Analysis of Impact of Technology on Employment from Text on Twitter,” International Journal of Interactive Mobile Technol- ogies, vol. 14, no. 7, pp. 88–103, 2020. https://doi.org/10.3991/ijim.v14i07.10600 [24] N. Z. Dina, R. T. Yunardi, and A. A. Firdaus, “Utilizing Text Mining and Feature-Sentiment-Pairs to Support Data-Driven Design Automation Massive Open Online Course,” International Journal of Emerging Technologies in Learning, vol. 16, no. 1, pp. 134–151, 2021. https://doi.org/10.3991/ijet.v16i01.17095 [25] H. Haleh, and A. Hamidi, “A fuzzy MCDM model for allocating orders to suppliers in a supply chain under uncertainty over a multi-period time horizon,” Expert Systems with Appli- cations, vol. 38, no. 8, pp. 9076–9083, 2011. https://doi.org/10.1016/j.eswa.2010.11.064 [26] J. J. H. Liou, C. Tsai, R. Lin, and G. Tzeng, “A modified VIKOR multiple-criteria decision method for improving domestic airlines service quality,” Expert Systems with Applications, vol. 17, no. 2, pp. 57–61, 2011. https://doi.org/10.1016/j.jairtraman.2010.03.004 [27] B. Muñoz, M. G. Romana, and J. Ordóñez, “Sensitivity analysis of multicriteria decision making methodology developed for selection of typologies of earth-retaining wall in an urban highway,” Transportation Research Procedia, vol. 18, pp. 135–139, 2016. https://doi. org/10.1016/j.trpro.2016.12.019 iJIM ‒ Vol. 15, No. 17, 2021 117 https://doi.org/10.1016/j.cie.2020.106421 https://doi.org/10.3991/ijim.v11i2.6563 https://doi.org/10.3991/ijim.v11i2.6563 https://doi.org/10.20473/jisebi.6.1.1-8 https://doi.org/10.5120/ijais14-451139 https://doi.org/10.5120/ijais14-451139 https://doi.org/10.1016/j.jksuci.2018.07.002 https://doi.org/10.1016/j.jksuci.2018.07.002 https://doi.org/10.3115/1073445.1073478 https://doi.org/10.1016/j.procs.2016.06.093 https://doi.org/10.3991/ijim.v14i07.10600 https://doi.org/10.3991/ijet.v16i01.17095 https://doi.org/10.1016/j.eswa.2010.11.064 https://doi.org/10.1016/j.jairtraman.2010.03.004 https://doi.org/10.1016/j.trpro.2016.12.019 https://doi.org/10.1016/j.trpro.2016.12.019 Paper—TF-IDF Decision Matrix to Measure Customers’ Satisfaction of Ride Hailing Mobile Application… [28] X. Liang, P. Liu, and Z. Wang, “Hotel Selection Utilizing Online Reviews: A Novel Decision Support Model based on Sentiment Analysis and DL-VIKOR Method,” Technological and Economic Development of Economy, vol. 25, no. 6, pp. 1139–1161, 2019. https://doi. org/10.3846/tede.2019.10766 [29] Emovon, and O. S. Oghenenyerovwho, “Application of MCDM method in material selection for optimal design: A review,” Results in Materials, vol. 7, pp. 1–21, 2020. https:// doi.org/10.1016/j.rinma.2020.100115 6 Authors Nasa Zata Dina is a lecturer in the Department of Engineering, Faculty of Voca- tional Studies at Universitas Airlangga, Surabaya, Indonesia. She obtained her Mas- ter of Science in Computer Science from Asian Institute of Technology, Thailand and Institut Teknologi Sepuluh Nopember, Indonesia. Her research interests include data mining, text mining, and engineering education. She is also an editorial board member of Journal of Information Systems Engineering and Business Intelligence. Ria Triwastuti is an Indonesian business scientist and educator, at Department of Business, Faculty of Vocational Studies, Universitas Airlangga, Surabaya, Indonesia. She obtained her Master of International Business from University of Adelaide, Aus- tralia. Her research interest includes risk management. She is also the Associate Editor of The International Journal of Applied Business. Mega Silfiani is an Indonesian scientist and a lecturer. Currently, she is pursuing Doctoral Degree in Department of Economics, University of Gdańsk, Gdańsk, Poland. Previously, she obtained her Master of Science in Industrial and Manufacturing Engi- neering from Asian Institute of Technology, Thailand and Institut Teknologi Sepu- luh Nopember, Indonesia. Her research interests are Economic Statistic and Applied Statistic. Article submitted 2021-03-08. Resubmitted 2021-06-27. Final acceptance 2021-06-27. Final version published as submitted by the authors. 118 http://www.i-jim.org https://doi.org/10.3846/tede.2019.10766 https://doi.org/10.3846/tede.2019.10766 https://doi.org/10.1016/j.rinma.2020.100115 https://doi.org/10.1016/j.rinma.2020.100115