INT J COMPUT COMMUN, ISSN 1841-9836 7(5):892-899, December, 2012. Mobile Network QoE-QoS Decision Making Tool for Performance Optimization in Critical Web Service C. Lozano-Garzon, C. Ariza-Porras, S. Rivera-Diaz, H. Riveros-Ardila, Y. Donoso Carlos Lozano-Garzon, Christian Ariza-Porras Sebastián Rivera-Díaz, Horacio Riveros-Ardila, Yezid Donoso Universidad de los Andes, Bogotá, Colombia E-mail: ca.lozano968@uniandes.edu.co cf.ariza975@uniandes.edu.co, s.rivera57@uniandes.edu.co lh.riveros102@uniandes.edu.co, ydonoso@uniandes.edu.co Abstract: Regardless of the type of service that a company offers the customer satisfaction is a factor for success, if these services are in a highly competitive environment. This situation encourages companies to develop strategies to improve the Quality of the Experience (QoE) of their users. Strategies include improving their processes, or in- frastructure for provisioning the services. Take these kind of decisions is very difficult because they ignore how the Key Performance Indicators (KPI) services are correlated with the information about user experience. This problem is approached from the perspective of mobile telecom operators, who have addressed this challenge through the Quality of Service (QoS) concept. Unfortunately, the QoS is only characterized by technical aspects, the user’s criteria are not included. Into a highly competitive environment, the user’s loyalty is a key component to be considered in the opera- tor’s development plan. Nowadays, the mobile telecom operators focus their efforts to ensure not only the QoS but also the QoE. The aim of this paper was the develop a decision making tool that allows the mo- bile telco operators support their determinations about the maintenance of network infrastructure, as well as the expansion of the same, specifically for their critical web services; based in a correlated information between QoS and QoE. This tool was devel- oped on the basis of the Pseudo Subjective Quality Assessment (PSQA) methodology. Keywords: Decision making tool, Pseudo subjective quality assessment, Quality of experience, Quality of service, Web services. 1 Introduction The rapid evolution that has made the telecommunications industry in the world, represented in technological development in networks and the emergence of IP as a fundamental part of both fixed and mobile networks, has led the sector to a converged environment that enables businesses to provide new services. This new environment base its performance in what is now known as Next Generation Mobile Networks (NGMN) which is intended to support the growing needs of both technical and quality demands for new services.This environment generates new challenges not only in the design and implementation of the network, also this should allow the provision of services independent of location, time or device from accessing this and besides challenges in the way that operators can implement mechanisms to ensure quality of Service (QoS) and quality of experience (QoE) under these different technology platforms. The term QoS is widely used in the environment for communications networks, it was defined by the ITU-T [1] as the collective effect of service performance which determine the satisfaction of a service user. Associated with the conceptualization of QoS Harry in [2] defines three concepts of QoS: intrinsic QoS perceived QoS and evaluated QoS, the definite relationship between these three concepts are general QoS model proposed by the ITU-T. Copyright c⃝ 2006-2012 by CCC Publications Mobile Network QoE-QoS Decision Making Tool for Performance Optimization in Critical Web Service 893 Stankiewicz, Cholda, & Jajszczyk in [3] describes the intrinsic QoS, it is known as "network performance" by the ITU and ETSI in Recommendation E.800, it covers all the features of service determined by the efficiency of the network. The intrinsic QoS is key to the quality perceived and evaluated by the customer. The perceived QoS reflects the customer experience in using a particular service through the ITU Recommendation G.1000 has four important perspectives: the QoS required by the client, the QoS offered by the provider, the QoS achieved by the supplier and the QoS perceived by the customer. Finally the evaluated QoS starts when the customer decides whether to continue using a service or not, this decision depends on the perceived quality, the price of the service, and supplier responses (problems and complaints), the ITU defines the guidelines of Quality of Experience (QoE) in Recommendation P.10. Even though QoS and QoE measurements are quite different, they have a high degree of correlation; nevertheless, some mobile operators have not yet implemented tools for incorporating the "feel" of a user based on the QoS parameters measured for a specific service. The mobile phone companies have improved the deployment and delivery of their products influenced by the quality of service (QoS), regardless of user perception. However, the mobile telco operators know that user satisfaction is a key success factor for the loyalty and positioning of the company against its competitors. This fact has led these operators to develop strategies for adoption of the perception of its users in their decision processes to the tuning of their infrastructure for the provision of services. This work aims to provide to mobile telecom operators a QoE-QoS decision making tool that will allow them to support their determinations about the maintenance of network infrastructure, as well as the expansion of the same, specifically for critical web services. The remainder of this paper is organized as follows. In Section 2 shown some works related with the measurement of quality of experience and the correlation of it with quality of service for some specific services. The PSQA methodology is presented in Section 3. The proposed methodology used to develop the QoS-QoE decision making tool is presented in Section 4 and the implementation of this methodology is described in Section 5. Finally, the experimental results of the proposed tool are presented in Section 6 and the Section 7 presented the conclusions and future work. 2 Related Works Stankiewicz, Cholda, and Jajszczyk in [3] notes that QoE assessment methods could be clas- sified into Qualitative (Subjective) and Quantitative (Objective) Methods. Qualitative methods are built with the participation of people, a representative sample of the population, whom used a particular service. In these methods the service is assessed in a controlled environment and people fill out a survey with numerical values qualification. Quantitative Methods provide a QoE assessment based on the measurement of several parameters related with service quality indicators in the signal at the output of the transmission channel. However, the Institut National de Recherche en Informatique et en Automatique (INRIA) in France proposed a Hybrid Method between subjective and objective assessments of QoE called PSQA [4]. Other papers examine the relation between QoS parameters and user perception. For ex- ample the research generated by Telenor in [5] about the conceptual difference between QoS, considered the quality associated with technical performance parameters; and QoE understood as a measure of user performance, based on objective and subjective measures of the use of Information and Communications Technology (ICT) product or service. In [5], [6], and [7] it is evident that between these two concepts clear relationship that allows us to express the quality of the experience in terms of quality of service, which is the starting point for further studies made in order to establish a relationship or a relational model between the two. Some of these studies 894 C. Lozano-Garzon, C. Ariza-Porras, S. Rivera-Diaz, H. Riveros-Ardila, Y. Donoso have focused on the relationship can be identified for a specific service such as the IPTV [8] and [9], transmission media [10] or public Internet [11], other studies have sought to establish a generic model of relationship between them as shown in [12], [13] and [14]. As a result of these studies, progress has been made in studies seeking to establish an objec- tive measure based on a subjective view by users. Within the work developed in this field we find the following: [15] focused on the development of the concept of the quality of experience focused on the measurement and communication requirements for industrial use, [16] measure- ment studies quality of experience based on ontologies and [17] which presents a look at the main techniques for measuring the quality of experience focused on the methodologies and tools available free. Finally in [18] the authors propose a new methodology called Pseudo-Subjective Quality Assesment (PSQA) based on Random Neural Networks, to quantify the quality of a video or audio transmission over the Internet. They discuss the results concerning PSQA-based dynamic quality control and conversational quality assessment. A general system developed to evaluate QoE on IP networks was shown in [19]. Their system architecture is designed to be capable of emulating multi agent networks and dynamically changing conditions, in a Web Browsing QoE experiment. The experiment was conducted on the basis of ITU-T Recommendation G.1030 , and aimed to update the perceptual model provided in this Recommendation to today context. As can be seen both of these studies: relationship between Quality of Service and Quality of Experience, as well as metrics for quality of experience; are an important part of an environment using All-IP network. 3 Pseudo-Subjective Quality Assessment (PSQA) Knowing that mobile operators require a method that allows them: correlate QoS parame- ters and subjective perception of the user, it can be used without creating a new testbed, and generating a set of reports that support their decisions about network infrastructure. After making a comparison between the existing metrics the Hybrid Model (specifically PSQA metric) was selected because it takes the best of the subjective and objective models, the results are in terms of Mean Opinion Score (MOS), is a not intrusive method, obtains real-time data and its implementation phase is low at the time. This model is divided into three big steps: Firstly the application of a subjective evaluation in a controlled environment where the samples are distorted in periods of time; in the second step the samples go through a statistical process where the elements out of range are detected and removed; and finally the results are used to train a statistical learning tool, a random neural network (RNN), that learns the correlation between configurations and MOS values defined, related among the parameters that cause distortion and the perceived quality. 4 Proposed methodology for development of QoE-QoS Decision Making Tool Based on the PSQA methodology we propose an adaptation of it, in order to develop a making decision tool that allow us to combine the knowledge of end-user experience and technical parameters values for the decision making in mobile operators. It is hoped that this new model assigns to the samples (QoS Parameters) a QoE value very close to the value that an average human observer would give.(see Figure 1). Mobile Network QoE-QoS Decision Making Tool for Performance Optimization in Critical Web Service 895 Figure 1: Flow Chart of QoE-QOS Decision Making Tool 4.1 Development of Subjective Test for assessment Web Services When we started the development of this work, we didn’t find any documented work related to the existence of a MOS test oriented to web services. For this reason, we decided the developing of a subjective test for critical web services based on some MOS existing tests. For the development of this test, we considered the following web contexts: a text-only page, a page with images and text, a video and download a file. 4.2 Determination of the Population Sample and Statistical analysis of the test Based in [20] the number of samples must be calculated so as to ensure a confidence interval of at least 95% and an error no greater than 5%, taking into consideration the following formula: n = (z1− α 2 )2 a2 · ( s mean(x) )2 (1) where n is the number of samples, z1− α 2 is the 1 − α 2 percentile of the standard normal distribution, s is the expected standard deviation, mean(x) is the expected mean value, and a is the relative accuracy. After we collect the samples through the designed test application, the results are passed through a statistical process in order to detect and remove users who present data out of range. 4.3 Development of agents to gathering QoS parameters The agents are the responsible of making the measurements of the QoS parameters defined for the Web browsing services. These parameters were selected according to [20]and are: bandwidth, latency, signal strength, trademark of device and the cell where the device is located. Another important characteristic of these agents is the need to send the measurements col- lected to the server in order to use these in a first time like initial information to training the 896 C. Lozano-Garzon, C. Ariza-Porras, S. Rivera-Diaz, H. Riveros-Ardila, Y. Donoso neural network and later as the principal source of information to calculate the QoE through the neural network. 4.4 Training of the Neural Network Given that the neural network should behave as a classifier, we propose the use of a multilayer perceptron whose input will correspond to the five (5) QoS parameters defined above and the output will correspond to one (1) value of QoE (excellent, very good, good, fair, or poor). Once the neural network has been trained and validate, it is hoped that the results produced will be very similar to the results of people’s subjective tests. 4.5 Storage of information In order to maintain a historical record of both: the values of QoS parameters collected and the QoE values calculated; it is necessary to implement a repository of information. This repository will be the primary source of information for the generation of reports that will support decision making. 4.6 Development of report manager The report manager should enable operators to generate the required documentation for making decisions regarding the maintenance of network infrastructure, as well as its expansion, specifically for critical web services. Some of the basic reports considered are the following: information of QoE by base station or by the device type and the correlation between QoS parameters and the estimated QoE. Also the operators need that the generator can produce new reports easily according to their requirements. 5 QoE-QoS Decision Making Tool implementation As a first step towards the implementation of this tool we designed the software architecture to be used, it is built by six (6) components: the mobile agent, the agent listener, the persistence component, the classifier, the report generator and the presentation component. These compo- nents and their connections can be seen in Fig. 2. Additionally, in order to support connectivity between the agents and the server, we propose a client - server network architecture of three layers (See Fig. 3). The implementation of the tool was performed according to previously pro- posed architectures. In the server component is used, among other tools: Weka for the classifier, webservices developed to receive data from agents and classified using the trained models and BIRT as a development tool for reports and report viewer. While the mobile agent components were implemented in Java. Figure 2: Software Architecture Connection Diagram Mobile Network QoE-QoS Decision Making Tool for Performance Optimization in Critical Web Service 897 Figure 3: Network Infrastructure Proposed 6 Experimental Results As a first step we performed the collection of about 120 subjective tests in 5 different cells of the operator. Subsequently we made a statistical analysis on data collected and all data that were outside three times the standard deviation were removed, because they were atypical behaviors that could influence the model training. With the remaining surveys, we conducted the relationship with data taken by the agents and estimated the value of QoE general survey taking an average of the evaluations of the test (text, text and images, video and downloads) and an average compared against the modified test. Once the data were purged we proceeded to do the training of the neural network. The classification algorithm used was the multilayer perceptron with 50 nodes in the hidden layer and 5000 epochs (times) as a limit. From 50 additional samples, which are selected to valid the model, after executing the classifier (neural network) had a success rate in the classification of 90% aand the mean absolute error is close to 4.7%. In Fig 4 we show the relationship between the calculated QoE and QoE assessment of the user to test Video. Figure 4: Relationship between the QoE assesment calculated and the test user to the video 7 Conclusions and Future Works In conclusion, through the decision making tool developed the mobile telecom operators could estimate the users’ subjective opinion based in network QoS parameters. This information allows the generation of specific reports with the aim of supporting decisions oriented to prevent product or service rejections by the users. Some of the most important decisions are related with the determinations about: the tuning of network infrastructure, the expansion of this infrastructure, the use of some specific equipment, among others. As future work we plan to work in two actions, the first is related to the exploration of other training algorithms for neural network that allows us to achieve better results, and the second seeks to expand this study to other data transmission services. 898 C. Lozano-Garzon, C. Ariza-Porras, S. Rivera-Diaz, H. Riveros-Ardila, Y. Donoso Bibliography [1] ITU - T. , Terms and Definitions Related to Quality of Service and Network Performance including Dependability, International Telecommunication Union., Recommendation E.800, 1995. [2] W. C. Hardy, QoS: Measurement and Evaluation of Telecommunications Quality of Ser- vice:Baffins Lane, Chichester, United Kingdom: John Wiley & Sons, Ltd., 2001. [3] R. Stankiewicz, P. Cholda, and A. Jajszczyk, QoX: What is It Really?, IEEE Communications Magazine, 49(4):148 - 158, 2011. [4] G. Rubino. The PSQA project. [Online]. http://www.irisa.fr/armor/lesmembres/Rubino/myPages/psqa.html [5] B. Hestnes, P. Brooks, and S. Heiestad, QoE (Quality of Experience) - measuring QoE for improving the usage of telecommunication services, Telenor, Research Report 2009. [6] A. van Moorsel, Metrics for the Internet Age: Quality of Experience and Quality of Business, Hewlett - Packard Laboratories, Palo Alto, California, USA, HPL-2001-179, 2001. [7] K. Bharrathsingh, Quality of experience as an integral part of network engineering, Focus in Convergence , vol. 1, February 2005. [8] J. Kim, T.W. Um, Ryu W., and B. Sun Lee, Heterogeneous Networks and Terminal-Aware QoS/QoE- Guaranteed Mobile IPTV Service, IEEE Communications Magazine, 46(5):110 - 117, 2008. [9] H.J. Kim and S.G. Choi, A Study on a QoS/QoE Correlation Model for QoE Evaluation on IPTV Service, in The 12th International Conference on Advanced Communication Technology (ICACT 2010), Gangwon-Do, Korea, 2:11077 - 1382, 2010. [10] M. Siller and J.C. Woods, QoS Arbitration for Improving the QoE in Multimedia Trans- mission, Int. Conf. on Visual Information Engineering (VIE 2003) , 238 - 241, 2003. [11] S. Khirman and P. Henriksen, Relationship between Quality-of-Service and Quality-of- Experience for Public Internet Service, Passive and Active Measurement Conference, Palo Alto, California, USA, 1 - 6, 2002. [12] M. Fiedler, T. Hossfeld, and Phuoc Tran-Gia, A Generic Quantitative Relationship between Quality of Experience and Quality of Service, IEEE Network, 24(2):36 -41, March - April 2010. [13] H.J. Kim et al., The QoE Evaluation Method through the QoS-QoE Correlation Model, Fourth Int. Conf. on Networked Computing and Advanced Information Management (NCM ’08), 2:719-725, 2008. [14] C. Guo, Y. Liu, and Y. Liu H. Du, Research on relationship between QoE and QoS based on BP Neural Network, IEEE Int. Conf. on Network Infrastructure and Digital Content (IC-NIDC 2009), 312 - 315, 2009. [15] P. Brooks and B. Hestnes, User Measures of Quality of Experience: Why Being Objective and Quantitative Is Important, IEEE Networks, 24(2): 8 - 13, March - April, 2010. Mobile Network QoE-QoS Decision Making Tool for Performance Optimization in Critical Web Service 899 [16] A. Sánchez-Macián, D. López, J. E. López de Vergara, and E. Pastor, A Framework for the Automatic Calculation of Quality of Experience in Telematic Services, Proc. of the 13th HP-OVUA Workshop, Côte d’Azur, 1-6, 2006. [17] R. Kooij, D. De Vleeschauwer, K. Brunnström, and F. Kuipers, Techniques for Measuring Quality of Experience, WWIC 2010, 216 - 217, 2010. [18] G. Rubino, P Tirilly, and M..Varela, Evaluating Users’ Satisfaction in Packet Networks Using Random Neural Networks, Proceedings of ICANN’06, Athens, Greece, 303-312, 2006. [19] E. Ibarrola, F. Liberal, I. Taboada, and R. Ortega, Web QoE Evaluation in Multi-agent Networks: Validation of ITU-T G.1030, Fifth Int. Conf. on Autonomic and Autonomous Systems (ICAS ’09), 289 - 294, 2009. [20] European Telecommunications Standards Institute, "Speech Processing, Transmission and Quality Aspects (STQ); User related QoS parameter definitions and measurements., European Telecommunications Standards Institute, Sophia Antipolis Cedex - FRANCE, Standard ETSI EG 202 057-2 V1.3.1, 2009.