146 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 © Global Society of Scientific Research and Researchers http://asrjetsjournal.org/ QoE Enhancement for Stereoscopic 3DVideo Quality Based on Depth and Color Transmission over IP Networks: A Review Wahiba Ismaiela*, Niema Osmanb, Is-Haka Mkwawac a,bSudan University, Khartoum and +249183, Sudan cUniversity of Plymouth, Plymouth, UK aEmail: whiba2008123@yahoo.com bEmail: niema.osman@gmail.com cEmail: Is-Haka.Mkwawa@Plymouth.ac.uk Abstract In this review paper we focus on the enhancement of Quality of Experience (QoE) for stereoscopic 3D video based on depth information. We focus on stereoscopic video format because it takes less bandwidth than other format when 3D video is transmitted over an error channel but it is easily affected by the network parameters such as packets loss, delay and jitter. The packet loss on 3D video has more impact in the depth information than other 3D video factors such as comfort, motion, disparity and discomfort. The packet loss on depth information causes undesired effect on color and depth maps. Therefore, in order to minimize quality degradation, the application of frame loss concealment technique is preferred. This technique is expected to improve the QoE for end users. In this paper we will also review 3D video factors and their challenges, methods of measuring the QOE, algorithms used for packets loss recovery. Keywords: 3D video; Stereoscopic; QoE; Color-plus-Depth; error concealment. ----------------------------------------------------------------------- *Corresponding author. http://asrjetsjournal.org/ American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 147 1. Introduction Multimedia communication field plays important role in the network technology revolution and future applications. Multimedia applications are used in several fields such as business, education, entertainment, home and public places. These applications add additional challenges especially when they are transmitted over IP network because they contain large volume of information. In recent years three-dimensional video (3DV) demand has increased in multimedia applications. 3DV plays a great role in services such as 3D cinema, 3D game and free viewpoint. Stereoscopic color-plus-depth is now more popular compared to the Left and Right views, therefore, devices for 3D stereoscopic are manufactured in a large numbers .It is a preferred for stereoscopic 3D video transmission because it requires less bandwidth [1]. This study presents Quality of Experience (QoE) for 3D video because it is an important aspect for end users. When transmitting video services over an IP network, challenges such as packet loss, limited bandwidth and traffic variation are inevitable and they affect 3D video quality. This survey focuses on how to improve stereoscopic quality caused by packets loss. We discuss some studies that enhance stereoscopic transmission in order to achieve better quality with concealment methods. This paper also reviews 3D video subjective and objective quality assessments. Some recommended methodologies to assess subjective quality stereoscopic video sequences are ITU-T BT.2021[2] and ITU-R BT 1483[3]. This paper is organized as follows: Section 2 provides presents an overview of 3D video. Section 3 introduces 3D video transmission, transmission protocols, transmission challenges and error resilience tools. Section 4 describes stereoscopic format. Section 5 presents 3D video quality by addressing subjective and objective methods. Section 6 discusses state of the art for frame concealment for 3D video stereoscopic color-plus-depth. The last section concludes the review. 2. 3D video 3D video displays from left and right views which add sensation of the depth by adding the depth dimension in order to solve complexity in 2D video viewing and some features such as (free view point and depth-controlled object insertion) [4]. 3D video technique provides us with new services and applications today such as 3D Game, 3DTV, 3D animal picture books, sport performance analysis, 3D surgery monitoring, 3D archive of traditional dances [5], video personal recording, 3D telepresence, 3D telemedicine, 3D surveillance and free point television (FTV) [6]. 3D video processing chain starts from 3D video production to display it on user devices. Figure 1 shows the processing steps [7]. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 148 Figure 1: 3D video processing chain • Acquisition is the first phase in 3D video communication chain which includes two stages, capturing and post-processing. The capturing stage requires setting up 2 or 3 cameras according to different 3D video formats. Multi camera system can be used but will face challenges such as temporal synchronization, geometrical calibration and color balance between the individual cameras [8]. Other things that can be taken into account when setting up camera are position and angle of camera, camera parameters and format [7]. The post-processing stage contributes to making color balancing between cameras by enabling the camera views to be aligned [1]. • The second phase is editing 3D video stream, this can use different types of editing operation such as color, texture, shape, and motion editing [5]. • The third phase is encoding, this phase requires compression techniques to represent 3D video in a compressed form without any loss of raw video information because 3D video needs to be packetized and transmitted over a limited bandwidth. In this operation, representation format has an effect on all phases in 3D video communication chain [5,8]. • The fourth phase is 3D video transmission, the transport format is important in transmitting 3D video content because 3D video faces different transmission challenges over the IP network. The presence of depth maps and multiple views affects the video reconstruction and end user QoE. They are also making different quality between left and right views [1,9]. • The fifth phase is decoding, it converts signals into raw 3D video for the left and right views, but it faces some problems such as block noise and reduced spatial resolution which influence the quality. This phase uses frame sequential format in order to avoid reduced spatial resolution problems [7]. • The sixth phase is rendering, this is the last phase in processing 3D video chain. In this phase user receives 3D video content. The transport format data must be converted into display format [8]. The display format is very important for 3D video output on the device, in addition to other requirements of displaying 3D video which are display type, glasses, monitor size, luminance and viewing distance/angle [7]. On the other hand, it causes a number of physiological cues such as binocular disparity, motion parallax, and ocular convergence in order to produce depth sensations in human visual system (HVS) [1]. 3. 3D video transmission The delivery of 3D video content has become one of the most important issues in recent years. Therefore, 3D video sequences sent over network channels would require mechanisms and concealment strategies to achieve end to end acceptable quality. Acquisition Editing Encoding Transmission Decoding Rendering American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 149 3.1 3D video transmissions over the IP Network Transmission of 3D video over the IP network uses protocol stack RTP/UDP/IP encapsulated video streams. In recent years, modern 3D video applications use the newest protocol rather than UDP which is named as Congestion Control Protocol (DCCP). DCCP has congestion control method that computes transmission rate by using TCP throughput equation. DCCP is suitable for services which have large size of data [10]. Authors in [11-12] have discussed DCCP in detail. 3.2 Challenges of transmitting 3D over IP networks The following part discusses some of the challenges of transmitting 3D video over IP networks. • Packet losses: are caused by network congestion or faulty in wire and wireless link [1,9]. • Bit error or Burst error: it happens by noisy and multi-path propagation. Multi-path propagation has more effects in mobile environment [9] • Temporal domain de-synchronization aspect: it is caused by delay in one view, this also causes missing frame and discomfort in 3D viewing [1] • Error on reconstructed video propagation: this error propagates from one frame to another due to prediction mechanism. • Error concealment method: error concealment mechanism is a very important aspect for 3D video transmission in order to mitigate error propagation. Error concealment techniques are limited to errors resulting from compression and codec [9]. 3.3 Error resilience tools Four types of error resilience tools are considered as follows [9], • Localization: It removes or decreases the spatial or temporal dependency between frames or slices in order to stop the propagation of errors. • Data partitioning: Frames can be merged or fragmented according to their importance if error in low priority data happens at the encoder side, while at decoder side the corresponding packets are removed in order get better group of highest priority received without errors. • Redundant coding: This tool provides flexibility at the decoder side by encapsulating the additional data with bit streams which results from duplicating primary slices as redundancy slices or Multiple Description Coding (MDC) or Reversible Variable Length Coding (RVLC). • Concealment driven coding: This tool can be used at the encoder and decoder; it will recover errors by utilizing the additional received information about general behavior of the lost data. 4. Stereoscopic format Stereoscopic 3D (S3D) video is more popular in presenting 3D video because it is simple. It consists of two views, left and right view which are received by left and right eyes of an observer, it depends on stereopsis American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 150 principle. It also has another representation called color-plus-depth, which has some advantages such as flexibility, low encoding cost and backwards compatibility compared to left-right stereoscopic video. S3D has some advantages compared to other 3D video representations. These advantages are shown below, • Easy data acquisition • More accuracy • It requires less storage and bandwidth • It is suited for applications which contain passive systems based on polarized glasses or active systems based on liquid crystal shutter glasses [13-14]. 5. 3D video QoE This section explains some concepts of QoE such as QoE assessment, factors which have impact on 3D video quality. 5.1. QoE Assessment Quality of Experience is an important issue for assessing 3D videos when transmitted over IP network. QoE means user perceptual of the performance of services [15]. The ITU-T recommendation P.10/G.100 defines QoE as “The overall acceptability of an applications or services, as perceived subjectively by the end-user. QoE includes the complete end-to-end system effects (client, terminal, network and services infrastructure) and overall acceptability may be influenced by user expectations and context”. Figure 2 shows QoE aspects which are technical factors that consist of the network performance, device characteristics and content representation format, the social and psychological factors containing prior experience, content preference, user expectations and environmental factors [16-17]. Figure 2: QoE aspects QoE is also defined by The European Telecommunications Standards Institute (ETSI) as "A measure of user American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 151 performance based on both objective and subjective psychological measures of using an ICT service or product" [18]. QoE has two methods to assess the video sequences which are subjective and objective methods. Video Quality Experts Group (VQMG) is a basic group of validation and measuring objective perceptual quality metrics. Their working results appear through International Telecommunication Union (ITU) standardization which is designed for the standard definition of television and for multimedia applications [19]. 5.1.1 The subjective method The ITU-T recommendation P.910 [20] recommended and contributed on assessing the video quality by providing four types of methods, • Absolute Category Rating (ACR): Test sequences are shown one at a time and are rated independently on category scale. The time duration is 10 seconds and the scale is ranging from 5 to 1 (5 Excellent, 4 Good, 3 Fair, 2 Poor and 1 Bad). • Absolute Category Rating with Hidden Reference (ACR-HR): Test sequences are presented one at a time and are rated separately on category scale. In this method, viewers do not notice the presence of the source sequence. The scale ranges from 5 to 1 (5 Excellent, 4 Good, 3 Fair, 2 Poor and 1 Bad). • Degradation Category Rating (DCR): Test sequences are shown in pairs, one is source reference and the second is degraded sequence (also known as the Double Stimulus Impairment Scale Method (DSIS). This method has five levels scale for rating the impairment (5 imperceptible, 4 perceptible but not annoying, 3 slightly annoying, 2 annoying and 1 very annoying). This method also uses explicit references. • Pair Comparison (PC): Test sequences are shown in pairs consisting of the same sequences and the general pair combined when the system under test are equal to n(n-1). The scale in this method has seven levels for rating the impairment which are: –3 Much worse , –2 Worse, –1 Slightly worse, 0 The same,1 Slightly better, 2 Better, 3 Much better [21]. In all the methods above, the average quality result from viewers is recorded by using one of two ways, Mean Opinion Score (MOS) and the Difference Mean Opinion Score (DMOS). The DMOS means that the mean of differential subjective scores (mean and the standard deviation are used). 5.1.2. The objective method This was classified into three methods, Full Reference (FR), Reduce Reference (RR) and No Reference (NR). The ITU-R Recommendation ITU-T J.247 (08/2008) represents the FR model which means comparing the original video sequence with degraded video sequence [22]. However, recommendation ITU-T J.246 (08/2008) defines RR by comparing the extracted features of original video sequence with degraded video sequence [23], but the NR does not need the original sequence, it only needs to assess the degraded sequence [24]. Objective assessment methods are classified into five categories according to the type of input data which is used to assess video quality. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 152 1. Media-layer model: It uses speech or video to give QoE 2. Parametric packet-layer model : It depends on packet-header information to give QoE, it doesn’t require media signal 3. Parametric planning model: It takes QoE for networks and terminals depending on prior information 4. Bit stream layer model: It takes information of encoded bit stream and information of packet-layer to achieve QoE 5. Hybrid model: It predicts QoE by merging other models 5.2. 3D video QoE Three aspects affect QoE on stereoscopic system such as visual comfort, depth quality and picture quality. ITU- R recommendation BT.2021 defined QoE 3D video by using some factors such as visual discomfort, depth perception and visual quality. Some researchers have different opinion for psychological impairment of stereoscopic imaging when assessed in general concepts such as naturalness and sense of presence [2]. The 3D video QoE factors were considered in [25] as general factors which have impact on 3D video services and are divided into three types: • Content production factors which include (camera performance, video shooting conditions, effects of the picture frame and magnitude/intensity of convergence change). • Video compression which include (transmission error, coding algorithm and format) • Delivery factors and viewing environment factors which include (viewing distance, viewing angle and display scheme). 5.2.1. Subjective 3D video methodology There are standards and recommendations that define subjective quality assessment of 3D video such as International Tele-communication Union Radio-communication Standards Section (ITU-R) BT 1438, ITU-R BT 500-1, and ITU Telecommunication Standards Sector (ITU-T) P.910. The (ITU-R) BT 1438 evaluated subjective quality according to factors that assess stereoscopic television system which are: resolution, color rendition, motion portrayal, overall quality, sharpness and depth [3]. Subjective methodology can be used to assess stereoscopic picture and it is recommended in ITU-R BT 500-12. All these methods relate to the depth quality, picture quality and visual comfort of stereoscopic imaging systems. These methods are explained in the Table 1 in terms of depth quality. 5.2.2. Objective 3D video Methodologies Objective assessment of 3D video is not simple compared to 2D video. 3D video has additional factors such as (depth perceptual and image quality), therefore, in severalt researches, conventional metrics were used to assess 3D video such as PSNR, VQM and SSIM while other researchers developed 2D video metrics. New methods are proposed in Section (5.3.2). American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 153 Table 1: Subjective methods for the assessment of depth quality Mode of presentation Sequence duration Binary scale Discrete scale Continuous scale Single-stimulus (SS) methods : In this method just degraded sequence appears and the subjects are asked to assess the quality ~10 s 5 Excellent 4 Good 3 Fair 2 Poor 1 Bad Double-stimulus impairment scale (DSIS) method or European Broadcasting Union(EBU): It displays multiple sequences containing source reference and degraded sequence and viewers are asked to assess the quality 5 Imperceptible 4 Perceptible, but not annoying 3 Slightly annoying 2 Annoying 1 Very annoying Double-stimulus continuous quality scale (DSCQS) method: It is used to assess the quality in terms of sharpness and overall depth of stereoscopic and monoscopic image streams [Rec BT.1438], in this method volunteers are showed a pair sequence twice then they rate the quality of each sequence in the pair at the second presentation. ~10 s Stimulus-comparison (SC) methods: Is the comparison of two degraded sequences against each other. ~10 s A vs. B −3 Much worse −2 Worse −1 Slightly worse 0 The same 1 Slightly better American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 154 2 Better 3 Much better Single-stimulus continuous quality evaluation (SSCQE) method: This method is using slider mechanism with an associated quality score and shows the degraded sequence only; the subjects rate the quality sequence. ~3-5 min Simultaneous double stimulus for continuous evaluation (SDSCE) method: It shows the reference sequence and degraded sequence together, subjects rate the quality of the sequence but it requires viewers’ attention on the left and right sconce. 5.3. Subjective and objective evaluation for stereoscopic depth information This section discusses several researches which are using methodologies for 2D video and other new methodologies to assess subjective and objective quality for stereoscopic depth information. 5.3.1. Subjective for stereoscopic depth information This section explains methods developed to assess subjective quality of 3D video. Some methods use 2D video quality assessment and others were only developed for 3D video Lebreton and his colleagues [26] evaluated the quality of distortion of depth perceptual 3D video stereoscopic which is caused by coding or transmission. In this study, the assessed depth included two aspects, depth quality and depth perception. Authors explained subjective experiment and proposed new objective model to assess the quality based on depth information. It also measured the interaction between binocular and monocular depth features and depth perception. Subjective tests environment was based on standardized ITU-R BT.500-12 recommendation. The experiment consisted of 64 source reference single (SRCS) of database content, which has different types of scenes, different values of (image quality, depth quality and visual discomfort). All sequences are full HD stereoscopic videos, high quality, slow or fast motion, each view has resolution 1920x1080, frame rate of 25, each sequence has 10 seconds length, viewing distance was set at 3H and the number of viewers was 24. Authors used ACR for quality rating; the depth was rated on five grades which were denoted to “very high”, “high”, “medium”, “low” or “very low.” While the evaluation of visual comfort was labeled as ''much more'', "more", "as", "less", “much less" and "comfortable than watching 2D video". The subjective tests took 50 minutes including training session. The results illustrated high correlation with the HVS. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 155 Umar and his colleagues [27] designed subjective method for 3D video human perception by using depth information. Authors designed questionnaires to gather data and assist in reaching some information about requirements, expectation, viewing comfort, needs, perception, preference, satisfaction of 3D user and overall 3D video quality. Authors used two datasets in the experiment. First dataset from at Ecole Politechnique Federale de Lausanne EPFL [53], it has resolution of 720x576 pixels, encoded using 3D MDC-SIMI with QP=5, sequence of 10 seconds long. They included stereoscopic images and 250 frames at 25 fps. The second dataset from visual media research group at Microsoft research [54] with resolution 1024x768 pixels with 15 fps, 150 frames and duration of 10 seconds. But authors didn’t specify what codec was used to encode the sequences and what 3D video format was used. Authors used 4x6 images for training, 3x6 for testing and 2x6 for validation and the subjects who participated in the assessment were 25 volunteers aged between 15 and 50 years old. The average age was 23 years old. There were18 males and 17 females, all were non experts, however, authors didn’t explain their educational levels and eyes condition. In the experiment, stereoscopic shutter glass, the screen resolution of 1152x900 pixels, frequency of 60 Hz x 60Hz, 2cm of viewing distance, assessment size of 360 x 270 mm, optical path length of 320 mm, the horizontal FOV of 0.8 rad and 17 inches laptop monitors were used for display. Volunteers were asked to evaluate the overall 3D video quality and overall perception (0-100) values according to ITU-R Recommendation (Excellent, Good, Fair, Don’t Know, bad), (High impact, moderate impact, less impact, Don’t know, No impact), (Very important, Important, Less important, Don’t know, Not important) and (Strongly Agree, Agree, Disagree, Don’t know, Strongly disagree). The experiment took 3 hours and each questionnaire took around 10 minutes. The authors computed the MOS using equation (1), where N denoted number of volunteers, 𝑚𝑚𝑖𝑖𝑖𝑖 is score by volunteer i of test condition j, and estimated mean value by using confidence interval is given in equation (2). MOSj= 1 𝑁𝑁 � 𝑚𝑚𝑖𝑖𝑖𝑖 𝑁𝑁 𝑖𝑖=1 (1) Cij= ((𝑡𝑡(1−∝) 2 ,𝑁𝑁)). 𝜎𝜎𝑗𝑗 √𝑁𝑁 (2) where 𝑡𝑡(1−∝) 2 ,𝑁𝑁 is the t-value corresponding to a two-tailed t-Student distribution with N-1 degree of freedom and a desired significance level α. The results showed the confidence interval of 0.05, this value corresponds to a degree of significance of 95%, and the results ensure the subjective test was well done. Authors used dataset from public dataset for 3D video. The subjective tests were based on ITU-Recommendation. Mysirlidis et al., [28] measured the impact of asymmetric packet losses on scalable Multiple Description Coding (MDC) and delivery of 3D video to the receiver over two different paths, MDC combined with Scalable Video Coding (SVC) to create two layers using Medium Gain Scale (MGS). MDC creates two sub video with half of the original spatial resolution from color plus depth format. The result illustrated that the MDC scheme performs better by using a single description with less packet loss than using double description. The subjective tests were based on Double Stimulus Continuous Quality Scale (DSCQS). However authors didn’t give any details of subjects such as the number of viewers, viewers’ gender and age. They also didn’t show the bit rates used for transmission. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 156 Kim et al., [29] presented a new approach which was called multimodal Interactive Continuous Scoring of quality (MICSQ) which they claimed to be better than the traditional method (Single Stimulus Continuous Quality Evaluation (SSCQE)). The aim of MICSQ was to decrease deviations between viewing and evaluation, and to facilitate visual comfort for the end user in order to assess the quality of the 3D video and improve the subjective evaluation. This was achieved by conducting several related experiments to illustrate the usefulness of the new method such as high reliability, multi user assessment, speed and direction of depth motion, comfortable viewing Zone (CVZ) and Artificial Depth of Field (DOF) naturalness. MICSQ used stereoscopic format and subjective score was set to 0-10 by using two ways. The first is between assessment tools (interaction devices) which are used to display 3D using wireless network protocol. The second is between viewers (human interaction) and tablet, single or multi viewers can be assessed at the same time. The proposed methodology results were better for human concentration and visual effort than traditional methods (SSCQE) in terms of disparity and motion. The authors found relationship between visual comfort for motion and disparity and the relationship between DOF and naturalness. 5.3.2. Objective for stereoscopic depth information Hossein et al., [30] presented new Reduce Reference (RR) metric of stereoscopic color plus depth for 3D video transmission. The idea was to merge the extracted edge information with spatial neighboring information to construct the proposed Grey level Co-occurrence Matrix (GLCM) metric. This metric is more efficient because it used GLCM in real time stereoscopic quality assessment, it requires less bandwidth (2 Kbps). Consequently, 725 videos were generated at 25 fps, the experiment used 29 references and subjective tests based on ITU guideline for experiment. Results showed that the proposed metric highly correlated with subjective quality score. However, the proposed metric needs more computation in all frames for both original and compressed video. Hewage and his colleagues [31] proposed a Reduced-Reference (RR) quality metric for compression depth maps associated with color plus depth 3D video according to edge detection which was divided into two sections. The first is gradient explore edge detection based on the minimum and the maximum in the first derivative of the image. The second is Laplacian at the location of zeros in the second derivative edges detection. All RR and Full-Reference (FR) use PSNR metric, the result showed that at the minimum of Quantization Parameter (QP), the RR is close to the FR but RR is not accurate at higher QP. The strength of the proposed method is that it takes less bandwidth than FR. However, PSNR is not good for QoE and does not give information about the depth perception. If subjective tests were done, they would reveal more realistic results. Chaminda and his colleagues [32] developed RR quality metric for color plus depth 3D video transmission. This method depends on the use of extracted edge information in the depth map. It used RR D-PSNR Metric to map the depth, and for color extracted edge information from the corresponding color image near the edge, it used RR C-PSNR. The strength of the method is that it reduces overhead (by decreasing bit rate) and bandwidth utilized compared with Full Reference (FR) method. Edge information can be used to describe the basic in front object. However, the authors used the bitrates without clarifying what bit rates range has been utilized. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 157 Joveluro and his colleagues [33] provided the accurate two dimensional (2D) objectives metric related with the Human Visual System (HVS) for evaluation 3D video quality. Using stereoscopic format which encodes the color and depth image by the Content Adaptive Binary Arithmetic Coding (CABAC) scheme in the Joint Scalable Video Model (JSVM) and representing left and right views by using Depth-Image-Base-Rendering (DIBR) before evaluating 2D metrics. The study powerfully presents the subjective method based on standard ITU-R WP6Q using SAMVSQ. However, the experiment didn’t determine the viewing distance. The proposed metric Perceptual Quality Metric (PQM) model has strong correlation when comparing their outcome with the result of Video Quality Metric (VQM) and its design is based on human visual system (HVS). This metric can be used for 2D and 3D quality. However, it is based only for stereoscopic measure in luminance component color and depth. Politis and his colleagues [34] suggested a metric to assess compression and packet loss. Two objective metrics were proposed, Video Quality Metric (VQM) base on ANSI T1.801.03 for perceived quality of stereoscopic. However, it didn’t assess Disparity factor. The metric VQMComp is taking weighted maximum and minimum VQMs of the stereoscopic views to assess compression quality using equation (3). VQM left denotes left view and VQM Right denotes right view. The authors got the best weight from merging the minimum and the maximum weights of left and right views. The second metric VQMpl measures packet loss using equation (4), in this equation authors selected three weight factors which are A, w and z. The value of A is 0.322 and w =z=0.072. These values are the result of linear regression of equation (4). The factors w and z are based on the Unequal Error Protection (UEP) strategy, the purpose of that is to decrease the impact on one of the two views according to the following rule, • w =z, in case of using High Priority to I-frames • w 0 (4) Yi Han and his colleagues [35] presented No Reference objective metric Video Quality Metric (NVQM) to 3D video stereoscopic real-time evaluation. It is based on the model in ITU_T G.1070 for 2D video assessment to evaluate the packet loss and various bit-rates. The model contains five sets of coefficient for MPEG-4 and ITUT H.264 and authors utilized one of the five set for MPEG-4 to build their model. The proposed metric was American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 158 compared with two FR metrics which are VQM and SSIM. This model used fixed frame rate explained by the equation (5). The coefficient a1 denoted the different MOS of the perceived 3D video quality, a2 is a fixed coefficient, PplV is the packet loss rate and BrV indicates bit rate. However, the coefficients from a1 to a5 are utilized from subjective test results. In the experiment, authors used different five video sequence with different motion (low, medium and high) and different packet loss such as (0%,0.5% ,1%,2%,3%,4%,5%,6%,7%,10% ). V3Dq = a1 + a2 ∗ e − Pplv a3+a4∗e − Bva5 (5) They used fixed frame rate of 18 fps and low and high bit-rate of 2 and 4 Mbps, respectively, with resolution of 1920x1080 pixel. Viewers used shutter glasses, 5mx5m quite room, distance between screen and viewers 1m and video duration from 6 to 14 seconds. Volunteers assessed quality ranging from 1 to 5 (bad, poor, fair, good and excellent). The subjective results illustrated that the MOS values were better at high bit-rate than low bit- rate and when PLR was less than 4%. However, when PLR was from 0 to 1%, the MOS values for both low and high bit-rate decreased by more than 1.0 until 2% PLR and after 4% PLR it decreased to 1.5. The proposed model achieved better result when compared with SSIM, VQM and G.1070 (c.f., Table 2). Table 2: Proposed method results compared to other metric SSIM VQM G.1070 NVQM High bit-rate 0.8712 0.7885 0.9059 0.9736 Low bit-rate 0.7474 0.8892 0.8606 , 0.8961 The proposed model (NVQM) achieves high accuracy when compared to SSIM, it has 11% and 19% for high and low bit-rate and it has 23% for high bit-rate compared to VQM, and 5% with G.1070. However, authors did not explain if it can be used for stereoscopic color plus depth. Han and his colleagues [36] developed a metric called extended No-reference 3D Video Quality Metric (eNVQM) based on packet loss rate, 3D video bit rate and 3D video frame rate. It aimed to assess stereoscopic 3D video quality including color and depth information. This model used three input parameters. It used most of the coefficients of the ITU-T G.1070 similar to 2D video. Authors used equations (6-8). In equation (6) the I3D coding includes frame rate and bit-rate t when packet loss (PplV) is 0%. 𝑉𝑉𝑞𝑞3𝐷𝐷 = 1 + 𝐼𝐼𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑐𝑐𝑐𝑐 3𝐷𝐷 𝑒𝑒 𝑃𝑃𝑝𝑝𝑝𝑝𝑝𝑝 𝐷𝐷𝑃𝑃𝑝𝑝𝑝𝑝𝑝𝑝 3𝐷𝐷 (6) Icoding=a1 ln Frv+a2 ln(a3+Brv) 3D (7) D PplV=a5++a6∗eFrva1 +a8∗e Brv a9 D3 (8) For quality of color and depth authors used equations (6) and (8) denoted for V D3 color and V D3 Depth and two set of coefficients A={a1,a2,...,a12} and the A color and A depth . However, a1 denoted the impact of frame rate and a2 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 159 denoted the impact of bit-rate when there is no packet loss. All coefficients from a1 to a9 are based on three parameters, video format, codec type and display size. In the experiment authors used frame rates of 18 fps and 11 fps, the bit rates with high (4 Mbps), medium (3 Mbps), and low 2 Mbps. The video was encoded into MPEG-4 SVC. Authors used different duration from 6 to 12 second. In this experiment different PLRs were used (0% ,0.1% , 0.5% , 1% , 2%, 3%, 4% ,5%, 6% ,8% ,10% ). The subjective test contain 10 set of video sequences chosen in order of video content, packet loss, bit rate and frame rate. A set included 33 video sequences with the resolution of 1920x1080 pixels. 40 volunteers assessed the quality over all 3D experience, 3D depth experience, eye comfort, 3D enjoyable level and 3D effect enhancement level buy using MOS values. The experimental results showed that at packet loss 0%, 1% and 3% MOS values were higher when bit rate and frame rate were increased and increasing bit rate had more effect than increased frame rate. The study was based on standard ITU-T G.1070 and compared to two previous studies. It achieved better result of 0.944 than SSIM 0.911 and VQM 0.932 in accuracy. Sawsen and his colleagues [37] developed a new NR metric called MD-QA for evaluating stereoscopic 3D video by taking features from stereoscopic frames containing motion vector lengths and depth map information. Depth information was generated by utilizing disparity (the horizontal distance value of each image pixel), but disparity for stereoscopic image is defined as the horizontal pixel between two similar pixels. The proposed metric MD-QA used H264/AVC MPEG1 and MPEG2. The proposed model was assessed by three FR metrics, SSIM, PSNR and VQM, it was compared to MOS score in the first step, then the second step extracted disparity map between the left and right views of each frame using sum of absolute differences method (SAD) in order to generate the depth information by using equation (9). Motion vectors map were also generated by using block matching method in order to contribute into 3D video quality. After that the author calculated weighting value of depth and motion vector using non-linear regression function. In the final step the author used Pearson Linear Correlation Coefficient PLCC benchmark to measure the proposed metric performance depending on the weighted values of depth and motion vectors features by equation (10) 𝑍𝑍𝑝𝑝 = 2𝑓𝑓𝑓𝑓 𝑐𝑐 = 𝑓𝑓𝑓𝑓 𝑐𝑐 (9) Qf = (w)Mf+ (1 w)Df (10) The database was used from EPFL and the authors used six videos, each has 5 scenes and frame rate of 25 fps, resolution of 1920x1080 pixel and duration was 10 second. 20 subjects evaluated the videos. The proposed model MD-QA was evaluated and correlated with MOS, the model achieved better results than others. Tselios and his colleagues [38] suggested a QoE objective model for 3D video according to network parameters. The model proposed PSNR metric to assess the perceptual quality of stereoscopic video of the two diminutions (2D), left and right views measured at the receiver. This paper used Double stimulus Continuous Quality Scale (DSCQS) subjective method, and the measured environmental luminance was based on the recommendations described in ITU-R BT.500. RTP packets were created according to standard (Single NAL Unit, Single-Timing Aggregation Packet). The study found that there is a strong correlation between objectivity and subjective quality. However, authors did not justify the use of different resolutions 1280x1024 for degraded and 640x480 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 160 for objective and the same quantization parameters (QP=24) for each left and right views with different bit rates. 6. Frame concealment for 3D stereoscopic based depth information This section includes three aspects, the first aspect discusses methods and algorithms for frame concealment due to packet loss of stereoscopic 3D video transmitted over IP network, the second aspect considers the preliminary results and the third aspect describes the tools. 6.1. Frame concealment and algorithms Carreira and his colleagues [39] considered three different algorithms which are named as Frame-Freeze (FF), Double-Freeze (DF) and Base-Copy (BC). In FF, the last frame that was received in the auxiliary view is copied to the temporal instant of the lost frame in the same view. DF is the base view where a frame temporally co- located within the last one that was received in the auxiliary view is copied into both views at the missing time instant. The BC copies the temporally co-located frame from the base view into the auxiliary view at the missing time instant. All concealment methods in this work are used to assess the subjective impact in temporary loss of depth information, in addition each of them have three different forms named as (1X2 means every other frame is lost, 2X3 means one of 2 consecutive frames are lost in each group of 3 and 3X4 consider that 3 of 4 frames are lost). These losses happen in two ways. First is (1 x 50) loss which occurs in the second half of the sequence 50%. Second is (1 x 100) loss frame which happens for the whole sequence 100%. In this study the base view is received without losing whereas the auxiliary view is prone to loss, therefore, only depth information is impacted. The experimental results of subjective test explain that all methods give higher MOS values in 1x50 patterns than in 1x100, also the FF method gives higher average MOS values than other methods in case the depth information loss decreases, and it decreases when percentage of loss increases. However, the concealment methods assessed in this paper use regular frame loss and therefore, the results might not be suitable for the random frame loss. Pinto and his colleagues [40] includes different frame loss methods which were mentioned in [39], i.e., FF, DF and BC. The DF method corresponds to freezing the last 3D image received without errors and BC method transforms 3D to 2D because both views consist of properly the same frames during the error time. However, this study assumes that the corresponding frame is lost whenever a packet is lost. The research in [40] aims to evaluate the effect of packet loss on the perceived depth whereby packet loss occurs more randomly than the previous study. Markov model examined two models of the packet loss in order to generate the packet loss forms. The author conducted subjective test using ACR methodology to find out the relationship between 3D quality and content features, frame concealment methods and packet loss statistics. The experimental results showed all concealment methods achieved similar MOS with very small error, but when error is increased, the BC gives the best MOS results compared to the FF and DF methods except for the highest disparity sequence, whereas 3D quality with low perceived depth is less annoying than high perceived depth. However, this work was not suitable to recover color because it focused only on concealing depth perception. Tae-Young and his colleagues [41] split loss frames into four error forms. Table.3 explains how to recover the American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2016) Volume 23, No 1, pp 146-171 161 error pattern. The authors encoded stereoscopic V+D of color and depth individually. Disparity Compensated Prediction (DCP) and intra-prediction for predicting the right color and depth frames from the left frame were used, whereas, using Motion Compensated Prediction (MCP) for predicting the left frames of color and depth temporally from previous frames. The proposed method assumed the first frame of GOP is received correctly. The performance of the proposed method assessed in the case without loss of the first frame, at frame loss rate of 10% the color and depth in the left and right views are randomly lost. The strength of the proposed method was based on error location. The simulation results showed the proposed algorithm which uses PSNR for objective test gave better results than four traditional methods (DBMA, MV sharing, PMVE +DFDP and PMVE + SCA-P). If the loss of color and depth happened at the same time, the proposed algorithm conceals by copying the temporally previous frames to replace the lost frames. Table 3: Error pattern and concealment methods Error pattern Left-color loss Left–depth loss Right-color loss Right-depth loss Concealment Methods Using Motion Vectors(MV) for the temporal error concealment Using MV for the corresponding left color frame methods First select matching pixels between left and right views using 3D warping scheme then utilize the high interview interconnection between the matching pixels Using MV for the right and left color frame Yang and his colleagues [42] presented slice loss method for intra frame 3D video called a depth-assisted error concealment method of 2D+depth video sequence. The authors used motion information directly from depth without considering the similarities in the motion information by using the MVs from the depth sequence, but in the case of the intra mode, the MV information cannot be used. The authors supposed that intra frames of the depth sequence are a temporal offset which is presented in Figure 3. The GoP denotes the size of G and GoP offset as G offset (offset 0