American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 Β© Global Society of Scientific Research and Researchers http://asrjetsjournal.org/ Fast and Robust Smoothing Based Estimation of Missing Data for Video Error Concealment Aniruddha D. Shelotkara*, Dr.P.V.Ingoleb Research Scholar, Sant Gadge Baba Amravati University, Amravati, Maharashtra, India Principal, G. H. Raisoni Institute of Technology and Management, Amravati, Maharashtra, India Email: anishelotkar@gmail.com Email: prashant.ingole@raisoni.net Abstract With advancement of wireless technology, every handheld device supports video streaming. Generally, in video transmission technology, user datagram protocol (UDP) is used which does not provide guaranteed quality of service (QoS). Therefore, there is demand for video post processing modules for error concealments. However, handheld devices impose two additional constraints on post processing modules, a) Fast processing algorithms to satisfy video latency requirements b) Low power requirements as devices are limited on battery power supply. Hence, we require a fast, robust and simple algorithm for error concealment. In this paper we propose such algorithm to recover lost block of data in video. The proposed algorithm is based on discrete cosine transform (DCT). DCT can be implemented completely in fixed point format and hence it speeds up operation also saves power as the number of instructions are less. In this algorithm we carry out an automatic estimation of required smoothing. An iteratively weighted algorithm is proposed to deal with missing data. The proposed algorithm is very fast, robust and requires low storage, thus satisfying requirements of many handheld devices. Experiments are carried with different YUV streams. The experimental results show improvements in PSNR as well as visual quality. Keywords: Missing Data Estimation; Error Concealment; DCT. ------------------------------------------------------------------ * Corresponding author. 272 http://asrjetsjournal.org/ American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 1. Introduction With advancements wireless technologies and increased usage of handheld devices for communication video on demand service requirements are increasing. However, internet uses UDP protocol for video transmission which does not provide guaranteed QoS. Traffic congestion usually results in the loss of data packets. The reasons for packet loss could vary from multipath fading, shadowing to noise disturbance of wireless channels. To handle poor bandwidth scenario video transmission technology send compressed video data over network. However, in compressed data loss of single packet over transmission channel introduces severe distortions because the compression algorithms use temporal and spatial dependency among adjacent frames. Therefore a single distorted block within a frame may lead to errors not only in present frame but also propagate error over several frames. Many error resilience and decoder error concealment techniques have been proposed to control amount of error in reconstructed frame [1,2]. A simple error resilience strategy is to use feedback channels and request for retransmission whenever there is error. This is the most robust technique and the recovered data would always be correct. However, it involves halting decoding process till error block of data is received again. This is an inefficient approach in terms of delay involved in process [4,5]. Another way to avoid errors is to embed error checks in encoded video bit streams and transmit over the channels. This method though avoids retransmission of video; it affects compression efficiency of the encoder and thus increased usage of network bandwidth. Hence, a set of post processing algorithms on the decoder side are proposed for error concealment. The advantage with decoder error concealment is that it does not require any change in encoding or decoding process. It simply appends a post processing block which recovers erroneous data. Hence, there is no increase in bit rate or delay. Fig.1 shows block diagram for post processing of video sequence to recover loss of macroblocks. Therefore these techniques can be used in real time video applications like video-voice over internet and streaming applications [3]. Temporal error concealment technique uses temporal neighbors to estimate erroneous block of data. It utilizes previous frame or next frame to conceal errors in current frame. Most of the temporal error concealment methods assume that only a few macroblocks (MB) or slices in video frame are lost. Typically, temporal reconstruction process recovers lost data with the help of motion field interpolation (MFI) [3]. However, using just one of the frame may not be sufficient in case of scene change during video sequence. In case of scene change, Mean Absolute Difference (MAD) between two frames is very large and hence the temporal estimation methods fail [1,7]. In real time scenario with low bit rate, with the video sequence, which involve very low motion, using MFI may not be a good idea because of computational complexity. Therefore in this paper, we propose a transformed domain approach for error concealment and recovery. The algorithm is based on a penalized least squares method, allows fast smoothing of data in one and higher dimensions by means of the discrete cosine transform. Automatic choice of the amount of smoothing is carried out by minimizing the generalized cross-validation score. An iteratively weighted robust version of the algorithm is proposed to deal with occurrences of missing and outlying values. 273 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 . Figure 1: Generic block diagram for video post processing The rest of the paper is organized as follows, Section 2 explains theory of smoothing function, section 3 explains algorithm of smoothing function for error concealment, section 4 shows experimental results and section 5 provides conclusion and future work in the same area. 2. Theory of Smoothing Function In statistics and data analysis smoothing function is used to reduce noise or small scale information while keeping most imprints of the datasets. Mathematically, noisy data can be represented as follows, 𝑦 = π‘¦π‘š+ ∈ (1) Where ∈ represents Gaussian noise with zero mean and unknown variance and π‘¦π‘š is the mean of the signal. Accuracy of estimation of 𝑦 depends upon accuracy ofπ‘¦π‘š. Also, π‘¦π‘šis supposed to be smooth function that is derivatives of π‘¦π‘š of particular order generally greater than 2 are continuous. Smoothing of 𝑦 relies upon smoothing ofπ‘¦π‘š. In this paper we use penalized least squares approach for smoothing of data. Mathematically, it can be expressed as, 𝐹(π‘¦π‘š) = 𝑅𝑆𝑆 + 𝑠 βˆ— 𝑃(π‘¦π‘š) (2) Where, RSS is residual sum of squares which is expressed as, 𝑅𝑆𝑆 = ||𝑦 βˆ’ π‘¦π‘š||2 𝑃(π‘¦π‘š)is the penalized term, and Sis scalar which indicates degree of smoothing, with increase in smoothing parameter, degree of smoothing also increases. In [3], it is explained that term 𝑃(π‘¦π‘š) can be expressed as, Encoder Decoder Post- Process Display 274 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 𝑃(π‘¦π‘š) = ||𝐷 βˆ— π‘¦π‘š||2 (3) Where Dis a tri-diagonal square matrix defined as, 𝐷𝑖,π‘–βˆ’1 = 2 β„Žπ‘–βˆ’1 βˆ— (β„Žπ‘–βˆ’1 + β„Žπ‘–) 𝐷𝑖,𝑖 = βˆ’2 β„Žπ‘–βˆ’1 βˆ— β„Žπ‘– π·π‘–βˆ’1,𝑖 = 2 β„Žπ‘– βˆ— (β„Žπ‘–βˆ’1 + β„Žπ‘–) Where β„Žπ‘– represents step betweenπ‘¦π‘š,𝑖 ,π‘¦π‘š,𝑖+1 To correctly estimate smoothing parameter, we minimize equation (2) with constraints of equation (3). Hence, smoothened data can be obtained as, (𝐼𝑛 + 𝑠 βˆ— 𝐷𝑇 βˆ— 𝐷) βˆ— π‘¦π‘š = 𝑦 (4) Where 𝐼𝑛 is 𝑛 π‘₯ 𝑛 identity matrix and 𝐷𝑇 is transpose of 𝐷. The smoothing parameter can be estimated by application of generalized cross validation method introduced in [9], 𝑠 = π‘Žπ‘Ÿπ‘”π‘£π‘šπ‘–π‘›(𝐺𝐢𝑉) Where GCV is expressed as, 275 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 𝐺𝐢𝑉 = 𝑅𝑆𝑆 𝑛� (1βˆ’ π‘‡π‘Ÿ(𝐻) 𝑛� )2 Where π‘‡π‘Ÿ(𝐻) is the trace of matrix It is shown in [8], that 𝐺𝐢𝑉 can be further simplified for weighted data as, 𝐺𝐢𝑉(𝑠) = 𝑀 βˆ— 𝑅𝑆𝑆 (𝑛 βˆ’ π‘›π‘šπ‘–π‘ π‘ )οΏ½ (1 βˆ’ π‘‡π‘Ÿ(𝐻) 𝑛 )2 Where 𝑛 is number of elements in yand π‘›π‘šπ‘–π‘ π‘  is the number of missing data. 3. Proposed Algorithm The stepwise algorithm is as follows: 1. Decoded video sequence is used as an input to the algorithm 2. Determine the block to be recovered in a given frame. 3. Estimate missing block of data using DCT based smoothing algorithm. 4. Apply smoothing filter to remove any blocking artifacts which may occur in the recovered frame. 5. Compute PSNR of smoothened frame. 4. Experimental Results This section describes experimental results. Proposed algorithm has been tested over CIF and QCIF resolution of video, containing medium motion. We also consider AVI, MP4 videos for compressed video error concealment. We have used coastguard and Stefan as standard input video sequences, in YUV format. Figure 5 shows frame number vs PSNR graph for MP4 video sequence. Figure 2, 3, and 4 shows reconstructed frames for the proposed algorithm for YUV DCIF and MP4 videos respectively. To show the comparative study we have used the algorithm given in [10], as it is conceptually similar to the proposed algorithm. Figure 5 Shows the PSNR comparison of PSNR of reconstructed MP4 for reference and proposed algorithm. 276 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 Figure 2: Input, corrupted and recovered frame of video sequence coastguard Figure 3: Input source frame, Error frame, and recovered frame for stefan Figure 4: Input source frame, Error frame, and recovered frame for MP4 video. Figure 5: PSNR Vs Frame number for MP4 sequence 277 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2015) Volume 14, No 3, pp 272-278 5. Conclusion and Future Work In this paper, we propose a low complexity approach for spatial error concealment. The proposed method exploits the information present in current frame to estimate pixel and avoids blocking artifacts by smoothing filter function. We have applied the method on QCIF, YUV and MP4 video sequence The proposed algorithm gives acceptable PSNR. Future work includes using spatial and temporal information adaptively to reconstruct lost pixel information. References [1] Y. Wang, S. Wenger, J. Wen and A. Katsaggelos, β€œError Resilient Video Coding Techniques,” IEEE Signal Processing Magazine, pp. 61-82, July 2000. [2] M.E. Al-Mualla, N. Canagarajah and D. R. Bull, β€œTemporal error concealment using motion field interpolation,” IEE Electronic Letters, pp. 215-217, Feb. 1999. [3] M.E. Al-Mualla, N. Canagarajah and D.R. Bull, β€œMultiple reference temporal error concealment,” Proc. IEEE ISCAS2001, vol. 5, pp. 149-152. [4] S.H. Lee, D.H. Choi and C.S. Hwang, β€œError concealment using affine transform for H.263 coded video transmissions,” Electronics Letters, 37(4), pp. 218-220, Feb.2001. [5] Yu Chen, Keman Yu, Jiang Li, Shipeng Li, β€œAn Error Concealment algorithm for entire frame loss in Video transmission,” [6] Bhalotra, Parul S. and Bhushan D. Patil, β€œShot Boundary Detection Using Radon Projection Method”, International Journal on Signal and Image Processing, 4.3(2013). [7] Bhalotra, Parul S. and Bhushan D. Patil, β€œ Video Shot Boundary Detection Using Finite Ridgelet Transform Method”, ICT and Annual Infrastructure: Processing of the 48th Annual Convention of Computer Society of India. Vol-11, Springer International Publishing. 2014. [8] Damien Garcia, β€œRobust smoothing of gridded data in one and higher dimensions with missing values”, Computational Statistics & Data Analysis, 2010; 54:1167-1178. [9] Craven, P., Wahba, G., β€œSmoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized crossvalidation”, NumerischeMathematik1978.31, 377_403 [10] Branislav H., Jan M. and Stanislav M., β€œExtended error concelmant algorithm for intra frame in H.264/AVC”, Acta Electrotechnica et Informatica, Vol. 10, No. 4, 2010, pp. 59–63. 278