Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 4, No. 2, 2023 27 A Lightweight Dual‐Branch Image Dehazing Network based on Associative Learning Xiaoxiao Xu * College of Computer Science and Technology, Qingdao University, Qingdao Shandong, 266000, China * Corresponding author Email: xxx77904@163.com Abstract: Haze degrades the clarity, contrast, and details of images, resulting in a decrease in image quality. Image dehazing provides a means to obtain clearer and more accurate image information. Traditional methods for haze removal typically rely on manually designed features and models, limiting their performance in complex scenes. In recent years, the rapid advancement of deep learning has offered new insights into addressing the image dehazing problem. This paper proposes a lightweight dual- branch image dehazing network based on associative learning (LDANet). The network consists of a lightweight dehazing sub- network (LDSN) and a lightweight image enhancement subnetwork (LESN). To reduce computational and parameter complexity, the Tied Block Convolution (TBC) is employed, allowing for parameter sharing among components. Lastly, through associative learning, their distinctive features are mapped. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of our approach in qualitative comparisons and quantitative evaluations compared to other state-of-the-art methods. Our method holds significant practical value for real-world image dehazing scenarios. Keywords: Associative Learning; Encoder-Decoder; Image Dehazing; Lightweight. 1. Introduction In domains such as transportation, aerospace, and road monitoring, haze significantly reduces visibility, affecting various aspects of real-life scenarios. Therefore, dehazing is a crucial task aimed at mitigating the impact of haze on different systems and enhancing their usability. The Atmospheric Scattering Model (ASM) [1] is a mathematical model that describes the scattering and absorption of light during its propagation through the atmosphere. It is employed to explain the process of light propagation in the atmosphere, particularly under conditions of haze, atmospheric pollution, or long-distance observations. The ASM can be formally expressed as follows: I(x) = J(x)t(x)+A(1-t(x)) (1) where I and J represent the observed hazy and haze-free images, respectively, t x e denotes the medium transmission map, β and d x represent the atmospheric scattering parameters and scene depth, respectively, while x denotes the pixel position. Based on different principles, current image dehazing algorithms can be categorized into two main types: image enhancement-based and image restoration-based dehazing algorithms. The remaining sections of this paper are structured as follows. In Section 2, we provide an exposition on the research status of image dehazing. In Section 3, the proposed network is presented in detail. In Section 4, we objectively analyze the experimental results, including evaluation metrics and comparative images. In Section 5, we summarize this paper. 2. Related Works 2.1. Image Enhancement-based Dehazing Algorithms By adjusting the grayscale levels, the contrast of an image can be enhanced, thus improving its visual effect. Histogram equalization algorithms are widely used in digital image processing, including both global and local histogram equalization methods [2]. Stark [2] and Kim et al. [4] proposed adaptive histogram equalization algorithms and partially overlapping sub-block histogram equalization algorithms, respectively. Russo [5] performed equalization on degraded images at multiple scales. Dippel et al. [6] compared two multi-resolution analysis methods, Laplacian pyramid and wavelet transform, which exhibit good local characteristics. The Retinex model [7], based on the theory of color constancy, includes both single-scale and multi-scale Retinex algorithms. However, these methods overlook the fundamental cause of image blur, leading to subpar results. 2.2. Image Restoration-based Dehazing Algorithms These methods mainly include those based on prior knowledge and those based on learning. He et al. [8] proposed the Dark Channel Prior (DCP) dehazing algorithm, which utilizes the Atmospheric Scattering Model (ASM) for uniform dehazing. Tan [9] constructed a Markov random field model to estimate the cost function of edge intensity, resulting in significant improvement in image details. Cai et al. [10] applied deep convolutional neural networks to dehazing scenarios, where their DehazeNet model, established using a deep CNN structure, provided a novel estimation of atmospheric degradation transmission. Li et al. [11] designed an end-to-end AOD-Net network, which directly obtains clear images without estimating medium transmission and atmospheric light. Ren et al. [12] proposed the MSCNN algorithm, which employs a multi-scale network structure to obtain clear images. Chen et al. [13] introduced the GCANet model to address the issue of grid artifacts in the image restoration process. Qin et al. [14] proposed the FFA-Net model, which incorporates attention mechanisms to effectively remove non-uniform haze. 28 3. Methodology 3.1. Network Architecture In this section, we will provide a brief overview of the proposed network, LDANet. As shown in Figure 1, the network takes a hazy image as input and outputs a clean image. The network consists of two main components: the dehazing subnetwork (LDSN) and the image enhancement subnetwork (LESN). The LDSN is designed as an encoder-decoder structure to roughly remove haze from the input image. It incorporates the Tied Block Convolution (TBC) lightweight convolution with shared parameters, which significantly reduces computational and parameter costs, saving time and resources. This design choice is practical and meaningful. The LESN serves as a complement to the LDSN features. It employs the SimAM attention mechanism to adaptively focus on different features and contextual information, highlighting salient features and improving the model's generalization ability. Importantly, no additional parameters are introduced in this process. Finally, through associative learning, the two subnetworks are trained to capture their correlation and jointly generate a clear image as the final output. Figure 1. Schematic Diagram of LDANet 3.2. Loss Function We use Charbonnier penalty loss [15] and Structural Similarity (SSIM) loss [16] to jointly compute the total loss function for network optimization, aiming to more accurately evaluate image quality. The mathematical expressions are as follows: , (2) 1 ∑ ̂ ̂ ̂ ̂ , (3) , (4) where represents the generated image and represents the target image. represents the total loss function, and are parameters. 4. Experiments 4.1. Datasets To thoroughly demonstrate the performance of the proposed model, we conducted experiments on several dehazing datasets, including synthetic datasets and heterogeneous datasets. The synthetic dataset utilized the RESIDE dataset [17], which is widely used in the field of image dehazing and consists of haze images synthesized using prior information. It consists of indoor training set (ITS), outdoor training set (OTS), and testing set (SOTS). The proposed model not only showed superior results on the synthetic dataset but, more importantly, achieved satisfactory performance on the heterogeneous dehazing datasets. Therefore, we conducted experiments and comparisons on the I-HAZE [18], O-HAZE [19], and NH-HAZE [20] datasets. 4.2. Quality Evaluation Metrics In the field of image dehazing, it is crucial to objectively evaluate the performance of an algorithm. Evaluating the performance of an image dehazing algorithm typically involves various evaluation metrics. In this paper, we used Peak Signal-to-Noise Ratio (PSNR) [21] and Structural Similarity Index (SSIM) [22] to measure the reconstruction quality of the images. The PSNR value reflects the difference between the original image and the reconstructed image, where a higher value indicates better image restoration. SSIM measures the similarity between the original image and the reconstructed image by comparing their contrast and structural information. The SSIM value ranges from 0 to 1, with a value closer to 1 indicating higher image similarity and better image restoration. 4.3. Experimental Results and Analysis Table 1 summarizes the image quality evaluation metrics of our proposed method compared to DCP, FFANet, and MSBDN methods on SOTS, I-HAZE, O-HAZE, and NH- HAZE datasets. Each value in the table represents the average result of the tests. It is evident that our proposed method outperforms the other methods by a significant margin in terms of numerical values. Table 1. Comparison of image quality evaluation metrics on SOTS, I-HAZE, O-HAZE and NH-HAZE datasets, where ↑ indicates a better value as it increases, bold means the optimal results Dataset Metric DCP FFA- Net MSBDN Ours SOTS SSIM↑ 0.832 0.944 0.927 0.967 PSNR↑ 16.98 31.37 29.53 28.29 I-HAZE SSIM↑ 0.651 0.663 0.739 0.806 PSNR↑ 12.94 16.28 16.29 19.28 O-HAZE SSIM↑ 0.575 0.719 0.639 0.856 PSNR↑ 16.17 18.27 18.38 24.08 NH- HAZE SSIM↑ 0.53 0.602 0.658 0.709 PSNR↑ 13.01 15.57 17.46 17.77 Efficiency Flops↓ — 288.3G 41.5G 4.2 Params↓ — 4.46M 31.35M 0.26k Figures 2-5 compare the results of our method and other methods on SOTS, I-HAZE, O-HAZE, and NH-HAZE datasets, which are visually more evident. Due to the non- uniformity of fog in the images, it means that different regions have varying degrees of scattering, making it challenging for traditional methods to obtain accurate transmission images. It is evident that when DCP processes heavily hazy images, the resulting images are often darker than the ground truth, affecting the visual effect. FFA-Net can effectively remove non-uniform fog, but the resulting images may have halos and artifacts. MSBDN exhibits local color distortion and poor stability, with residual haze still present. In contrast, our method can capture rich details and local features, effectively handling heterogeneous haze and achieving superior haze removal results. 29 Figure 2. Visualized Results on SOTS (outdoor) Dataset Figure 3. Visualized Results on I-HAZE Dataset Figure 4. Visualized Results on O-HAZE Dataset Figure 5. Visualized Results on NH-HAZE Dataset 5. Conclusion Inspired by deep learning and influenced by popular neural networks, this paper proposes a lightweight dual-branch image dehazing network based on correlation learning to eliminate the effects of haze in images and improve the usability of systems in domains such as transportation, surveillance, and aviation. Specifically, the proposed network consists of a haze removal sub-network and an image enhancement sub-network. The Tied Block Convolution (TBC) is employed with parameter sharing to reduce computational and parameter complexity. Finally, through correlation learning, the different features of these sub- networks are mapped. The proposed model demonstrates qualitative and quantitative advantages, yielding superior dehazing results. Moreover, it has low computational and parameter requirements, saving computational resources, and thus holds significant practical value for real-world image dehazing. Future applications and improvements can be explored in the domain of video dehazing. Additionally, this paper does not cover the scenario of nighttime dehazing, which is another area worthy of investigation. References [1] W. E. K. Middleton. Vision through the atmosphere. University of Toronto Press, 1952. [2] Turgay Celik. Spatial entropy-based global and local image contrast enhancement[J]. IEEE Transactions on Image Processing, 2014, 23(12): 5298-5308. [3] STARK J A. Adaptive image contrast enhancement using generalizations of histogram equalization[J]. IEEE Transactions on Image Processing,2000,9(5):889-896. [4] KIM J Y,KIM L S,HWANG S H. An advanced contrast enhancement using partially overlapped sub-block histogram equalization [J]. IEEE Transactions on Circuits and Systems for Video Technology,2001,11(4):475-484. [5] Russo F. An image enhancement technique combining sharpening and noise reduction. IEEE Transactions on Instrumentation and Measurement, 2002, 51(4): 824−828. [6] Dippel S, Stahl M, Wiemker R, Blaffert T. Multiscale contrast enhancement for radiographies: Laplacian pyramid versus fast wavelet transform. IEEE Transactions on Medical Imaging, 2002, 21(4): 343−353. 30 [7] Land E H, Mccann J J. Lightness and retinex theory[J]. Journal of the Optical Society of America, 1971,61(1):1-11. [8] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(12): 2341-2353. [9] Tan R T. Visibility in bad weather from a single image[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 2347-2354. [10] Cai B L, Xu X M, Jia K, et al. DehazeNet: an endto-End system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11):5187-5198. [11] LI B Y,PENG X L,WANG Z Y,et al. AOD-net:All-in-one dehazing network[C]. IEEE International Conference on Computer Vision,2017:4770-4778. [12] REN W Q,LIU S,ZHANG H,et al. Single image dehazing via multiscale convolutional neural networks[C]. European Conference on Computer Vision,2016:154-169. [13] Chen D, He M, Fan Q, et al. Gated context aggregation network for image dehazing and deraining[C]. Proceedings of the 2019 IEEE winter conference on applications of computer vision. 2019: 1375-1383. [14] Qin X, Wang Z, Bai Y, et al. FFA-Net: Feature fusion attention network for single image dehazing[C]. Proceedings of the AAAI conference on artificial intelligence. 2020, 34(07): 11908-11915. [15] Lai, Wei Sheng , et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution [C]. IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2017:5835-5843. [16] Wang Z , Bovik A C, Sheikh H R ,et al.Image quality assessment: from error visibility to structural similarity [J]. IEEE Trans Image Process, 2004, 13(4). [17] Li B, Ren W, Fu D, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2018, 28 (1): 492-505. [18] Ancuti C, Ancuti C O, Timofte R, et al. I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images[C]. Proceedings of the Advanced Concepts for Intelligent Vision Systems. 2018: 620-631. [19] Ancuti C O, Ancuti C, Timofte R, et al. O-haze: a dehazing benchmark with real hazy and haze-free outdoor images[C]. Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018: 754-762. [20] Ancuti C O, Ancuti C, Timofte R. NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze- free images[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020: 444- 445. [21] Li L, Zhou Y, Wu J, et al. Color-enriched gradient similarity for retouched image quality evaluation[J]. IEICE Transactions on Information and Systems, 2016, 99(3): 773-776. [22] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing, 2004, 13(4): 600-612. [23] Chen D, He M, Fan Q, et al. Gated context aggregation network for image dehazing and deraining[C]// Proceedings of the 2019 IEEE winter conference on applications of computer vision. 2019: 1375-1383. [24] Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming- Hsuan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR,pages 624–632, 2017. [25] Di You, Jian Zhang, Jingfen Xie, Bin Chen, and Siwei Ma.COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30: 6066–6080, 2021.