EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS
Vol. 16, No. 2, 2023, 975-982
ISSN 1307-5543 – ejpam.com
Published by New York Business Global

Computational Experience with Modified Coefficients
Conjugate Gradient for Image Restoration

Basim A. Hassan1,∗, Maysoon M. Aziz1

1 Department of Mathematics, College of Computers Sciences and Mathematics,
University of Mosul, Iraq

Abstract. The Conjugate Gradient Method is a numerical optimization technique that finds the
optimal solution by focusing on the coefficient conjugate. This paper presents a new coefficients
conjugate gradient method for removing impulse noise from images, which is based on a quadratic
function and is proven to be globally convergent. Results show that it is an effective method for
image restoration.

2020 Mathematics Subject Classifications: 65K10, 49M37, 90C06
Key Words and Phrases: Conjugate gradient method, Coefficient conjugate gradient, Image
restoration

1. Introduction

This work presents a class of iterative methods for optimization problems where the
objective function is an edge-preserving regularization (EPR) functional. The goal is to
find the optimal solution to the problem. Impulse noise can be identified and filtered
using an adaptive median filter (AMF) equation. The true image is denoted by X and
the index set of X is denoted by A = 1, 2, 3, ...,M × 1, 2, 3, ..., N . The sets N ⊂ A and
Pi,j represent the indices of noise pixels detected in the first phase and the four closest
neighbors of the pixel at position (i, j) ∈ A, respectively. Given a pixel value yi,j at
position (i, j), ui,j = [ui,j ](i,j)∈N is a vector of length c that is ordered lexicographically.
the number of elements in a set N , represented by the letter c. Here c is the number
of elements of N . A two-step process for recovering noise pixels. The first step involves
minimizing a functional, and the second step involves minimizing a different functional to
recover the noise pixels.

fa(u) =
∑

(i,j)∈N

[
|ui,j − yi,j |+

β

2
(2× S1

i,j + S2
i,j)

]
(1)

∗Corresponding author.
DOI: https://doi.org/10.29020/nybg.ejpam.v16i2.4754

Email addresses: basimah@uomosul.edu.iq (B. A. Hassan),
aziz maysoon@uomosul.edu.iq (M. M. Aziz)

https://www.ejpam.com 975 © 2023 EJPAM All rights reserved.



976

where, β is the regularization parameter, and: S1
i,j = 2

∑
(m,n)∈Pi,j∩Nc φα(ui,j − ym,n),

S2
i,j =

∑
(m,n)∈Pi,jN

φα(ui,j − ym,n) and the φα =
√
a+ x2, α > 0 edge-preserving potential

function is an example of a potential function that preserves edges in an image. It is used
to reduce noise and enhance the edges of an image, making it easier to identify features.
Optimization methods can be used to minimize a smooth edge-preserving regularization
(EPR) functional. It is noted that the non-smooth data-fitting term is not needed in the
second phase, where only noisy pixels are restored:

fa(u) =
∑

(i,j)∈N

[
(2× S1

i,j + S2
i,j

]
(2)

Referring to [14] of additional information.
Conjugate gradient method is commonly used in image restoration, where it is used to

minimize a given objective function:

Minf(x), u ∈ Rn (3)

where a function is continuously differentiable. The Conjugate Gradient Method works
by taking a starting point and then iteratively improving the solution:

uk+1 = uk + αkdk (4)

until it converges to the optimal solution, a search direction dk and exact step size αk can
be used to optimize the quadratic case. These two elements are essential for achieving the
best possible results:

αk =
−gTk dk

dTkQdk
. (5)

In order to solve for general nonlinear functions, an iterative procedure must be used [11].
The Wolfe conditions are a set of criteria used to determine the step length in a given
work:

f(uk + αkdk) ≤ f(uk) + δαkg
T
k dk (6)

dTk g(uk + akdk) ≥ σdTk gk (7)

where 0 < δ < σ < 1. Additional information. The following is the search direction
equation:

dk+1 = −gk+1 + βkdk (8)

where the coefficient βk is selected in such a way that and must satisfy the conjugacy
property. Fletcher and Reeves (FR) [2] and Dai and Yuan (DY) [1] are two well-known
βk that are thought to be the most effective approaches, as stated by:

βFR
k =

gTk+1gk+1

gTk gk
, βDY

k =
gTk+1gk+1

dTk yk
(9)



977

Alternatively, by other equations (see, for example, [3],[10], [12], [13]. Some computer-
generated imagery conjugate gradient approaches are numerically efficient, while others
are conceptually efficient. All of these strategies are equivalent if the objective function is
strictly convex quadratic. When applied to generic non-quadratic functions, they behave
differently, as shown in [4]. The conjugate gradient approach was later extended to larger
unconstrained optimization problems in formula [2]. Conjugate gradient approaches are
now recognized as being particularly useful for dealing with large-scale unconstrained
optimization problems since they do not require the storing of matrices. Many conjugate
gradient techniques have lately been proposed that offer both strong global convergence
properties and good numerical performance. These approaches are based on the conjugacy
requirement of multiple conjugate gradient parameters see, [7], [9], [8], [6]. Previous
conjugate gradient algorithms made advantage of the conjugacy requirement:

dTk+1Qdk = 0. (10)

this is essential for the mathematical experiment and the convergence analysis [8].
Based on the quadratic function, a novel conjugate gradient method was developed,

rationalized, and studied. Numerical studies demonstrated that the suggested technique
outperformed other conjugate gradient methods.

2. Modification of Coefficients Conjugate Gradient

We utilizes the quadratic model to derivation the new coefficients conjugate gradient:

fk+1 = fk + sTk gk +
1

2
sTkQ(uk)sk (11)

where Q(uk) is the Hessian matrix. Taking the derivative of (11) on both sides for sk, we
get:

∇fk+1 = gk +Q(uk)sk (12)

By combining 5 and 12 in 11, we get:

sTkQ(uk)sk = fk − fk+1 +
1

2
sTk yk (13)

We may express the equation 13 using 3 as follows:

sTkQ(xk)sk = 2
(gTk sk)

2

(sTk yk + 2(fk − fk+1))
. (14)

The conjugacy condition, yields:
dTk+1Qsk = 0 (15)

From 14 through 15, and after some algebra, we get:

βk =
2

(gTk sk)
2

sTk yk(s
T
k yk+2(fk−fk+1))

gTk+1yk

dTk yk
(16)



978

If exact line search is utilized in 16, then βk is such that:

βBE1
k =

2
(gTk sk)

2

sTk yk(s
T
k yk+2(fk−fk+1))

∥gk+1∥2

dTk yk
(17)

and

βBE2
k =

2
(gTk sk)

2

sTk gk(2(fk−fk+1)−sTk gk)
∥gk+1∥2

dTk gk
(18)

By the above modification, it is not difficult to see that:

βBE3
k =

2
(gTk sk)

2

ak∥gk∥2(ak∥gk∥2+2(fk−fk+1))
∥gk+1∥2

∥gk∥2
(19)

from 17, 18 and 19 we define new formula denote by BE1, BE2 and BE3. Inspired by the
above direction, we refer to the proposed algorithm as:

Algorithm (2.1): (BE1,BE2 and BE3).
Stage 1. Choose the first starting location u1. Set d1 = −g1.
Stage 2. if ∥gk∥ = 0 stop.
Stage 3. State Determine if αk meets the requirements 6 and 7.
Stage 4. Compute βk as given in formula (2.17-2.19).
Stage 5. Create the search direction as dk+1 = −gk+1 + βkdk.
Stage 6. Set k = k + 1 and go to Stage 2.

3. Convergence analysis

To give the convergence result, the following assumptions are given. Assumptions:
In an open convex set Ψ that contains the level set Ψ = x ∈ Rn : f(x) ≤ f(x0), where

x0 is given, f is differentiable and its gradient is Lipschitz continuous, namely, there exists
a constant L > 0 such that:

∥g(z)− g(u)∥ = L ∥z − u∥ ,∀z, u ∈ Rn. (20)

With a constant ¥ ≥ 0 such that ∥∇f(x)∥ = ¥ , see [9].
The following important result was obtained by Zoutendijk [16].

Lemma 1. Any iteration method in which ak is obtained by the Wolfe line search and the
Assumptions holds. Then:

∞∑
k=1

(gTk dk)
2

∥dk∥2
< ∞. (21)

Theorem 1. Let the direction dk+1 be yielded by the new Algorithms. Then, we obtained:

dTk+1gk+1 < 0 and dTk+1gk+1 = βkd
T
k gk. (22)



979

Proof.
Using 8, it holds that:

dTk+1gk+1 = −gTk+1gk+1 + βkd
T
k gk+1 (23)

On 17 and after some algebra another is propose as:

∥gk+1∥2 =
βBE1
k dTk yk

2
(gTk sk)2

sTk yk(s
T
k yk+2(fk−fk+1))

(24)

Applying 24 on 23 will lead to the following:

dTk+1gk+1 = βBE1
k [dTk gk+1 − dTk yk] = βBE1

k dTk gk < 0 (25)

This inequality implies 22 is satisfied for k + 1.

Theorem 2. Suppose that Assumptions hold. Let {uk} be generated by new Algorithm.
Then one has:

lim
k→∞

inf |gk| = 0. (26)

Proof. Similar to the proof done in Hassan [6].

4. Computational Experience

After that, we do numerical experiments to evaluate Algorithm (2.1) and compare the
performance of the FR technique to salt-and-pepper impulse noise reduction. Table 1
shows the techniques used to evaluate photographs: placeLena, Home, Cameraman, and
Elaine. To assess restoration success qualitatively, we use the PSNR (peak signal to noise
ratio):

PSNR = 10log10
2552

1
MN

∑
i,j (u

r
i,j − u∗i,j)

2
(27)

where uri,j and u∗i,j represent the restored and original image pixel values.
Both approaches’ stopping criteria are:

|f(uk)− f(uk−1)|
|f(uk)|

≤ 10−4 and ∥f(uk)∥ = 10−4(1 + |f(uk)|) (28)

For more details see [5],[15].
Table (1) displays the PSNR, total number of iterates (NI), and function evaluation

(NF).



980

Table 1: Numerical results of FR, BE1, BE2 and BE3 algorithms.

Table (1), shows that the suggested algorithms beat the FR technique in terms of
iterations and function evaluations, as well as peak signal to noise ratio.

Figure 1: Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, (b)
recovered images through FR, (c), (d) and (e) restored images using BE1, BE2 and BE3 of 256× 256.

Figure 2: Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, (b)
recovered images through FR, (c), (d) and (e) restored images using BE1, BE2 and BE3 of 256× 256.

Figure 3: Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, (b)
recovered images through FR, (c), (d) and (e) restored images using BE1, BE2 and BE3 of 256× 256.



981

Figure 4: Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, (b)
recovered images through FR, (c), (d) and (e) restored images using BE1, BE2 and BE3 of 256× 256.

5. Conclusions

In this study, we introduce a parameter conjugate gradient strategy based on the
quadratic function for picture restoration problems, and we analyze its global convergence
under various mild circumstances. Our solution is promising and practical, as demon-
strated by numerical tests for restricted picture restoration challenges.

References

[1] Yu-Hong Dai and Yaxiang Yuan. A nonlinear conjugate gradient method with a
strong global convergence property. SIAM Journal on optimization, 10(1):177–182,
1999.

[2] Reeves Fletcher and Colin M Reeves. Function minimization by conjugate gradients.
The computer journal, 7(2):149–154, 1964.

[3] Roger Fletcher. Practical methods of optimization. John Wiley & Sons, 2013.

[4] Dai Y. H. and Yuan Y. onlinear conjugate gradient method. Beijing, 1998.

[5] Basim A. Hassan and Ali Ahmed A. Abdullah. Improvement of conjugate gradi-
ent methods for removing impulse noise images. Indonesian Journal of Electrical
Engineering and Computer Science, 29(1):245–251, 2023.

[6] Basim A Hassan and Haneen Alashoor. A new type coefficient conjugate on the
gradient methods for impulse noise removal in images. European Journal of Pure and
Applied Mathematics, 15(4):2043–2053, 2022.

[7] Basim A. Hassan and Haneen A. Alashoor. On image restoration problems using
new conjugate gradient methods. Indonesian Journal of Electrical Engineering and
Computer Science, 29(3):1438–1445, 2023.

[8] Basim A Hassan and Hameed Sadiq. Efficient new conjugate gradient methods for
removing impulse noise images. European Journal of Pure and Applied Mathematics,
15(4):2011–2021, 2022.



REFERENCES 982

[9] Basim A Hassan and Hameed M Sadiq. A new formula on the conjugate gradient
method for removing impulse noise images. Bulletin of the South Ural State Univer-
sity. Ser. Mathematical Modelling, Programming and Computer Software, 15(4):123–
130, 2022.

[10] Magnus R Hestenes, Eduard Stiefel, et al. Methods of conjugate gradients for solving
linear systems. Journal of research of the National Bureau of Standards, 49(6):409–
436, 1952.

[11] Nocedal Jorge and J Wright Stephen. Numerical optimization, 2006.

[12] Y Liu and C Storey. Efficient generalized conjugate gradient algorithms, part 1:
theory. Journal of optimization theory and applications, 69:129–137, 1991.

[13] Elijah Polak and Gerard Ribiere. Note sur la convergence de méthodes de direc-
tions conjuguées. Revue française d’informatique et de recherche opérationnelle. Série
rouge, 3(16):35–43, 1969.

[14] Wei Xue, Junhong Ren, Xiao Zheng, Zhi Liu, and Yueyong Liang. A new dy conjugate
gradient method and applications to image denoising. IEICE TRANSACTIONS on
Information and Systems, 101(12):2984–2990, 2018.

[15] Gaohang Yu, Liqun Qi, Yimin Sun, and Yi Zhou. Impulse noise removal by a non-
monotone adaptive gradient method. Signal processing, 90(10):2891–2897, 2010.

[16] G Zoutendijk. Nonlinear programming, computational methods. Integer and nonlin-
ear programming, pages 37–86, 1970.