EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS
Vol. 16, No. 1, 2023, 233-242
ISSN 1307-5543 – ejpam.com
Published by New York Business Global

A New Scalar of Conjugate Gradient Methods for
Solving Unconstrained Minimization

Saja O. Mohammad T.1, Hamsa Th. Saeed Chilmeran2, Rana Z. Al-Kawaz3,∗

1 Department of Mathematics, College of Basic Education, University of Mosul, Mosul, Iraq
2 Department of Mathematics, College of Computer Sciences and Mathematics,
University of Mosul, Iraq
3 Department of Mathematics, College of Basic Education, University of Telafer, Mosul,
Iraq

Abstract. In this paper, we derive a search direction for the conjugate-gradient method based
on the use of the self-scaling Quasi Newton-method, and the usefulness of the new method is to
solve unconstrained optimization problems with large dimensions. To clarify the importance of
the proposed method, we have shown its characteristics in terms of the sufficient descent condition
and the theoretically global convergence condition. Numerically, we applied the proposed method
to a variety of known test functions to prove its effectiveness. When compared with some previous
methods in the same direction, the proposed method proved to be superior to them that the tools
used for this purpose.

2020 Mathematics Subject Classifications: 65K05, 90C53, 65K10, 49M37

Key Words and Phrases: Conjugate-gradient, self-scaling, Quasi Newton-method, sufficient
descent, global convergence

1. Introduction

Optimization methods are divided into two types: methods that depend on the exis-
tence of a target function and a set of restrictions, regardless of the number of variables
used in the issue, and they have much wide life and engineering applications to this day
called (constrained optimization methods) and a second type that depends only on the
presence of an objective function without restrictions is called (unconstrained optimization
methods) also, regardless of the number of the issue variables, and it has wider applica-
tions than the first type. The conjugate-gradient methods were influential in the field of
finding minimization for quadratic and convex functions because they do not need to store

∗Corresponding author.
DOI: https://doi.org/10.29020/nybg.ejpam.v16i1.4619

Email addresses: saja-othmanmt@uomosul.edu.iq (S. Mohammad T.), hamsathrot@uomosul.edu.iq
(H. Chilmeran), rana.alkawaz@yahoo.com, ranazidan@uotelafer.edu.iq (R. Al-Kawaz)

https://www.ejpam.com 233 © 2023 EJPAM All rights reserved.



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 234

any matrix during their implementation. An unconstrained optimization problem known
as minimizing a function is defined as:

minimizex∈Rnf(x) (1)

As the function in 1 is for the real variable, which can use iterative methods to solve
it, such as the conjugate-gradient method (CG) or Newton’s methods or Quasi-Newton
(QN), which use the update new-point through:

xk+1 = xk + τkpk (2)

Most of the iterative methods focus on the length of the step τk specified for the necessary
distance to the search direction pk+1 which is obtained from (CG) i.e.:

pk+1 = −µk+1 + γkpk (3)

Where µk+1 = ∇f(xk+1). In the past several decades, γk has been defined in several
scientific papers such as [15, 17, 18, 21]. To ensure the convergence and descent direction
of this method when solving different types of general functions, we need to calculate the
step length with one of the inexact lines search such as the Wolfe line search (WLS), from
which two main types emerge:

(i) Weak type called Wolfe’s weak line search (WWLS) and it defines:

fk+1 − fk ≤ δτkµk
T pk (4)

µk+1
T pk ≥ σµk

T pk (5)

(ii) A strong type is called Wolfe’s strong line search (WSLS) and it defines as:

fk+1 − fk ≤ δτkµk
T pk (6)

|µk+1
T pk| ≤ −σµk

T pk (7)

Where 0 < δ < σ < 1. This is now a very mature field to be now converted into many
updated and expanded formats and with different applications as in [1–3, 8, 11–14, 19].
Later, the field of application gradually expanded to include conjugate gradient directions
for solving non-linear systems as in [5–7].

2. A New Parameter for γk

In this experiment, we introduce a new parameter, γk, to improve the CG orientation
based on a Quasi-Newton update of matrix approximation Ak known as a modified (DFP-
QN) consisting of:

Ak+1 = Ak −
Akωkω

T
k Ak

ωT
k Akωk

+
sks

T
k

sTk ωk
(8)



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 235

where sk = xk+1 − xk , yk = µk+1 − µk and

ωk = yk +
ρk

∥sk∥2
sk (9)

ρk = 2[fk − fk+1] + (µk+1 + µk)
T sk (10)

To learn more about equations (9) and (10) you can see [22]. We suggest multiplying the

updated DFB matrix by self-scaling ηk+1 =
yTk sk

µT
k Akµk

which is defined in [4] to get:

Ak+1 =
yTk sk

µT
kAkµk

(
Ak −

Akωkω
T
k Ak

ωT
k Akωk

+
sks

T
k

sTk ωk

)
(11)

Also,replace yk in the ηk+1 formula to the ωk. As for the search direction for a method
QN, it is:

pk+1 = −Ak+1µk+1 (12)

Now we substitute equation (11) into the equation (12), that is:

pk+1 = −
(

ωT
k sk

µT
kAkµk

(
Ak −

Akωkω
T
k Ak

ωT
k Akωk

+
sks

T
k

sTk ωk

))
µk+1 (13)

Then we make the search direction memoryless, i.e. by replacing each Ak = I, we get:

pk+1 = −
ωT
k sk

∥µk∥2

(
µk+1 −

ωT
k µk+1

∥ωk∥2
ωk +

sTk µk+1

sTk ωk
sk

)
(14)

To increase the efficiency of the search direction for CG methods, we equate it with the new
search direction for QN methods, and we deduce from the equation a new γk parameter
as in the following steps:

−µk+1 + γksk = −
ωT
k sk

∥µk∥2

(
µk+1 −

ωT
k µk+1

∥ωk∥2
ωk +

sTk µk+1

sTk ωk
sk

)
(15)

We multiply both sides of the equation (15) by ωk to get:

−ωT
k µk+1 + γkω

T
k sk = −

ωT
k sk

∥µk∥2
ωT
k µk+1 +

ωT
k sk

∥µk∥2
ωT
k µk+1

∥ωk∥2
ωT
k ωk −

ωT
k sk

∥µk∥2
sTk µk+1

sTk ωk
ωT
k sk (16)

By simplifying equation (16):

γk =
ωT
k µk+1

ωT
k sk

−
ωT
k sk

∥µk∥2
ωT
k µk+1

ωT
k sk

+
ωT
k sk

∥µk∥2
ωT
k µk+1

∥ωk∥2
∥ωk∥2

ωT
k sk

−
ωT
k sk

∥µk∥2
sTk µk+1

sTk ωk

ωT
k sk

ωT
k sk

(17)

then

γSHR
k =

ωT
k µk+1

ωT
k sk

−
sTk µk+1

∥µk∥2
(18)



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 236

Equation (18) is the new formula of the parameter γSHR
k that can be inserted into the

search direction of the CG (3) to give a new search direction formula as in:

pk+1 = −µk+1 + γSHR
k sk (19)

The following theorem is to test the property of sufficient descent for the new search
direction mentioned in the equations (18) and (19) that is:

Theorem 1. Suppose xk is generated by the new algorithm in equations (18) and (19)
with the step τk is computed by applying the (WSLS) condition in (6) and (7) then the
direction is sufficiently-descent i.e.:

pTk+1µk+1 ≤ − (1− u) ∥µk+1∥2 (20)

Proof. From the new search direction equation (19) and multiplying it from both

directions by
(

µk+1

∥µk+1∥2

)
we get:

pTk+1µk+1

∥µk+1∥2
+ 1 =

(
ωT
k µk+1

ωT
k sk

−
sTk µk+1

∥µk∥2

)
sTk µk+1

∥µk+1∥2
(21)

Since ωT
k µk+1 ≤ ∥ωk∥∥µk+1∥ and sTk µk+1 ≤ sTk ωk, that is

pTk+1µk+1

∥µk+1∥2
+ 1 ≤

(
∥ωk∥∥µk+1∥

ωT
k sk

−
sTk ωk

∥µk∥2

)
sTk ωk

∥µk+1∥2

pTk+1µk+1

∥µk+1∥2
+ 1 ≤ ∥ωk∥∥µk+1∥

∥µk+1∥2
−

(
sTk ωk

)2
∥µk∥2∥µk+1∥2

pTk+1µk+1

∥µk+1∥2
+ 1 ≤ ∥ωk∥

∥µk+1∥
−

(
sTk ωk

)2
∥µk∥2∥µk+1∥2

pTk+1µk+1

∥µk+1∥2
+ 1 ≤ ∥ωk∥

∥µk+1∥

let u = ∥ωk∥
∥µk+1∥ . So we got the equation (20) we need to prove.

3. Global Convergence Property

Before starting this section, we need to make some assumptions necessary to prove the
convergence of the new algorithm (SHR-CG), as shown:



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 237

3.1. Assumption[20]

(i) Assume f(x) is bound on the level specified by φ = {x ∈ Rn, f(x) ≤ f(x0)} from
below. Whereas x0 is the point of departure.

(ii) In some neighborhoods N of φ, the objective function is continuously-differentiable,
and its gradient is continuous-Lipschitz, which is, there is such a constant L > 0
such that:

∥µ(x)− µ(y)∥ ≤ L∥x− y∥,∀x, y ∈ φ. (22)

We can conclude that ζ, ζ̄ > 0 exists by applying assumptions (i) and (ii) to f .

ζ ≤ ∥µ(x)∥ ≤ ζ̄ (23)

(iii) The following requirement is fulfilled:

∥ (µ(x)− µ(y))T (x− y)∥ ≥ ς∥sk∥2,∀x, y ∈ φ, ς > 0. (24)

Lemma 1. If the assumption is correct and each conjugated gradient method has a de-
scending direction pk+1 with a step length τk of equations (6) and (7) if so,∑

k≥1

1

∥pk+1∥2
= ∞ (25)

So the next equation is right:
lim inf
k→∞

∥µk∥ = 0 (26)

Theorem 2. If the assumption is adhered to and the new direction pk+1 described by
equation (19) is descent, then

lim inf
k→∞

∥µk∥ = 0 (27)

Proof. From the formula of the new algorithm, we take the absolute value of the
proposed parameter γSHR

k i.e. equation (18) as in:

|γSHR
k | =

∣∣∣∣ωT
k µk+1

ωT
k sk

−
sTk µk+1

∥µk∥2

∣∣∣∣
|γSHR

k | ≤
∣∣∣∣ωT

k µk+1

ωT
k sk

∣∣∣∣+ ∣∣∣∣sTk µk+1

∥µk∥2

∣∣∣∣ (28)

Since ωT
k µk+1 ≤ ∥ωk∥∥µk+1∥ and from equation (7) substitute it into equation (28) i.e.:

|γSHR
k | ≤ ∥ωk∥∥µk+1∥

ς∥sk∥2
+

∣∣∣∣σµT
k pk

∥µk∥2

∣∣∣∣ (29)



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 238

Since pk = −µk and ωk = yk +
ρk

∥sk∥2
sk

|γSHR
k | ≤

∥yk + ρk
∥sk∥2

sk∥∥µk+1∥
ς∥sk∥2

+
σ∥µk∥∥sk∥

∥µk∥2

|γSHR
k | ≤

∥yk∥∥µk+1∥+ ρk
∥sk∥2

∥sk∥∥µk+1∥
ς∥sk∥2

+
σ∥sk∥
∥µk∥

(30)

from the Lipschitz condition:

|γSHR
k | ≤ L∥sk∥∥µk+1∥∥sk∥2 + ρk∥sk∥∥µk+1∥

ς∥sk∥4
+

σ∥sk∥
∥µk∥

|γSHR
k | ≤ L∥µk+1∥

ς∥sk∥
+

ρk∥µk+1∥
ς∥sk∥3

+
σ∥sk∥
∥µk∥

(31)

As the assumption is fulfilled

|γSHR
k | ≤ Lζ

ς∥sk∥
+

ρkζ

ς∥sk∥3
+ στk (32)

let Lζ
ς∥sk∥ + ρkζ

ς∥sk∥3
+ στk = ξ, by taking the absolute value of equation (19) we get:

∥pk+1∥ ≤ ∥µk+1 + γSHR
k sk∥

∥pk+1∥ ≤ ∥µk+1∥+ ∥γSHR
k ∥∥sk∥

∥pk+1∥ ≤ ζ̄ + ξ∥sk∥ = D̆∑
k≥1

1

∥pk+1∥2
≥ 1

D̆

∑
k≥1

1 = ∞ (33)

lim inf
k→∞

∥µk∥ = 0 (34)

So the proof is complete.

4. Numerical Result

We present a numerical comparison in this section between the new algorithm proposed
in equation (19) concerning (35) standard test functions from [9, 10] with the algorithms
(HS) [21] and (HZ) [18]. The tests were based on the following tools:

• The number of times the iteration is calculated (ITC).

• The number of times the function and its derivative are calculated (FDC).

• The time it takes to execute for the processor (TEP).



S. Mohammad T., H. Chilmeran, R. Al-Kawaz / Eur. J. Pure Appl. Math, 16 (1) (2023), 233-242 239

These comparisons were carried out by code in the FORTRAN program and with a stop
scale µk ≤ 10−5. These results are plotted for better comparison based on the Dolan-
More method [16]. We all of the algorithms mentioned, we tried on small (1000) and large
dimensions (10000) of the variables with Wolfe’s search line (WSLS), and the results were
as in the following figures:

1 2 3 4 5 6 7

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P
e

rf
o

rm
a

n
c
e

 o
f 

IT
C

SHR

HS

HZ

Figure 1: The number of times the iteration is calculated (ITC).

Figure 1 shows the progress of the new algorithm (SHR) over the basic algorithms of
Hestenes and Stiefel (HS) and Hager and Zhang (HZ) about the calculated iterations of
the test functions during the implementation by Dolan-More method.

1 2 3 4 5 6 7

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P
e

rf
o

rm
a

n
c
e

 o
f 

F
D

C

SHR

HS

HZ

Figure 2: The number of times the function and its derivative are calculated (FDC).

In Figure 2, the new algorithm (SHR) is the best in performance and overcomes the ba-
sic algorithms concerning the number of times the function is calculated and it’s derivative
for each of the test functions computed during implementation through the Dolan-More
method.



REFERENCES 240

1 2 3 4 5 6 7

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P
e

rf
o

rm
a

n
c
e

 o
f 

T
E

P

SHR

HS

HZ

Figure 3: The time it takes to execute for the processor (TEP).

Finally, the Figure 3 confirms that the new algorithm (SHR) is the most efficient of the
basic algorithms compared with it within the figure in relation to the time spent during
the implementation of the mentioned algorithms.

5. Conclusions

This may be considered a promising aspect of the derivations in this manner used
within the paper. The numerical results in the figures presented in the previous section
show the distinction and efficiency of the new algorithm (SHR) when compared with the
previous basic algorithms and within standard functions for this purpose. Also, for the
theoretical side of this proposed algorithm, we have achieved sufficient-descent and its
global-convergence within some assumptions.

Acknowledgements

The paper is supported by the College of Computer Science and Mathematics and
the College of Basic Education at the University of Mosul and Talafar University in the
Republic of Iraq.

References

[1] H I Ahmed, R Z Al-Kawaz, and A Y Al-Bayati. Spectral three-term constrained
conjugate gradient algorithm for function minimizations. Journal of Applied Mathe-
matics, 2019, 2019.

[2] H I Ahmed and Gh M Al-Naemi. A modified dai-yuan conjugate gradient method
and its global convergence. Iraqi Journal of Science, 53(3):620–628, 2012.

[3] H I Ahmed, E T Hamed, and H Th Saeed Chilmeran. A modified bat algorithm
with conjugate gradient method for global optimization. International Journal of
Mathematics and Mathematical Sciences, 2020, 2020.



REFERENCES 241

[4] A Y Al-Bayati and M S Al-Salih. New vm-methods for nonlinear unconstrained
optimization. Journal of Education and Science, Mosul University, 21:169–182, 1994.

[5] R Z Al-Kawaz and A Y Al-Bayati. An adaptive search direction algorithm for the
modified projection minimization optimization. In IOP Conference Series: Materials
Science and Engineering, volume 1152, page 012019. IOP Publishing, 2021.

[6] R Z Al-Kawaz and A Y Al-Bayati. An effective new iterative cg-method to solve
unconstrained non-linear optimization issues. TELKOMNIKA (Telecommunication
Computing Electronics and Control), 19(6):1847–1856, 2021.

[7] R Z Al-Kawaz and A Y Al-Bayati. Show off the efficiency of the dai-liao method
in merging technology for monotonous non-linear problems. Indonesian Journal of
Electrical Engineering and Computer Sciences, 21(1):505–515, 2021.

[8] R Z Al-Kawaz, A Y Al-Bayati, and M S Jameel. Interaction between un updated
fr-cg algorithms with an optimal cuckoo algorithm. Indonesian Journal of Electrical
Engineering and Computer Science, 19(3):1497–1504, 2020.

[9] N Andrei. An unconstrained optimization test functions collection. Adv. Model.
Optim, 10(1):147–161, 2008.

[10] N Andrei. An adaptive conjugate gradient algorithm for large-scale unconstrained
optimization. Journal of Computational and Applied Mathematics, 292:83–91, 2016.

[11] H Th Saeed Chilmeran, H I Ahmed, E T Hamed, and A Y Al-Bayati. A new modifica-
tion of the quasi-newton method for unconstrained optimization. Indonesian Journal
of Electrical Engineering and Computer Science, 21(3):1683–1691, 2021.

[12] H Th Saeed Chilmeran, E T Hamed, H I Ahmed, and A Y Al-Bayati. A method
of two new augmented lagrange multiplier versions for solving constrained problems.
International Journal of Mathematics and Mathematical Sciences, 2022, 2022.

[13] H Th Saeed Chilmeran, E S Khaleel, H I Ahmed, and E T Hamed. A modified
sufficient descent spectral conjugate gradient method for minimization. In 2021 7th
International Conference on Contemporary Information Technology and Mathematics
(ICCITM), pages 239–242. IEEE, 2021.

[14] H Th Saeed Chilmeran and H Y Najem. New conjugacy coefficient for conjugate
gradient method for unconstrained optimization. AL-Rafidain Journal of Computer
Sciences and Mathematics, 10(2):33–46, 2013.

[15] Y H Dai and Y Yuan. A nonlinear conjugate gradient method with a strong global
convergence property. SIAM Journal on optimization, 10(1):177–182, 1999.

[16] E D Dolan and J J Moré. Benchmarking optimization software with performance
profiles. Mathematical programming, 91(2):201–213, 2002.



REFERENCES 242

[17] R Fletcher and C M Reeves. Function minimization by conjugate gradients. The
computer journal, 7(2):149–154, 1964.

[18] W W Hager and H Zhang. A new conjugate gradient method with guaranteed descent
and an efficient line search. SIAM Journal on optimization, 16(1):170–192, 2005.

[19] E T Hamed, R Z Al-Kawaz, and A Y Al-Bayati. New investigation for the liu-story
scaled conjugate gradient method for nonlinear optimization. Journal of Mathematics,
2020, 2020.

[20] M Hamoda, M Mamat, M Rivaie, and Z Salleh. A conjugate gradient method with
strong wolfe-powell line search for unconstrained optimization. Appl. Math. Sci,
10:721–734, 2016.

[21] M R Hestenes and E Stiefel. Methods of conjugate gradients for solving. Journal of
research of the National Bureau of Standards, 49(6):409, 1952.

[22] Z Wei, G Yu, G Yuan, and Z Lian. The superlinear convergence of a modified
bfgs-type method for unconstrained optimization. Computational optimization and
applications, 29(3):315–332, 2004.