EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS
Vol. 15, No. 2, 2022, 784-795
ISSN 1307-5543 – ejpam.com
Published by New York Business Global

New Spectral Idea for Conjugate Gradient Methods
and Its Global Convergence Theorems

Aseel M. Qasim1, Zinah F. Salih2,∗

1 Department of Mathematics, College of Education of Pure Sciences, University of Mosul,
Mosul, Nineveh, Iraq
2 Department of Mathematics, College of Computers Sciences and Mathematics,
University of Mosul, Mosul, Nineveh, Iraq

Abstract. Recently, the unconstrained optimization conjugate gradient methods have been widely
utilized, especially for problems that are known as large-scale problems. This work proposes a new
spectral gradient coefficient obtained from a convex linear combination of two different gradient
coefficients to solve unconstrained optimization problems. One of the most essential features of
our suggested strategy is to guarantee the suitable subsidence direction of the line search precision.
Furthermore, the proposed strategy is more effective than previous conjugate gradient approaches
and stationery, which have been observed in the test problem. However, when it is compared
to other conjugate gradient methods, such as FR methods, the proposed method confirmed the
globally convergent, indicating that it can be used in scientific data computation.

2020 Mathematics Subject Classifications: 65K10

Key Words and Phrases: Conjugate gradient, Spectral, Unconstrained optimization, Global
convergence, Descent property

1. Introduction

The spectral gradient approaches presented by Barzilai and Browein [5] and later
researched by Raydan [23] have proven to be useful and valuable in this area, which are
utilized to determine the local minimizers of large-scale problems [2]. In this paper, we
describe a spectral gradient technique to solve unconstrained optimization problems of the
form:

min
x∈R

f(x) (1)

In most cases, the conjugate gradient approach creates a sequence {xk} in such a way:

xk+1 = xk + αkdk (2)

∗Corresponding author.
DOI: https://doi.org/10.29020/nybg.ejpam.v15i2.4364

Email addresses: aseel.albazaz@uomosul.edu.iq (A. M. Qasim),
zn f2020@uomosul.edu.iq (Z. F. Salih)

https://www.ejpam.com 784 © 2022 EJPAM All rights reserved.



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 785

For k = 0, x0 denotes the starting position, αk denotes the specified step size by a line
search, dk signifies the direction of search, which is specified by:

dk+1 = −gk+1 + βkdk (3)

where d0 = −g0 for k = 0 and βk is a scalar, gk+1 refers to the gradient ∇f(xk+1) at
the new point. The most familiar βk formulation is the Fletcher-Reeves (FR), Polak-
Ribire, Hestenes-Stiefel (HS), Dai and Yan (DY) formulations which are supplied by: see
[12][20][21][17][6]

βFR
k =

gTk+1gk+1

gTk gk
(4)

βPRP
k =

gTk+1yk

||gk||2
(5)

βHS
k =

gTk+1 gk+1

dTk yk
(6)

βDY
k =

gTk+1gk+1

yTk dk
(7)

Also, recently, Hassan [13] suggests a new formula which is denoted in equations (8), (9)
respectively.

βY
k =

gTk+1yk

(fk − fk+1)/αk−g
T
k dk/2

(8)

βG
k =

gTk+1gk+1

(fk − fk+1)/αk−g
T
k dk/2

(9)

Where the last two equations of βk are used to suggest a new form of βk, which we
call βNN

k . The other βk parameters have been offered in the literature see for example
[1][3][4][10][11]. To avoid the non-convergence in the nonlinear function which is used with
inexact line search the following condition is used:∣∣gTk+1gk

∣∣ > 0.2||gk+1|| (10)

Known as the Powell restart condition [7]. Also, to avoid the negative case of βk we
use Wolfe conditions to guarantee the convergence of non-linear conjugate methods for
the research [24]. More information on these line search methods can be found in the
literature [8][19][26][27]

f(xk + αkdk) ≤ f(xk) + δ1αkd
T
k gk (11)

g(xk + αkdk)
Tdk ≥ δ2αkd

T
k gk (12)



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 786

Where dk is descant direction, gTk dk < 0, and 0 < δ1 < δ2 < 1. Dai and Yuan [6] show
that the FR method satisfies the global convergence feature if the strong Wolfe conditions
are satisfied. Recently, combining the well numerical results execution of PRP and HS
with the globally convergent properties of the FR and DY methods [6], are the way that
is adopted in this paper, which will be discussed in the second section. the systematic
sequence of this paper is: The second section proposes a new spectral CG formula and the
section three establishes the new method’s global convergence. The numerical experiments
are shown in the fourth Section, and finally, the fifth Section presents the comparison of
the results and conclusion.

2. New Spectral formula for CG Method

The new concept is to redirect the search path so that the new path guarantees the
condition.

gTk+1 dk+1 ≤ −c ||gk+1||2 (13)

Zhang et al. [28] suggested a conjugate gradient approach based on the modification of
the Fletcher-Reeves method in such a way that the direction dk which is given by [14]:

dk+1 = −θgk+1 + βk dk (14)

and the scalar θ is defined as, θ =
(
1 + βk

dTk gk+1

||gk+1||2

)
Now to improve the proposed CG method, we combine the good computational properties
of βY

k and βG
k method which is given in equations (8) and (9) respectively. These meth-

ods have strong convergence properties, then rewrite the new formula of βk as a linear
combination of (8) and (9), therefor our new formula βNN

k becomes:

βNN
k = (1− r)βY

k + r βG
k (15)

Where 0 < r < 1, βY
k =

gTk+1 yk

(fk−fk+1)/αk− gTk dk/2
, and βG

k =
gTk+1 gk+1

(fk−fk+1)/αk− gTk dk/2

Now by multiplying (14) by yTk and using the Conjugacy condition [25]
yTk dk+1 = 0 in order to compute the value of r we get:

yTk (−θgk+1 + βk dk) = 0 (16)

−[1 + βk
dTk gk+1

||gk+1||2
]yTk gk+1 + βky

T
k dk = 0 (17)

By using (15) we get:

−
[
1 + [(1− r)βY

k + rβG
k

] dTk gk+1

||gk+1||2
]yTk gk+1 + [(1− r)βY

k + rβG
k ]y

T
k dk = 0

r

[
−βG

k

dTk gk+1

||gk+1||2
yTk gk+1 + βY

k

dTk gk+1

||gk+1||2
yTk gk+1 − βY

k yTk dk + βG
k y

T
k dk

]



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 787

= βY
k

dTk gk+1

||gk+1||2
yTk gk+1 − βY

k yTk dk + yTk gk+1

∴ r =
βY
k

(
dTk gk+1

||gk+1||2
yTk gk+1 − yTk dk

)
+ yTk gk+1(

βG
k − βY

k

) (
yTk dk −

dTk gk+1

||gk+1||2
yTk gk+1

) (18)

This is the final form of r which is denoted in the equation above. the explanation of
the outlines of the new algorithm might be stated as follows:

2.1. The NN-CG Algorithm

• 1st: take x1 ∈ Rn, ε > 0, d1 = −g1 if ||g1|| < ε , then quit.

• 2nd: if ||gk+1|| < ε then come to the end; otherwise proceed to 3th step.

• 3rd: By satisfying Wolfe conditions (11) and (12), find αk and take xk+1 = xk+αkdk

• 4th: find βk by equation (15), then compute dk+1 by (3).

• 5th: put k = k + 1, go-to 2nd .

3. The Global Convergence Theorem and hypothesis

The following hypothesis is required to investigate a new convergence method:

hypothesis number one:

(i) The set Ω = x ∈ Rn|f(x) ≤ f(x0) is bounded when x0 ∈ Rn, is an initial point.

(ii) There exists a constant W>0 in an open convex set N that includes f is continuously
differentiable and the gradient that satisfies Lipschitz condition is continuous. For
more detail see [29]. Such as:

||g (x)− g(y)|| ≤ W ||x− y||, ∀x, y ∈ Ω (19)

Furthermore, we can deduce from the hypothesis that p and ℓ > 0 are constants
such that:

||x|| ≤ p, ||g(x)|| < ℓ ,∈ Ω

Lemma 1. Consider that hypothesis number one is holding, and let any step of the type
(3) where dk is decline direction and αk fulfills conditions (11) and (12), so Zoutedijk
condition hold. [30][22].



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 788

∞∑
k=1

(gTk dk)
2

||dk||2
< ∞ (20)

Following that, we present additional lemmas that are critical for the global convergence
dissection.

Lemma 2. Suppose that hypothesis number one holds, and take into account any iteration
of the form (2) then the direction dk+1 which is given in (14) satisfies the sufficient descant
condition

dTk+1 gk+1 = −||gk+1||2 (21)

Proof. From (15), for k = 0 the result is hold and we have dT1 g1 = −||g1||2, now when
k > 0 we obtain:

gTk+1 dk+1 = −

(
1 + βk

gTk+1dk

||gk+1||2

)
||gk+1||2 + βkg

T
k+1dk

= −||gk+1||2 − βk
gTk+1 dk

||gk+1||2
.||gk+1||2 + βkg

T
k+1 dk = −||gk+1||2

(22)

We see that (21) holds for all k > 0. So, the proof is complete.

Lemma 3. Consider that hypothesis (1) and (2) hold and let αk satisfies Wolfe’s condition
(11), (12) then βk which is determined by (15) satisfies 0 < βk

Proof. from the line of inquiry condition (11), (12) as well as the sufficient descent
condition (21), and since dTk gk = −||gk||2 it is possible to show that:

βY
k =

gTk+1yk

(fk − fk+1)/αk−g
T
k dk/2

>
||gk+1||2 − gTk+1gk

−δ1dTk gk +
||gk||2

2

(23)

βG
k =

gTk+1gk+1

(fk − fk+1)/αk − gTk dk/2
>

||gk+1||2

−δ1dTk gk +
||gk||2

2

>
||gk+1||2

δ||gk||2 + ||gk||2
2

=
||gk+1||2(

δ + 1
2

)
||gk||2

> 0

(24)

Then from (23), (24) and (15) we have:

βNN
k = (1− r)βY

k + r βG
k

= (1− r)
||gk+1||2 − gTk+1gk(

δ + 1
2

)
||gk||2

+ r
||gk+1||2(

δ + 1
2

)
||gk||2

=
||gk+1||2 − gTk+1gk + rgTk+1gk(

δ + 1
2

)
||gk||2



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 789

From Powell restart condition (10) we have

βNN
k >

||gk+1||2 − 0.2||gk+1||2 + 0.2r||gk+1||2(
δ + 1

2

)
||gk||2

>
0.8||gk+1||2 + 0.2r||gk+1||2(

δ + 1
2

)
||gk||2

∴ βNN
k =

n||gk+1||2(
δ + 1

2

)
||gk||2

> 0

(25)

by assuming n = (0.8 + 0.2r), and since 0 < r < 1 then it is clearly that βNN
k > 0

Theorem 1. Assume that hypothesis number one is true, {xk} is a sequence generated by
the algorithm (2.1) and αk fulfill the Wolfe’s conditions (11) and (12), Then we need to
prove that limk→∞ inf ||gk+1|| = 0

Proof. we use the disagreement method to complete the proof which assumes that
||gk+1|| > γ for γ > 0 from (3) it follows that: dk+1 + gk+1 = βkdk by squaring the
equation (3) we have:

||dk+1||2 + 2gTk+1dk+1 + ||gk+1||2 = β2
k||dk||2

Since the sufficient descant condition (21) is held, we obtain:

||dk+1||2 = (βk)
2||dk||2 − ||gk+1||2 − 2gTk+1 dk+1

= (βk)
2||dk||2 − ||gk+1||2 + 2||gk+1||2

= (βk)
2||dk||2 + ||gk+1||2

(26)

Both sides are divided by (gTk+1 dk+1)
2
we have:

||dk+1||2

(gTk+1dk+1)
2 ≤

(
β2
k

) ||dk||2(
gTk+1dk+1

)2 +
||gk+1||2(
gTk+1dk+1

)2
≤ (βk)

2 ||dk||2(
gTk+1dk+1

)2 +
1

||gk+1||2

(27)

Now from (25), (27) we obtain:

||dk+1||2

(gTk+1dk+1)
2 ≤

(
n||gk+1||2(
δ + 1

2

)
||gk||2

)2
||dk||2

||gk+1||4
+

1

||gk+1||2

=
n2||dk||2(

δ + 1
2

)2 ||gk||4 +
1

||gk+1||2

=
n2||dk||2(

δ + 1
2

)2
(gTk dk)

2
+

1

||gk+1||2

(28)



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 790

Since that:
||d1||2

(gT1 d1)
2 =

1

||g1||2
(29)

From this and ||gk+1|| > ε2 ∀k we have:

||dk+1||2

(gTk+1dk+1
)
2 ≤

k+1∑
i=1

1

(δ + 1/2)2||gi||2
≤ k

(δ + 1/2)2ε2

∴
(gTk+1dk+1

)
2

||dk+1||2
≥ ( δ + 1/2)2ε2

k

Which indicates:
∞∑
k=1

(gTk+1dk+1
)
2

||dk+1||2
≥

∞∑
k=1

(δ + 1
2)

2
ε2

k
= ∞ (30)

And this is in direct opposition to the Zoutendijk condition (20), with this contradiction
we complete proof.

4. Numerical Experiments

Now we set up the numeral experiments, Compute and make a comparison between
our method vs. βFR

k . The comparison is written using the Fortran 90 program, and the
test function is implemented using the functions chosen from Andrei [2]. The measure
of stooping the algorithm is denoted as ||gk+1|| ≤ 10−6, We use 27 test problems with
different dimensions to experiment with the execution of the new method. Table 1 shows
the calculation result, with NOI, NOR, and NOF which stand for the amount of iterations
total, restart, and function evaluation, respectively. While Table 2 explains the execution
percentage of the new method opposite FR method, also the execution was analyzed by
the performance profile software which is developed by Dolan and Mor’e [9] which are seen
in Figures 1, 2, 3.

Figure 1: The performance profile of iteration.



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 791

Figure 2: The performance profile of function evaluation.

Figure 3: The performance profile of restart.



A. M. Qasim, Z. F. Salih / Eur. J. Pure Appl. Math, 15 (2) (2022), 784-795 792

Test Problem Dim NN FR
NOI NOR NOF NOI NOR NOF

Trigonometric
100 20 11 37 18 10 34
1000 33 19 63 40 24 69

Penalty
100 9 6 25 11 6 28
1000 51 43 959 210 202 6193

Perturbed Quadratic
100 102 35 156 102 31 156
1000 340 95 559 325 111 494

Raydan 2
100 4 4 9 4 4 9
1000 4 4 9 4 4 9

Diagonal 2
100 61 18 105 70 24 113
1000 221 74 377 211 74 351

Generalized Tridiagonal 1
100 23 7 46 22 6 44
1000 32 17 263 49 29 758

Extended Tridiagonal 1
100 11 6 23 10 5 21
1000 13 7 26 13 7 26

Generalized Tridiagonal 2
100 42 17 64 37 11 59
1000 62 25 107 67 28 102

Diagonal 4
100 4 2 8 4 2 8
1000 4 2 8 4 2 8

Diagonal 5
100 4 4 9 4 4 9
1000 4 4 9 4 4 9

Extended Himmelblau
100 22 11 38 10 6 19
1000 12 6 23 22 12 35

Extended PSC1
100 8 6 17 8 6 17
1000 7 5 15 7 5 15

Extended Powell
100 69 20 129 66 22 127
1000 100 31 232 89 31 167

Extended BD1
100 63 63 98 63 63 98
1000 67 67 105 67 67 105

Quadratic Diagonal Perturbed
100 73 9 128 52 11 89
1000 151 31 268 181 39 321

Extended Wood WOODS
100 32 13 60 25 9 48
1000 46 16 83 30 13 55

Quadratic QF1
100 95 30 153 85 22 134
1000 364 111 588 362 101 568

Extended EP1
100 2 2 5 2 2 5
1000 2 2 5 2 2 5

Extended Tridiagonal 2
100 39 15 63 36 12 57
1000 42 18 64 40 17 64

NONDIA
100 12 8 23 15 8 30
1000 12 7 25 9 6 19

DQDRTIC
100 8 1 17 6 1 13
1000 11 0 23 7 1 15

DIXMAANA
100 7 7 14 7 7 14
1000 7 3 14 7 5 14

DIZMAANB
100 10 10 18 10 10 18
1000 11 11 19 11 11 19

DIXMAANC
100 13 13 24 13 13 24
1000 14 14 25 14 14 25

DIXMAANE
100 79 28 119 82 27 126
1000 216 67 344 239 60 378

Partial Perturbed Quadratic
100 85 26 135 77 26 120
1000 250 65 410 249 68 412

Broyden Tridiagonal
100 38 16 58 30 10 48
1000 40 17 68 33 8 63

Total 3051 1149 6272 3165 1303 11767

Table 1: Results for the NN and FR algorithm



REFERENCES 793

FR NN
NOI 100% 96.398%
NOR 100% 88.18%
NOF 100% 53.30%

Table 2: Efficiency of the NN method

5. Conclusion

In this paper, we searched for a new conjugate gradient method that depends on the
spectral strategy. We established, under acceptable assumptions, that the global con-
vergence for one of the offered approaches is satisfied. The arithmetical computation
explained in Table 1 shows the efficiency of the proposed algorithm outperformed the reg-
ular FR method on average, according to the numerical results and Figures 1, 2, 3, which
are denoted by the performance profile iteration and the number of function evaluations
and CPU time respectively. Furthermore, there are exist many good progress in conjugate
gradient algorithms see for example [16][15][18]

References

[1] Mehiddin Al-Baali, Yasushi Narushima, and Hiroshi Yabe. A family of three-term
conjugate gradient methods with sufficient descent property for unconstrained opti-
mization. Computational Optimization and Applications, 60(1):89–110, 2015.

[2] Neculai Andrei. An unconstrained optimization test functions collection. Adv. Model.
Optim, 10(1):147–161, 2008.

[3] Neculai Andrei. A new three-term conjugate gradient algorithm for unconstrained
optimization. Numerical Algorithms, 68(2):305–321, 2015.

[4] Saman Babaie-Kafaki. Two modified scaled nonlinear conjugate gradient methods.
Journal of Computational and Applied Mathematics, 261:172–182, 2014.

[5] Jonathan Barzilai and Jonathan M Borwein. Two-point step size gradient methods.
IMA journal of numerical analysis, 8(1):141–148, 1988.

[6] YH Dai and Ya-xiang YUAN. Convergence properties of the fletcher-reeves method.
IMA Journal of Numerical Analysis, 16(2):155–164, 1996.

[7] Yuhong Dai and Yaxiang Yuan. Convergence properties of beale-powell restart algo-
rithm. Science in China Series A: Mathematics, 41(11):1142–1150, 1998.

[8] Zhifeng Dai and Fenghua Wen. Another improved wei–yao–liu nonlinear conjugate
gradient method with sufficient descent property. Applied Mathematics and Compu-
tation, 218(14):7421–7430, 2012.



REFERENCES 794

[9] Elizabeth D. Dolan and Jorge J. Moré. Benchmarking optimization software with
performance profiles. Mathematical Programming, 91(2):201–213, jan 2002.

[10] XiaoLiang Dong, Hongwei Liu, and Yubo He. A self-adjusting conjugate gradient
method with sufficient descent condition and conjugacy condition. Journal of Opti-
mization Theory and Applications, 165(1):225–241, 2015.

[11] Yunda Dong. A practical pr+ conjugate gradient method only using gradient. Applied
Mathematics and Computation, 219(4):2041–2052, 2012.

[12] Reeves Fletcher and Colin M Reeves. Function minimization by conjugate gradients.
The computer journal, 7(2):149–154, 1964.

[13] Basim A Hassan. A new formula for conjugate parameter computation based on
the quadratic model. Indonesian Journal of Electrical Engineering and Computer
Science, 3(3):954–961, 2019.

[14] Basim A Hassan, Sawsan S Ismael, and Osam M Taher. A new spectral gradient co-
efficient is based on the convex combination of the two different conjugate coefficients
for optimization. International Journal of Enhanced Research in Science, Technology
Engineering, 7(2):44–50, 2018.

[15] Basim Abbas Hassan, Zeyad M. Abdullah, and Hawraz N. Jabbar. A descent exten-
sion of the dai - yuan conjugate gradient technique. Indonesian Journal of Electrical
Engineering and Computer Science, 16(2):661, nov 2019.

[16] Basim Abbas Hassan, Hussein O. Dahawi, and Azzam S. Younus. A new kind of
parameter conjugate gradient for unconstrained optimization. Indonesian Journal of
Electrical Engineering and Computer Science, 17(1):404, jan 2020.

[17] Magnus R Hestenes and Eduard Stiefel. Methods of conjugate gradients for solving.
Journal of research of the National Bureau of Standards, 49(6):409, 1952.

[18] Hawraz N. Jabbar and Basim A. Hassan. Two-versions of descent conjugate gradient
methods for large-scale unconstrained optimization. Indonesian Journal of Electrical
Engineering and Computer Science, 22(3):1643, jun 2021.

[19] Hailin Liu. A mixture conjugate gradient method for unconstrained optimization.
In 2010 Third International Symposium on Intelligent Information Technology and
Security Informatics, pages 26–29. IEEE, 2010.

[20] Elijah Polak and Gerard Ribiere. Note sur la convergence de méthodes de directions
conjuguées. ESAIM: Mathematical Modelling and Numerical Analysis-Modélisation
Mathématique et Analyse Numérique, 3(R1):35–43, 1969.

[21] Boris Teodorovich Polyak. The conjugate gradient method in extremal problems.
USSR Computational Mathematics and Mathematical Physics, 9(4):94–112, 1969.



REFERENCES 795

[22] Aseel M Qasim, Zinah F Salih, and Basim A Hassan. A new conjugate gradient algo-
rithms using conjugacy condition for solving unconstrained optimization. Indonesian
Journal of Electrical Engineering and Computer Science, 24(3):1654–1660, 2021.

[23] Marcos Raydan. On the barzilai and borwein choice of steplength for the gradient
method. IMA Journal of Numerical Analysis, 13(3):321–326, 1993.

[24] Jian Wang and Xuebin Chi. Cg global convergence properties with goldstein line-
search. Bulletin of the Brazilian Mathematical Society, 36(2):197–204, 2005.

[25] Shengwei Yao, Xiwen Lu, and Bin Qin. A modified conjugacy condition and related
nonlinear conjugate gradient method. Mathematical Problems in Engineering, 2014:1–
9, 2014.

[26] Gonglin Yuan and Xiwen Lu. A modified prp conjugate gradient method. Annals of
Operations Research, 166(1):73–90, 2009.

[27] Li Zhang, Weijun Zhou, and Dong-Hui Li. A descent modified polak–ribière–polyak
conjugate gradient method and its global convergence. IMA Journal of Numerical
Analysis, 26(4):629–640, 2006.

[28] Li Zhang, Weijun Zhou, and Donghui Li. Global convergence of a modified fletcher–
reeves conjugate gradient method with armijo-type line search. Numerische mathe-
matik, 104(4):561–572, 2006.

[29] Anwa Zhou, Zhibin Zhu, Hao Fan, and Qian Qing. Three new hybrid conjugate
gradient methods for optimization. Applied Mathematics, 2(3):303, 2011.

[30] G Zoutendijk. Nonlinear programming, computational methods. Integer and nonlin-
ear programming, pages 37–86, 1970.