Mathematics in Applied Sciences and Engineering https://ojs.lib.uwo.ca/mase
Volume 2, Number 1, March 2021, pp.22-31 https://doi.org/10.5206/mase/13393

TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF

SUBLINEARITY

SRIJANA GHIMIRE AND XIANG-SHENG WANG

Abstract. In this paper, we investigate traveling wave solutions of a diffusive predator-prey model

which takes into consideration hunting cooperation. The sublinearity condition is violated for the

function of cooperative predation. When the basic reproduction number for the diffusion-free model

is greater than one, we find a critical wave speed below which no positive traveling wave solution

shall exist. On the other hand, if the wave speed exceeds this critical value, we prove the existence

of a positive traveling wave solution connecting the predator-free equilibrium to the unique positive

equilibrium under a technical assumption of weak cooperative predation. The key idea of the proof

contains two major steps: (i) we construct a suitable pentahedron and find inside it a trajectory

connecting the predator-free equilibrium; and (ii) we construct a suitable Lyapunov function and use

LaSalle invariance principle to prove that the trajectory also connects the positive equilibrium. At the

end of this paper, we propose five open problems related to traveling wave solutions in cooperative

predation.

1. Introduction

The Lotka-Volterra system has been widely used in the models of predation ever since Lotka and

Volterra did two independent studies [17, 20] near one century ago. Taking spatial diffusion into

consideration, a general diffusive Lotka-Volterra system can be written as

∂tu = d1∂xxu + b(u) −f(u,v), (1.1)

∂tv = d2∂xxv + rf(u,v) −µv, (1.2)

where u(t,x) and v(t,x) correspond to the densities of prey and predator, respectively. The constants

d1 and d2 are two non-negative diffusion rates. In the absence of the predator, the dynamics of prey

is determined by the function b(u), which usually possesses a stable positive equilibrium K. The

constant K is also referred to as the carrying capacity in the literature. The only nonlinearity in

the model system is the predation rate f(u,v) which is an increasing function of both u and v, and

f(0,v) = f(u, 0) = 0. The constants r and µ are rates of energy conversation and predator death,

respectively. It is obvious that the model system has a predator-free equilibrium (K, 0). We also

assume that it has a positive equilibrium (U,V ), where V = rb(U)/µ and U is a positive root of

b(U) − f(U,rb(U)/µ) = 0. A traveling wave solution connecting the predator-free equilibrium to
the positive equilibrium is a solution of the special form (u(x + ct),v(x + ct)) with c > 0 such that

(u(−∞),v(−∞)) = (K, 0) and (u(∞),v(∞)) = (U,V ). There are plenty of results on the existence

Received by the editors 6 December 2020; accepted 29 January 2021; published online 1 February 2021.

2000 Mathematics Subject Classification. Primary 92D25; Secondary 35K57, 34B40.

Key words and phrases. Predator-prey model; cooperative predation; traveling waves; sublinearity.

22



TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF SUBLINEARITY 23

theory of traveling wave solutions to the Lotka-Volterra system of predation; see [5, 10, 11, 12, 13, 16, 22]

and references therein. In the aforementioned literature, sublinearity of the predation rate is a crucial

condition to prove existence of traveling wave solutions; that is, one should assume that there exists

M > 0 such that

f(u,v) ≤ Mv (1.3)

for all u ∈ [0,K] and v ≥ 0. The objective of this paper is to relax this sublinearity condition and
establish existence theory of travelling waves for a diffusive model of cooperative predation.

Cooperative hunting plays an important role in Phylogenetics [2] and is necessary for some predators

[19]. For example, the Yellowstone wolves of larger group size have bigger success rates in capturing

bison, their most formidable prey [18]. During the season of nonbreeding, Harris’ hawks in New Mexico

will improve capture success by cooperative hunting [3]. D. discoideum, which is a soil amoeba feeding

on bacteria, lives as single cells during most of the time, but will develop social cooperation if it is

under starvation conditions [4, 15]. A two-dimensional Lotka-Volterra system was proposed in [1] which

considers logistic growth of the prey and cooperative hunting on the predator. This model was extended

in [14] to investigate Allee effects on the prey. In this paper, we will introduce spatial diffusion to the

predator and study the following diffusive predator-prey model with cooperative predation.

∂tu = α− δu−puv −quv2, (1.4)

∂tv = d∂xxv + r(puv + quv
2) −µv, (1.5)

where α and δ are the constant birth rate and per capita death rate of the prey. The nonlinear predation

rate is composed of mass-action predation puv and cooperative predation quv2. For simplicity, we

neglect the spatial diffusion of the prey and assume that the predator has a constant diffusion rate

d > 0. The conversation rate of predation energy is r > 0 and the per capita death rate of the predator

is µ > 0.

It is obvious that the model system has a predator-free equilibrium (K, 0), where K = α/δ is the

carrying capacity of the prey. Define

R0 =
rpK

µ
=
rpα

µδ
(1.6)

as the basic reproduction number for the diffusion-free ordinary differential system. Throughout this

paper, we will assume that R0 > 1. A simple argument shows that the model system possesses a unique

positive equilibrium (U,V ) with

α− δU = pUV + qUV 2 = µV/r. (1.7)

We are interested in the traveling wave solution (u(x + ct),v(x + ct)) that connects the predator-free

equilibrium (K, 0) to the positive equilibrium (U,V ). As we mentioned earlier, the nonlinear predation

rate f(u,v) = puv +quv2 does not satisfy the sublinearity condition (1.3) and the results in the existing

literature do not apply. To overcome this difficulty, we will modify the shooting method which was

introduced by Dunbar [6, 7], and later developed by Hosono and Ilyas [8, 9] and by Huang [12, 13].

Especially, we will apply an extension of the technique introduced by Huang [12, 13] to prove the

existence of a positive traveling wave connecting the predator-prey equilibrium.

The rest of this paper is organized as follows. In Section 2, we introduce non-dimensional scales for

system (1.4)-(1.5) and state our main theorem. In Section 3, we convert the traveling wave equations to

a three-dimensional dynamical system and construct a suitable pentahedron. In Section 4, we analyze



24 SRIJANA GHIMIRE AND XIANG-SHENG WANG

the unstable manifold of the predator-free equilibrium and find a trajectory which approaches the

predator-free equilibrium as ξ → −∞ and stays in the pentahedron for all ξ ∈ R. In Section 5, we
use Lyapunov function technique to prove that this trajectory converges to the positive equilibrium as

ξ →∞. In Section 6, we give a proof of our main theorem. In Section 7, we conclude our paper with a
brief discussion and propose some open problems.

2. Nondimensionalization and main theorem

For convenience, we nondimensionalize the model by scaling the variables:

t̃ = δt, x̃ =
x√
d/δ

, ũ =
u

K
, ṽ =

v

rK
. (2.1)

Recall that K = α/δ. The system (1.4)-(1.5) is rewritten as

∂t̃ũ = 1 − ũ− p̃ũṽ − q̃ũṽ
2, (2.2)

∂t̃ṽ = ∂x̃x̃ṽ + p̃ũṽ + q̃ũṽ
2 − µ̃ṽ, (2.3)

where

p̃ =
prK

δ
, q̃ =

qr2K2

δ
, µ̃ =

µ

δ
. (2.4)

Hence, we may assume without loss of generality that α = δ = d = r = 1 and drop the tilde in the

above system. Denote ξ = x + ct. The traveling wave solution is a solution to the boundary value

problem

cu′ = 1 −u−puv −quv2, (2.5)

cv′ = v′′ + puv + quv2 −µv, (2.6)

together with the boundary conditions

u(−∞) = 1, v(−∞) = 0, u(∞) = U, v(∞) = V, (2.7)

where

1 −U = pUV + qUV 2 = µV. (2.8)

The condition R0 > 1 is simplified as p > µ. Our main theorem is stated as below.

Theorem 2.1. Let α = δ = d = r = 1. If p > µ, then for any c > 2
√
p−µ, there exists Q > 0 such

that the boundary value problem (2.5)-(2.7) with q ∈ [0,Q] has a positive solution which corresponds
to a traveling wave solution to (1.4)-(1.5) connecting the predator-free equilibrium to the positive

equilibrium. On the other hand, if p > µ and 0 ≤ c < 2
√
p−µ, then the boundary value problem

(2.5)-(2.7) with q ≥ 0 does not have a positive solution.

3. Dynamical system approach

We convert the boundary value problem (2.5)-(2.6) into a three-dimensional dynamical system

cu′ = 1 −u−puv −quv2, (3.1)

v′ = cv − cw, (3.2)

cw′ = puv + quv2 −µv. (3.3)



TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF SUBLINEARITY 25

We intend to find a trajectory such that the boundary conditions (2.7) are satisfied. Let ε > 0 be a

positive constant to be determined later. We obtain from the above equations

(cu + εv + cw)′ = 1 −ε(cu + εv + cw) − (1 − cε)u− (µ− cε−ε2)v. (3.4)

We will choose ε to be sufficiently small so that

1 − cε > 0, µ− cε−ε2 > 0; (3.5)

namely, we require

ε < min{
1

c
,
c +

√
c2 + 4µ

2
}. (3.6)

Let 0 < k1 < k2 be two positive constants to be determined later. We introduce a pentahedron P with

two parallel triangular bases

B0 := {u = 0, k1v < w < k2v, cu + εv + cw < 1/ε}, (3.7)

B1 := {u = 1, k1v < w < k2v, cu + εv + cw < 1/ε}, (3.8)

and three trapezoid sides

S0 := {cu + εv + cw = 1/ε, 0 < u < 1, k1v < w < k2v}, (3.9)

S1 := {w = k1v, 0 < u < 1, cu + εv + cw < 1/ε}, (3.10)

S2 := {w = k2v, 0 < u < 1, cu + εv + cw < 1/ε}. (3.11)

We have the following lemma.

Lemma 3.1. Assume p > µ, c > 2
√
p−µ and q < ε2[c2/4 − (p−µ)]. Define

k1 :=
1 −

√
1 − 4(p + q/ε2 −µ)/c2

2
, k2 :=

1 +
√

1 + 4µ/c2

2
. (3.12)

The direction fields of the ordinary differential system (3.1)-(3.3) point inward on B0 ∪ B1 ∪ S0 and
outward on S1 ∪S2. In other words, if (u(ξ1),v(ξ1),w(ξ1)) ∈ B0 ∪B1 ∪S0 for some ξ1 ∈ R, then there
exists δ > 0 such that (u(ξ),v(ξ),w(ξ)) ∈ P for all ξ ∈ (ξ1,ξ1 + δ) and (u(ξ),v(ξ),w(ξ)) /∈ P for all
ξ ∈ (ξ1 −δ,ξ1); and if (u(ξ1),v(ξ1),w(ξ1)) ∈ S1 ∪S2 for some ξ1 ∈ R, then there exists δ > 0 such that
(u(ξ),v(ξ),w(ξ)) /∈ P for all ξ ∈ (ξ1,ξ1 + δ) and (u(ξ),v(ξ),w(ξ)) ∈ P for all ξ ∈ (ξ1 −δ,ξ1).

Proof. It is obvious that 0 ≤ u ≤ 1, 0 ≤ v ≤ 1/ε2 and 0 ≤ w ≤ 1/(cε) for any (u,v,w) ∈ P .
If (u(ξ1),v(ξ1),w(ξ1)) ∈ B0 for some ξ1 ∈ R, then u(ξ1) = 0. It follows from (3.1) that u′(ξ1) =

1/c > 0. Hence, there exists δ > 0 such that u(ξ) > 0 for ξ ∈ (ξ1,ξ1 +δ) and u(ξ) < 0 for ξ ∈ (ξ1−δ,ξ1).
This implies that the direction field on B0 is inward.

If (u(ξ1),v(ξ1),w(ξ1)) ∈ B1 for some ξ1 ∈ R, then u(ξ1) = 1. It follows from (3.1) that u′(ξ1) =
−[pv(ξ1) + qv2(ξ1)]/c < 0. Hence, there exists δ > 0 such that u(ξ) < 1 for ξ ∈ (ξ1,ξ1 + δ) and u(ξ) > 1
for ξ ∈ (ξ1 − δ,ξ1). This implies that the direction field on B1 is inward.

If (u(ξ1),v(ξ1),w(ξ1)) ∈ S0 for some ξ1 ∈ R, then cu(ξ1) + εv(ξ1) + cw(ξ1) = 1/ε. It follows from
(3.4) that cu′(ξ1) + εv

′(ξ1) + cw
′(ξ1) = −(1 − cε)u(ξ1) − (µ − cε − ε2)v(ξ1) < 0. Hence, there exists

δ > 0 such that cu(ξ) + εv(ξ) + cw(ξ) < 1/ε for ξ ∈ (ξ1,ξ1 + δ) and cu(ξ) + εv(ξ) + cw(ξ) > 1/ε for
ξ ∈ (ξ1 −δ,ξ1). This implies that the direction field on S0 is inward.



26 SRIJANA GHIMIRE AND XIANG-SHENG WANG

If (u(ξ1),v(ξ1),w(ξ1)) ∈ S1 for some ξ1 ∈ R, then w(ξ1) = k1v(ξ1). It follows from (3.2) and (3.3)
that

w′(ξ1) −k1v′(ξ1)
c

<
p + q/ε2 −µ

c2
v(ξ1) −k1v(ξ1) + k21v(ξ1) = 0.

Hence, there exists δ > 0 such that w(ξ) < k1v(ξ) for ξ ∈ (ξ1,ξ1+δ) and w(ξ) > k1v(ξ) for ξ ∈ (ξ1−δ,ξ1).
This implies that the direction field on S1 is outward.

If (u(ξ1),v(ξ1),w(ξ1)) ∈ S2 for some ξ1 ∈ R, then w(ξ1) = k2v(ξ1). It follows from (3.2) and (3.3)
that

w′(ξ1) −k2v′(ξ1)
c

> −
µ

c2
v(ξ1) −k2v(ξ1) + k22v(ξ1) = 0.

Hence, there exists δ > 0 such that w(ξ) > k2v(ξ) for ξ ∈ (ξ1,ξ1+δ) and w(ξ) < k2v(ξ) for ξ ∈ (ξ1−δ,ξ1).
This implies that the direction field on S2 is outward. �

4. Unstable manifold of predator-free equilibrium

Throughout this section, we assume that the conditions in Lemma 3.1 are satisfied; namely, p > µ,

c > 2
√
p−µ and q < ε2[c2/4 − (p−µ)]. The predator-free equilibrium (1, 0) of the system (2.5)-(2.6)

corresponds to the equilibrium E0 = (1, 0, 0), which is also referred to as the predator-free equilibrium,

of the dynamical system (3.1)-(3.3). The Jacobian matrix at E0 is calculated as

J0 =


−1/c −p/c 00 c −c

0 (p−µ)/c 0


 . (4.1)

In addition to a negative eigenvalue −1/c, the matrix J0 has two positive eigenvalues

λ± =
c±

√
c2 − 4(p−µ)

2
. (4.2)

The eigenvectors associated with λ± are

e± =


−p/(cλ± + 1)1

1 −λ±/c


 . (4.3)

By [21, Theorem 3.2.1], the predator-free equilibrium E0 possesses a smooth two-dimensional local

invariant unstable manifold Wu(E0) tangent to the plane spanned by e+ and e−. Note that

1 −
λ±
c

=
1 ∓

√
1 − 4(p−µ)/c2

2
∈ (k1,k2),

where k1 and k2 are given in (3.12). Hence, both vectors e± starting at the equilibrium E0 = (1, 0, 0)

point inward the pentahedron P . We then find a smooth curve γ ∈ Wu(E0)∩P with two end points on
S1 and S2, respectively. For each i = 1, 2, we denote γi to be the point on γ such that the trajectories

starting from these points will remain in P until touching Si at a finite time. Obviously, γ1 and γ2

are disjoint open subsets of γ. Since γ is smooth and connected, there exists at least one point on

γ \ (γ1 ∪γ2). The trajectory starting from this point will never touch S1 or S2, and hence stays in P
all the time. We summarize our argument in the following lemma.



TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF SUBLINEARITY 27

Lemma 4.1. Assume p > µ, c > 2
√
p−µ and q < ε2[c2/4 − (p − µ)]. There exists a trajectory

(u(ξ),v(ξ),w(ξ)) of the dynamical system (3.1)-(3.3) such that

lim
ξ→−∞

(u(ξ),v(ξ),w(ξ)) = (1, 0, 0),

and (u(ξ),v(ξ),w(ξ)) ∈ P for all ξ ∈ R.

5. Heteroclinic orbit and Lyapunov function technique

Let (u(ξ),v(ξ),w(ξ)) be the trajectory which connects to the predator-free equilibrium E0 = (1, 0, 0)

as ξ →−∞ and stays in the pentahedron P for all ξ ∈ R; see Lemma 4.1. We will prove in this section
that this trajectory converges to the positive equilibrium E1 = (U,V,V ) as ξ →∞, where (U,V ) is the
positive equilibrium of the original system (1.4)-(1.5). From (2.8), we calculate

U =
q + pµ−

√
(q + pµ)2 − 4qµ2

2q
, V =

q −pµ +
√

(q + pµ)2 − 4qµ2
2qµ

.

It is easy to verify that 0 < U < 1 and k1 < 1 < k2. Moreover, in view of (2.8) and (3.6), we obtain

cU + εV + cV <
U

ε
+
µV

ε
=

1

ε
.

Consequently, (U,V,V ) ∈ P .

Lemma 5.1. Assume p > µ, c > 2
√
p−µ and q ≤ Q with

Q := min{
ε4c2(p−µ)

2µ2
,

2ε4c2

µ
,
ε2[c2 − 4(p−µ)]

5
} (5.1)

and

ε := min{
1

2c
,
−c +

√
c2 + 4µ

4
,

µ3/4

c[4/c2 + (µ− 1)2/(4µ)]1/4
}. (5.2)

Let (u(ξ),v(ξ),w(ξ)) be the trajectory given in Lemma 4.1 and (U,V,V ) be the positive equilibrium

calculated in (2.8). We have

lim
ξ→∞

(u(ξ),v(ξ),w(ξ)) = (U,V,V ).

Proof. For any (u,v,w) ∈ P , we construct a Lyapunov function as

L(u,v,w) := c(u−U ln u + w −V
w

v
−V ln v) +

κ

2c
[(cu +

µ

c
v + cw) − (cU +

µ

c
V + cV )]2, (5.3)

where κ > 0 is a positive constant to be determined later. Restricting the Lyapunov function on the

trajectory (u(ξ),v(ξ),w(ξ)) and taking derivative with respect to ξ yield

d

dξ
L(u(ξ),v(ξ),w(ξ)) =c[(1 −

U

u(ξ)
)u′(ξ) + (1 −

V

v(ξ)
)w′(ξ) + (

V w

v(ξ)2
−

V

v(ξ)
)v′(ξ)]

+
κ

c
[c(u(ξ) −U) +

µ

c
(v(ξ) −V ) + c(w(ξ) −V )][cu′(ξ) +

µ

c
v′(ξ) + cw′(ξ)].

For simplicity, we will drop the dependence on ξ and write u(ξ), v(ξ) and w(ξ) as u, v and w, respec-

tively.. It then follows from (3.1)-(3.3) that

L′ =[
u−U
u

(1 −u−puv −quv2) +
v −V
v

(puv + quv2 −µv) +
V (w −v)

v2
c2(v −w)]

+ κ[(u−U) +
µ

c2
(v −V ) + (w −V )](1 −u−µw).



28 SRIJANA GHIMIRE AND XIANG-SHENG WANG

Denote ū = u−U, v̄ = v −V and w̄ = w −V . It is readily seen from (2.8) that
u−U
u

(1 −u−puv −quv2) = −
1

uU
ū2 −pūv̄ −qūv̄2 − 2qV ūv̄,

v −V
v

(puv + quv2 −µv) = pūv̄ + qūv̄2 + qV ūv̄ + qUv̄2,

V (w −v)
v2

c2(v −w) = −
c2V

v2
(v̄2 − 2v̄w̄ + w̄2),

and

[(u−U) +
µ

c2
(v −V ) + (w −V )](1 −u−µw) = −ū2 −

µ

c2
ūv̄ − (µ + 1)ūw̄ −

µ2

c2
v̄w̄ −µw̄2.

Note that U ≤ 1, u ≤ 1 and v ≤ 1/ε2. We obtain from the above equations that

L′ ≤[−ū2 −qV ūv̄ + qUv̄2 −ε4c2V (v̄2 − 2v̄w̄ + w̄2)]

+ κ[−ū2 −
µ

c2
ūv̄ − (µ + 1)ūw̄ −

µ2

c2
v̄w̄ −µw̄2].

Now, we choose κ = 2ε4c4V/µ2 to eliminate v̄w̄ on the right-hand side of the above inequality. It follows

that

L′ ≤−(1 + κ)ū2 − (qV +
κµ

c2
)ūv̄ − (

κµ2

2c2
−qU)v̄2 −κ(µ + 1)ūw̄ − (

κµ2

2c2
+ κµ)w̄2 ≤ 0,

provided q < ε4c2V/U = κµ2/(2c2U) and

(qV + κµ/c2)2

2κµ2/c2 − 4qU
+

κ2(µ + 1)2

2κµ2/c2 + 4κµ
≤ 1 + κ. (5.4)

Note from (2.8) that µV + U = 1 ≥ pU/µ. Especially, V/U ≥ (p − µ)/µ2. For any q ≤ Q, we have
qV ≤ κµ/c2 and qU ≤ κµ2/(4c2). Consequently,

(qV + κµ/c2)2

2κµ2/c2 − 4qU
+

κ2(µ + 1)2

2κµ2/c2 + 4κµ
−κ ≤

4κ

c2
+

κ(µ + 1)2

2µ2/c2 + 4µ
−κ ≤

2ε4c4

µ3
[

4

c2
+

(µ− 1)2

4µ
],

where we have made use of κ = 2ε4c4V/µ2 and V ≤ 1/µ in the last inequality. By the choice of ε in
(5.2), the right-hand side of the above inequality is no more than 1. Hence, (5.4) is satisfied and L′ ≤ 0
for all q ≤ Q.

We claim that

lim
ξ→∞

L(u(ξ),v(ξ),w(ξ)) > −∞.

Assume to the contrary that L(u(ξ),v(ξ),w(ξ)) → −∞ as ξ → ∞. It follows from the definition of L
in (5.3) that v(ξ) → 0 as ξ → ∞. For any small ε0 > 0, there exists ξ0 ∈ R such that pu(ξ)v(ξ) +
qu(ξ)v(ξ)2 < ε0 for all ξ > ξ0. In view of (3.1), we have cu

′ > 1 − u − ε0 for all ξ > ξ0. By
comparison principle, we obtain lim infξ→∞u(ξ) ≥ 1 − ε0. Since ε0 > 0 is arbitrary, letting ε0 → 0+

gives lim infξ→∞u(ξ) ≥ 1. This together with u(ξ) ≤ 1 implies that limξ→∞u(ξ) = 1. Note that p > µ
and v(ξ) ≥ 0. There exists ξ1 ∈ R such that pu(ξ)v(ξ) −µv(ξ) ≥ 0 for all ξ ≥ ξ1. On account of (3.3),
we obtain cw′(ξ) ≥ 0 for all ξ ≥ ξ1. Let w∞ = limξ→∞w(ξ) ≥ 0. If w∞ > 0, then there exists ξ2 ∈ R
such that v(ξ) −w(ξ) < −w∞/2 for all ξ > ξ2. It follows from (3.2) that v′(ξ) < −cw∞/2 for ξ > ξ2,
which contradicts to limξ→∞v(ξ) = 0. Hence, w∞ = limξ→∞w(ξ) = 0. Since w(ξ) ≥ 0 and cw′(ξ) ≥ 0
for all ξ ≥ ξ1, we obtain w(ξ) = 0 for all ξ ≥ ξ1. Choose ξ3 ≤ ξ1 such that w(ξ) = 0 for all ξ ≥ ξ3
and w(ξ) > 0 if ξ is smaller than and close to ξ3. In view of (3.2) and limξ→∞v(ξ) = 0, we obtain



TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF SUBLINEARITY 29

v(ξ) = 0 for all ξ ≥ ξ3. Since the line v = w = 0 is negatively invariant for the system (3.1)-(3.3), we
have v(ξ) = w(ξ) = 0 for all ξ ∈ R, a contradiction.

The above argument implies that

L∞ := lim
ξ→∞

L(u(ξ),v(ξ),w(ξ)) > −∞.

Now, we apply LaSalle invariance principle to show that the trajectory (u(ξ),v(ξ),w(ξ)) converges to

the positive equilibrium (U,V,V ) as ξ → ∞. Let (ũ0, ṽ0, w̃0) be any point in the omega limit set of
the trajectory (u(ξ),v(ξ),w(ξ)); namely, there exists a subsequence ξ1 < ξ2 < · · · < ξn → ∞ such
that (u(ξn),v(ξn),w(ξn)) → (ũ0, ṽ0, w̃0) as n →∞. Let (ũ(ξ), ṽ(ξ), w̃(ξ)) be the solution of (3.1)-(3.3)
with (ũ(0), ṽ(0), w̃(0)) = (ũ0, ṽ0, w̃0). For any ξ ∈ R, we then have (u(ξn + ξ),v(ξn + ξ),w(ξn + ξ)) →
(ũ(ξ), ṽ(ξ), w̃(ξ)) ∈ P as n →∞. Especially, we obtain

L(ũ(ξ), ṽ(ξ), w̃(ξ)) = lim
n→∞

(u(ξn + ξ),v(ξn + ξ),w(ξn + ξ)) = L∞,

and thus
d

dξ
L(ũ(ξ), ṽ(ξ), w̃(ξ)) = 0.

From the proof of L′ ≤ 0, we conclude that (ũ(ξ), ṽ(ξ), w̃(ξ)) = (U,V,V ) for all ξ ∈ R. Therefore,
the omega limit set of the trajectory (u(ξ),v(ξ),w(ξ)) is a singleton (U,V,V ), which implies that

(u(ξ),v(ξ),w(ξ)) → (U,V,V ) as ξ →∞. This completes the proof. �

6. Proof of Theorem 2.1

The existence result follows from a sequence of Lemma 3.1, Lemma 4.1 and Lemma 5.1. To establish

the nonexistence result, we assume to the contrary that (u(ξ),v(ξ)) is a positive solution to the boundary

value problem (2.5)-(2.7). Define w(ξ) = v(ξ) − v′(ξ)/c. Then, (u(ξ),v(ξ),w(ξ)) is a solution to the
dynamical system (3.1)-(3.3) such that v(ξ) > 0 and (u(ξ),v(ξ),w(ξ)) → (1, 0, 0) as ξ → −∞. Recall
that the Jacobian matrix J0 about the predator-free equilibrium E0 = (1, 0, 0) has three eigenvalues

−1/c and λ±; see (4.1) and (4.2). Since c < 2
√
p−µ, the eigenvalues λ± are complex with positive real

parts and nonzero imaginary parts. There exist two (complex conjugate) constants c± such that
u(ξ)v(ξ)
w(ξ)


 =


10

0


 + c+eλ+ξe+ + c−eλ−ξe− + o(eRe λ+ξ),

as ξ → −∞, where e± are the eigenvectors associated with the eigenvalues λ±; see (4.3). Especially,
v(ξ) oscillates around 0 as ξ → −∞. This contradicts to the assumption that v(ξ) > 0 for all ξ ∈ R.
The proof is complete.

7. Discussion

In this paper, we have established an existence theory of traveling wave solutions for a diffusive

predator-prey system with cooperative hunting. The main challenge lies in the fact that the function

of cooperative predation is not dominated by a linear function; namely, the sublinearity condition is

not satisfied. We first transform the traveling wave equations to a dynamical system. By using an

extension of the approach introduced in [12, 13], we construct a suitable pentahedron, and find inside

this pentahedron a trajectory connecting the predator-free equilibrium at one end. Lyapunov function

technique and LaSalle invariance principle are applied to prove that this trajectory connects the positive



30 SRIJANA GHIMIRE AND XIANG-SHENG WANG

equilibrium at the other end. Our result indicates that there exists a critical wave speed c∗ such that

no positive traveling wave solution exists when c < c∗. On the other hand, if c > c∗, then a positive

traveling wave solution exists for any sufficiently small cooperative predation rate q.

There are some open problems related to traveling waves in cooperative predation. (1) Strong

cooperative predation. In our analysis, we need a technical assumption q < Q, where Q is a positive

constant depending on the model parameters and the wave speed. It is conjectured that when q ≥ Q,
there still exists a positive traveling wave solution if c > c∗. (2) Prey diffusion. In our model, we simply

assume that the diffusion rate of the prey is zero. It is reasonable to expect that a similar result holds

when the prey has a positive diffusion rate. In this case, one needs to investigate a four-dimensional

dynamical system and construct a suitable polychoron (i.e., 4-polytope). We leave this problem as

future work. (3) Critical case. We have proved the existence of traveling wave solution when c > c∗ and

the non-existence of traveling wave solution when c < c∗. We conjecture that the traveling wave solution

exists for the critical case c = c∗. (4) Bistable equilibria. We assume that the basic reproduction number

R0 is greater than one and hence there exists a unique positive equilibrium for the diffusion-free system.

However, if R0 < 1, the diffusion-free system may have two positive equilibria with one unstable and

the other locally asymptotically stable. It is interesting to study the traveling wave solution connecting

the unstable positive equilibrium to the stable positive equilibrium. One may also want to investigate

the traveling wave solution connecting the unstable positive equilibrium to the (stable) predation-free

equilibrium. (5) General prey growth. In our model, we assume that the prey growth function is linear.

It is natural to ask whether one could extend our study to the general case when the growth rate of the

prey is a general function such as the logistic (quadratic) function or the Allee effect (cubic) function.

Acknowledgment

We would like to thank two anonymous referees for careful reading and helpful suggestions which led

to an improvement to our original manuscript.

References

[1] M. Alves and F.M. Hilker, Hunting cooperation and Allee effects in predators, J. Theo. Biol. 419 (2017), 13–22.

[2] G. Beauchamp, Social Predation: How Group Living Benefits Predators and Prey, Academic Press, 2014.

[3] J. Bednarz, Cooperative hunting Harris’ hawks (Parabuteo unicinctus), Science 239 (1988), 1525–1527.

[4] R. Chisholm and R. Firtel, Insights into morphogenesis from a simple developmental system, Nat. Rev. Mol. Cell.

Biol. 5 (2004), 531–541.

[5] W. Ding and W. Huang, Traveling wave solutions for some classes of diffusive predator-prey models, J. Dynam.

Differential Equations 28 (2016), 1293–1308.

[6] S. R. Dunbar, Traveling wave solutions of diffusive Lotka-Volterra equations, J. Math. Biol. 17 (1983), 11–32.

[7] , Traveling wave solutions of diffusive Lotka-Volterra equations: A heteroclinic connection in R4, Trans.

Amer. Math. Soc. 286 (1984), 557–594.

[8] Y. Hosono and B. Ilyas, Existence of traveling waves with any positive speed for a diffusive epidemic model, Nonlinear

World 1 (1994), 277–290.

[9] , Traveling waves for a simple diffusive epidemic model, Math. Models Methods Appl. Sci. 5 (1995), 935–966.

[10] C.-H. Hsu, C.-R. Yang, T.-H. Yang, and T.-S. Yang, Existence of traveling wave solutions for diffusive predator-prey

type systems, J. Differential Equations 252 (2012), 3040–3075.

[11] J. Huang, G. Lu, and S. Ruan, Existence of traveling wave solutions in a diffusive predator-prey model, J. Math.

Biol. 46 (2003), 132–152.

[12] W. Huang, Traveling wave solutions for a class of predator-prey systems, J. Dynam. Differential Equations 24 (2012),

633–644.



TRAVELING WAVES IN COOPERATIVE PREDATION: RELAXATION OF SUBLINEARITY 31

[13] , A geometric approach in the study of traveling waves for some classes of non-monotone reaction-diffusion

systems, J. Differential Equations 260 (2016), 2190–2224.

[14] S. R.-J. Jang, W. Zhang, and V. Larriva, Cooperative hunting in a predator-prey system with Allee effects in the

prey, Nat. Resour. Model. 31 (2018), e12194.

[15] S. I. Li and M. D. Purugganan, The cooperative amoeba: Dictyostelium as a model for social evolution, Trends

Genet. 27 (2011), 48–54.

[16] W.-T. Li and S.-L. Wu, Traveling waves in a diffusive predator-prey model with holling type-III functional response,

Chaos Solitons Fractals 37 (2008), 476–486.

[17] A. Lotka, Elements of Physical Biology, Williams and Wilkins, Baltimore, 1925.

[18] D. R. MacNulty, A. Tallian, D. R. Stahler, and D. W. Smith, Influence of group size on the success of wolves hunting

bison, PLOS ONE 9 (2014), e112884.

[19] C. Packer and L. Ruttan, The evolution of cooperative hunting, Am. Nat. 132 (1988), 159–198.

[20] V. Volterra, Fluctuations in the abundance of a species considered mathematically, Nature 118 (1926), 538–560.

[21] S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos, Texts in Applied Mathematics, vol. 5,

Springer, New York, 1990.

[22] T. Zhang, W. Wang, and K. Wang, Minimal wave speed for a class of non-cooperative diffusion-reaction system, J.

Differential Equations 260 (2016), 2763–2791.

Department of Mathematics, University of Louisiana at Lafayette, Lafayette, LA 70503, USA.

E-mail address: srijana.ghimire1@louisiana.edu

Corresponding author, Department of Mathematics, University of Louisiana at Lafayette, Lafayette, LA

70503, USA.

E-mail address: xswang@louisiana.edu