Electronic Journal of Differential Equations, Vol. 2022 (2022), No. 79, pp. 1–42. ISSN: 1072-6691. URL: https://ejde.math.txstate.edu or https://ejde.math.unt.edu EXISTENCE OF POSITIVE SOLUTIONS FOR FRACTIONAL LAPLACIAN SYSTEMS WITH CRITICAL GROWTH JEZIEL N. CORREIA, CLAUDIONEI P. OLIVEIRA Abstract. In this article, we show the existence of positive solution to the nonlocal system (−∆)su + a(x)u = 1 2∗s Hu(u, v) in RN , (−∆)sv + b(x)v = 1 2∗s Hv(u, v) in RN , u, v > 0 in RN , u, v ∈ Ds,2(RN ). We also prove a global compactness result for the associated energy functional similar to that due to Struwe in [26]. The basic tools are some information from a limit system with a(x) = b(x) = 0, a variant of the Lion’s principle of concentration and compactness for fractional systems, and Brouwer degree theory. 1. Introduction In this article, we study the existence of positive solutions for the nonlocal elliptic system (−∆)su+ a(x)u = 1 2∗s Hu(u, v) in RN , (−∆)sv + b(x)v = 1 2∗s Hv(u, v) in RN , u, v > 0 in RN , u, v ∈ Ds,2(RN ) , (1.1) with s ∈ (0, 1), N > 2s, Hu and Hv are the partial derivatives of the function H, where H(u, v) ∈ C1(R2 +,R) is a homogeneous function satisfying suitable conditions that will be presented throughout later. The fractional Laplacian (−∆)s, of a smooth function u : RN → R, is defined by (−∆)su(x) := C(N, s) P.V. ∫ RN u(x)− u(y) |x− y|N+2s dy, 2020 Mathematics Subject Classification. 35J20, 35J47, 35J50, 35J91. Key words and phrases. Fractional Laplacian; concentration-compactness; critical nonlinearity global compactness. ©2022. This work is licensed under a CC BY 4.0 license. Submitted March 29, 2022. Published November 22, 2022. 1 2 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 where P.V. is a commonly used abbreviation for “in the Cauchy principal value sense” and C(N, s) > 0 denotes the normalization constant. The work space Ds,2(RN ) is defined as the completion of u ∈ C∞c (RN ) with respect to the Gagliardo semi-norm [u] := (∫∫ R2N |u(x)− u(y)|2 |x− y|N+2s dx dy )1/2 According to [24, Propositions 3.4 and 3.6], we have that ‖u‖2 = |(−∆)s/2u|2L2 = [u]2, by omitting the normalization C(N, s). Notice that this space can be also charac- terized as Ds,2(RN ) := { u ∈ L2∗s (RN ); [u] < +∞ } , where 2∗s = 2N/(N − 2s) is the fractional critical Sobolev exponent. For an ele- mentary introduction to the fractional Laplacian and fractional Sobolev spaces, we refer the interested readers to [22, 24] and references therein. In recent years, the fractional Laplace operator has received attention, for both its applicability and for its purely mathematical properties. This operator can be seen as the infinitesimal generators of Lévy stable processes (see [4]) and arises in several areas such as physics, biology, anomalous diffusion, chemistry, and finance; see [4, 5, 18, 20]. For more details and applications, see [9, 17, 28, 29, 30] and the references therein. In the case s = 1, u = v, and H(u, u) = |u|2∗ with 2∗ = 2N/(N − 2), system (1.1) reduces to the critical Schrödinger equation −∆u+ a(x)u = u N+2 N−2 in RN , u ∈ D1,2(RN ), u ≥ 0, N ≥ 3, (1.2) which was studied by Benci and Cerami in the seminal paper [6]. In this article, we prove that (1.2) does not have a ground state solution and this fact generates some additional difficulties. To overcome these difficulties, the authors investigate the behavior of a Palais-Smale sequence estimate of the energy levels where the Palais-Smale condition fails. In that article, they proved that if N ≥ 3 and ‖a‖N/2 is small enough, then the problem (1.2) has at least one positive solution. After this pioneering work, several other authors studied problems related to (1.2); see for example [2, 7, 8, 11, 13, 19, 21, 23] and references therein. Correia and Figueiredo [13] studied the following version of problem (1.2) for the fractional Laplacian, (−∆)su+ a(x)u = |u|2 ∗ s−2u in RN , u > 0, in RN , u ∈ Ds,2(RN ). (1.3) They first proved a global compactness result for fractional Laplacian in RN , and then, by the compactness result above, and the Linking Theorem, they obtained the existence of high energy solutions for (1.3), provided that a(x) ≥ 0 in RN and |a|LN/2s ≤ S(22s/N − 1), EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 3 where S is the best constant for the Sobolev embedding Ds,2(RN ) ↪→ L2∗s (RN ); that is, S := inf u∈Ds,2(RN )\{0} ∫ RN |(−∆)s/2u|2 dx ( ∫ RN |u|2 ∗ sdx)2/2∗s . (1.4) If a(x) ≡ 0, problem (1.3) reduces to (−∆)su = |u|2 ∗ s−2u in RN , u > 0, in RN , u ∈ Ds,2(RN ). (1.5) It is known that problem (1.5) has the positive solution Φδ,b(x) = c ( δ δ2 + |x− b|2 )(N−2s)/2 , x, b ∈ RN , δ > 0, (1.6) and satisfies ‖Φδ,b‖2 = S, |Φδ,b|2∗s = 1. (1.7) Moreover, all positive solutions of (1.5) can be obtained by translation and scale changes, see [12]. Recently, Figueiredo and Silva [15] considered a variant of the Benci and Cerami’s problem for the system of equations −∆u+ a(x)u = 1 2∗ Ku(u, v) in RN , −∆v + b(x)v = 1 2∗ Kv(u, v) in RN , u, v > 0 in RN , u, v ∈ D1,2(RN ), (1.8) where the nonlinearity K(u, v) ∈ C1(R2 +,R) is a homogeneous function with certain assumptions (for more details see [14]). In that article, using the same techniques introduced by Benci and Cerami [6], they obtained the existence of high energy solutions for system (1.8), provided that a(x), b(x) ≥ 0 in RN and s2 0|a|LN/2 + t20|b|LN/2 < SK(22/N − 1), where SK denote the best constant of the embedding D1,2(RN ) × D1,2(RN ) ↪→ L2∗(RN )× L2∗(RN ); that is, SK := inf u,v∈D1,2(RN )\{0} ∫ RN [|∇u|2 + |∇v|2]dx ( ∫ RN K(u, v)dx)2/2∗ , with s0, t0 positives constant such that the pair (s0Ψδ,y, t0Ψδ,y) reaches SK (see [14, Lemma 3]) and Ψδ,y are Talenti functions (see [1, 27]). Motivated by the works mentioned above, mainly by the ideas found in Benci and Cerami [6], Correia and Figueiredo [13] and Figueiredo and Silva [15], and that a bibliography review did not find any paper dealing with (1.1), we decided to investigate the this class of systems. This article concerns the existence of positive solution for system (1.1). However, we would like to point out that some estimates made in [6, 13, 15] are not immediate for our case because of the nonlocal character of fractional Laplacian. Some refined estimates were necessary, see Section 3 and Section 4. In this article, we consider the following assumptions on H = H(`, t): 4 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 (H0) H is 2∗s-homogeneous, that is, H(θ`, θt) = θ2∗sH(`, t), for each θ > 0, (`, t) ∈ R2 +; (H1) there exists c1 > 0 such that |H`(`, t)|+ |Ht(`, t)| ≤ c1(`2 ∗ s−1 + t2 ∗ s−1) for each (`, t) ∈ R2 +; (H2) ∇H(0, 1) = ∇H(1, 0) = (0, 0); (H3) H(`, t) > 0 for each `, t > 0; (H4) H`(`, t) ≥ 0, Ht(`, t) ≥ 0 for each (`, t) ∈ R2 +; (H5) the 1-homogeneous function Ψ(`2 ∗ s , t2 ∗ s ) = H(`, t) is concave in R2 +. On the functions a, b : RN → R, we assume the following conditions: (H6) The functions a, b are positive in a set of positive measure; (H7) a, b ∈ Lq(RN ) for all q ∈ [p1, p2] with 1 < p1 < N/2s < p2 and p2 < N/(4s−N) if N < 4s; (H8) `20|a|LN/2s(RN ) + t20|b|LN/2s(RN ) < SH(22s/N − 1). In assumption (H8), the expression SH denotes the best constant of the immer- sion Ds,2(RN )×Ds,2(RN ) ↪→ L2∗s (RN )× L2∗s (RN ), namely SH := inf u,v∈Ds,2(RN )\{0} ∫ RN [|(−∆)s/2u|2 + |(−∆)s/2v|2]dx ( ∫ RN H(u, v)dx)2/2∗s . Moreover, by [3, Lemma 2.3] there are `0 and t0 positive such that SH is attained by (`0Φδ,b, t0Φδ,b) and MHSH = S, (1.9) where MH = max`2+t2=1H(`, t)2/2∗s = H(`0, t0)2/2∗s . To state the main result this article, we consider the energy functional of calss C1, J : Ds,2(RN )×Ds,2(RN )→ R, given by J (u, v) = 1 2 ‖(u, v)‖2 + 1 2 ∫ RN (a(x)u2 + b(x)v2)dx− 1 2∗s ∫ RN H(u, v)dx, where ‖(u, v)‖2 = ‖u‖2 + ‖v‖2 is the norm in the space Ds,2(RN )×Ds,2(RN ) and J ′(u, v)(ϕ,ψ) = ∫ RN [(−∆)s/2u(−∆)s/2ϕ+ (−∆)s/2v(−∆)s/2ψ]dx + ∫ RN [a(x)uϕ+ b(x)vψ]dx− 1 2∗s ∫ RN [Hu(u, v)ϕ+Hv(u, v)ψ]dx for all (ϕ,ψ) ∈ Ds,2(RN )×Ds,2(RN ). We have the following existence result. Theorem 1.1. Assume that (H0)–(H8) hold. Then (1.1) has a positive solution (u0, v0) ∈ Ds,2(RN )×Ds,2(RN ) with s N S N/2s H < J (u0, v0) < 2s N S N/2s H . In some sense, the main result of this article expands the study made in [6, 13, 15], because we are considering a version of a paper for the fractional Laplacian. Moreover, we prove the version for fractional system in RN of Struwe’s Global Compactness result [26], which may be useful also in other context and has never appeared in the literature, to the best of our knowledge. Before we finish this introduction, let us comment on some difficulties encoun- tered in problem (1.1). EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 5 • The “double” lack of compactness due to the unboundedness of the domain, and the presence of the critical Sobolev exponent, which is related to the fact that embedding Ds,2(RN ) ↪→ L2∗s (RN ) is not compact. Thus, the associated energy functional does not satisfy the Palais-Smale condition in general. • The extension to nonlocal system involves some technical difficulties which are overcome with some refined estimates, as can be seen in Lemma 3.1, Theorem 3.2, and Section 4. This article is organized as follows. In Section 2, we study the limit system associated with (1.1). In Section 3, we give the complete descriptions for the Palais- Smale (PS) sequences for the functional J . In Section 4, we prove some technical lemmas. In Section 5, we show the main result. 2. Limit problem In this section, we give some results involving the limit problem that will be useful in our approach. We start with example of the function H(u, v) that satisfies the conditions (H0)–(H5). Let H be the function H(u, v) := a|u|2 ∗ s + ∑ αi+βi=2∗s bi|u|αi |v|βi + c|v|2 ∗ s , where a, bi, c ∈ R, αi + βi = 2∗s, αi, βi ≥ 1, i ∈ I with I a finite subset of N. Then H satisfies conditions (H0)–(H5). From the homogeneity condition (H0), have the so called Euller identity, (u, v) · ∇H(u, v) = 2∗sH(u, v). (2.1) Let us introduce the limit problem associated with (1.1), (−∆)su = 1 2∗s Hu(u, v) in RN , (−∆)sv = 1 2∗s Hv(u, v) in RN , u, v > 0 in RN , u, v ∈ Ds,2(RN ) (2.2) whose associated energy functional J∞ : Ds,2(RN )×Ds,2(RN )→ R is J∞(u, v) = 1 2 ‖(u, v)‖2 − 1 2∗s ∫ RN H(u, v)dx. The next lemma states that the functional associated with the limit problem satis- fies the Palais-Smale condition. Lemma 2.1 ((PS)-condition for J∞). Let (un, vn) be sequence (PS)c for J∞. Then (a) The sequence (un, vn) is bounded in Ds,2(RN )×Ds,2(RN ); (b) If (un, vn) ⇀ (u, v) in Ds,2(RN )×Ds,2(RN ), then J ′∞(u, v) = 0; (c) If c ∈ (−∞, sN S N/2s H ), the J∞ satisfies the (PS)c condition, i.e, up to a subsequence (un, vn)→ (u, v) in Ds,2(RN )×Ds,2(RN ). 6 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Proof. (a) Since J∞(un, vn)→ c and J ′∞(un, vn)→ 0, and using (2.1), there exists d > 0 such that d+ ‖(un, vn)‖ ≥ J∞(un, vn)− 1 2∗s J∞(un, vn)(un, vn) = s N ‖(un, vn)‖2 + on(1); thus s N ‖(un, vn)‖2 + on(1) ≤ d+ ‖(un, vn)‖ . which proves part (a). (b) Since (un, vn) ⇀ (u, v) in Ds,2(RN ) × Ds,2(RN ), up to a subsequence, we have (un, vn)→ (u, v) in Lqloc(RN )× Lqloc(RN ), (un, vn)→ (u, v) a.e. in RN × RN . Using a denseness argument we obtain∫ RN Hu(un, vn)ϕdx+ ∫ RN Hv(un, vn)ψ dx→ ∫ RN Hu(u, v)ϕdx+ ∫ RN Hv(u, v)ψ dx for all ϕ,ψ ∈ Ds,2(RN ), which implies (b). (c) Consider the sequence (wn, zn) = (un − u, vn − v) and note that on(1) = J ′∞(un, vn)(un, vn) = ‖(un, vn)‖2− 1 2∗s ∫ RN [Hu(un, vn)un+Hv(un, vn)vn] dx or on(1) = ‖(wn, zn)‖2 + ‖(u, v)‖2 − 1 2∗s ∫ RN Hu(wn + u, zn + v)(wn + u)dx − 1 2∗s ∫ RN Hv(wn + u, zn + v)(zn + v) dx . From [3, Lemma 7.2], we have ‖(wn, zn)‖2 + ‖(u, v)‖2 − 1 2∗s ∫ RN Hu(wn, zn)wn dx− 1 2∗s ∫ RN Hv(wn, zn)zn dx − 1 2∗s ∫ RN Hu(u, v)u dx− 1 2∗s ∫ RN Hv(u, v)v dx = on(1) Now using (b) and (2.1) we have ‖(wn, zn)‖2 − ∫ RN H(wn, zn)dx = on(1). Up to a subsequence, we conclude that there exists L ≥ 0 such that lim n→+∞ ‖(wn, zn)‖2 = lim n→+∞ ∫ RN H(wn, zn)dx = L. Suppose, by contradiction, that L > 0. Using the inequality SH (∫ RN H(wn, zn)dx )2/2∗s ≤ ‖(wn, zn)‖2 we obtain L ≥ SHL2/2∗s =⇒ L ≥ SN/2sH . Since J∞(u, v) = s N ‖(u, v)‖2 ≥ 0 and c = s N ‖(wn, zn)‖2 + J∞(u, v) + on(1), EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 7 it follows that c = s N ‖(wn, zn)‖2 + J∞(u, v) + on(1) ≥ s N ‖(wn, zn)‖2 + on(1) ≥ s N L ≥ s N S N/2s H , which is a contradiction. Therefore, L = 0 and so ‖un − u‖2 → 0 and ‖vn − v‖2 → 0. � 3. A compactness result We start this section by establishing the following technical lemma for J∞ which will be useful for proving our compactness theorem. Lemma 3.1. Let (un, vn) be a (PS)c sequence for the functional J∞ with (un, vn) ⇀ (0, 0) and (un, vn) 6→ (0, 0). Then, there are sequences (Rn) ⊂ R+, (xn) ⊂ RN and (u0, v0) ∈ Ds,2(RN ) × Ds,2(RN ) nontrivial solution of (S∞) and a sequence (τn, ζn) which is (PS)c̃ for the J∞ such that, up to a subsequence of (un, vn), we have τn(x) = un(x)−R N−2s 2 n u0(Rn(x− xn)) + on(1), ζn(x) = vn(x)−R N−2s 2 n v0(Rn(x− xn)) + on(1). Proof. Let (un, vn) ⊂ Ds,2(RN )×Ds,2(RN ) be a (PS)c sequence for the functional J∞, i.e., J∞(un, vn)→ c and J ′∞(un, vn)→ 0. (3.1) From Lemma 2.1(a), we obtain that (un, vn) is bounded in Ds,2(RN )× Ds,2(RN ). Since (un, vn) ⇀ (0, 0) and (un, vn) 6→ (0, 0), by the Lemma 2.1(c) it follows that c ≥ s N S N/2s H . Note that c+ on(1) = J∞(un, vn)− 1 2∗s J ′∞(un, vn)(un, vn) = s N ∫ RN [ |(−∆)s/2un|2 + |(−∆)s/2vn|2 ] dx, which implies lim n→+∞ s N ∫ RN [ |(−∆)s/2un|2 + |(−∆)s/2vn|2 ] dx ≥ SN/2sH . (3.2) Let L be a number such that B2(0) is covered by L balls of radius 1, (Rn) ⊂ R, (xn) ⊂ RN such that sup y∈RN ∫ B R −1 n (y) [ |(−∆)s/2un|2 + |(−∆)s/2un|2 ] dx = ∫ B R −1 n (xn) [ |(−∆)s/2un|2 + |(−∆)s/2vn|2 ] dx = S N/2s H 2L . We define the sequence (wn(x), zn(x)) = ( R 2s−N 2 n un ( x Rn + xn ) , R 2s−N 2 n vn ( x Rn + xn )) . 8 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Using a change of variable, we can prove that∫ B1(0) [ |(−∆)s/2wn|2 + |(−∆)s/2zn|2 ] dx = S N/2s H 2L = sup y∈RN ∫ B1(y) [ |(−∆)s/2wn|2 + |(−∆)s/2zn|2 ] dx. Now, for each (Φ1,Φ2) ∈ Ds,2(RN )×Ds,2(RN ), we define (Φ̃1,n, Φ̃2,n)(x) = ( R N−2s 2 n Φ1(Rn(x− xn)), R N−2s 2 n Φ2(Rn(x− xn)) ) which satisfies∫ RN [ (−∆)s/2un(−∆)s/2Φ̃1,n + (−∆)s/2vn(−∆)s/2Φ̃2,n ] dx = ∫ RN [ (−∆)s/2wn(−∆)s/2Φ1 + (−∆)s/2zn(−∆)s/2Φ2 ] dx (3.3) and ∫ RN [Hu(un, vn)Φ̃1,n +Hv(un, vn)Φ̃2,n]dx = ∫ RN [Hw(wn, zn)Φ1 +Hzn(wn, zn)Φ2]dx. (3.4) These limits yield that J∞(wn, zn)→ c and J ′∞(wn, zn)→ 0. (3.5) From Lemma 3.1, there exists (u0, v0) ∈ Ds,2(RN )×Ds,2(RN ) such that, up to a subsequence, (un, vn) ⇀ (u0, v0) in Ds,2(RN )×Ds,2(RN ) and J ′∞(u0, v0) = 0. As a consequence from following variant of the Concentration-Compactness Li- ons’s Lemma [3, Lemma 4.3], we obtain∫ RN H(wn, zn)φdx→ ∫ RN H(u0, v0)φdx+ ∑ j∈J φ(xj)νj , ∀φ ∈ C∞0 (RN ) (3.6) and |(−∆)s/2wn|2 + |(−∆)s/2zn|2 ⇀ µ+ σ ≥ |(−∆)s/2u0|2 + |(−∆)s/2v0|2 + ∑ j∈J φ(xj)µj + ∑ j∈J φ(xj)σj ,∀φ ∈ C∞0 (RN ) for some {xj}j∈J ⊂ RN and for some {νj}j∈J , {µj}j∈J , {σj}j∈J ⊂ R+ with SHν 2/2∗s j ≤ µj + σj , where J is at most a countable set. Indeed, J is finite. To see this, consider φ ∈ C∞0 (RN ) such that 0 ≤ φ(x) ≤ 1, for all x ∈ RN , φ(x) = 0 for all x ∈ Bc2(0) and φ(x) = 1 for all x ∈ B1(0). Now fix xj ∈ RN , j ∈ J and define φρ(x) = φ( x−xj ρ ), for each ρ > 0. Thus, 0 ≤ φρ(x) ≤ 1, for all x ∈ RN , φρ(x) = 0 EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 9 for all x ∈ Bc2ρ(xj) and φρ(x) = 1 for all x ∈ Bρ(xj). We have that (wnφρ, znφρ) is bounded in Ds,2(RN )×Ds,2(RN ) and J ′∞(wn, zn)(wnφρ, znφρ) = on(1). Then∫ RN (−∆)s/2wn(−∆)s/2(wnφρ)dx+ ∫ RN (−∆)s/2zn(−∆)s/2(znφρ)dx = ∫ RN Hw(wn, zn)(wnφρ)dx+ ∫ RN Hz(wn, zn)(znφρ)dx+ on(1). (3.7) As∫ RN (−∆)s/2wn(−∆)s/2(wnφρ) dx dy + ∫ RN (−∆)s/2zn(−∆)s/2(znφρ) dx dy = ∫ R2N (wn(x)− wn(y))2φρ(y) |x− y|N+2s dx dy + ∫ R2N (wn(x)− wn(y))(φρ(x)− φρ(y))wn(x) |x− y|N+2s dx dy + ∫ R2N (zn(x)− zn(y))2φρ(y) |x− y|N+2s dx dy + ∫ R2N (zn(x)− zn(y))(φρ(x)− φρ(y))zn(x) |x− y|N+2s dx dy, (3.8) it is easy to verify that∫ R2N (wn(x)− wn(y))2φρ(y) |x− y|N+2s dx dy + ∫ R2N (zn(x)− zn(y))2φρ(y) |x− y|N+2s dx dy = ∫ RN |(−∆)s/2wn|2φρ(y)dy + ∫ RN |(−∆)s/2zn|2φρ(y)dy → ∫ RN φρ(y)dµ+ ∫ RN φρ(y)dσ as n→ +∞ (3.9) and∫ RN φρ(y)dµ+ ∫ RN φρ(y)dσ → µ({xj}) + σ({xj}) = µj + σj as ρ→ 0. (3.10) Also, by Hölder inequality∣∣∣ ∫ R2N (wn(x)− wn(y))(φρ(x)− φρ(y))wn(x) |x− y|N+2s dx dy ∣∣∣ ≤ ∫ R2N |wn(x)− wn(y)||φρ(x)− φρ(y)||wn(x)| |x− y|N+2s dx dy ≤ C1 (∫ R2N |φρ(x)− φρ(y)|2|wn(x)|2 |x− y|N+2s dx dy )1/2 (3.11) and ∣∣∣ ∫ R2N (zn(x)− zn(y))(φρ(x)− φρ(y))zn(x) |x− y|N+2s dx dy ∣∣∣ ≤ ∫ R2N |zn(x)− zn(y)||φρ(x)− φρ(y)||zn(x)| |x− y|N+2s dx dy ≤ C2 (∫ R2N |φρ(x)− φρ(y)|2|zn(x)|2 |x− y|N+2s dx dy )1/2 . (3.12) 10 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Arguing as in [32, Lemma 3.6], we see that lim ρ→0 lim n→+∞ ∫ R2N |φρ(x)− φρ(y)|2|wn(x)|2 |x− y|N+2s dx dy = 0, (3.13) lim ρ→0 lim n→+∞ ∫ R2N |φρ(x)− φρ(y)|2|zn(x)|2 |x− y|N+2s dx dy = 0. (3.14) On the other hand, by (2.1) we have∫ RN Hw(wn, zn)(wnφρ)dx+ ∫ RN Hz(wn, zn)(znφρ)dx = ∫ RN ∇H(wn, zn) · (wnφρ, znφρ) = 2∗s ∫ RN H(wn, zn)φρdx→ 2∗s ∫ RN φρ(y)dν (3.15) and ∫ RN φρ(y)dν → ν({xj}) = νj as ρ→ 0. (3.16) From (3.8), (3.9), (3.10), (3.11), (3.12), (3.13), (3.14), (3.15) and (3.16), it follows that SHν 2/2∗s j ≤ µj + σj ≤ 2∗sνj . Since that νj > 0, we see that S N/2s H ≤ (µj + σj) N/2s ≤ Cνj , ∑ j∈J ν 2/2∗s j <∞ and so νj does not converge to zero, which means that J is finite. From now on, we denote by J = {1, 2, . . . ,m} and Γ ⊂ RN the set given by Γ = {xj ∈ {xj}j∈J ; |xj | > 1}, with (xj given by (3.6). Note that we can consider xj , j = 1, . . . ,m, belonging to Γ, otherwise, we choose the smallest distance point for zero in this set. We are going to show that (u0, v0) 6= (0, 0). Suppose, by contradiction, that (u0, v0) = (0, 0). Then, by (3.6) we have∫ RN H(wn, zn)φdx→ 0, ∀φ ∈ C∞0 (RN \ {x1, x2, . . . , xm}). (3.17) Since (φ1,n, φ2,n) = (φwn, φzn) with φ ∈ C∞0 (RN \ {x1, x2, . . . , xm}) is bounded, we obtain J ′∞(wn, zn)(φ1,n, φ2,n) = on(1); that is,∫ RN [(−∆)s/2wn(−∆)s/2φ1,n + (−∆)s/2zn(−∆)s/2φ2,n]dx − 1 2∗s ∫ RN Hw(wn, zn)φ1,ndx− 1 2∗s ∫ RN Hz(wn, zn)φ2,ndx = on(1), (3.18) or ∫ R2N (wn(x)− wn(y))(φ1,n(x)wn(x)− φ1,n(y)wn(y)) |x− y|N+2s dx dy + ∫ R2N (zn(x)− zn(y))(φ2,n(x)(zn(x)− φ2,n(y)zn(y)) |x− y|N+2s dx dy − 1 2∗s ∫ RN Hw(wn, zn)φ1,ndx− 1 2∗s ∫ RN Hz(wn, zn)φ2,ndx = on(1). (3.19) EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 11 But the above equality is equivalent to∫ R2N wn(x) (wn(x)− wn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy + ∫ R2N φ(y) (wn(x)− wn(y))2 |x− y|N+2s dx dy + ∫ R2N zn(x) (zn(x)− zn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy + ∫ R2N φ(y) (zn(x)− zn(y))2 |x− y|N+2s dx dy − ∫ RN H(wn, zn)φdx = on(1). Then∣∣∣ ∫ R2N φ(y) |wn(x)− wn(y)|2 |x− y|N+2s dx dy + ∫ R2N φ(y) |zn(x)− zn(y)|2 |x− y|N+2s dx dy ∣∣∣ = ∣∣∣ ∫ RN H(wn, zn)φdx− ∫ R2N wn(x) (wn(x)− wn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy − ∫ R2N zn(x) (zn(x)− zn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy + on(1) ∣∣∣ ≤ ∣∣∣ ∫ RN H(wn, zn)φdx ∣∣∣+ ∣∣∣ ∫ R2N wn(x) (wn(x)− wn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy ∣∣∣ + ∣∣∣ ∫ R2N zn(x) (zn(x)− zn(y))(φ(x)− φ(y)) |x− y|N+2s dx dy ∣∣∣+ on(1) ≤ ∣∣∣ ∫ RN H(wn, vn)φdx ∣∣∣+ ‖wn‖ (∫ R2N |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy )1/2 + ‖zn‖ (∫ R2N |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy )1/2 + on(1). (3.20) Now, we show that ∫ R2N |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = on(1), (3.21)∫ R2N |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = on(1). (3.22) For this, let R be a positive number such that supp(φ) ⊂ BR(0) and write R2N as R2N = [ (RN \BR(0))× (RN \BR(0)) ] ∪ [ BR(0)× (RN \BR(0)) ] ∪ [ (RN \BR(0))×BR(0) ] = Ω1 ∪ Ω2 ∪ Ω3. Thus, we have∫ R2N |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = ∫ Ω1 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy + ∫ Ω2 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy + ∫ Ω3 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy (3.23) 12 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 and∫ R2N |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = ∫ Ω1 |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy + ∫ Ω2 |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy + ∫ Ω3 |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy. (3.24) We will prove (3.23), the case (3.24) follows in an analogous way. To do this we estimate each integral in (3.23). Since φ = 0 in RN \BR(0), we have∫ Ω1 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = 0. (3.25) Using |φ| ≤ C1, |∇φ| ≤ C2 and using the mean value theorem, we infer that∫ Ω2 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = ∫ BR(0) |wn(x)|2dx ∫ {y∈RN :|x−y|≤R} |φ(x)− φ(y)|2 |x− y|N+2s dy + ∫ BR(0) |wn(x)|2dx ∫ {y∈RN :|x−y|>R} |φ(x)− φ(y)|2 |x− y|N+2s dy ≤ C|∇φ|2L∞(RN ) ∫ BR(0) |wn(x)|2dx ∫ {y∈RN :|x−y|≤R} 1 |x− y|N+2s−2 dy + C ∫ BR(0) |wn(x)|2dx ∫ {y∈RN :|x−y|>R} 1 |x− y|N+2s dy = CR2−2s ∫ BR(0) |wn(x)|2dx+ CR−2s ∫ BR(0) |wn(x)|2dx = on(1). (3.26) Moreover, for the integral on Ω3, we have∫ Ω3 |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = ∫ RN\BR(0) |wn(x)|2dx ∫ {y∈BR(0):|x−y|≤R} |φ(x)− φ(y)|2 |x− y|N+2s dy = ∫ RN\BR(0) |wn(x)|2dx ∫ {y∈BR(0):|x−y|>R} |φ(x)− φ(y)|2 |x− y|N+2s dy =: T 1 R + T 2 R . (3.27) It is not difficult to verify that if (x, y) ∈ (RN \ BR(0)) × BR(0) and |x − y| ≤ R, then |x| ≤ 2R, thus T 1 R = ∫ RN\BR(0) |wn(x)|2dx ∫ {y∈BR(0):|x−y|≤R} |φ(x)− φ(y)|2 |x− y|N+2s dy ≤ C|∇φ|2L∞(RN ) ∫ B2R(0) |wn(x)|2dx ∫ {z∈BR(0):|z|≤R} 1 |z|N+2s−2 dz = CR2−2s ∫ B2R(0) |wn(x)|2dx = on(1). (3.28) EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 13 Note that, there exists k > 4 such that Ω3 = [ (RN \BR(0))× (BR(0)) ] ∪ [BkR(0)× (BR(0))] ∪ [ (RN \BkR(0))×BR(0) ] . Therefore, ∫ BkR(0) |wn(x)|2dx ∫ {y∈BR(0):|x−y|>R} |φ(x)− φ(y)|2 |x− y|N+2s dy ≤ C ∫ BkR(0) |wn(x)|2dx ∫ {z∈RN :|z|>R} 1 |z|N+2s dz = CR−2s ∫ BkR(0) |wn(x)|2dx = on(1). (3.29) If (x, y) ∈ (RN \BkR(0))×BR(0), then |x− y| ≥ |x| − |y| ≥ kR 2 −R > |x| 2 , and using Hölder’s inequality, we obtain∫ RN\BkR(0) |wn(x)|2dx ∫ {y∈BR(0):|x−y|>R} |φ(x)− φ(y)|2 |x− y|N+2s dy ≤ C ∫ RN\BkR(0) dx ∫ {y∈BR(0):|x−y|>R} |wn(x)|2 |x− y|N+2s dy ≤ CRN ∫ RN\BkR(0) |wn(x)|2 |x|N+2s dx ≤ (∫ RN\BkR(0) |wn(x)|2 ∗ sdx )2/2∗s (∫ RN\BkR(0) |x|−(N+2s) 2∗s 2∗s−2 ) 2∗s−2 2∗s ≤ Ck−N (∫ RN\BkR(0) |wn(x)|2 ∗ sdx )2/2∗s ≤ Ck−N . (3.30) From (3.29) and (3.30), we obtain T 2 R ≤ Ck−N + on(1). (3.31) Combining (3.20)-(3.28) and (3.31), we deduce that lim sup n→+∞ ∫ RN |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = lim k→+∞ lim sup n→+∞ ∫ RN |wn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = 0 and lim sup n→+∞ ∫ RN |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = lim k→+∞ lim sup n→+∞ ∫ RN |zn(x)|2 |φ(x)− φ(y)|2 |x− y|N+2s dx dy = 0. Combining (3.18), (3.20), (3.21), (3.22), and (3.17), we conclude that∫ R2N φ(y) |wn(x)− wn(y)|2 |x− y|N+2s dx dy + ∫ R2N φ(y) |zn(x)− zn(y)|2 |x− y|N+2s dx dy → 0 (3.32) 14 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 for all φ ∈ C∞0 (RN \ {x1, . . . , xm}), which leads to∫ RN |(−∆)s/2wn|2φdx+ ∫ RN |(−∆)s/2zn|2φdx = on(1). (3.33) Let ρ ∈ R be a number that satisfies 0 < ρ < min{dist(Γ, B̄1(0), 1)}. We will show that ∫ B1+ρ(0)\B1+ ρ 3 (0) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]dx→ 0. (3.34) To do this, we consider φ ∈ C∞0 (RN ) such that 0 ≤ φ(x) ≤ 1 and φ(x) = 1 if x ∈ B1+ρ(0). If φ̃ = φ|RN\{x1,...,xm}, follows by (3.33) that 0 ≤ ∫ B1+ρ(0)\B1+ ρ 3 (0) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]dx ≤ ∫ B1+ρ(0) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]dx = ∫ B1+ρ(0) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]φ̃dx ≤ ∫ RN [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]φ̃dx→ 0, which implies that (3.34) occurs. Let Φ ∈ C∞0 (RN ) be such that 0 ≤ Φ(x) ≤ 1, |∇Φ| ≤ 2 for all x ∈ RN and Φ(x) = { 1, x ∈ B1+ ρ 3 (0), 0, x ∈ Bc 1+ 2ρ 3 (0) and consider the sequence (Φ1,n,Φ2,n) given by (Φ1,n(x),Φ2,n(x)) = (Φ(x)wn(x),Φ(x)zn(x)). Using (3.21) and (3.22), we have∫ RN\B1+ρ(0) |(−∆)s/2Φ1,n|2dx+ ∫ RN\B1+ρ(0) |(−∆)s/2Φ2,n|2dx ≤ 2 ∫ (RN\B1+ρ(0))×RN |wn(y)|2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (RN\B1+ρ(0))×RN |Φ(y)|2|wn(x)− wn(y)|2 |x− y|N+2s dx dy ≤ 2 ∫ (RN\B1+ρ(0))×RN |zn(y)|2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (RN\B1+ρ(0))×RN |Φ(y)|2|zn(x)− zn(y)|2 |x− y|N+2s dx dy = on(1) + 2 ∫ RN\B1+ρ(0) Φ(x)2|(−∆)s/2wn(x)|2dx + on(1) + 2 ∫ RN\B1+ρ(0) Φ(x)2|(−∆)s/2zn(x)|2dx = on(1). (3.35) EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 15 Similarly, we can obtain the estimate∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2Φ1,n|2dx+ ∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2Φ2,n|2dx ≤ 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN wn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN Φ(y)2|wn(x)− wn(y)|2 |x− y|N+2s dx dy + 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN zn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN Φ(y)2|zn(x)− zn(y)|2 |x− y|N+2s dx dy ≤ 2 ∫ B1+ρ(0)×RN wn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN |wn(x)− wn(y)|2 |x− y|N+2s dx dy + 2 ∫ B1+ρ(0)×RN zn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ (B1+ρ(0)\B1+ ρ 3 (0))×RN |zn(x)− zn(y)|2 |x− y|N+2s dx dy = 2 ∫ B1+ρ(0)×RN wn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2wn|2dx + 2 ∫ B1+ρ(0)×RN zn(x)2|Φ(x)− Φ(y)|2 |x− y|N+2s dx dy + 2 ∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2zn|2dx = on(1), (3.36) where in the last equality we made use of estimates (3.21), (3.22), and (3.34). Since (Φ1,n,Φ2,n) is bounded in Ds,2(RN )×Ds,2(RN ), we derive that∫ B1+ρ(0)\B1+ ρ 3 (0) (−∆)s/2wn(−∆)s/2Φ1,ndx+ ∫ B1+ ρ 3 (0) (−∆)s/2wn(−∆)s/2Φ1,ndx + ∫ B1+ρ(0)\B1+ ρ 3 (0) (−∆)s/2zn(−∆)s/2Φ2,ndx+ ∫ B1+ ρ 3 (0) (−∆)s/2zn(−∆)s/2Φ2,ndx − 1 2∗s ∫ B1+ρ(0)\B1+ ρ 3 (0) Φ1,nHw(wn, zn)dx− 1 2∗s ∫ B1+ ρ 3 (0) Φ1,nHw(wn, zn)dx 16 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 − 1 2∗s ∫ B1+ρ(0)\B1+ ρ 3 (0) Φ2,nHz(wn, zn)dx− 1 2∗s ∫ B1+ ρ 3 (0) Φ2,nHz(wn, zn)dx = on(1), which implies∫ B1+ρ(0)\B1+ ρ 3 (0) (−∆)s/2wn(−∆)s/2Φ1,ndx+ ∫ B1+ ρ 3 (0) |(−∆)s/2Φ1,n|2dx + ∫ B1+ρ(0)\B1+ ρ 3 (0) (−∆)s/2zn(−∆)s/2Φ2,ndx+ ∫ B1+ ρ 3 (0) |(−∆)s/2Φ2,n|2 dx − 1 2∗s ∫ B1+ρ(0)\B1+ ρ 3 (0) Φ1,nHw(wn, zn)dx− 1 2∗s ∫ B1+ ρ 3 (0) Φ1,nHw(Φ1,n,Φ1,n)dx − 1 2∗s ∫ B1+ρ(0)\B1+ ρ 3 (0) Φ2,nHz(wn, zn)dx− 1 2∗s ∫ B1+ ρ 3 (0) Φ2,nHz(Φ2,n,Φ2,n)dx = on(1). (3.37) Note that from Hölder inequality, (3.35) and (3.36) we obtain∫ B1+ρ(0)\B1+ ρ 3 (0) [ (−∆)s/2wn(−∆)s/2Φ1,n + (−∆)s/2zn(−∆)s/2Φ2,n ] dx = on(1). (3.38) Moreover, combining (2.1) and (3.17) we deduce∫ B1+ρ(0)\B1+ ρ 3 (0) Φ1,nHw(wn, zn)dx+ ∫ B1+ρ(0)\B1+ ρ 3 (0) Φ2,nHz(wn, zn)dx = on(1). (3.39) From (3.37), (3.38), and (3.39), we obtain∫ B1+ ρ 3 (0) |(−∆)s/2Φ1,n|2dx+ ∫ B1+ ρ 3 (0) |(−∆)s/2Φ2,n|2 dx − 1 2∗s ∫ B1+ ρ 3 (0) Φ1,nHw(Φ1,n,Φ1,n)dx− 1 2∗s ∫ B1+ ρ 3 (0) Φ2,nHz(Φ2,n,Φ2,n)dx = on(1). Note that ∫ RN [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx = ∫ B1+ ρ 3 (0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx = ∫ B1+ρ(0)\B1+ ρ 3 (0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx + ∫ B1+ ρ 3 (0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx = on(1) + ∫ B1+ ρ 3 (0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx . EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 17 Using (2.1), we obtain∫ RN H(Φ1,n,Φ2,n)dx = ∫ B1+ρ(0) H(Φ1,n,Φ2,n)dx = ∫ B1+ρ(0)\B1+ ρ 3 (0) H(Φ1,n,Φ2,n)dx+ ∫ B1+ ρ 3 (0) H(Φ1,nΦ2,n)dx, from where we deduce∫ RN [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx− ∫ RN H(Φ1,n,Φ2,n)dx = on(1), i.e., ‖Φ1,n‖2 + ‖Φ2,n‖2 − ∫ RN H(Φ1,n,Φ2,n)dx = on(1). From the definition of SH , we have (‖Φ1,n‖2 + ‖Φ2,n‖2) [ 1− 1 S 2∗s/2 H [‖Φ1,n‖2 + ‖Φ2,n‖2]2 ∗ s−2 ] = ‖Φ1,n‖2 + ‖Φ2,n‖2 − 1 S2∗s/2 [‖Φ1,n‖2 + ‖Φ2,n‖2]2 ∗ s ≤ ∫ RN [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx− ∫ RN H(Φ1,n,Φ2,n)dx = on(1). (3.40) On the other hand, ‖Φ1,n‖2 + ‖Φ2,n‖2 = ∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2Φ1,n|2dx+ ∫ B1+ ρ 3 (0) |(−∆)s/2Φ1,n|2dx + ∫ B1+ρ(0)\B1+ ρ 3 (0) |(−∆)s/2Φ2,n|2dx+ ∫ B1+ ρ 3 (0) |(−∆)s/2Φ2,n|2dx = on(1) + ∫ B1+ ρ 3 (0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx. Since Φ1,n = wn, Φ2,n = zn in B1+ ρ 3 (0) and that B1+ ρ 3 (0) ⊂ B2(0), we obtain ‖Φ1,n‖2 + ‖Φ2,n‖2 ≤ on(1) + ∫ B2(0) [|(−∆)s/2Φ1,n|2 + |(−∆)s/2Φ2,n|2]dx, which implies ‖Φ1,n‖2 + ‖Φ2,n‖2 ≤ on(1) + ∫ ⋃L k=1 B1(yk) [|(−∆)s/2zn|2 + |(−∆)s/2wn|2]dx ≤ on(1) + L∑ k=1 ∫ B1(yk) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]dx ≤ on(1) + L sup y∈RN ∫ B1(y) [|(−∆)s/2wn|2 + |(−∆)s/2wn|2]dx 18 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 ≤ on(1) + S N/2s H 2 . Then ( ‖Φ1,n‖2 + ‖Φ2,n‖2 )1/2 ≤ on(1) + SN/4s√ 2 , i.e., (‖Φ1,n‖+ ‖Φ2,n‖)2∗s−2 ≤ on(1) + (SN/4s√ 2 )2∗s−2 or on(1)− (SN/4s√ 2 )2∗s−2 ≤ −(‖Φ1,n‖2 + ‖Φ2,n‖2)2∗s−2. (3.41) Using (3.40) and (3.41), we have that (‖Φ1,n‖2 + ‖Φ2,n‖2) [ 1 + on(1)− 1 S 2∗s/2 H (SN/4sH√ 2 )2∗s−2] = (‖Φ1,n‖2 + ‖Φ2,n‖2) { 1 + 1 S 2∗s/2 H [ on(1)− (SN/4sH√ 2 )2∗s−2]} ≤ (‖Φ1,n‖2 + ‖Φ2,n‖2) [ 1− 1 S 2∗s/2 H (‖Φ1,n‖2 + ‖Φ2,n‖2)2∗s−2 ] = on(1). But the equality N 4s (2∗s − 2)− 2∗s 2 = N 4s ( 4s N − 2s ) − N N − 2s = 0 implies (‖Φ1,n‖2 + ‖Φ2,n‖2) [ 1− (1 2 )(2∗s−2)/2] ≤ on(1), and then (Φ1,n,Φ2,n)→ (0, 0) in Ds,2(RN )×Ds,2(RN ). Since wn = Φ1,n, zn = Φ2,n in B1(0), we deduce that 0 ≤ ∫ B1(0) [|(−∆)s/2wn|2 + |(−∆)s/2zn|2]dx = ‖Φ1,n‖2 + ‖Φ2,n‖2, which implies∫ B1(0) [ |(−∆)s/2wn|2 + |(−∆)s/2zn|2 ] dx→ 0 as n→∞. But this convergence contradicts that∫ B1(0) [ |(−∆)s/2wn|2 + |(−∆)s/2zn|2 ] dx = S N/2s H 2L , ∀n ∈ N. Therefore, (u0, v0) 6= (0, 0). Now we show that there is (τn, ζn) ∈ Ds,2(RN )×Ds,2(RN ) such that (τn, ζn) is a (PS)c̃ sequence for J∞ satisfying τn(x) = un(x)−R(N−2s)/2 n u0(Rn(x− xn)) + on(1), ζn(x) = vn(x)−R(N−2s)/2 n v0(Rn(x− xn)) + on(1), EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 19 up to a subsequence of (un, vn). For this, we consider ψ ∈ C∞0 (RN ) such that 0 ≤ ψ(x) ≤ 1 for all x ∈ RN and ψ(x) = { 1, if x ∈ B1(0), 0, if x ∈ Bc2(0) and consider (τn, ζn) a sequence defined by τn(x) = un(x)−R(N−2s)/2 n u0(Rn(x− xn))ψ(R̄n(x− xn)), (3.42) ζn(x) = vn(x)−R(N−2s)/2 n v0(Rn(x− xn))ψ(R̄n(x− xn)), (3.43) where (R̄n) satisfies R̃n = Rn R̄n →∞. From (3.42) and (3.43), we obtain R(2s−N)/2 n τn(x) = R(2s−N)/2 n un(x)− u0(Rn(x− xn))ψ(R̄n(x− xn)), R(2s−N)/2 n ζn(x) = R(2s−N)/2 n vn(x)− v0(Rn(x− xn))ψ(R̄n(x− xn)). Making a change of variable, we conclude that R(2s−N)/2 n τn ( z Rn + xn ) = R(2s−N)/2 n un ( z Rn + xn ) − u0ψ ( z R̃n ) , R(2s−N)/2 n ζn ( z Rn + xn ) = R(2s−N)/2 n vn ( z Rn + xn ) − v0ψ ( z R̃n ) . Now we define τ̃n = R(2s−N)/2 n τn ( z Rn + xn ) and ζ̃n = R(2s−N)/2 n ζn ( z Rn + xn ) . Since wn(x) = R(2s−N)/2 n un ( x Rn + xn ) and zn(x) = R(2s−N)/2 n vn ( x Rn + xn ) implies τ̃n(z) = wn(z)− u0(z)ψ ( z R̃n ) , (3.44) ζ̃n(z) = zn(z)− v0(z)ψ ( z R̃n ) . (3.45) If ψn(z) = ψ ( z R̃n ) (3.46) then ψn(z) = { 1, if z ∈ BR̃n(0), 0, if z ∈ Bc 2R̃n (0). From (3.45), (3.44) and (3.46), we derive that τ̃n(z) = wn(z)− u0(z)ψn(z), ζ̃n(z) = zn(z)− v0(z)ψn(z). 20 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 The result is proved if we show that u0ψn → u0 in Ds,2(RN ) and v0ψn → v0 in Ds,2(RN ), and that (wn, zn) is a (PS)c̃ sequence for J∞. For this, we note that ‖u0ψn − u0‖2 = ∫ RN |(−∆)s/2(u0ψn − u0)|2dx = ∫ R2N |u0(x)ψn(x)− u0(x)− u0(y)ψn(y) + u0(y)|2 |x− y|N+2s dx dy = ∫ R2N |u0(x)(ψn(x)− ψn(y)) + (ψn(y)− 1)(u0(x)− u0(y))|2 |x− y|N+2s dx dy ≤ 4 ∫ R2N |u0(x)|2|ψn(x)− ψn(y)|2 |x− y|N+2s dx dy + 4 ∫ R2N |ψn(y)− 1|2|u0(x)− u0(y)|2 |x− y|N+2s dx dy. (3.47) Arguing as in the proof of (3.21), if we replace wn by u0, and φ by ψn, since supp(ψn) ⊂ B2R̃(0), we can see that∫ R2N |u0(x)|2|ψn(x)− ψn(y)|2 |x− y|N+2s dx dy = on(1). (3.48) Moreover, taking into account that |ψn − 1| ≤ 2, |ψn − 1| → 0 a.e. in RN and u0 ∈ Ds,2(RN ), the Dominated Convergence Theorem implies that∫ R2N |ψn(y)− 1|2|u0(x)− u0(y)|2 |x− y|N+2s dx dy = on(1). (3.49) Combining (3.47), (3.48), and (3.49), we obtain u0ψn → u0 in Ds,2(RN ). Similarly arguing, we obtain v0ψn → v0 in Ds,2(RN ). Hence, τ̃n(z) = wn(z)− u0(z) + on(1), ζ̃n(z) = zn(z)− v0(z) + on(1). Since wn → u0 in Ds,2(RN ), zn → v0 in Ds,2(RN ), wn → u0 in RN and zn → v0, by [10, Lemma 2.2],∫ RN |(−∆)s/2wn|2dx = ∫ RN |(−∆)s/2u0|2dx+ ∫ RN |(−∆)s/2(wn − u0)|2dx+ on(1),∫ RN |(−∆)s/2zn|2dx = ∫ RN |(−∆)s/2v0|2dx+ ∫ RN |(−∆)s/2(zn − v0)|2dx+ on(1). By [3, Lemma 7.2], we have∫ RN H(wn, zn)dx = ∫ RN H(u0, v0)dx+ ∫ RN H(wn − u0, zn − v0)dx+ on(1) which implies that J∞(τn, ζn) = J∞(wn, zn)− J∞(u0, v0) + on(1). Therefore, J∞(τn, ζn)→ c̃ as n→ +∞, where c̃ = c−J∞(u0, v0). Moreover, using Hölder’s inequality and [3, Lemma 7.2] a direct calculation gives us ‖J ′∞(τ̃n, ζ̃n)− J ′∞(wn, zn) + J ′∞(u0, v0)‖(D×D)′ → 0. EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 21 Since (u0, v0) is a nontrivial critical point of J∞, we conclude that J ′∞(τ̃n, ζ̃n) = J ′∞(wn, zn) + J ′∞(u0, v0) + on(1) = J ′∞(wn, zn) + on(1). Since 0 ≤ ‖J ′∞(τn, ζn)‖(D×D)′ ≤ ‖J ′∞(τ̃n, ζ̃n)‖(D×D)′ , it follows that J ′∞(τn, ζn)→ 0 and the proof of Lemma 3.1 is complete. � The next result is a version of nonlocal global compactness result for a fractional Laplacian system in RN of the result due to Struwe that can be found in [26]. Theorem 3.2 (A global compactness result). Let (un, vn) be a (PS)c sequence for J with (un, vn) ⇀ (u0, v0) in Ds,2(RN ) × Ds,2(RN ). Then, up to a subsequence, (un, vn) satisfies either, (a) (un, vn)→ (u0, v0) in Ds,2(RN )×Ds,2(RN ) or, (b) there exists k ∈ N and nontrivial solutions (z1 0 , ζ 1 0 ), (z2 0 , ζ 2 0 ), . . . , (zk0 , ζ k 0 ) for the system (2.2), such that ‖(un, vn)‖2 → ‖(u0, v0)‖2 + k∑ j=1 ‖(zj0, ζ j 0)‖2, J (un, vn)→ J (u0, v0) + k∑ j=1 J∞(zj0, ζ j 0). Proof. From the weak convergence and a density argument, we have that (u0, v0) is a critical point of J . Suppose that (un, vn) 6→ (u0, v0) in Ds,2(RN ) × Ds,2(RN ) and let (w1 n, z 1 n) ⊂ Ds,2(RN ) × Ds,2(RN ) be the sequence given by (w1 n, z 1 n) = (un − u0, vn − v0). Then by hypothesis, (w1 n, z 1 n) ⇀ (0, 0) in Ds,2(RN )×Ds,2(RN ) and (w1 n, v 1 n) 6→ (0, 0). Applying [16, Lema 4.6] and [3, Lemma 7.2], we obtain J∞(w1 n, z 1 n) = J (un, vn)− J (u0, v0) + on(1), (3.50) J ′∞(w1 n, z 1 n) = J ′(un, vn)− J ′(u0, v0) + on(1). (3.51) Then, we conclude from (3.50) and (3.51) that (w1 n, z 1 n) is a (PS)c1 sequence for J∞. Hence, by Lemma 3.1, there are sequences Rn,1 ⊂ R, xn,1 ⊂ RN , (z1 0 , ζ 1 0 ) ∈ Ds,2(RN )×Ds,2(RN ) nontrivial solution for the system (2.2) and a (PS)c2 sequence (w2 n, z 2 n) ⊂ Ds,2(RN )×Ds,2(RN ) for J∞ such that w2 n(x) = w1 n(x)−R(N−2s)/2 n,1 z1 0(Rn,1(x− xn,1)) + on(1), z2 n(x) = z1 n(x)−R(N−2s)/2 n,1 ζ1 0 (Rn,1(x− xn,1)) + on(1). If we define Φ1 n(x) = R (2s−N)/2 n,1 w1 n ( x Rn,1 + xn,1 ) Ψ1 n(x) = R (2s−N)/2 n,1 z1 n ( x Rn,1 + xn,1 ) , w̃2 n(x) = R (2s−N)/2 n,1 w2 n ( x Rn,1 + xn,1 ) , z̃2 n(x) = R (2s−N)/2 n,1 z2 n ( x Rn,1 + xn,1 ) , 22 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 then we have w̃2 n(x) = Φ1 n(x)− z1 0(x) + on(1), (3.52) z̃2 n(x) = Ψ1 n(x)− ζ1 0 (x) + on(1), (3.53) ‖Φ1 n‖ = ‖w1 n‖, ‖Ψ1 n‖ = ‖z1 n‖, (3.54)∫ RN H(Φ1 n,Ψ 1 n)dx = ∫ RN H(w1 n, z 1 n)dx. (3.55) Hence, J∞(Φ1 n,Ψ 1 n) = J∞(w1 n, z 1 n), (3.56) J ′∞(Φ1 n,Ψ 1 n)→ 0 in (Ds,2(RN )×Ds,2(RN ))′. (3.57) By (3.56) and (3.57) and from item (a) of Lemma 2.1, we have that (Φ1 n,Ψ 1 n) is a bounded sequence in Ds,2(RN )×Ds,2(RN ) and, up to a subsequence, we have (Φ1 n,Ψ 1 n) ⇀ (z1 0 , ζ 1 0 ) in Ds,2(RN )×Ds,2(RN ) (3.58) Again, using [16, Lema 4.6] and [3, Lemma 7.2], we obtain J∞(w̃2 n, z̃ 2 n) = J∞(Φ1 n,Ψ 1 n)− J∞(z1 0 , ζ 1 0 ) + on(1) = J (un, vn)− J (u0, v0)− J∞(z1 0 , ζ 1 0 ) + on(1), (3.59) J ′∞(w̃2 n, z̃ 2 n) = J ′∞(Φ1 n,Ψ 1 n)− J ′∞(z1 0 , ζ 1 0 ) + on(1). (3.60) If (w̃2 n, z̃ 2 n)→ (0, 0) in Ds,2(RN )×Ds,2(RN ) the proof is complete for k = 1, because in this case, we have ‖(un, vn)‖2 → ‖(u0, v0)‖2 + ‖(z1 0 , ζ 1 0 )‖2. Moreover, using continuity of J∞, we obtain J (un, vn)→ J (u0, v0) + J∞(z1 0 , ζ 1 0 ). If (w̃2 n, z̃ 2 n) 6→ (0, 0) in Ds,2(RN ) × Ds,2(RN ), by (3.52)-(3.53) and (3.58) we have (w̃2 n, z̃ 2 n) ⇀ (0, 0) in Ds,2(RN ) × Ds,2(RN ), and using (3.59)and(3.60) we conclude that (w̃2 n, z̃ 2 n) is a (PS)c2 sequence for J∞. By Lemma 3.1, there are sequences (Rn,2) ⊂ R, (xn,2) ⊂ RN , (z2 0 , ζ 2 0 ) ∈ Ds,2(RN )×Ds,2(RN ) nontrivial solutions of (2.2), and a (PS)c3 sequence (w3 n, z 3 n) ⊂ Ds,2(RN )×Ds,2(RN ) for J∞ such that w3 n(x) = w̃2 n(x)−R(N−2s)/2 n,2 z2 0 (Rn,2(x− xn,2)) + on(1) z3 n(x) = z̃2 n(x)−R(N−2s)/2 n,2 ζ2 0 (Rn,2(x− xn,2)) + on(1). If Φ2 n(x) = R (2s−N)/2 n,2 w̃2 n ( x Rn,2 + xn,2 ) , Ψ2 n(x) = R (2s−N)/2 n,2 z̃2 n ( x Rn,2 + xn,2 ) , w̃3 n(x) = R (2s−N)/2 n,2 w3 n ( x Rn,2 + xn,2 ) , z̃3 n(x) = R (2s−N)/2 n,2 z3 n ( x Rn,2 + xn,2 ) , EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 23 then w̃3 n(x) = Φ2 n(x)− z2 0(x) + on(1), (3.61) z̃3 n(x) = Ψ2 n(x)− ζ2 0 (x) + on(1). (3.62) Arguing as before, we conclude that ‖(w̃3 n, z̃ 3 n)‖2 = ‖(un, vn)‖2 − ‖(u0, v0)‖2 − ‖(z1 0 , ζ 1 0 )‖2 − ‖(z2 0 , ζ 2 0 )‖2 + on(1), (3.63) J∞(w̃3 n, z̃ 3 n) = J (un, vn)− J (u0, v0)− J∞(z1 0 , ζ 1 0 )− J∞(z2 0 , ζ 2 0 ) + on(1), (3.64) J ′∞(w̃3 n, z̃ 3 n) = J ′∞(Φ2 n,Ψ 2 n)− J ′∞(z2 0 , ζ 2 0 ) + on(1). (3.65) If (w̃3 n, z̃ 3 n) → (0, 0) in Ds,2(RN ) × Ds,2(RN ), the proof is complete for k = 2, because in this case ‖(w̃3 n, z̃ 3 n)‖2 → 0 and from (3.63), we have ‖(un, vn)‖2 → ‖(u0, v0)‖2 + 2∑ j=1 ‖(zj0, ζ j 0)‖2. Similarly, if (w̃3 n, z̃ 3 n) → (0, 0) in Ds,2(RN ) × Ds,2(RN ) then the continuity of J∞ assures us that J∞(w̃3 n, z̃ 3 n)→ 0, and by (3.64) we obtain J (un, vn)→ J (u0, v0) + 2∑ j=1 J∞(zj0, ζ j 0). If (w̃3 n, z̃ 3 n) 6→ (0, 0) in Ds,2(RN ) × Ds,2(RN ), we can repeat the same arguments before and we can find (z1 0 , ζ 1 0 ), (z2 0 , ζ 2 0 ), . . . , (zk−1 0 , ζk−1 0 ) nontrivial solutions for the system (2.2) satisfying ‖(w̃kn, z̃kn)‖2 = ‖(un, vn)‖2 − ‖(u0, v0)‖2 − k−1∑ j=1 ‖(zj0, ζ j 0)‖2 + on(1), (3.66) J∞(w̃kn, z̃ k n) = J (un, vn)− J (u0, v0)− k−1∑ j=1 J∞(zj0, ζ j 0) + on(1). (3.67) From the definition of constant SH , we obtain(∫ RN H(zj0, ζ j 0)dx )2/2∗s SH ≤ ‖(zj0, ζ j 0)‖2, j = 1, 2, . . . , k − 1. (3.68) Since (zj0, ζ j 0) is a nontrivial solution of (2.2), for j = 1, 2, . . . , k − 1, we have ‖(zj0, ζ j 0)‖2 = ∫ RN H(zj0, ζ j 0)dx. Hence, − ‖(zj0, ζ j 0)‖ ≤ −SN/2sH , j = 1, 2, . . . , k − 1. (3.69) From (3.66) and (3.69), we have ‖(w̃kn, z̃kn)‖2 = ‖(un, vn)‖2 − ‖(u0, v0)‖2 − k−1∑ j=1 ‖(zj0, ζ j 0)‖2 + on(1) ≤ ‖(un, vn)‖2 − ‖(u0, v0)‖2 − k−1∑ j=1 S N/2s H + on(1) = ‖(un, vn)‖2 − ‖(u0, v0)‖2 − (k − 1)S N/2s H + on(1). 24 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Since (un, vn) is bounded in Ds,2(RN )×Ds,2(RN ) for k sufficient large, we conclude that (w̃kn, z̃ k n)→ (0, 0) in Ds,2(RN )×Ds,2(RN ) and the proof is complete. � The following two corollaries are immediate consequences of Theorem 3.2. Corollary 3.3. Let (un, vn) be a (PS)c sequence for J with c ∈ ( 0, sN S N/2s H ) . Then, up to a subsequence, (un, vn) converges strongly in Ds,2(RN )×Ds,2(RN ). Proof. Since (un, vn) is bounded in Ds,2(RN )×Ds,2(RN ), we have (un, vn) ⇀ (u0, v0) in Ds,2(RN )×Ds,2(RN ) and a denseness argument implies that J ′(u0, v0) = 0. Suppose, by contradiction, that (un, vn) 6→ (u0, v0) in Ds,2(RN )×Ds,2(RN ). From Theorem 3.2, there are nontrivial solutions (z1 0 , ζ 1 0 ), (z2 0 , ζ 2 0 ), . . . , (zk0 , ζ k 0 ) of system (2.2) and k ∈ N such that ‖(un, vn)‖2 → ‖(u0, v0)‖2 + k∑ j=1 ‖(zj0, ζ j 0)‖2, J (un, vn)→ J (u0, v0) + k∑ j=1 J∞(zj0, ζ j 0). By (2.1), we have J (u0, v0) = 1 2 ‖(u0, v0)‖2 + 1 2 ∫ RN (a(x)u2 0 + b(x)v2 0)dx− 1 2∗s ∫ RN H(u0, v0)dx = 1 2 ‖(u0, v0)‖2 + 1 2 (∫ RN H(u0, v0)dx− ‖(u0, v0)‖2 ) − 1 2∗s ∫ RN H(u0, v0)dx = (1 2 − 1 2∗s ) ∫ RN H(u0, v0)dx = s N ∫ RN H(u0, v0)dx ≥ 0. Then c = J (u0, v0) + k∑ j=1 J∞(zj0, ζ j 0) ≥ k∑ j=1 J∞(zj0, ζ j 0) ≥ ks N S N/2s H ≥ s N S N/2s H which contradicts c ∈ (0, sN S N/2s H ). � The next corollary tells us that the functional J satisfies the Palais-Smale con- dition. Corollary 3.4. The functional J : Ds,2(RN )×Ds,2(RN )→ R satisfies the Palais- Smale condition in ( s N S N/2s H , 2s N S N/2s H ) . Proof. Let (un, vn) ⊂ Ds,2(RN )×Ds,2(RN ) be a sequence such that J (un, vn)→ c and J ′(un, vn)→ 0. Since (un, vn) is bounded in Ds,2(RN )×Ds,2(RN ), up to a subsequence, we have (un, vn) ⇀ (u0, v0) in Ds,2(RN )×Ds,2(RN ). EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 25 Moreover, J (u0, v0) ≥ 0. Suppose, by contradiction,that (un, vn) 6→ (u0, v0) in Ds,2(RN )×Ds,2(RN ). From Theorem 3.2, there are nontrivial solutions (z1 0 , ζ 1 0 ), (z2 0 , ζ 2 0 ), .., (zk0 , ζ k 0 ) of sys- tem (2.2), and a k ∈ N such that ‖(un, vn)‖2 → ‖(u0, v0)‖2 + k∑ j=1 ‖(zj0, ζ j 0)‖2, J (un, vn)→ J (u0, v0) + k∑ j=1 J∞(zj0, ζ j 0) = c. Since J (u0, v0) ≥ 0, it follows that k = 1 and (z1 0 , ζ 1 0 ) cannot change sign. Hence, c = J (u0, v0) + J∞(u0, v0) = J (u0, v0) + s N S N/2s H . By the definition of SH , J ′(u0, v0) = 0, and J (u0, v0) = s N ∫ RN H(u0, v0)dx we have 2s N S N/2s H ≤ J (u0, v0) + s N S N/2s H = c, which contradicts c ∈ ( sN S N/2s H , 2s N S N/2s H ). � Corollary 3.5. Let (un, vn) be a (PS)c sequence for J with c ∈ (ks N S N/2s H , (k + 1)s N S N/2s H ) , where k ∈ N. Then the weak limit (u0, v0) of (un, vn) is not trivial. Proof. Suppose, by contradiction, that (u0, v0) ≡ (0, 0). Since c > 0, it follows that (un, vn) 6→ (0, 0) in Ds,2(RN ) × Ds,2(RN ). By Theorem 3.2, up to a subsequence, we obtain ‖(un, vn)‖2 → ‖(u0, v0)‖2 + k∑ j=1 ‖(zj0, ζ j 0)‖2 = k∑ j=1 ‖(zj0, ζ0)‖2, J (un, vn)→ J (u0, v0) + k∑ j=1 J∞(zj0, ζ j 0) = k∑ j=1 J∞(zj0, ζ j 0) = c ≥ (k + 1)s N S N/2s H which contradicts c ∈ (ksN S N/2s H , (k+1)s N S N/2s H ). � Next we consider the functional f : Ds,2(RN )×Ds,2(RN )→ R given by f(u, v) := ‖(u, v)‖2 + ∫ RN (a(x)u2 + b(x)v2)dx and the manifold M⊂ Ds,2(RN )×Ds,2(RN ) given by M := { (u, v) ∈ Ds,2(RN )×Ds,2(RN ) : ∫ RN H(u, v)dx = 1 } . 26 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Remark 3.6. Note that if (un, vn) ⊂M satisfies f(un, vn)→ c and f ′|M(un, vn)→ 0, then the sequence (wn, zn) ⊂ Ds,2(RN )×Ds,2(RN ), where (wn, zn) = ( c(N−2s)/4sun, c (N−2s)/4svn ) satisfies J (wn, zn)→ s N c N/2s and J ′(wn, zn)→ 0. The remark above combined with Corollary 3.4 leads us to the following result. Corollary 3.7. Suppose that there are a sequence (un, vn) ⊂ M, and a number c ∈ ( SH , 2 2s/NSH ) such that f(un, vn) → c and f ′|M(un, vn) → 0. Then, up to a subsequence, (un, vn)→ (u0, v0) in Ds,2(RN )×Ds,2(RN ) for some (u0, v0) ∈ Ds,2(RN )×Ds,2(RN ). rom Corollaries 3.4 and 3.7 we have the following result. Corollary 3.8. Suppose that there are a sequences (un, vn) ⊂ M and a number c ∈ ( SH , 2 2s/NSH ) such that f(un, vn) → c and f ′(un, vn) → 0. Then J has a critical point (w0, z0) ∈ Ds,2(RN )×Ds,2(RN ) with J (w0, z0) = s N c N/2s. 4. Technical lemmas In this subsection, we prove some properties of the function Φδ,b given in (1.6). Note that (Φδ,b,Φδ,b) ⊂ Σ := {(u, v) ∈ Ds,2(RN )×Ds,2(RN ) : u, v ≥ 0}. (4.1) Moreover, making a change of variable we can prove that Φδ,b ∈ Lq(RN ) for q ∈ ( N N − 2s , 2∗s ] , ∀δ > 0, ∀b ∈ RN . (4.2) Lemma 4.1. For each b ∈ RN , we have (i) ‖Φδ,b‖H1,∞(RN ) → 0 as δ → +∞; (ii) ‖Φδ,b‖H1,∞(RN ) → +∞ as δ → 0; (iii) |Φδ,b|q → 0 as δ → 0, for all q ∈ ( N N−2s , 2 ∗ s); (iv) |Φδ,b|q → +∞ as δ →∞, for all q ∈ ( N N−2s , 2 ∗ s). Proof. Using the definition of Φδ,b, we have |∇Φδ,b(x)| = Cδ N−2s 2 [δ2 + |x− b|2] N−2s+2 2 , where C is a positive constant. Thus ‖Φδ,b‖H1,∞(RN ) = C̃δ− N+2−2s 2 , C̃ > 0 and consequently (i) and (ii) follow. Now, note that |Φδ,b|qq = Ĉqδ q(2s−N) 2 +N ∫ RN ( 1 1 + |z|2 ) q(N−2s) 2 dz, Ĉ > 0, and so, for all q ∈ ( N N−2s , 2 ∗ s), (iii) and (iv) follow. � EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 27 Lemma 4.2. For each ε > 0, we have∫ RN\Bε(0) |(−∆)s/2Φδ,0|2dx→ 0, as δ → 0. The proof of the above lemma can be found in [13, Lemma 4.2]. Lemma 4.3. Assume condition (H7). Then: (i) for each ε > 0, there are δ = δ(ε) > 0 and δ̄ = δ̄(ε) > 0 such that sup b∈RN f(`0Φδ,b, t0Φδ,b) < SH + ε, ∀δ ∈ (0, δ] ∪ [δ̄,∞); (ii) for each δ > 0, we have lim |b|→+∞ f(`0Φδ,b, t0Φδ,b) = SH . Proof. (i) Consider b ∈ RN , q ∈ (N2s , p2] and t ∈ (1,+∞) with 1 q + 1 t = 1. By a simple calculations, N N − 2s < 2t < 2∗s. Since Φδ,b ∈ Ld(RN ) for all d ∈ ( N N−2s , 2 ∗ s ) , we obtain |Φδ,b|2 ∈ Lt(RN ). Then, using Hölder’s inequality and a change of variable, we have∫ RN a(x)|Φδ,b|2dx ≤ |a|q (∫ RN ∣∣ cδ(N−2s)/2 [δ2 + |x− b|2](N−2s)/2 ∣∣2tdx)1/t = |a|q (∫ RN ∣∣ cδ(N−2s)/2 [δ2 + |z|2](N−2s)/2 ∣∣2tdz)1/t = |a|q (∫ RN |Φδ,0|2tdz )1/t = |a|q|Φδ,0|22t, ∀b ∈ RN . Arguing in the same way, we have∫ RN b(x)|Φδ,b|2dx ≤ |b|q|Φδ,0|22t, ∀b ∈ RN . From Lemma 4.1(iii), given ε > 0 there exists δ = δ(ε) > 0 such that sup b∈RN f(`0Φδ,b, t0Φδ,b) ≤ SH + ε 2 + ε 2 ≤ SH + ε, ∀δ ∈ (0, δ]. On the other hand, suppose q ∈ [p1, N 2s ) with t ∈ (1,+∞) and 1 q + 1 t = 1. In these conditions we have 2t− 2∗s > 0, |Φδ,y|2 ∗ s ∈ L1(RN ) and for δ > 1, |Φδ,y| ∈ L∞(RN ), and so |Φδ,y|2 ∈ Lt(RN ). Thus, using Hölder’s inequality with exponents q and t and remembering that ‖Φδ,0‖2∗s = 1, we deduce `20 ∫ RN a(x)|Φδ,y|2dx ≤ `20|a|q (∫ RN |Φδ,0|2tdz )1/t = `20|a|q|Φδ,0| 2t−2∗ t∞ (∫ RN |Φδ,0|2 ∗ sdx )1/t ≤ `20|a|qCδ 2s−N 2 2t−2∗s t , ∀b ∈ RN . (4.3) 28 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Once ( 2s−N 2 )( 2t−2∗s t ) < 0, given ε > 0, there is δ̄ = δ̄(ε) > 1 such that δ 2s−N 2 2t−2∗s t < ε 2`20C|a|q ∀δ ∈ [δ̄,∞). (4.4) Arguing in the same way, we have t20 ∫ RN b(x)|Φδ,b|2dx ≤ t20|a|qCδ 2s−N 2 2t−2∗s t , ∀b ∈ RN . (4.5) Combining (4.3), (4.4), and (4.5), we obtain `20 sup b∈RN ∫ RN a(x)|Φδ,b|2dx < ε 2 , ∀δ ∈ [δ̄,∞), t20 sup b∈RN ∫ RN b(x)|Φδ,b|2dx < ε 2 , ∀δ ∈ [δ̄,∞). Therefore, f(`0Φδ,, t0Φδ,) = ∫ RN |(−∆)s/2`0Φδ,b|2dx+ ∫ RN |(−∆)s/2t0Φδ,b|2dx + `20 ∫ RN a(x)|Φδ,b|2dx+ t20 ∫ RN b(x)|Φδ,b|2dx < SH + ε, ∀b ∈ RN , ∀δ ∈ [δ̄,∞). (ii) Since f(`0Φδ,b, t0Φδ,b) = ∫ RN |(−∆)s/2`0Φδ,b|2dx+ ∫ RN |(−∆)s/2t0Φδ,b|2dx + `20 ∫ RN a(x)|Φδ,b|2dx+ t20 ∫ RN b(x)|Φδ,b|2dx = SH + `20 ∫ RN a(x)|Φδ,b|2dx+ t20 ∫ RN b(x)|Φδ,b|2dx, it suffices to prove that lim |b|→∞ ( `20 ∫ RN a(x)|Φδ,b|2dx+ t20 ∫ RN b(x)|Φδ,b|2dx ) = 0, ∀δ > 0. (4.6) Note that given ε > 0, there are k1, k2 > 0 such that(∫ RN\Bρ(0) a(x)N/2sdx )2s/N < ε, ∀ρ > k1, (4.7)(∫ RN\Bρ(0) |Φδ,b|2 ∗ sdx )1/2∗s = (∫ RN\Bρ(0) |Φδ,0|2 ∗ sdz )1/2∗s < ε, ∀ρ > k2. (4.8) Let k0 = max{k1, k2} and consider k0 < 2ρ < |b| (ρ fixed) (4.9) and note that Bρ(0) ∩Bρ(b) = ∅. (4.10) Using Hölder’s inequality with exponents N/2s and N/(N − 2s), and taking into account (4.7), (4.8), (4.9), and (4.10), we obtain∫ RN a(x)|Φδ,b|2dx EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 29 ≤ (∫ RN\(Bρ(0)∪Bρ(b)) aN/2sdx )2s/N(∫ RN\(Bρ(0)∪Bρ(b)) |Φδ,b|2 ∗ sdx )N−2s N + (∫ Bρ(0) aN/2sdx )2s/N(∫ Bρ(0) |Φδ,b|2 ∗ sdx )N−2s N + (∫ Bρ(b) aN/2sdx )2s/N(∫ Bρ(b) |Φδ,b|2 ∗ sdx )N−2s N ≤ (∫ RN\Bρ(0) aN/2sdx )2s/N(∫ RN\Bρ(b) |Φδ,b|2 ∗ sdx )N−2s N + (∫ RN aN/2sdx )2s/N(∫ RN\Bρ(b) |Φδ,b|2 ∗ sdx )N−2s N + (∫ RN\Bρ(0) aN/2sdx )2s/N(∫ RN |Φδ,b|2 ∗ sdx )N−2s N < εε2 + ε2|a|N/2s + ε. Arguing similarly for the second part of (4.6), we obtain∫ RN b(x)|Φδ,b|2dx < εε2 + ε2|b|N/2s + ε and the proof is complete. � Lemma 4.4. Under Assumption (H8), sup δ>0,b∈RN f(`0Φδ,b, t0Φδ,b) < 22s/NSH . Proof. Using the definition of F , Hölder’s inequality with N/2s and N/(N − 2s), and condition (H8), we obtain sup δ>0,b∈RN f(`0Φδ,b, t0Φδ,b) ≤ SH + `20|a|N/2s + t20|b|N/2s < SH + SH(22s/N − 1) = 22s/NSH . � In what follows, we consider the function ξ(x) = { 0, if |x| < 1 1, if |x| ≥ 1 and define κ : Ds,2(RN )×Ds,2(RN )→ RN+1 by κ(u, v) = 1 SH ∫ RN ( x |x| , ξ(x) ) [`20|(−∆)s/2u|2+t20|(−∆)s/2v|2]dx = (β(u, v), γ(u, v)), where β(u, v) = 1 SH ∫ RN x |x| [`20|(−∆)s/2u|2 + t20|(−∆)s/2v|2]dx, γ(u, v) = 1 SH ∫ RN ξ(x)[`20|(−∆)s/2u|2 + t20|(−∆)s/2v|2]dx. Lemma 4.5. If |b| ≥ 1/2, then β(Φδ,b,Φδ,b) = b |b| + oδ(1) as δ → 0. 30 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Proof. By Lemma 4.2, there is δ̂ > 0 such that∣∣∣β(Φδ,b,Φδ,b)− 1 SH ∫ Bε(b) x |x| [`20|(−∆)s/2Φδ,b|2 + t20|(−∆)s/2Φδ,b|2dx] ∣∣∣ = ∣∣∣`20 + t20 SH ∫ RN\Bε(b) x |x| |(−∆)s/2Φδ,b|2 ∣∣∣ ≤ `20 + t20 SH ∫ Bε(b) |(−∆)s/2Φδ,b|2dx < ε, (4.11) for all δ ∈ (0, δ̂). On the other hand, for ε > 0 sufficiently small and |b| > 1/2, we have ∣∣ x |x| − b |b| ∣∣ < 4ε, ∀x ∈ Bε(x), and so∣∣∣ b|b| − `20 + t20 SH ∫ Bε(b) x |x| |(−∆)s/2Φδ,b|2dx ∣∣∣ < 4ε+ ε, ∀δ ∈ (0, δ̂). (4.12) From (4.11) and (4.12), it follows that∣∣β(Φδ,b,Φδ,b)− b |b| ∣∣ ≤ ∣∣∣β(Φδ,b,Φδ,b)− `20 + t20 SH ∫ Bε(b) x |x| |(−∆)s/2Φδ,b|2dx ∣∣∣ + ∣∣∣`20 + t20 SH ∫ Bε(b) x |x| |(−∆)s/2Φδ,b|2 − b |b| ∣∣∣ < Cε, ∀δ ∈ (0, δ̂). This completes the proof. � Now we define the set = = {(u, v) ∈M;κ(u, v) = (0, 1 2 )}. Lemma 4.6. The set = is not empty. Proof. Since that Φδ,0 is an odd function and Br(0) is symmetric, we have that β(Φδ,0,Φδ,0) = 0. From Lemma 4.2, we see that γ(Φδ,0,Φδ,0)→ 0 as δ → 0. (4.13) On the other hand γ(Φδ,0,Φδ,0) = 1− `20 + t20 SH ∫ B1(0) |(−∆)s/2Φδ,0|2dx, (4.14) and moreover, by [24, Proposition 2.2], we see that∫ B1(0) |(−∆)s/2Φδ,0|2dx ≤ ∫ B1(0) |∇Φδ,0|2dx ≤ Cδ2s−2 ∫ B1(0) |z|2 [1 + |z|2]N−2s+2 dz ≤ C̃δ2s−2 → 0, as δ → +∞. (4.15) Combining (4.14) and (4.15) we have γ(Φδ,0,Φδ,0)→ 1, as δ → +∞. (4.16) By (4.13) and (4.16) there is δ1 > 0 such that (Φδ1,0,Φδ1,0) ∈ =. � EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 31 Lemma 4.7. The number c0 = inf(u,v)∈= f(u, v) satisfies the inequality c0 > SH . Proof. Since = ⊂M, we have SH ≤ c0. (4.17) Suppose, by contradiction, that SH = c0. By Ekeland variational principle [31], there is (un, vn) ∈ Ds,2(RN )×Ds,2(RN ) such that∫ RN H(un, vn)dx = 1, τ(un, vn)→ ( 0, 1 2 ) , (4.18) f(un, vn)→ SH , f ′|M(un, vn)→ 0. (4.19) Then, (un, vn) is bounded in Ds,2(RN ) × Ds,2(RN ), and, up to a subsequence, (un, vn) ⇀ (ū, v̄) in Ds,2(RN )×Ds,2(RN ). If (wn, zn) = (S N−2s 4s H un, S N−2s 4s H vn) and (w̄, z̄) = (S N−2s 4s H ū, S N−2s 4s H v̄), we see that (wn, zn) ⇀ (w̄, z̄) in Ds,2(RN ) ×Ds,2(RN ), and so, by (4.19) and Remark 3.6, we obtain J (wn, zn)→ s N S N 2s H and J ′(wn, zn)→ 0. We are going to show that (w̄, z̄) ≡ (0, 0). First of all, note that (un, vn) 6→ (ū, v̄) in Ds,2(RN )×Ds,2(RN ), (4.20) because otherwise (ū, v̄) 6= (0, 0) and SH ≤ ∫ RN |(−∆)s/2ū|2dx+ ∫ RN |(−∆)s/2v̄|2dx < ∫ RN |(−∆)s/2ū|2dx+ ∫ RN |(−∆)s/2v̄|2dx+ ∫ RN a(x)|ū|2dx+ ∫ RN b(x)|v̄|2dx = SH , which is a contradiction. Thus, (wn, zn) 6→ (w̄, z̄) in Ds,2(RN ) × Ds,2(RN ) and, since (wn, zn) is a (PS)c sequence for J , by Theorem 3.2, we have J (wn, zn)→ J (w̄, z̄) + k∑ j=1 J∞(uj0, v j 0) = s N S N 2s H . Using J ′(uj0, v j 0) = 0, we obtain J (w̄, z̄) = 0, k = 1, uj0, v j 0 > 0, J∞(w̄, z̄) = s N ∫ RN H(w̄, z̄)dx, (4.21) which implies that (w̄, z̄) = (0, 0). Then (wn, zn) is a (PS)c sequence for J such that (wn, zn) ⇀ (0, 0), (wn, zn) 6→ (0, 0), ∫ RN a(x)|wn|2dx = on(1) and∫ RN b(x)|zn|2dx = on(1). Therefore, s N S N 2s H + on(1) = J (wn, zn) = J∞(wn, zn) + ∫ RN a(x)|wn|2dx+ ∫ RN b(x)|zn|2dx = J∞(wn, zn) + on(1) (4.22) and ‖J ′∞(wn, zn)‖(D×D)′ ≤ ‖J ′(wn, zn)‖(D×D)′ + on(1). (4.23) 32 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 From (4.22) and (4.23) we conclude that (wn, zn) is a (PS)c sequence for J∞ and by Lemma 3.1, there are sequences (Rn) ⊂ R, (xn) ⊂ RN , u1 0, v 1 0 nontrivial solution of (2.2) and (τn, ζn) a (PS)c sequence for J∞ such that wn(x) = τn(x) +R N−2s 2 n u1 0(Rn(x− xn)) + on(1), zn(x) = ζn(x) +R N−2s 2 n v1 0(Rn(x− xn)) + on(1). Setting τ̃n(x) = R N−2s 2 n u1 0(Rn(x− xn)) and ζ̃n(x) = R N−2s 2 n v1 0(Rn(x− xn)) and making change of variable, we have J ′∞(τ̃n, ζ̃n)(ϕ1, ϕ2) = J ′∞(u1 0, v 1 0)(ϕ1,n, ϕ2,n) = 0, for all (ϕ1, ϕ2) ∈ Ds,2(RN )×Ds,2(RN ) and for all n ∈ N; thus (τ̃n, ζ̃n) is a solution of (2.2), for all n ∈ N. Moreover, from definition of (τ̃n, ζ̃n) and by (4.21), we obtain τ̃n(x) = ζ̃n(x) = c ( δn δ2 n + |x− bn|2 )N−2s 2 , x ∈ RN . Therefore, un(x) = τn(x) + Φδn,bn(x) + on(1) and vn(x) = ζn(x) + Φδn,bn(x) + on(1), where τn(x) = S 4s N−2s H τn(x), ζn(x) = S 4s N−2s H ζn(x), Φδn,bn(x) = S 4s N−2s H τ̃n(x) = S 4s N−2s H ζ̃n(x). By (4.21), we derive that τn → 0 and ζn → 0 in Ds,2(RN ), which implies that τn → 0 and ζn → 0 in Ds,2(RN ). Therefore, from (4.18), we have( 0, 1 2 ) + on(1) = κ(un, vn) = κ(Φδn,bn ,Φδn,bn) which implies that (i) β(Φδn,bn ,Φδn,bn)→ 0, (ii) γ(Φδn,bn ,Φδn,bn)→ 1/2. Passing to a subsequence, one of the following cases must occur. (a) δn → +∞ when n→ +∞; (b) δn → δ 6= 0 when n→ +∞; (c) δn → 0 and bn → b when n→ +∞ with |b| < 1/2; (d) δn → 0 when n→ +∞ and |bn| ≥ 1/2 for n sufficiently large. Suppose that (a) is true. Then γ(Φδn,bn ,Φδn,bn) = 1− `20 + t20 SH ∫ B1(0) |(−∆)s/2Φδn,bn |2dx and by Lemma 4.1 we deduce that |γ(Φδn,bn ,Φδn,bn)− 1| = `20 + t20 SH ∫ B1(0) |(−∆)s/2Φδn,bn |2dx = on(1) which contradicts (ii). EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 33 Suppose that (b) is true. In this case we may suppose that |bn| → +∞ because if bn → b, we can prove that Φδn,bn → Φδ,b in Ds,2(RN ). Since τn, ζn → 0 in Ds,2(RN ) and un = τn+Φδn,bn+on(1), vn = ζn+Φδn,bn+on(1), we see that (un, vn) converges in Ds,2(RN )×Ds,2(RN ) but this is a contradiction with (4.20). Hence, γ(Φδn,bn ,Φδn,bn) = `20 + t20 SH ∫ B1(0) |(−∆)s/2Φδn,bn |2dx = `20 + t20 SH ∫ RN\B1(0) |(−∆)s/2Φδn,bn |2dx = 1− `20 + t20 SH ∫ B1(−bn) |(−∆)s/2Φδn,0|2dx. (4.24) Applying Lebesgue’s theorem we can show that∫ B1(−bn) |(−∆)s/2Φδn,0|2dx→ 0 as n→ +∞ and from (4.24) we obtain γ(Φδn,bn ,Φδn,bn)→ 1, as n→ +∞, which again contra- dicts (ii). Suppose that (c) is true. Note that γ(Φδn,bn ,Φδn,bn) = `20 + t20 SH ∫ B1(0) |(−∆)s/2Φδn,bn |2dx = `20 + t20 SH ∫ RN\B1(0) |(−∆)s/2Φδn,bn |2dx = 1− `20 + t20 SH ∫ B1(−bn) |(−∆)s/2Φδn,0|2dx. (4.25) Therefore, using again the Lebesgue theorem, we deduce that lim n→+∞ `20 + t20 SH ∫ B1(−bn) |(−∆)s/2Φδn,0|2dx = 1 From (4.25) we obtain γ(Φδn,bn ,Φδn,bn)→ 0, which again contradicts (ii). Suppose that (d) is true. Since |bn| ≥ 1/2 for n large, we have that bn 6→ 0 in RN . From Lemma 4.5 we have β(Φδn,bn ,Φδn,bn) = bn |bn| + on(1). Thus, β(Φδn,bn ,Φδn,bn) 6→ 0, which contradicts (i). So, SH < c0 and the proof is complete. � Lemma 4.8. There is δ1 ∈ (0, 1/2) such that (a) f(`0Φδ1,b, t0Φδ1,b) < c0+SH 2 , ∀b ∈ RN ; (b) γ(Φδ1,b,Φδ1,b) < 1/2 for all b ∈ RN such that |b| < 1/2; (c) |β(Φδ1,b,Φδ1,b)− b |b| | < 1/4 for all b ∈ RN such that |b| ≥ 1/2. 34 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Proof. From Lemma 4.3, we can choose ε = c0−SH 2 > 0, δ2 < min{δ, 1/2}, and conclude that f(`0Φδ3,b, t0Φδ3,b) ≤ sup b∈RN f(`0Φδ3,b, t0Φδ3,b) < SH + c0 − SH 2 = c0 + SH 2 , (4.26) for all b ∈ RN . Now by the definition of ξ and Lemma 4.1, we have γ(Φδ,b,Φδ,b) = 1− `0 + t0 SH ∫ B1(−b) |(−∆)s/2Φδ,0|2dz. and by Lebesgue Theorem `0 + t0 SH ∫ B1(−b) |(−∆)s/2Φδ,0|2dz → 1, thus γ(Φδ,b,Φδ,b) → 1 as δ → 0. The above convergence assures us that there is δ̂ > 0 such that γ(Φδ,b,Φδ,b) < 1 2 for all δ ∈ (0, δ̂). Choosing δ4 < min{δ̂, 1/2} we have γ(Φδ4,b,Φδ4,b) < 1 2 , ∀b ∈ RN with |b| < 1 2 . (4.27) Furthermore, by Lemma 4.5, there is δ̃ > 0 such that∣∣∣∣β(Φδ,b,Φδ,b)− b |b| ∣∣∣∣ < 1 4 , ∀δ ∈ (0, δ̃), with |b| ≥ 1 2 . Thus, choosing δ5 < min{δ̃, 1/2} we obtain∣∣β(Φδ5,b,Φδ5,b)− b |b| ∣∣ < 1 4 , ∀b ∈ RN , with |b| ≥ 1 2 . (4.28) Finally, choosing δ1 = min{δ3, δ4, δ5} the result follows from (4.26), (4.27), and (4.28). � Lemma 4.9. There is δ2 > 0 such that (a) f(`0Φδ2,b, t0Φδ2,b) < c0+SH 2 for all b ∈ RN ; (b) γ(Φδ2,b,Φδ2,b) > 1 2 for all b ∈ RN . Proof. Given ε = c0−SH 2 > 0, by Lemma 4.3, we can choose δ3 > max{δ, 1/2} such that f(`0Φδ3 , t0Φδ3) ≤ sup b∈RN f(`0Φδ3 , t0Φδ3) < SH + c0 2 , ∀b ∈ RN . (4.29) On other hand, γ(Φδ,b,Φδ,b) = 1− `20 + t20 SH ∫ B1(−b) |(−∆)s/2Φδ,0|2dx and applying [24, Proposition 2.2] and Lemma 4.1 see that∫ B1(−b) |(−∆)s/2Φδ,0|2dx→ 0 as δ → +∞. Thus, for each b ∈ RN , γ(Φδ,b,Φδ,b) → 1 as δ → +∞; hence, there is δ̂ > 0 such that γ(Φδ,b,Φδ,b) > 1 2 , ∀δ ∈ (δ̂,+∞). EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 35 Choosing δ4 > max{δ̂, 1/2}, we have γ(Φδ4,b,Φδ4,b) > 1 2 , ∀b ∈ RN . (4.30) Now, choosing δ2 = max{δ3, δ4} the result follows of (4.29) and (4.30). � Lemma 4.10. There is R > 0 such that (a) f(`0Φδ,b, t0Φδ,b) < c0+SH 2 for all b for which |b| ≥ R and δ ∈ [δ1, δ2]; (b) (β(Φδ,b,Φδ,b)|b)RN > 0 for all b for which |b| ≥ R and δ ∈ [δ1, δ2]. Proof. From Lemma 4.3 assuming δ = c0−SH 2 > 0, we can find R1 > 0, big enough, such that f(`0Φδ,b, t0Φδ,b) < SH + δ = SH + c0 2 , ∀b : |b| ≥ R1, and δ ∈ [δ1, δ2], (4.31) and item (a) follows. Now, for each b ∈ RN we consider the sets (RN )+ b = {x ∈ RN ; (x|b)RN > 0} and (RN )−b = RN \ (RN )+ b . Since ε varies in the compact set [δ1, δ2], we can prove there is R2 > 0 big enough and r ∈ (0, 1 4 ) such that the following things are true if |b| ≥ R2 and |b− b0| = 1 2 , Br(b0) = {x ∈ RN ; |b− b0| < r} ⊂ (RN )+ b . Initially, note that for every x ∈ Br(b0), we have |(−∆)s/2Φδ,b|2 = ∫ RN |Φδ,b(x)− Φδ,b(y)|2 |x− y|N+2s dy ≥ ∫ B r 2 (b)\B r 4 (b) ∣∣ cδ N−2s 2 [δ2+|x−b|2] N−2s 2 − cδ N−2s 2 [δ2+|y−b|2] N−2s 2 ∣∣2 |x− y|N+2s dy ≥ ∫ B r 2 (b)\B r 4 (b) cδN−2s 1 ( 7 8 )N+2s ∣∣∣ 1 [δ2 2 + 9 16 ]N−2s 2 − 1 [δ2 1 + 1 64 ] N−2s 2 ∣∣∣2dy := H1 > 0. Thus, (β(Φδ,b,Φδ,b)|b)RN ≥ `20 + t20 SH {∫ Br(b0) (x|b) |x| H1dx+ ∫ (RN )−b (x|b) |x| |(−∆)s/2Φδ,b|2dx } ≥ `20 + t20 SH { |b| ∫ Br(b0) (x|b) |x||b| H1dx− |b| ∫ (RN )−b |(−∆)s/2Φδ,b|2dx } ≥ `20 + t20 SH { |b|C1 ∫ Br(b0) H1 |x| dx− |b| ∫ (RN )−b |(−∆)s/2Φδ,b|2dx } = `20 + t20 SH { |b|H2 − |b| ∫ (RN )−b |(−∆)s/2Φδ,b|2dx } , (4.32) where H2 := C1 ∫ Br(b0) H1 |x| dx. 36 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Moreover, using [24, Propositin 2.2] and spherical coordinates, we deduce that∫ (RN )−b |(−∆)s/2Φδ,b(x)|2dx ≤ ∫ BcR2 (b) |(−∆)s/2Φδ,b(x)|2dx ≤ C2 ∫ BcR2 (b) |(−∆)s/2Φδ,b(x)|2dx ≤ C2 ∫ BcR2 (0) |(−∆)s/2Φδ,0(x)|2dx ≤ C2δ 2s−2 ∫ BcR2 (0) |z| (1 + |z|2)N+2s+2 dz = C3δ N−2sR −(N+2−4s) 2 , where we can choose R2 > 0 large, such that for all b ∈ RN with |b| > R2, we have∫ (RN )−b |(−∆)s/2Φδ,b(x)|2dx < H2. (4.33) Therefore, from (4.32) and (4.33), it follows that (β(Φδ,b,Φδ,b)|b) ≥ `20 + t20 SH |b| { H2 − ∫ (RN )−b |(−∆)s/2Φδ,b(x)|2dx } > 0, (4.34) for all |b| > R2 and for all δ ∈ [δ1, δ2]. Now, choosing R = max{R1, R2} the result of (4.31) and (4.34). � 5. Proof of main theorem To prove Theorem 1.1, we first fix some notation and give some more technical lemmas. Consider the set V := {(b, δ) ∈ RN × (0,∞) : |b| < R and δ ∈ (δ1, δ2)}, where δ1, δ2 and R are given by Lemmas 4.8, 4.9, and 4.10, respectively. Let Q : RN × (0,∞)→ Ds,2(RN ) be the continuous function given by Q(b, δ) = Φδ,b. With the above notation, we define the sets Θ := {(Q(b, δ), Q(b, δ)) : (b, δ) ∈ V}, H := {h ∈ C(Σ ∩M) : h(u, v) = (u, v), ∀(u, v) ∈ Σ ∩M : f(`0u, t0v) < c0 + SH 2 }, Γ := {A ⊂ Σ ∩M : A = h(Θ), h ∈ H}. Note that Θ ⊂ Σ ∩ M, Θ = Q(V) × Q(V) is compact and H 6= 0, because the identity function is in H. Lemma 5.1. Let F : V → RN+1 be the function defined by F(b, δ) = (κ ◦ (Q,Q))(b, δ) = `20 + t20 SH ∫ RN ( x |x| , ξ(x) ) |(−∆)s/2Φb,δ|2 dx. Then the topological degree is d(F ,V, (0, 1/2)) = 1. EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 37 Proof. Define Z : [0, 1]× V → RN+1 the homotopy by Z(t, (b, δ)) = tF(b, δ) + (1− t)IV(b, δ) where I is the identity operator. Using Lemma 4.8 and Lemma 4.9, we can show that (0, 1/2) /∈ Z([0, 1]× (∂V)), i.e., tβ(Φb,δ,Φb,δ) + (1− t)b 6= 0, ∀t ∈ [0, 1] and ∀(b, δ) ∈ ∂V (5.1) or tγ(Φb,δ,Φb,δ) + (1− t)δ 6= 1 2 , ∀t ∈ [0, 1] and ∀(b, δ) ∈ ∂V. (5.2) Since, (0, 1/2) /∈ Z([0, 1]×(∂V)), we have that d(F ,V, (0, 1/2)), d(Z(t, ·),V, (0.1/2)) is well defined and by properties of the topological degree d(F ,V, (0, 1/2)) = d(IV ,V, (0, 1/2)). Since (0, 1/2) ∈ V, we deduce that d(F ,V, (0, 1/2)) = d(IV ,V, (0, 1/2)) = 1. � Lemma 5.2. If A ∈ Γ, then A ∩ = 6= ∅. Proof. It is sufficient to prove that for all h ∈ H, there exists (b0, δ0) such that (κ ◦ H ◦ (Q,Q))(b0, δ0) = ( 0, 1 2 ) . Given h ∈ H, let Fh : V → RN+1 be the continuous function given by Fh(b, δ) = (κ ◦ h ◦ (Q,Q))(b, δ). Now we show that Fh = F in ∂V. Note that ∂V = Π1 ∪Π2 ∪Π3, (5.3) where Π1 := {(b, δ1) : |b| ≤ R}, Π2 := {(b, δ2) : |b| ≤ R}, Π3 := {(b, δ3) : |b| = R and δ ∈ [δ1, δ2]}. If (b, δ) ∈ Π1, then (b, δ) = (b, δ1). and by Lemma 4.8-(a), we have f(`0Q(b, δ), t0Q(b, δ)) = f(`0Q(b, δ1), t0Q(b, δ1)) = f(`0Φδ1,b, t0Φδ1,b) < SH + c0 2 , ∀(b, δ) ∈ Π1. (5.4) If (b, δ) ∈ Π2, then (b, δ) = (b, δ2), and by Lemma 4.9(a), we have f(`0Q(b, δ), t0Q(b, δ)) = f(`0Q(b, δ2), t0Q(b, δ2)) = f(`0Φδ2,b, t0Φδ2,b) < SH + c0 2 , ∀(b, δ) ∈ Π2. (5.5) If (b, δ) ∈ Π3, then |b| = R and δ ∈ [δ1, δ2] and by Lemma 4.10(a), we obtain f(`0Q(b, δ), t0Q(b, δ)) = f(`0Φδ,b, t0Φδ,b) < SH + c0 2 , ∀(b, δ) ∈ Π3. (5.6) 38 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Combining (5.3), (5.4), (5.5), and (5.6), we obtain f(`0Φδ,b, t0Φδ,b) < SH + c0 2 , ∀(b, δ) ∈ ∂V. Thus, Fh(b, δ) = (κ ◦ h ◦ (Q,Q))(b, δ) = (κ ◦ h)(Q(b, δ), Q(b, δ)) = κ(h((Q(b, δ), Q(b, δ)))) = κ((Q(b, δ), Q(b, δ))) = (κ ◦ (Q,Q))(b, δ) = F(b, δ), ∀(b, δ) ∈ ∂V. Since (0, 1/2) /∈ F(∂V), we obtain d(Fh,V, (0, 1/2)) = d(F ,V, (0, 1/2)). By Lemma 5.1, we have d(Fh,V, (0, 1/2)) = d(F ,V, (0, 1/2)) = 1, and there is (b0, δ0) ∈ V such that Fh(b0, δ0) = (κ ◦ h ◦ (Q,Q))(b0, δ0) = ( 0, 1 2 ) and the proof is complete. � Proof of Theorem 1.1. We define the number c = inf A∈Γ max (u,v)∈A f(u, v) and for each q ∈ R, we define the set fq := {(u, v) ∈ Σ ∩M : f(u, v) ≤ q}. We start our analysis by noting that SH < c < 22s/NSH . (5.7) In fact, by Lemma 4.4, c = inf A∈Γ max (u,v)∈A f(u, v) ≤ max (u,v)∈Θ f(u, v) ≤ sup (b,δ)∈RN×(0,+∞) f(`0Φδ,b, t0Φδ,b) < 22s/NSH . On the other hand, by Lemmas 4.7 and 5.2, we obtain SH < c0 = inf (u,v)∈= f(u, v) = inf A∈Γ max (u,v)∈A f(u, v) ≤ sup (b,δ)∈RN×(0,+∞) f(`0Φδ,b, t0Φδ,b) < 22s/NSH , (5.8) from where it follows (5.7). EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 39 Using the definition of c, there exists the sequence (un, vn) ∈ Σ ∩M such that f(un, vn) → c. Suppose, by contradiction, that f ′|M(un, vn) 6→ 0. Then, there exists (unj , vnj ) ⊂ (un, vn) such that ‖f ′|M(unj , vnj )‖∗ ≥ C > 0, ∀j ∈ N. By a deformation Lemma [31], there exists a continuous application η : [0, 1]×Σ∩ M→ Σ ∩M and ε0 > 0 such that (a) η(0, (u, v)) = (u, v); (b) η(t, (u, v)) = (u, v) for all (u, v) ∈ f c−ε0∪{(Σ∩M)\f c+ε0} and all t ∈ [0, 1]; (c) η(1, f c+ε0/2) ⊂ f c−ε0/2. From the definition of c, there exists à ∈ Γ such that c ≤ max (u,v)∈à f(u, v) < c+ ε0 2 , where à ⊂ f c+ ε0 2 . (5.9) Since à ∈ Γ we have à ⊂ (Σ ∩M) and there exists h ∈ H such that h(Θ) = Ã. (5.10) From the definition of η, we have η(1, Ã) ⊂ (Σ ∩M). (5.11) Let h∗ : (Σ∩M)→ (Σ∩M) be the function given by h∗(u, v) = η(1, h(1, v)). Note that h∗ ∈ C(Σ ∩M,Σ ∩M). We are going to show that f c+ε0 \ f c−ε0 ⊂ f22s/NSH \ f (SH+c0)/2. (5.12) Indeed, given (u, v) ∈ f c+ε0 \ f c−ε0 , we have c− ε0 < f(u, v) ≤ c+ ε0 and by (5.7), for ε0 sufficiently small, we obtain c− ε0 < f(u, v) ≤ c+ ε0 < 22s/NSH . Now, combining Lemma 4.7 with (5.8), we have SH + c0 2 < c0 − ε0 < c− ε0 < 22s/NSH , SH + c0 2 < c0 − ε0 ≤ c− ε0 < f(u, v) which implies (u, v) ∈ f22s/NSH \ f (SH+c0)/2, from where it follows (5.12). Consider (u, v) ∈ (Σ ∩M) such that f(u, v) < SH + c0 2 . (5.13) Then h(u, v) = (u, v) and from (5.13), we have that (u, v) 6∈ f22s/NSH \ f SH+c0 2 and by (5.12), we have (u, v) 6∈ f c+ε0 \ f c−ε0 . Thus, (u, v) ∈ f c−ε0 ∪ {(Σ ∩M) \ f c+ε0} 40 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 and by (b), we obtain η(1, (u, v)) = (u, v). Therefore, h∗(u, v) = η(1, h(u, v)) = η(1, (u, v)) = (u, v), which shows that h∗ ∈ H, and so, h∗(Θ) = η(1, h(Θ)) = η(1, Ã) ∈ Γ. Hence, c = inf A max (u,v)∈A f(u, v) ≤ max (u,v)∈A f(u, v). (5.14) On the other hand, by (c) and (5.9), we obtain η(1, Ã) ⊂ η(1, f c+ ε0 2 ) ⊂ f c− ε0 2 . That is, f(u, v) ≤ c− ε0 2 , ∀(u, v) ∈ η(1, Ã), which implies that max (u,v)∈η(1,Ã) f(u, v) ≤ c− ε0 2 , which is a contradiction. Therefore, we must have f(un, vn)→ c and f ′|M(un, vn)→ 0 and from Corollary 3.7, up to a subsequence, we have un → u0, vn → v0 in Ds,2(RN ), and satisfies f(u0, v0) = c and f ′|M(u0, v0) = 0. By Corollary 3.8, J has a critical point (w0, z0) ∈ Ds,2(RN )×Ds,2(RN ) such that J (w0, z0) = s N cN/2s, and by (5.7), we obtain s N S N 2s H < J (w0, z0) < 2s N S N 2s H . The positivity of (w0, z0) is a consequence of maximum principle that can be see in [25, Proposition 2.17]. � References [1] Aubin, T.; Problèmes isopérimétriques et espaces de Sobolev. J. Differen- tial Geometry 11(4) (1976), 573–598. [2] Alves, C. O.; Existence of positive solutions for a problem with lack of compactness involving the p-Laplacian, Nonlinear Anal., 51 (2002), 1187–1206. [3] Ambrosio, V.; Concentration phenomena for critical fractional Schrödinger systems, Com- mun. Pure Appl. Anal., 17 (2018), 2085–2123. [4] Applebaum, D.; Lévy processes-from probability to finance and quantum groups, Not. Am. Math. Soc., 51 (2004), 1336–1347. [5] Applebaum, D.; Lévy Processes and Stochastic Calculus, second edition, Cambridge Studies in Advanced Mathematics, vol.116, Cambridge University Press, Cambridge, 2009. [6] Benci, V.; Cerami, G.; Existence of positive solutions of the equation −∆u + a(x)u = u N+2 N−2 in RN , J. Funct. Anal., 88 (1990), 90-117. [7] Benci, V.; Grisanti, C. R.; Micheletti, A. M.; Existence of solutions for the nonlinear Schrödinger equation with V (∞) = 0, Progr. Nonlinear Differential Equations Appl., 66 (2005), 53-65. [8] Ben-Naoum, A. K.; Troestler, C.; Willem, M.; Extrema problems with critical Sobolev expo- nents on unbounded domains, Nonlinear Analysis, 26 (1996), 823–833. EJDE-2022/79 FRACTIONAL SYSTEMS SYSTEMS WITH CRITICAL GROWTH 41 [9] Berton, J.; Lévy Processes, Cambridge Tracts in Mathematics, vol.121, Cambridge University Press, Cambridge, 1996. [10] Brasco, L.; Squassina, M.; Yang, Y.; Global compactness result for nonlocal problems, Discrete & Continuous Dynamical Systems - S, 11 (3) (2018), 391-424. doi: 10.3934/ dcdss.2018022. [11] Chabrowski, J.; Concentration-compactness principle at infinity and semilinear elliptic equa- tions involving critical and subcritical Sobolev exponents, Calc. Var. Partial Differ. Eq., 3 (1995), 493-512. [12] Chen, W.; Li, C.; Ou, B.; Classifications of solutions for an integral equation, Commun. Pure Appl.Math., 59 (3) (2006), 330–343. [13] Correia, J. N.; Figueiredo, G. M.; Existence of positive solution of the equation (−∆)su + a(x)u = |u|2∗s−2u, Calc. Var. PDE, v. 58, p. 63, (2019). [14] De Morais, D. C.; Filho, M. A. S. Souto.; Systems of p-Laplacean equations involving ho- mogeneous nonlinearities with critical Sobolev exponent degrees, Comm. Partial Differential Equations 24 (1999), no. 7-8, 1537–1553. [15] Figueiredo, G. M.; Silva, L. S.; Existence of positive solutions of a critical system in RN , Palestine Journal of Mathematics, v. 10, p. 502-532, (2021). [16] Kavian, O.; Introduction è la théorie des points critiques, Springer-Verlag, 1991. [17] Majda, A.; Tabak, E.; A two-dimensional model for quasigeostrophic flow: comparisonwith the two-dimensional Euler flow, Nonlinear Phenomena in Ocean Dynamics (Los Almos, NM, 1995), Phys, D, 98 (2–4) (1996) 515522. [18] Mellet, A.; Mischler, S.; Mouhot, C.; Fractional diffusion limit for collisional kinetic equations, Arch. Rational Mech. Anal., 199 (2011), 493–525. [19] Mercuri, C.; Willem, M.; A global compactness result for the p-Laplacian involving critical nonlinearities, Discrete & Continuous Dynamical Systems - A, 28 (2010), 469-493. [20] Metzler, R.; Klafter, J.; The random walk’s guide to anomalous diffusion: a fractional dy- namics approach, Phys. Rep., 339 (2000), 1–77. [21] Miyagaki, O. H.; On a class of semilinear elliptic problems in RN with critical growth, Non- linear Analysis, 29 (1997), 773–781. [22] Molica Bisci, G.; Radulescu, V. D.; Servadei, R.; Variational Methods for Nonlocal Fractional Problems, Encyclopedia of Mathematics and Its Applications, vol. 162, Cambridge University Press, Cambridge, 2016. [23] Molle, R.; Passaseo, D.; Multispike solutions of nonlinear elliptic equations with critical Sobolev exponent, Comm. in PDE., 32 (2007), 797–818. [24] Nezza, E. Di.; Palatucci, G.; Valdinoci, E.; Hitchhiker’s guide to the fractional Sobolev spaces, Bull. Sci. math., 136 (2012), 521–573. [25] Silvestre, L.; Regularity of the obstacle problem for a fractional power of the Laplace operator, Commun. Pure Appl. Math., 60 (2007), 67–112. [26] Struwe, M.; A global compactness result for elliptic boundary value problems involving lim- iting nonliarities, Math. Z., 187, no. 4, (1984), 511-517. [27] Talenti, G.; Best constant in Sobolev inequality, Ann. Mat. Pula Appl., 110 (1976), 353-372. [28] Tankov, P.; Cont, R.; Financial Modelling With Jump Processes, Chapman & Hall/CRC Financial Mathematics Series, Boca Raton, FI, ISBN1584884134, (2004). [29] Valdinoci, E.; From the long jump randomwalk to the fractional Laplacian, Bol. Soc. Esp. Mat. Apl., Se MA 49 (2009), 3344. [30] Vlahos, L.; Isliker, H.; Kominis, Y.; Hizonidis, K.; Normal and anomalous diffusion: a tuto- rial. In order and chaos, 10th volume, in: T. Bountis (Ed.), Patras University Press, 2008. [31] Willem, M.; Minimax Theorems, Progress in Nonlinear Differential Equations and their Ap- plications, 24. Birkhäuser, Basel, 1996. [32] Zhang, X.; Zhang, B.; Xiang, M.; Ground states for fractional Schrödinger equations involving a critical nonlinearity, Adv. Nonlinear Anal., 5 (2016), 293–314. Jeziel N. Correia Departamento de Matemètica, Universidade Federal do Parè, Salinèpolis, CEP 68721- 000, Brazil Email address: Jeziel@ufpa.br 42 J. N. CORREIA, C. P. OLIVEIRA EJDE-2022/79 Claudionei P. Oliveira Faculdade de Matemètica, Universidade Federal do Sul e Sudeste do Parè, Marabè, CEP 68507-590, Brazil Email address: clauunifesspa@unifesspa.edu.br 1. Introduction 2. Limit problem 3. A compactness result 4. Technical lemmas 5. Proof of main theorem References