Electronic Journal of Differential Equations, Vol. 2020 (2020), No. 108, pp. 1–20. ISSN: 1072-6691. URL: http://ejde.math.txstate.edu or http://ejde.math.unt.edu POSITIVE VORTEX SOLUTIONS AND PHASE SEPARATION FOR COUPLED SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL JIN DENG, ALIANG XIA, JIANFU YANG Abstract. We consider the existence of rotating solitary waves (vortices) for a coupled Schrödinger equations by finding solutions to the singular system −∆u+ λ1u+ u |x|2 = µ1u 3 + βuv2, x ∈ R2, −∆v + λ2v + v |x|2 = µ2v 3 + βu2v, x ∈ R2, u, v ≥ 0, x ∈ R2, where λ1, λ2, µ1, µ2 are positive parameters, β 6= 0. We show that this system has a positive least energy solution for the cases when either β is negative or β is positive and small or large. Moreover, if λ1 = λ2, then the solution is unique. We also study the limiting behavior of the least energy solutions in the repulsive case for β → −∞, and phase separation. 1. Introduction In this article, we consider solitary wave solutions of the time-dependent coupled nonlinear Schrödinger equations: −i∂Φ1 ∂t = ∆Φ1 + µ1|Φ1|2Φ1 + β|Φ2|2Φ1, x ∈ RN , t > 0, −i∂Φ2 ∂t = ∆Φ2 + µ2|Φ2|2Φ1 + β|Φ1|2Φ2, x ∈ RN , t > 0, Φj = Φj(x, t) ∈ C, j = 1, 2, (1.1) where i is the imaginary unit, µ1, µ2 > 0 and β 6= 0 is a coupling constant. When N ≤ 3, system (1.1) appears in many physical problems. Especially in nonlinear optics, the solution Φj denotes the j-th component of the beam in Kerr-like photo- refractive media, the positive parameter µj is for self-focusing in the j-th component of the beam, see for instance [2]. System (1.1) also arises in the Hartree-Fock theory for a double condensate, that is, a binary mixture of Bose-Einstein condensates in two different hyperfine states |1〉 and |2〉, see [13]. Physically, Φj values are the corresponding condensate amplitudes, µj and β are the intraspecies and interspecies scattering lengths. The sign of β determines whether the interactions of states |1〉 and |2〉 are repulsive or attractive: the interaction is attractive if β > 0, and the 2010 Mathematics Subject Classification. 35J20, 35B08, 35B40. Key words and phrases. Schödinger equation; singular potential; Nehari manifold. c©2020 Texas State University. Submitted March 5, 2020. Published October 30, 2020. 1 2 J. DENG, A. XIA, J. YANG EJDE-2020/108 interaction is repulsive if β < 0, where the two states are in strong competition. If the condensates repel, the spatially separate. This phenomenon is called phase separation and has been described in [28]. By a solitary wave solution of system (1.1), we mean a solution of (1.1) with the form Φ1(x, t) = u(x)eiλ1t and Φ2(x, t) = v(x)eiλ2t. Then (u, v) satisfies −∆u+ λ1u = µ1u 3 + βuv2, x ∈ RN , −∆v + λ2v = µ2v 3 + βu2v, x ∈ RN , u, v ≥ 0, x ∈ RN . (1.2) When N ≤ 3, the existence of solutions has received great interest. Particularly, it is considered the existence of a ground state solution in [1, 6, 10, 17, 21, 26], the existence of semiclassical states or singular perturbed settings in [16, 18, 19, 22, 24, 25], and the existence of multiple solutions in [5, 12, 20, 23, 29]. When N = 4, this problem becomes critical case, one can find related results in [9, 11] and references therein. However, if ψ0(x) ∈ R, the angular momentum of ψ(t, x) = ψ0(x)eiω0t, ω0 > 0, is trivial, that is, M(ψ) = 0, where M(ψ) = Re ∫ RN ∂tψ(x×∇ψ)dx. (1.3) Therefore, a solution (u, v) with vortices should be complex valued. In this article, we are interested in finding a standing wave solution with non- trivial angular momentum in the dimension N = 2, that is, a solution with vortices. Making an ansatz of the form Φ1(x, t) = u(x)ei(k0θ(x)+λ1t) and Φ2(x, t) = v(x)ei(k0θ(x)+λ2t), where θ(x) ∈ R/2πZ, k0 6= 0, we see that system (1.1) is equivalent to the system −∆u+ λ1u+ k2 0u|∇θ|2 = µ1u 3 + βuv2, x ∈ RN , 2∇θ · ∇u− u∆θ = 0, x ∈ RN , −∆v + λ2v + k2 0v|∇θ|2 = µ2v 3 + βu2v, x ∈ RN , 2∇θ · ∇v − v∆θ = 0, x ∈ RN , u, v ≥ 0, x ∈ RN . (1.4) If we assume u(x) = u(|x|) and choose the angular coordinate in R2 as phase function, see [3, 4], that is, θ(x) :=  arctan x2 x1 , if x1 > 0, π + arctan x2 x1 , if x1 < 0, π/2, if x1 = 0 and x2 > 0, −π/2, if x1 = 0 and x2 < 0, (1.5) we obtain ∆θ = 0, ∇θ · ∇u = 0, |∇θ|2 = 1 |x|2 , EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 3 and the system reduces to −∆u+ λ1u+ k2 0 u |x|2 = µ1u 3 + βuv2, x ∈ R2, −∆v + λ2v + k2 0 v |x|2 = µ2v 3 + βu2v, x ∈ R2, u, v ≥ 0, x ∈ R2. (1.6) Noting that x×∇ψ = x1∂2ψ − x2∂2ψ if N = 2, we may verify that M ( u(x)ei(k0θ(x)+λ1t) ) = λ1k0 ∫ R2 u2dx, M ( v(x)ei(k0θ(x)+λ2t) ) = λ2k0 ∫ R2 v2dx. We point out that (1.2) is a special case of system (1.6), that is, k0 = 0. In this article, we consider the case k0 > 0. For simplicity, we always assume k0 = 1 in (1.6), and thus we study the following coupled singular Schrödinger system −∆u+ λ1u+ u |x|2 = µ1u 3 + βuv2, x ∈ R2, −∆v + λ2v + v |x|2 = µ2v 3 + βu2v, x ∈ R2, u, v ≥ 0, x ∈ R2. (1.7) It is well known that solutions of (1.7) are the critical points of the functional E : H → R, where E(u, v) = 1 2 ∫ R2 ( |∇u|2 + λ1u 2 + u2 |x|2 + |∇v|2 + λ2v 2 + v2 |x|2 ) dx − 1 4 ∫ R2 ( µ1u 4 + 2βu2v2 + µ2v 4 ) dx, (1.8) where H := Hλ1 × Hλ2 is given in section 2. We call a solution (u, v) nontrivial if both u 6≡ 0 and v 6≡ 0; and semi-trivial if (u, v) is a type of solution (u, 0) or (0, v). A solution (u, v) of (1.7) is a least energy solution if (u, v) is nontrivial and E(u, v) ≤ E(ϕ,ψ) for any other nontrivial solutions (ϕ,ψ) of (1.7). We define M = { (u, v) ∈ H \ {0, 0} :∫ R2 ( |∇u|2 + λ1u 2 + u2 |x|2 ) dx = ∫ R2 ( µ1u 4 + βu2v2 ) dx,∫ R2 ( |∇v|2 + λ2v 2 + v2 |x|2 ) dx = ∫ R2 ( µ2v 4 + βu2v2 ) dx } , which is the Nehari manifold for system (1.7), and contains all nontrivial solution of (1.7). We consider the minimization problem I = inf (u,v)∈M E(u, v) = inf (u,v)∈M 1 4 ∫ R2 ( |∇u|2 + λ1u 2 + u2 |x|2 ) + ( |∇v|2 + λ2v 2 + v2 |x|2 ) dx. (1.9) 4 J. DENG, A. XIA, J. YANG EJDE-2020/108 Let Q(x) = Q(|x|) be the unique positive ground solution of scalar equation −∆Q+Q+ Q |x|2 = Q3, x ∈ R2. (1.10) The function Q is well studied, see [15]. Our first result deals with the case λ1 = λ2. In this case, the ground state solution of (1.7) can be constructed from the solution of the scalar equation (1.10), and a more explicit expression of positive ground state solutions can be obtained as follows. Theorem 1.1. Assume that λ1 = λ2 > 0. (1) If 0 < β < min{µ1, µ2} or β > max{µ1, µ2}, then I is attained at ( √ kwλ1 , √ lwλ1 ), where k, l > 0 satisfy µ1k + βl = 1, βk + µ2l = 1, (1.11) and wλ1 (x) = √ λ1Q (√ λ1x ) . (1.12) That is, ( √ kwλ1 , √ lwλ1 ) is a positive least energy solution of (1.7). (2) If β ∈ [min{µ1, µ2},max{µ1, µ2}] and µ1 6= µ2, then (1.7) does not have a nontrivial nonnegative solution. Taking advantage of λ1 = λ2, we can prove the uniqueness of positive ground state solution of (1.7). Precisely, we have the following result. Theorem 1.2. Assume that λ1 = λ2 > 0, and let 0 < β < min{µ1, µ2} or β > max{µ1, µ2}. Let (u, v) be any positive least energy solution of (1.7), then (u, v) = ( √ kwλ1 , √ lwλ1), where (k, l) satisfies (1.11) and wλ1 is given in (1.12). Next, we consider the general case λ2 ≥ λ1 > 0 and β ∈ R which covers all negative value, that is, the repulsive case. For the existence, we have that following result. Theorem 1.3. Assume that λ2 ≥ λ1 > 0. Let χ0 be the smaller root of the equation λ−1/2 ( 2− λ−1/2 ) x2 − (ν1 + ν2)x+ ν1ν2 = 0, (1.13) where ν1 = µ1λ 1/2, ν2 = µ2λ −1/2, λ = λ2 λ1 . (1.14) If −∞ < β < χ0, then (1.7) possesses a positive ground state solution. In Theorem 1.3, the existence of positive radial ground state solution is shown for β < χ0. Indeed we can prove the existence of radial ground state solution for β ∈ (−∞, χ1) where χ1 > χ0, see Proposition 2.5 for the definition of χ1. In next section, we also show that χ0 < χ1 < min{ν1, ν2}. Our last result concerns with the limiting behavior of positive ground state so- lutions of (1.7) as β → −∞. Denote {w > 0} := {x ∈ R2 : w(x) > 0}. Then we have the following result. Theorem 1.4. Assume that λ2 ≥ λ1 > 0. Let βn < 0, n ∈ N satisfy βn → −∞ as n → ∞, and let (un, vn) be the positive least energy solutions of (1.7) with β = βn, finding by Theorem 1.3. Then, after passing to a subsequence, we have that EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 5 un → u∞ and vn → v∞ strongly in H, the functions u∞ and v∞ are continuous, u∞ ≥ 0, v∞ ≥ 0, u∞v∞ ≡ 0, u∞ solves the problem −∆u+ λ1u+ u2 |x|2 = µ1u 3 in {u∞ > 0}, and v∞ solves the problem −∆v + λ2v + v2 |x|2 = µ2v 3 in {v∞ > 0}. Furthermore, both {u∞ > 0} and {v∞ > 0} are connected domains and {u∞ > 0} = R2 \ {v∞ > 0}. As shown in Theorem 1.4, the components of the limiting profile tend to separate in different regions of R2, and thus the phenomena of phase separation happens. The paper is organized as follows. After presenting preliminary results in Section 2, we prove the existence and non-existence results in Section 3. Section 4 is devoted to prove the uniqueness result. Finally, we prove the phenomena of phase separation in Section 5. 2. Preliminary results In this section, we show some preliminary results for future reference. Let H := { u ∈ H1 r (R2) : ∫ R2 u2 |x|2 dx <∞ } , where H1 r (R2) = {u ∈ H1(R2) : u(x) = u(|x|)}. We denote by Hλ the Hilbert spaces H endowed with the norm defined by ‖u‖2λ := ∫ R2 ( |∇u|2 + λu2 + u2 |x|2 ) dx for all u ∈ Hλ, which is induced by the inner product 〈u, v〉λ := ∫ R2 ( ∇u∇v + λuv + uv |x|2 ) dx for all u, v ∈ Hλ. Apparently, Hλ ↪→ H1 r (R2), and by well known compact embedding of H1 r (R2), one has Hλ ↪→↪→ L4(R2) is compact. The following proposition shows that minimizers of I defined by (1.9) are solutions of (1.7). Proposition 2.1. If I is attained by a couple (u, v) ∈M, then (u, v) is a solution of (1.7) provided −∞ < β < √ µ1µ2. Proof. We need to show that any minimizer (u, v) of I satisfies dE(u, v) = E′(u, v) = 0. We write M = M1 ∩M2, where Mi is the set of pairs (u, v) ∈ H such that u 6≡ 0, v 6≡ 0, fi(u, v) = 0, for i = 1, 2, where f1(u, v) := ∫ R2 ( |∇u|2 + λ1u 2 + u2 |x|2 ) dx− ∫ R2 ( µ1u 4 + βu2v2 ) dx, f2(u, v) := ∫ R2 ( |∇v|2 + λ2v 2 + v2 |x|2 ) dx− ∫ R2 ( µ2v 4 + βu2v2 ) dx. 6 J. DENG, A. XIA, J. YANG EJDE-2020/108 For each (ϕ,ψ) ∈ H, 〈E′(u, v), (ϕ,ψ)〉 = ∫ R2 ( ∇u∇ϕ+ λ1uϕ+ uϕ |x|2 ) dx+ ∫ R2 ( ∇v∇ψ + λ2vψ + vψ |x|2 ) dx − ∫ R2 ( µ1u 3ϕ+ βuv(uψ + vϕ) + µ2v 3ψ ) dx, 〈f ′1(u, v), 1 2 (ϕ,ψ)〉 = ∫ R2 ( ∇u∇ϕ+ λ1uϕ+ uϕ |x|2 ) dx− ∫ R2 ( 2µ1u 3ϕ+ βuv(uψ + vϕ) ) dx, 〈f ′2(u, v), 1 2 (ϕ,ψ)〉 = ∫ R2 ( ∇v∇ψ + λ2vψ + vψ |x|2 ) dx− ∫ R2 ( 2µ2v 3ψ + βuv(uψ + vϕ) ) dx. We can verify that f ′i(u, v) 6= 0 for (u, v) ∈M, since u 6≡ 0 and v 6≡ 0 in Mi. Let (u, v) ∈ M be a minimizer for E restricted on M, there are two Lagrange multipliers L1, L2 ∈ R such that E′(u, v) + L1f ′ 1(u, v) + L2f ′ 2(u, v) = 0. Taking into account f1(u, v) = 0, from 〈E′(u, v) + L1f ′ 1(u, v) + L2f ′ 2(u, v), (u, 0)〉 = 0, (2.1) we deduce that L1 ∫ R2 µ1u 4dx+ L2 ∫ R2 βu2v2dx = 0. (2.2) Similarly, from 〈E′(u, v) + L1f ′ 1(u, v) + L2f ′ 2(u, v), (u, 0)〉 = 0, we obtain L1 ∫ R2 βu2v2dx+ L2 ∫ R2 µ2v 4dx = 0. (2.3) Using that f1(u, v) = f2(u, v) = 0, if β < 0, A = det ( ∫ R2 µ1u 4dx ∫ R2 βu 2v2dx∫ R2 βu 2v2dx ∫ R2 µ2v 4dx ) (2.4) is diagonally dominant, see [18, Lemma 2.1]. By the Hölder inequality, A is pos- itively definite if β2 < µ1µ2. Hence, the only solution of system (2.2)-(2.3) is L1 = L2 = 0, which implies E′(u, v) = 0 by (2.1). � The existence and properties of semi-trivial solutions of (1.7) are well-studied. Let us recall some facts. Consider the minimization problems Sλ,µ = inf u∈H\{0} ‖u‖2λ( ∫ R2 µu4dx )1/2 (2.5) and Tλ,µ = inf N0 {1 2 ‖u‖2λ − 1 4 ∫ R2 µu4dx } , where N0 = {u ∈ H : u 6≡ 0, ‖u‖2λ = ∫ R2 µu 4dx}. EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 7 Proposition 2.2. The function wλ,µ = µ−1/2 √ λ Q (√ λx ) is a minimizer for Tλ,µ, and it is the unique positive solution of the equation −∆w + λw + w |x|2 = µw3 in R2. In addition, Tλ,µ = 1 4 S2 λ,µ, Sλ,µ = µ−1/2λ1/2S1,1. The assertion of the above proposition follows by scaling arguments for Q since Q(x) is the unique positive ground state solution of equation (1.10). In the following, we set wλ(x) = wλ,1(x) = √ λQ( √ λx), Tλ = Tλ,1 and Sλ = Sλ,1. We introduce a function h : R+ → R+ defined by h(λ) := ∫ R2 Q 2(x)w2 λ(x)dx∫ R2 Q4(x)dx . (2.6) Proposition 2.3. For every λ ≥ 1, we have 1 ≤ h(λ) ≤ λ1/2. (2.7) Proof. Since Q(x) is radial and strictly decreasing in |x|, we have Q(x) ≥ Q( √ λx) for λ ≥ 1 and x ∈ R2. Using this fact and a scaling argument, (2.7) readily follows. � Next, we find a bound for I. Lemma 2.4. Let h(λ) be defined in (2.6) with λ = √ λ2/λ1 for λ2 ≥ λ1 > 0. If k, l > 0 satisfy the linear system µ1k + βh(λ)l = 1, βh(λ)k + µ2λl = λ. (2.8) Then (a) ( √ kwλ1 , √ lwλ2 ) ∈M; (b) there exists a ρ0 > 0 such that 0 < ρ0 ≤ I ≤ 1 4 (λ1k + λ2l)S 2 1 . (2.9) Proof. To prove that ( √ kwλ1 , √ lwλ2) ∈ M, it is suffices to show that (u, v) = ( √ kwλ1 , √ lwλ2 ) satisfies ‖u‖2λ1 = ∫ R2 ( µ1u 4 + βu2v2 ) dx and ‖v‖2λ2 = ∫ R2 ( µ2v 4 + βu2v2 ) dx. (2.10) Apparently, ‖ √ kwλ1 ‖2λ1 = kλ1 (∫ R2 |∇Q|2 +Q2 + Q2 |x|2 dx ) = kλ1 ∫ R2 Q4dx. Substitution x by x√ λ1 , one can see that µ1 ∫ R2 ( √ kwλ1 )4dx+ β ∫ R2 ( √ kwλ1 )2( √ lwλ2 )2dx 8 J. DENG, A. XIA, J. YANG EJDE-2020/108 = µ1k 2λ2 1 ∫ R2 (Q( √ λ1x))4dx+ βklλ1λ2 ∫ R2 (Q( √ λ1x))2(Q( √ λ2x))2dx = µ1k 2λ1 ∫ R2 Q4(x)dx+ βklλ2 ∫ R2 Q2(x) ( Q (√λ2 λ1 x ))2 dx = kλ1 (µ1k + βh(λ)l) ∫ R2 Q4(x)dx, So the first equality in (2.10) holds if µ1k + βh(λ)l = 1. Similarly, we can prove that the second equality in (2.10) also satisfied if βh(λ)k + µ2λl = λ. Since ( √ kwλ1 , √ lwλ2 ) ∈M, by Proposition 2.2, I ≤ E( √ kwλ1 , √ lwλ2 ) = k 4 ‖wλ1 ‖2λ1 + l 4 ‖wλ2 ‖2λ2 = kTλ1 + lTλ2 = k 4 S2 λ1 + l 4 S2 λ2 = 1 4 (kλ1 + lλ2)S2 1 . (2.11) On the other hand, if (u, v) ∈M, we have ‖u‖2λ1 + ‖v‖2λ2 = µ1‖u‖4L4 + µ2‖v‖4L4 + 2β‖uv‖2L2 . By Sobolev inequality H ↪→ H1 r (R2) ↪→ L4(R2) and Hölder inequality, we have ‖u‖2λ1 + ‖v‖2λ2 ≤ c0(‖u‖4λ1 + ‖v‖4λ2 + 2‖u‖2λ1 ‖v‖2λ2 ), where c0 = c0(µ1, µ2, β) is a positive constant. Therefore, for (u, v) ∈M, E(u, v) = 1 4 (‖u‖2λ1 + ‖v‖2λ2 ) ≥ 1 4c0 , (2.12) which implies I ≥ ρ0 > 0 for some ρ0. Item (b) follows from (2.11) and (2.12). � Now, we solve (2.8). It follows from (2.8) that (µ1µ2λ− β2h2(λ))k = µ2λ− λh(λ)β. (2.13) Hence, (2.13) is solvable for k > 0 and l > 0 if either µ1µ2λ− β2h2(λ) > 0 and βh(λ) < min{µ2, µ1λ}, (2.14) or µ1µ2λ− β2h2(λ) < 0 and βh(λ) > max{µ2, µ1λ}. (2.15) By Proposition 2.3, we know that (2.14) is satisfied if −√µ1µ2 < β < λ−1/2 min{µ2, µ1λ} = min{ν1, ν2}, (2.16) where ν1 and ν2 are defined in (1.14). Similarly, (2.15) is satisfied if β > max{µ2, µ1λ} = λ1/2 max{ν1, ν2}. (2.17) Hence, k = λ(µ2 − βh(λ)) µ1µ2λ− β2h2(λ) and l = µ1λ− βh(λ) µ1µ2λ− β2h2(λ) (2.18) if either (2.16) or (2.17) holds. Define a(λ) = g(λ)(2− g(λ)) where g(λ) = λ−1/2h(λ). (2.19) EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 9 By Proposition 2.3, λ−1/2 ≤ g(λ) ≤ 1 and λ−1/2(2− λ−1/2) ≤ a(λ) ≤ 1 for λ ≥ 1. (2.20) We consider now the minimization problem I. Proposition 2.5. Suppose that λ = λ2/λ1 ≥ 1. Let χ1 be the smaller root of the quadratic equation a(λ)x2 − (ν1 + ν2)x+ ν1ν2 = 0. Assume that −∞ < β < χ1. (2.21) Let {(un, vn)} ⊂ M be a sequence such that E(un, vn)→ I as n→∞. Then there exists a constant c0 > 0 such that ‖un‖L4 ≥ c0 and ‖vn‖L4 ≥ c0 for all n ∈ N. Remark 2.6. Recall the constant χ0 defined in Theorem 1.3 and (1.13), we see from (2.20) that χ0 ≤ χ1 ≤ min{ν1, ν2} . Proof of Proposition 2.5. Let {(un, vn)} ⊂ M be a minimizing sequence for I, that is, E(un, vn) = 1 4 ( ‖un‖2λ1 + ‖vn‖2λ2 ) = 1 4 ∫ R2 µ1u 4 n + 2βu2 nv 2 n + µ2v 4 ndx→ I, as n → ∞. It follows that {(un, vn)} is bounded in H. We recall that un 6≡ 0 and vn 6≡ 0, then we define that z1,n = (∫ R2 u4 n dx )1/2 , z2,n = (∫ R2 v4 n dx )1/2 . By the Sobolev and Hölder inequalities, the definition of Sλ and Proposition 2.2, we have λ 1/2 1 S1z1,n ≤ ‖un‖2λ1 = ∫ R2 µ1u 4 n + βu2 nv 2 ndx ≤ µ1z 2 1,n + β+z1,nz2,n , (2.22) λ 1/2 2 S1z2,n ≤ ‖vn‖2λ2 = ∫ R2 µ2v 4 n + βu2 nv 2 ndx ≤ µ2z 2 2,n + β+z1,nz2,n , (2.23) where β+ = max{β, 0}. If β ≤ 0, the conclusion follows from (2.22)-(2.23). Therefore, we assume that β > 0. By (2.22)-(2.23), we have S1 ( λ 1/2 1 z1,n + λ 1/2 2 z2,n ) ≤ ∫ R2 µ1u 4 n + 2βu2 nv 2 n + µ2v 4 ndx = 4I + on(1), (2.24) where on(1)→ 0 as n→∞. Let z̃i,n = λ −1/2 1 S−1 1 zi,n for i = 1, 2. By (2.9), (2.22) and (2.23), we obtain the following inequalities z̃1,n + λ1/2z̃2,n ≤ k + λl + on(1), µ1z̃1,n + βz̃2,n ≥ 1, βz̃1,n + µ2z̃2,n ≥ λ1/2, (2.25) where λ = √ λ2/λ1, k, l are given in Lemma 2.4. To prove that the two sequences {z̃1,n}, {z̃2,n} stay uniformly away zero, we need to show that each two of the following lines l1 = { (z1, z2) ∈ R2 : z1 + λ1/2z2 = k + λl } , 10 J. DENG, A. XIA, J. YANG EJDE-2020/108 l2 = { (z1, z2) ∈ R2 : µ1z1 + βz2 = 1 } , l3 = { (z1, z2) ∈ R2 : βz1 + µ2z2 = λ1/2 } meet, and their crossing points have strictly positive coordinates. This can be achieved if the following set of conditions are met: βλ1/2 < µ2, β < µ1λ 1/2, (2.26) µ1(k + λl) > 1, (2.27) µ2(k + λl) > λ, (2.28) β(k + λl) < λ1/2. (2.29) By Remark 2.6, β < χ1 ≤ min{ν1, ν2}, then (2.26) holds. Next, we deduce from (2.14) and (2.18) that µ1(k + λl)− 1 = (µ1λ1 − βh(λ))2 µ1µ2λ− β2h2(λ) > 0, (2.30) this implies (2.27) holds. Similarly, (2.28) holds. Finally, (2.29) is equivalent to[2h(λ)λ1/2 − h2(λ) λ ] β2 − ( λ−1/2µ2 + λ1/2µ1 ) β + µ1µ2 > 0, that is, a(λ)β2 − (ν1 + ν2)β + ν1ν2 > 0. Therefore, by the definition of χ1, one sees that (2.26)-(2.29) are satisfied if 0 < β < χ1. This completes the proof. � 3. Proof of Theorems 1.1 and 1.3 In this section we assume λ1 = λ2 > 0. Proof of Theorem 1.1. If λ = λ2/λ1 = 1, then (2.25) becomes z̃1,n + z̃2,n ≤ k + l + on(1), µ1z̃1,n + βz̃2,n ≥ 1, βz̃1,n + µ2z̃2,n ≥ 1. (3.1) If either 0 < β < min{µ1, µ2}, or β > max{µ1, µ2} holds, equation (1.11) has a solution (k, l) satisfying k > 0 and l > 0. We set w1,n = z̃1,n−k and w2,n = z̃2,n− l. By (1.11) and (3.1), we deduce that w1,n + w2,n ≤ on(1), µ1w1,n + βw2,n ≥ 0, βw1,n + µ2w2,n ≥ 0. (3.2) Therefore, w1,n → 0 and w2,n → 0 as n → +∞, that is, z̃1,n → k and z̃2,n → l as n→ +∞. Noting z̃i,n = λ −1/2 1 S−1 1 zi,n for i = 1, 2 and passing to the limit in (2.24) with λ1 = λ2, we obtain I ≥ 1 4 λ1(k + l)S2 1 . On the other hand, by Lemma 2.4, we have I ≤ E (√ kwλ1 , √ lwλ1 ) = 1 4 λ1 (k + l)S2 1 . (3.3) EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 11 This implies I = E (√ kwλ1 , √ lwλ1 ) = 1 4 λ1 (k + l)S2 1 , (3.4) which proves part (1) in Theorem 1.1. For part (2), multiplying the u-equation in (1.7) by v, and the v-equation by u, subtracting and integrating over R2, we obtain∫ R2 uv[(µ1 − β)u2 + (β − µ2)v2]dx = 0. Thus, (1.7) does not have nontrivial nonnegative solutions if β ∈ [min{µ1, µ2},max{µ1, µ2}] and µ1 6= µ2. That is, (2) in Theorem 1.1 holds. This completes the proof. � Proof of Theorem 1.3. Let {(un, vn)} ⊂ M be a minimizing sequence for I. Since {(un, vn)} is bounded in H, we may assume that (un, vn) ⇀ (u, v) in H, (un, vn)→ (u, v) in L4(R2)× L4(R2), with (u, v) ∈ H. The weak continuity of norms yields ‖u‖2λ1 + ‖v‖2λ2 ≤ lim inf n→+∞ ( ‖un‖2λ1 + ‖vn‖2λ2 ) = 4I. (3.5) By Proposition 2.5 and Remark 2.6, both {‖un‖L4} and {‖vn‖L4} are bounded away from zero, so the limit (u, v) is nontrivial. Moreover,∫ R2 µ1u 4 + 2βu2v2 + µ2v 4dx = lim n→+∞ ∫ R2 µ1u 4 n + 2βu2 nv 2 n + µ2v 4 ndx (3.6) = 4 lim n→+∞ E(un, vn) = 4I. (3.7) Since (u, v) is nontrivial and the matrix A in (2.4) is positively definite, there exists a unique couple (t1, t2) satisfying(∫ R2 µ1u 4dx ) t1 + (∫ R2 βu2v2dx ) t2 = ‖u‖2λ1 ,(∫ R2 βu2v2dx ) t1 + (∫ R2 µ2v 4dx ) t2 = ‖v‖2λ2 . (3.8) We claim that t1 > 0 and t2 > 0. If β ≤ 0, the claim is obvious. Now we deal with the case β > 0. Let us prove that t1 > 0. The case t2 > 0 can be proved in the same way. We deduce from (3.8) that{(∫ R2 µ1u 4dx )(∫ R2 µ2v 4dx ) − (∫ R2 βu2v2dx )2} t1 = ‖u‖2λ1 (∫ R2 µ2v 4dx ) − ‖v‖2λ2 (∫ R2 βu2v2dx ) . (3.9) Since the matrix A in (2.4) is diagonally dominate, in order to prove t1 > 0, it is sufficient to show that ‖u‖2λ1 (∫ R2 µ2v 4dx ) > ‖v‖2λ2 (∫ R2 βu2v2dx ) . (3.10) By the Hölder inequality and the definition of Sλ1 in (2.5), we have∫ R2 βu2v2dx ≤ β (∫ R2 u4dx )1/2(∫ R2 v4dx )1/2 (3.11) 12 J. DENG, A. XIA, J. YANG EJDE-2020/108 and Sλ1 ≤ ‖u‖2λ1( ∫ R2 u4dx )1/2 . Therefore, (3.10) follows once we prove µ2 (∫ R2 v4dx )1/2 > β Sλ1 ‖v‖2λ2 . (3.12) Now we show (3.12). Since (un, vn) ⇀ (u, v) in H and (un, vn) → (u, v) in L4(R2)× L4(R2), we have ‖u‖2λ1 ≤ lim inf n→+∞ ‖un‖2λ1 = lim inf n→+∞ ∫ R2 µ1u 4 n + βu2 nv 2 ndx = ∫ R2 µ1u 4 + βu2v2dx. (3.13) Similarly, ‖v‖2λ2 ≤ ∫ R2 µ2v 4 + βu2v2dx. (3.14) We deduce from (3.11) that ‖v‖2λ2 ≤ ∫ R2 µ2v 4dx+ β (∫ R2 u4dx )1/2(∫ R2 v4dx )1/2 . (3.15) Hence, we see that (3.12) holds if 1 > β Sλ1 ((∫ R2 v4dx )1/2 + β µ2 (∫ R2 u4dx )1/2) . (3.16) By Proposition 2.2, Sλ1 = λ 1/2 1 S1, equation (3.16) can be written as lim n→+∞ β ( β µ2 z̃1,n + z̃2,n ) < 1. (3.17) We claim that (3.17) is valid. Indeed, by the first inequality in (2.25) and (2.26), as well as (2.29) we see that lim n→+∞ β ( β µ2 z̃1,n + z̃2,n ) ≤ lim n→+∞ β λ1/2 ( z̃1,n + λ1/2z̃2,n ) ≤ β λ1/2 (k + λl) < 1. Similarly, by (3.13) and the second inequality in (2.26) we can prove that t2 > 0. Since t1 > 0 and t2 > 0, we know from (3.8) that ( √ t1u, √ t2v) ∈M. So I ≤ E( √ t1u, √ t2v) = 1 4 ( t1‖u‖2λ1 + t2‖v‖2λ2 ) . (3.18) Equations (3.5) and (3.18) yield ‖u‖2λ1 + ‖v‖2λ1 ≤ t1‖u‖2λ1 + t2‖v‖2λ2 . (3.19) Substituting (3.19) into (3.8), we obtain t1 ( ‖u‖2λ1 − ∫ R2 (µ1u 4 + βu2v2)dx ) + t2 ( ‖v‖2λ2 − ∫ R2 (µ2v 4 + βu2v2)dx ) ≥ 0. By (3.13)-(3.15) and t1, t2 > 0, we have ‖u‖2λ1 − ∫ R2 (µ1u 4 + βu2v2)dx = 0, ‖v‖2λ2 − ∫ R2 (µ2v 4 + βu2v2)dx = 0, EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 13 which means (u, v) ∈ M. Thus, by (3.5)-(3.7), we know (u, v) is a minimizer of I. Furthermore, by Remark 2.6, β < χ0 ≤ min{ν1, ν2} < √ µ1µ2, Proposition 2.1 then implies that (u, v) is a nontrivial solution of (1.7). � 4. Proof of Theorem 1.2 In this section, we prove the uniqueness of solution for problem (1.7) inspired by [9]. Proof of Theorem 1.2. There are two cases to be considered, the first one is µ1 > 0, µ2 > 0 and 0 < β < min{µ1, µ2}, another one is µ1 > 0, µ2 > 0 and β > max{µ1, µ2}. In the first case: µ1 > 0, µ2 > 0 and 0 < β < min{µ1, µ2}, suppose (u0, v0) is a positive least energy solution of (1.7). By Theorem 1.1, ( √ kwλ1 , √ lwλ1) is a least energy solution, we claim that∫ R2 u4 0 dx = k2 ∫ R2 w4 λ1 dx, (4.1)∫ R2 v4 0 dx = l2 ∫ R2 w4 λ1 dx, (4.2)∫ R2 βu2 0v 2 0 dx = kl ∫ R2 w4 λ1 dx. (4.3) To prove the claim, we perturb the parameter µ. In fact, there exists a δ > 0, such that 0 < β < min{µ, µ2} for any µ ∈ (µ1 − δ, µ1 + δ). We can show as the proof of Theorem 1.1 that I is attained if we replace µ1 by µ. Since E,M and I are all depend on µ, we denote them by Eµ,Mµ and I(µ). Hence, we infer from (1.11) and (3.4) that I(µ) = µ+ µ2 − 2β 4(µµ2 − β2) λ1S 2 1 , and so I ′(µ1) := d dµI(µ)|µ=µ1 exists. Define f(t, s, µ) := tµ ∫ R2 u4 0dx+ s ∫ R2 βu2 0v 2 0dx− ∫ R2 ( |∇u0|2 + λ1u 2 0 + u2 0 |x|2 ) dx, g(t, s, µ) := s ∫ R2 µ2v 4 0dx+ t ∫ R2 βu2 0v 2 0dx− ∫ R2 ( |∇v0|2 + λ2v 2 0 + v2 0 |x|2 ) dx. Then f(1, 1, µ1) = g(1, 1, µ1) = 0 and ∂f ∂t (1, 1, µ1) = µ1 ∫ R2 u4 0dx, ∂f ∂s (1, 1, µ1) = β ∫ R2 u2 0v 2 0dx, ∂g ∂t (1, 1, µ1) = β ∫ R2 u2 0v 2 0dx, ∂g ∂s (1, 1, µ1) = µ2 ∫ R2 v4 0dx. Set B = ( ∂f ∂t (1, 1, µ1) ∂f ∂s (1, 1, µ1) ∂g ∂t (1, 1, µ1) ∂g ∂s (1, 1, µ1) ) . Then det(B) > 0. By the implicit function theorem, we know functions t(µ) and s(µ) are well defined and belongs to the class C1 on (µ1 − δ1, µ1 + δ1) for some 14 J. DENG, A. XIA, J. YANG EJDE-2020/108 δ1 ≤ δ. Moreover, t(µ1) = s(µ1) = 1, so we can assume that t(µ) > 0 and s(µ) > 0 for all µ ∈ (µ1 − δ1, µ1 + δ1) by choosing a small δ1 > 0. We also know f(t(µ), s(µ), µ) ≡ g(t(µ), s(µ), µ) ≡ 0. (4.4) It can be verified that t′(µ1) = − 1 B ∫ R2 u4 0dx ∫ R2 µ2v 4 0dx, s′(µ1) = 1 B ∫ R2 u4 0dx ∫ R2 βu2 0v 2 0dx. By Taylor expansion, we see that t(µ) = 1 + t′(µ1)(µ− µ1) +O((µ− µ1)2), s(µ) = 1 + s′(µ1)(µ− µ1) +O((µ− µ1)2). By (4.4), ( √ t(µ)u0, √ s(µ)v0) ∈Mµ. We find that I(µ) ≤ Eµ (√ t(µ)u0, √ s(µ)v0 ) = t(µ) 4 ∫ R2 ( |∇u0|2 + λ1u 2 0 + u2 0 |x|2 ) dx+ s(µ) 4 ∫ R2 ( |∇v0|2 + λ2v 2 0 + v2 0 |x|2 ) dx = I(µ1) + 1 4 Θ · (µ− µ1) +O((µ− µ1)2), where Θ := t′(µ1) ∫ R2 ( |∇u0|2 + λ1u 2 0 + u2 0 |x|2 ) dx + s′(µ1) ∫ R2 ( |∇v0|2 + λ2v 2 0 + v2 0 |x|2 ) dx = − 1 B ∫ R2 u4 0dx ∫ R2 µ2v 4 0dx ∫ R2 (µ1u 4 0 + βu2 0v 2 0)dx + 1 B ∫ R2 u4 0dx ∫ R2 βu2 0v 2 0dx ∫ R2 (µ2v 4 0 + βu2 0v 2 0)dx = − ∫ R2 u4 0dx. It follows that I(µ)− I(µ1) µ− µ1 ≥ Θ 4 +O(µ− µ1), as µ↗ µ1. As a result, B′(µ1) ≥ Θ 4 . Similarly, I(µ)− I(µ1) µ− µ1 ≤ Θ 4 +O(µ− µ1), as µ↘ µ1, that is, B′(µ1) ≤ Θ 4 . Therefore, I ′(µ1) = Θ 4 = −1 4 ∫ R2 u4 0dx. (4.5) On the other hand, by Theorem 1.1, ( √ kwλ1 , √ lwλ1) is also a positive least energy solution of (1.7). Hence, I ′(µ1) = −k 2 4 ∫ R2 w4 λ1 dx. (4.6) Consequently, (4.1) is true. EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 15 Similarly, using I ′(µ2) and I ′(β) respectively, we can prove that∫ R2 v4 0dx = l2 ∫ R2 w4 λ1 dx, ∫ R2 βu2 0v 2 0dx = kl ∫ R2 w4 λ1 dx. Therefore, ∫ R2 βu2 0v 2 0dx = l k ∫ R2 u4 0dx = k l ∫ R2 v4 0dx. Let (ũ, ṽ) = ( 1√ k u0, 1√ l v0). Since (u0, v0) ∈M, by (1.11), we can verify that∫ R2 |∇ũ|+ λ1ũ 2 + ũ2 |x|2 dx = ∫ R2 ũ4dx,∫ R2 |∇ṽ|+ λ2ṽ 2 + ṽ2 |x|2 dx = ∫ R2 ṽ4dx. (4.7) Noting ũ, ṽ ∈ N0, by Proposition 2.2, we have∫ R2 |∇ũ|+ λ1ũ 2 + ũ2 |x|2 dx ≥ λ1S 2 1 , ∫ R2 |∇ṽ|+ λ2ṽ 2 + ṽ2 |x|2 dx ≥ λ1S 2 1 . Therefore, I = 1 4 λ1(k + l)S2 1 = 1 4 ∫ R2 ( |∇u0|+ λ1u 2 0 + u2 0 |x|2 + |∇v0|+ λ2v 2 0 + v2 0 |x|2 ) dx = k 4 ∫ R2 ( |∇ũ|+ λ1ũ 2 + ũ2 |x|2 ) dx+ l 4 ∫ R2 ( |∇ṽ|+ λ2ṽ 2 + ṽ2 |x|2 ) dx ≥ 1 4 λ1(k + l)S2 1 . This implies ∫ R2 |∇ũ|+ λ1ũ 2 + ũ2 |x|2 dx = λ1S 2 1 = Sλ1 ,∫ R2 |∇ṽ|+ λ2ṽ 2 + ṽ2 |x|2 dx = λ1S 2 1 = Sλ2 . By (4.7), we know ũ and ṽ are positive ground state solutions of −∆w + λ1w + w |x|2 = w3 in R2. The uniqueness of positive ground solution of (1.10) implies ũ(x) = ṽ(x) = √ λ1Q (√ λ1x ) = wλ1(x), namely, (u0, v0) = ( √ kũ, √ lṽ) = ( √ kwλ1 √ lwλ1 ). Finally, the case µ1 > 0, µ2 > 0 and β > max{µ1, µ2} can be treated in the same way since det(B) < 0, the implicit function theorem can also be used. � 16 J. DENG, A. XIA, J. YANG EJDE-2020/108 5. Proof of Theorem 1.4 This section is devoted to prove Theorem 1.4. To highlight the dependence on β, we write Eβ ,Mβ instead of E,M. Let Iβ := inf (u,v)∈Mβ Eβ(u, v). The energy functional Φ of the problem −∆w + ( λ1 + 1 |x|2 ) w+ + ( λ2 + 1 |x|2 ) w− = µ1(w+)3 + µ2(w−)3 in R2, (5.1) where w+ := max{w, 0} and w− := min{w, 0}, is given by Φ(w) := 1 2 ∫ R2 [ |∇w|2 + ( λ1 + 1 |x|2 ) (w+)2 + ( λ2 + 1 |x|2 ) (w−)2 ] dx − 1 4 ∫ R2 [ µ1(w+)4 + µ2(w−)4 ] dx, and we define the corresponding Nehari manifold N by N := {w ∈ H : w 6= 0, Φ′(w)w = 0} = { w ∈ H : w 6= 0,∫ R2 [ |∇w|2 + ( λ1 + 1 |x|2 ) (w+)2 + ( λ2 + 1 |x|2 ) (w−)2 ] dx = ∫ R2 [µ1(w+)4 + µ2(w−)4]dx } . Sign-changing solutions of (5.1) belong to the set E := {w ∈ H : w+ ∈ N , w− ∈ N}. Observe that, if u, v ∈ H\{0}, u ≥ 0, v ≥ 0, there exist unique numbers s, t ∈ (0,∞) such that su ∈ N and −tv ∈ N , that is, s2 = ∫ R2 |∇u|2 + λ1u 2 + u2 |x|2 dx∫ R2 µ1u4dx and t2 = ∫ R2 |∇v|2 + λ2v 2 + v2 |x|2 dx∫ R2 µ2v4dx . (5.2) If, moreover, supp(u) ∩ supp(v) = ∅, then su− tv ∈ E . Hence, E 6= ∅. We define I∞ := inf w∈E Φ(w). Then I∞ is finite. Proposition 5.1. For βn → −∞, let (un, vn) ∈ Mβn satisfy un ≥ 0, vn ≥ 0 and Eβn(un, vn) = Iβn . Then, after passing to a subsequence, we have un → u∞ and vn → v∞ strongly in H, and (u∞, v∞) satisfies (a) u∞, v∞ ∈ N , u∞ ≥ 0, v∞ ≥ 0, u∞v∞ = 0. Then, u∞ − v∞ ∈ E. (b) limn→+∞ Iβn = Φ(u∞ − v∞) = I∞. (c) u∞ − v∞ solves problem (5.1). Proof. If w ∈ E , we have w+w− = 0, and then (w+, w−) ∈Mβ , Φ(w) = Eβ(w+, w−) EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 17 for every β < 0. Therefore, Iβ ≤ I∞ for every β < 0. This implies, in particular, that 1 4 (‖un‖2λ1 + ‖vn‖2λ1 ) = Eβn(un, vn) ≤ I∞ for all n ∈ N. So, after passing to a subsequence, there exist u∞, v∞ ∈ H such that (un, vn) ⇀ (u∞, v∞) weakly in H, (un, vn)→ (u∞, v∞) strongly in L4(R2)× L4(R2), (un, vn)→ (u∞, v∞) a.e. in R2 × R2. Hence, u∞ ≥ 0 and v∞ ≥ 0. Since (un, vn) ∈Mβn , we see that 0 ≤ −2βn ∫ R2 u2 nv 2 ndx ≤ µ1 ∫ R2 u4 ndx+ µ2 ∫ R2 v4 ndx ≤ C0. Using Fatou’s lemma, we obtain∫ R2 u2 ∞v 2 ∞dx ≤ lim inf n→+∞ ∫ R2 u2 nv 2 ndx ≤ C0 2 lim n→+∞ 1 (−βn) = 0. Hence, u∞v∞ = 0 a.e. in R2. On the other hand, by Proposition 2.5, we know u∞ 6= 0 and v∞ 6= 0. Then, we may show as (5.2) that there exists s, t ∈ (0,∞) such that su∞, tv∞ ∈ N and su∞ − tv∞ ∈ E . By the fact that E(u, v) = max{E(su, tv) : s > 0, t > 0} if (u, v) ∈M, seeing (d) of Proposition 2.1 in [11], we deduce that I∞ ≤ 1 2 ∫ R2 [ |∇su∞|2 + |∇tv∞|2 + ( λ1 + 1 |x|2 ) (su∞)2 + ( λ2 + 1 |x|2 ) (tv∞)2 ] dx − 1 4 ∫ R2 [µ1(su∞)4 + µ2(tv∞)4]dx ≤ 1 2 lim inf n→+∞ ∫ R2 [ |∇sun|2 + |∇tvn|2 + ( λ1 + 1 |x|2 ) (sun)2 + ( λ2 + 1 |x|2 ) (tvn)2 ] dx − 1 4 lim n→+∞ ∫ R2 [µ1(sun)4 + µ2(tvn)4]dx ≤ 1 2 lim inf n→+∞ ∫ R2 [ |∇sun|2 + |∇tvn|2 + ( λ1 + 1 |x|2 ) (sun)2 + ( λ2 + 1 |x|2 ) (tvn)2 ] dx − 1 4 lim n→+∞ ∫ R2 [µ1(sun)4 + µ2(tvn)4]dx+ lim n→+∞ (−βn) ∫ R2 (sun)2(tvn)2dx ≤ lim inf n→+∞ Eβn(sun, tvn) ≤ lim inf n→+∞ Eβn(un, vn) = lim inf n→+∞ Iβn ≤ lim sup n→+∞ Iβn ≤ I∞, It follows that lim n→+∞ (−βn) ∫ R2 u2 nv 2 ndx = 0 and that lim n→+∞ ∫ R2 [ |∇sun|2 + |∇tvn|2 + ( λ1 + 1 |x|2 ) (sun)2 + ( λ2 + 1 |x|2 ) (tvn)2 ] dx = ∫ R2 [ |∇su∞|2 + |∇tv∞|2 + ( λ1 + 1 |x|2 ) (su∞)2 + ( λ2 + 1 |x|2 ) (tv∞)2 ] dx. 18 J. DENG, A. XIA, J. YANG EJDE-2020/108 Since (sun, tvn)→ (su∞, tv∞) weakly in H, we conclude that (un, vn)→ (u∞, v∞) strongly in H. As a result, I∞ = lim n→+∞ Eβn(un, vn) = 1 2 ∫ R2 [ |∇u∞|2 + |∇v∞|2 + ( λ1 + 1 |x|2 ) (u∞) 2 + ( λ2 + 1 |x|2 ) (v∞)2 ] dx − 1 4 ∫ R2 [µ1 (u∞) 4 + µ2(v∞)4]dx = Φ(u∞ − v∞). The fact (un, vn) ∈ Mβn yields u∞, v∞ ∈ N . This completes the proof of (a) and (b). We have shown that u∞ − v∞ is a minimizer for Φ on E . By the Sobolev compact embedding, we know Φ satisfies the Palais-Smale condition on N . The same argument of the proof of Lemma 2.6 in [8] leads to the conclusion that u∞−v∞ is a critical point of Φ. This proves (c). � Proof of Theorem 1.4. Let βn → −∞. Correspondingly, we have (un, vn) ∈ Mβn satisfying un ≥ 0 and vn ≥ 0 and Eβn(un, vn) = Iβn . By Proposition 5.1, after passing to a subsequence, we have that (un, vn)→ (u∞, v∞) strongly in H, u∞ ≥ 0, v∞ ≥ 0, and u∞ − v∞ is a nontrivial solution to the problem (5.1). Observing that −∆un ≤ µ1u 3 n, −∆vn ≤ µ2v 3 n in R2, by a Brézis-Kato argument we can verify that the uniform boundedness of (un, vn) in H1(R2) × H1(R2) implies the uniform boundedness of (un, vn) in L∞(R2) × L∞(R2), see [7]. By the interior W 2,2-regularity, see Theorem 1 in p.329 of [14], we obtain that (un, vn) ∈ W 2,2 loc (R2) ×W 2,2 loc (R2). It follows from the Lp-regularity , see Theorem B.2 in [27], and the fact that (un, vn) is bounded in ∈ L∞(R2) × L∞(R2) that (un, vn) ∈W 2,p loc (R2)×W 2,p loc (R2) for all p ≥ 2. Thanks to the Sobolev embedding theorem, we have (un, vn) ∈ C1(R2) × C1(R2). It follows from Arsela- Ascoli theorem that (un, vn) → (u∞, v∞) strongly in Cloc(R2) × Cloc(R2) as n → +∞ with (u∞, v∞) ∈ C(R2)×C(R2). Now, for x, y ∈ R2, we can see from the fact that ∇u∞ is bounded in L∞(R2) that |u∞(x)− u∞(y)| ≤ |u∞(x)− un(x)|+ |un(x)− un(y)|+ |un(y)− u∞(y)| ≤ ‖∇un‖L∞(R2)|x− y|+ on(1) ≤M |x− y|+ on(1). Lettin n → +∞, we obtain that u∞ is locally Lipschitz in R2. Similarly, v∞ is locally Lipschitz in R2. Therefore, u∞ − v∞ is locally Lipschitz in R2. As u∞ = (u∞−v∞)+ and v∞ = (u∞−v∞)−, these functions are continuous and the sets {u∞ > 0} and {v∞ > 0} are both open. Since u∞ − v∞ is a minimizer of Φ in N , these sets are connected. Moreover, we have {u∞ > 0} ∪ {v∞ > 0} = R2 because, otherwise, u∞ − v∞ would vanish in an open set, contradicting with the unique continuation principle. Obviously, u∞ solves the problem −∆u+ λ1u+ u2 |x|2 = µ1u 3 in {u∞ > 0}, EJDE-2020/108 SCHRÖDINGER SYSTEM WITH SINGULAR POTENTIAL 19 and v∞ solves the problem −∆v + λ2v + v2 |x|2 = µ2v 3 in {v∞ > 0}. This completes the proof. � Acknowledgments. A. Xia was supported by the Foundation of Jiangxi Provin- cial Education Department (NoĠJJ160335), and by the National Natural Science Foundation of China (Nos. 11701239 and 11871253). J. Yang was supported by the National Natural Science Foundation of China (Nos. 11671179 and 11771300). References [1] A. Ambrosetti, E. Colorado; Standing waves of some coupled nonlinear Schrödinger equations, J. Lond. Math. Soc., 75 (2007), 67-82. [2] N. Akhmediev, A. Ankiewicz; Partially coherent solitons on a finite background, Phys. Rev. Lett., 82 (1999), 2661-2664. [3] M. Badiale, V. Benci, S. Rolando; A nonlinear elliptic equation with singular potential and applications to nonlinear field equations, J. Eur. Math. Soc., 9 (2007), no. 3, 355-381. [4] M. Badiale, V. Benci, S. Rolando; Three dimensional vortices in the nonlinear wave equation, Boll. Unione Mat. Ital., (9) 2 (2009), no. 1, 105-134. [5] T. Bartsch, N. Dancer, Z.-Q. Wang; A Liouville theorem, a priori bounds, and bifurcating branches of positive solutions for a nonlinear elliptic system, Calc. Var. Partial Differential Equations, 37 (2010), 345-361. [6] T. Bartsch, Z.-Q. Wang; Note on ground states of nonlinear Schrödinger systems, J. Partial Differ. Equ., 19 (2006), 200-207. [7] H. Brézis, T. Kato; Remarks on the Schrödinger operator with singular complex potentials, J. Math. Pures Appl., (9) 58 (2) (1979), 137-151. [8] A. Castro, J. Cossio, J. M. Neuberger; A sign-changing solution for a superlinear Dirichlet problem, Rocky Mt. J. Math., (4) 27 (1997), 1041-1053. [9] Z. Chen, W. Zou; Positive least energy solutions and phase separation for coupled Schrödinger equations with critical exponent, Arch. Ration. Mech. Anal., 205 (2012), no. 2, 515-551. [10] Z. Chen, W. Zou; An optimal constant for the existence of least energy solutions of a coupled Schrödinger system, Calc. Var. Partial Differential Equations, 48 (2013), no. 3-4, 695-711. [11] M. Clapp, A. Pistoia; Existence and phase separation of entire solutions to a pure critical competitive elliptic system, Calc. Var. Partial Differential Equations, 57 (2018), no. 1, Art. 23, 20 pp. [12] N. Dancer, J. Wei, T. Weth; A priori bounds versus multiple existence of positive solutions for a nonlinear Schrödinger systems, Ann. Inst. H. Poincaré Anal. Non Linéaire, 27 (2010), 953-969. [13] B. Esry, C. Greene, J. Burke, J. Bohn; Hartree-Fock theory for double condensates, Phys. Rev. Lett., 78 (1997), 3594-3597. [14] L. C. Evans; Partial Differential Equations. Second edition, Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, Vol. 19 (2010). [15] G. Fibich, N. Gavish; Theory of singular vortex solutions of the nonlinear Schrödinger equa- tion, Phys. D, 237 (2008), no. 21, 2696-2730. [16] N. Gialelis; Inviscid limit of linearly damped and forced nonlinear Schrödinger equations, Electron. J. Differential Equations, 2020 (2020), No. 66, pp. 1-18. [17] T. Lin, J. Wei; Ground state of N coupled nonlinear Schrödinger equations in Rn, n ≥ 3, Commun. Math. Phys., 255 (2005), 629-653. [18] T. Lin, J. Wei; Spikes in two coupled nonlinear Schrödinger equations, Ann. Inst. H. Poincaré Anal. Non Linéaire, 22 (2005), 403-439. [19] T. Lin, J. Wei; Spikes in two-component systems of nonlinear Schrödinger equations with trapping potentials, J. Differ. Equ., 229 (2006), 538-569. [20] Z. Liu, Z.-Q. Wang; Multiple bound states of nonlinear Schrödinger systems, Commun. Math. Phys., 282 (2008), 721-731. 20 J. DENG, A. XIA, J. YANG EJDE-2020/108 [21] L. Maia, E. Montefusco, B. Pellacci; Positive solutions for a weakly coupled nonlinear Schrödinger systems, J. Differ. Equ., 229 (2006), 743-767. [22] L. Maia, B. Pellacci, M. Squassina; Semiclassical states for weakly coupled nonlinear Schrödinger systems, J. Eur. Math. Soc., 10 (2007), 47-71. [23] B. Noris, M. Ramos; Existence and bounds of positive solutions for a nonlinear Schrödinger system, Proc. Am. Math. Soc., 138 (2010), 1681-1692. [24] B. Noris, H. Tavares, S. Terracini, G. Verzini; Uniform Hölder bounds for nonlinear Schrödinger systems with strong competition, Comm. Pure Appl. Math., 63 (2010), no. 3, 267-302. [25] A. Pomponio; Coupled nonlinear Schrödinger systems with potentials, J. Differ. Equ., 227 (2006), 258-281. [26] B. Sirakov; Least energy solitary waves for a system of nonlinear Schrödinger equations in Rn, Commun. Math. Phys., 271 (2007), 199-221. [27] M. Struwe; Variational Methods. Second edition, Springer-Verlag, Berlin-Heidelberg, 1996. [28] E. Timmermans; Phase separation of Bose-Einstein condensates, Phys. Rev. Lett., 81 (1998), 5718-5721. [29] J. Wei, T. Weth; Radial solutions and phase separation in a system of two coupled Schrödinger equations, Arch. Ration. Mech. Anal., 190 (2008), 83-106. Jin Deng School of Mathematics and Statistics, Jiangxi Normal University, Nanchang, Jiangxi 330022, China Email address: Dengjin@jxnu.edu.cn Aliang Xia School of Mathematics and Statistics, Jiangxi Normal University, Nanchang, Jiangxi 330022, China Email address: aliang xia@jxnu.edu.cn Jianfu Yang School of Mathematics and Statistics, Jiangxi Normal University, Nanchang, Jiangxi 330022, China Email address: jfyang 2000@yahoo.com 1. Introduction 2. Preliminary results 3. Proof of Theorems ?? and ?? 4. Proof of Theorem ?? 5. Proof of Theorem ?? Acknowledgments References