Orbital stability of ground state solutions of coupled nonlinear Schrödinger equations Liliane Maia∗, Eugenio Montefusco†, Benedetta Pellacci† June 6, 2008

Abstract In this paper orbital stability of solutions of weakly coupled nonlinear Schrödinger equations is studied. It is proved that ground state solutions-scalar or vector ones-are orbitally stable, while bound states with Morse index strictly greater than one are not stable. Moreover, an instability result for large exponent in the nonlinearity is presented.

1 Introduction We consider the following Cauchy problem for two coupled nonlinear Schrödinger equations

(1.1)

¡ ¢ 2p−2 + β|φ2 |p |φ1 |p−2 φ1 = 0 i ∂t φ1 + ∆φ1 + ¡|φ1 | ¢ i ∂t φ2 + ∆φ2 + |φ2 |2p−2 + β|φ1 |p |φ2 |p−2 φ2 = 0 φ1 (0, x) = φ01 (x), φ2 (0, x) = φ02 (x)

,

where φi : × n → , φ0i : n → , p > 1 and β is a real positive constant. In the following we will denote Φ = (φ1 , φ2 ). Coupled nonlinear Schrödinger equations appear in the study of many physical processes. For instance, such equations with cubic nonlinearity model the nonlinear interaction of two wave packets (see [6]), optical pulse propagation in birefringent fibers (see [25, 26]) or wavelengthdivision-multiplexed optical systems [1, 18]. A soliton or standing wave solution is a solution of the form Φ(x, t ) = (u 1 (x)e i ω1 t , u 2 (x)e i ω2 t ) where U (x) = (u 1 (x), u 2 (x)) is a solution of the elliptic system (1.2)

( ¡ ¢ −∆u 1 + ω1 u 1 = |u 1 |2p−2 + β|u 1 |p−2 |u 2 |p u 1 ¡ ¢ −∆u 2 + ω2 u 2 = |u 2 |2p−2 + β|u 2 |p−2 |u 1 |p u 2

.

Among all the standing waves we can distinguish between ground states and bound states. A ground state corresponds to a least energy solution U of (1.2); while all the critical points of the action functional give rise to bound states (or excited states) of (1.1). The existence of ground and bound states have been investigated by many authors using different methods. In [2, 11, 17, 19, 30, 35, 38, 39] by numerical arguments or analytical expansions ∗ Research partially supported by Projeto Universal/CNP

q and FAPDF. † Research supported by MIUR project Metodi Variazionali ed Equazioni Differenziali non lineari.

1

different families of vector and scalar solitons are found. In [3, 4, 21, 23, 24, 34] the mathematical analysis using variational methods has been pursued to prove the existence of ground and bound states with both nontrivial components. Moreover, in [3] it is clarified the difference between ground and bound states in dependence to different geometrical properties of the action functional. A ground state can be viewed as a one-hump soliton of (1.1) because it is nonnegative, radially symmetric and decades exponentially at infinity ([7]). On the other hand, vector multihump solitons are of much interest in the applications, for example they have been observed in photorefractive crystals [10, 27] and are considered of much interest in applications. The stability of a soliton is an important issue both from the mathematical point of view or applications. Moreover, since the problem is invariant with respect of the action of the map θ → e i θt , it is natural to investigate stability properties that take into account this rotation invariance and this is done by the orbital stability property. In the case of a single equation it has been proved (see [8, 9, 20, 37]) that the ground state is unique and orbitally stable, that is, roughly speaking, that if an initial datum Φ0 is close to the ground state U then all the orbit generated by Φ0 remains close to the soliton generated by U up to translations or phase rotations. Moreover, in [16, 33] it has been showed that a solution V is orbitally stable if and only if the charge of V viewed as a function of the parameter ω is convex, and this holds if and only if V is a ground state solution. Finally in [36] it is proved that under the same condition of convexity, orbital stability and linear stability are equivalent, so that the task is completely understood for the single Schrödinger equation. For the coupled nonlinear Schrödinger equations, the linear stability (or instability) of one-hump and multi-hump solitons have been recently studied in some interesting papers using numerical and analytical methods. In particular, for the cubic NLS equations single-hump vector solitons are known to be linearly stable [30, 40], while in [39] it is conjectured, based on numerical evidence, that multi-hump vector solitons are all linearly unstable and this is proved by numerical and analytical arguments in [31, 41] for p = 2 in (1.1) and for any p for special families of multihump vector solitons. Moreover, in [22] it has been studied orbital stability for different systems assuming a convexity condition similar to the one of [16]. On the other hand, to our knowledge the orbital stability has not been investigated for coupled nonlinear Schrödinger equations. In this paper we prove that there exists a strict relation between orbital stability and geometrical properties of the energy functional associated to (1.2). More precisely, we will show that if p < 1 + 2/n, every minimum point of the energy functional ´ 1 1 ³ 1 2p 2p p ku 1 k2p + ku 2 k2p + 2βku 1 u 2 kp . E (u 1 , u 2 ) = k∇u 1 k22 + k∇u 2 k22 − 2 2 2p on the manifold © ª Mω = (u 1 , u 2 ) ∈ H 1 × H 1 : ω1 ku 1 k2L 2 + ω2 ku 2 k2L 2 = γ gives rise to a orbitally stable solution; while every critical point of E on Mω on which E 00 has at least one negative direction in the tangent space, generates orbital instability. Our results add to the results in [3, 23] implying that for small values of β the orbitally stable solutions are scalar, while for β sufficiently large the stable solitons have both nontrivial components. Note that, in [5], this kind of arguments is used to study the orbital stability of the solitary waves of the nonlinear Klein-Gordon equation. Finally, we will also show an instability result in dependence of the exponent p as a consequence of blowing up in finite time. The paper is organized as follows. In Section 2 we state our main results. The definitions and preliminary results, preparatory to the proofs, are presented in section 3. In section 4 we give the proofs of our main results. A conclusion comments the results obtained. 2

2 Setting of the problem and main results In order to study the orbital stability of the ground state solutions of (1.1) let us consider the functional spaces 2 = L 2 (n , ) × L 2 (n , ) and 1 = H 1 (n , ) × H 1 (n , ). We recall that the ¯ = 1/2(u v¯ + v u). ¯ Then for ω = (ω1 , ω2 ), inner product between u, v ∈ is given by u · v = ℜ(u v) ωi ∈ , ωi > 0, we can define an equivalent inner product in 2 given by Z ¤ £ ∀ Φ = (φ1 , φ2 ), Ψ = (ψ1 , ψ2 ) (Φ|Ψ) = ℜ ω1 φ1 ψ1 + ω2 φ2 ψ2 , and an equivalent norm kΦk22,ω

= kφ1 k22,ω1

+ kφ2 k22,ω2 ,

where

kφi k22,ωi

= ωi kφi k22

= ωi

Z

φi φi

for i = 1, 2. It is known (see [8, 37]) that (1.1) is well locally posed in time, for p < n/(n − 2) when n > 2 and for any p for n = 1, 2, in the space 1 endowed with the norm kΦk21 = k∇Φk22 + kΦk22,ω p

p

p

for every Φ = (φ1 , φ2 ) ∈ 1 . Moreover we set the p norm as kΦkp = kφ1 kp +kφ2 kp , for p ∈ [1, +∞). It is well known that the mass of a solution and its total energy are preserved in time, that is the following conservation laws hold (see [13, 29]): kφ1 k22 = kφ01 k22 ,

(2.1)

kφ2 k22 = kφ02 k22 ,

and the total energy of the system E (Φ(t )) =

(2.2)

°2 ¡ ¢ 1° 1 k∇Φ(t )k22 − F (Φ(t )) = °∇Φ0 °2 − F Φ0 = E (0), 2 2

where F (Φ) =

(2.3)

´ 1 ³ 2p p kΦk2p + 2βkφ1 φ2 kp . 2p

We will denote with ( f 1 , f 2 ) the vector whose components are the partial derivatives of the function F . Note that it is possibile to write problem (1.1) in a vectorial form as follows i ∂Φ = E 0 (Φ) . (2.4) ∂t Φ(0) = Φ0 In [13] it is proved that, under the assumption p < 1+

(2.5)

2 , n

the solution of the Cauchy problem (1.1) exists globally in time. Our main result is concerned with the orbital stability of a ground state solution. In order to prove this, we will use the functional energy (see (2.2)) and action 1 I (U ) = kU k21 − F (U ). 2 Definition 2.1 Given ω = (ω1 , ω2 ) with ωi > 0 for i = 1, 2, we will say that a ground state solution U of (1.2) is a solution obtained as a scaling of the solution of the following minimization problem © ª (2.6) min E (U ) = c ω , where Mω = U ∈ 1 : kU k22,ω = γ , for some γ > 0. Mω

Moreover, we will denote with G the set of the ground state solutions. 3

The orbital stability property of a ground state solution is defined in the following Definition 2.2 The set G of the ground state solutions is orbitally stable if for any ε > 0 there exists δ = δ(ε) > 0 such that if inf kΨ0 −U k1 < δ, U ∈G

then

sup inf kΨ(t , ·) − V k1 < ε, t ≥0 V ∈G

where Ψ is a solution of (1.1) with initial datum Ψ0 . Roughly speaking the set of the ground state solution is orbitally stable if any other orbit generated from an initial datum Ψ0 close to U remains close to G uniformly with respect to time. Up to now the uniqueness of the ground state solution is an open problem for system (1.1), so that in the definition of orbital stability we have to take into account the possibility of a solution Ψ to go from a ground state close to Ψ0 to a different ground state solution V . Our main result is the following. Theorem 2.3 Assume (2.5). For any β, ω1 , ω2 > 0 a ground state solution U ∈ G is orbitally stable. Remark 2.4 This result and the conservation laws (2.1) imply that solutions that starts from initial data close to ground states with both nontrivial components remain close to orbits generated by ground states with both nontrivial components. While, solutions whose initial data is close to a ground state with one trivial component live close to orbits associated to ground states with one trivial component. Theorem 2.5 Assume (2.5). For any β, ω1 , ω2 > 0 a bound state V of (1.1), such that I 00 (V ) has at least two negative eigenvalues, is not orbitally stable. Remark 2.6 The preceding statement reads as follows: the instability of a solution is generated by the existence of a direction in the space TV (Mω ) on which the energy E (or, equivalently, the action I ) is negative definite. Since the manifold Mω has codimension 1 in 1 , the negative eigenspace of I 00 (V ) has a nonempty intersection with the tangent space. Remark 2.7 The result above means that if we take an initial datum arbitrarily near to a bound state, which is not a local minimum of the functional, the solution of the system does not remain close to the orbit generated by the excited state. In particular this implies that, for β ¿ 1, positive vector solutions are not orbitally stable since in [3, 23] it is proved that the scalar solution are local minimima while the positive solution is a mountain pass on the Nehari manifold. The local minima of I on N are scalar solution, i.e. with only one component different from zero. Remark 2.8 In [16, 33] it is proved that, for the single Schrödinger equation i φt + ∆φ + |φ|2p−2 φ = 0, the existence of a direction in the tangent space T v (Mω ) generating instability of a standing wave e i ωt v, it is equivalent to the concavity of the function of one real variable ω 7−→ c ω . In our case ω = (ω1 , ω2 ) ∈ 2 and the cited papers do not apply, so whether there exists a link between the geometrical properties of the map ω 7−→ c ω and the instability of the bound state is an open question. Moreover, it is an open problem also if the orbital stability property of a solution of (1.2) is equivalent to the linear stability property. 4

We will also prove an instability result in dependence of the exponent p. More precisely, we will show the following result. Theorem 2.9 Assume 1 + 2/n¡≤ p < n/(n − 2). Let U ¢be a solution of the system (1.2). Then, for any ω1 , ω2 , β > 0 the solution Φ = e i ω1 t u 1 (x), e i ω2 t u 2 (x) is unstable in the following sense: For any ε > 0 there exists Uε0 with kUε0 −U k1 ≤ ε such that the solution Φε satisfying Φε (0) = Uε0 blows up in a finite time in 1 .

3 Preliminary results In this section we will present some general results which will be useful in proving Theorems 2.3, 2.5 and 2.9. In proving many of the results of this section we will make use of the following lemma which proof is straightforward. Lemma 3.1 For any u ∈ H 1 and for any positive numbers λ, µ, we can define the scaling u µ,λ = µu(λx) such that the following equalities hold. (3.1)

ku µ,λ k22 = µ2 λ−n kuk22 ,

k∇u µ,λ k22 = µ2 λ2−n k∇uk22 ,

ku µ,λ k2p = µ2p λ−n kuk2p . 2p

2p

Proof. The proof is an immediate consequence of a change of variables. In [3, 23] the existence of a ground state solution is proved by solving the following minimization problem (3.2)

© ª min I (U ) = m ω , where N = U , (0, 0) : 〈I 0 (U ),U 〉 = 0 ,

(u,v)∈N

is called in the literature Nehari manifold (see [28, 23, 32]). Here we want to show the equivalence between this problem and problem (2.6). In order to do this, we denote by A the set of the solution of Problem (3.2) and define the sets © ª K E = c ∈ : ∃U ∈ Mω : E (U ) = c, ∇Mω E (U ) = 0 , (3.3)

© ª K˜E = U ∈ Mω : ∇Mω E (U ) = 0 ,

© ª K I = m ∈ : ∃U ∈ N : I (U ) = m, I 0 (U ) = 0 ,

© ª K˜I = U ∈ N : I 0 (U ) = 0

where we have denoted with ∇Mω E the tangential derivative of E on Mω . The following result holds. Theorem 3.2 Assume (2.5). For any β, ω1 , ω2 > 0 the following conclusions hold: a) there exists a bijective correspondence between the sets K˜E and K˜I , b) there exists a bijective map T : K I → K E ∩ − given by (3.4)

n 1 − T (m) = − p −1 2 ·

¸·

γ 2p 0 − n

¸

2p 0 −n 2/(p−1)−n

where p 0 = p/(p − 1) stands for the conjugate exponent of p.

5

·

1 m

¸

2 2/(p−1)−n

.

Proof. In order to prove assertion a), take V = (v 1 , v 2 ) ∈ Mω such that V satisfies 〈E 0 (V ),V 〉 = −νγ,

(3.5)

E (V ) = c < 0.

Then, using that f 1 (V )v 1 + f 2 (V )v 2 = 2pF (V ), it follows 〈E 0 (V ),V 〉 − 2E (V ) = −2pF (V ) +

1 F (V ) + 2c = −2νγ, p

then ν is a positive real number, and we can define the map T˜ : Mω → N by T µ,λ (V ) = V µ,λ where µ, λ, are given by µ = ν−1/2(p−1) ,

(3.6)

λ = ν−1/2 .

Using this and (3.5) one obtains that V µ,λ solves (1.2), so that T µ,λ (V ) belongs to K˜I . Vice-versa if U ∈ K˜I let us take ν > 0 such that ν1/(p−1)−n/2 = and λ, µ > 0 given by

λ = ν1/2

γ kU k22,ω

µ = ν1/2(p−1)

so that U µ,λ belongs to Mω , ∇Mω E (U ) = 0. This shows that (T µ,λ )−1 = T 1/µ,1/λ . In order to prove assertion b), note first that any m ∈ K I is positive. Indeed, since there exists U ∈ N such that I (U ) = m and I 0 (U ) = 0, it follows µ ¶ 1 1 1 1− kU k21 > 0, 〈I 0 (U ),U 〉 = m = I (U ) − 2p 2 p so that T is a well defined and injective map. Let us first show that if c ∈ K E ∩ − , then c = T (m). Indeed take V ∈ Mω corresponding to such c, and take T µ,λ (V ) = V µ,λ . Recalling Pohozaev identity (see (5.9) in [23]) and since V µ,λ ∈ N we get µ ¶ ´ 1 1 ³ − k∇V µ,λ k22 + kV µ,λ k22,ω = m, 2 2p (n − 2)k∇V µ,λ k22 + nkV µ,λ k22,ω =

´ n³ k∇V µ,λ k22 + kV µ,λ k22,ω , p

where m = I (V µ,λ ). We derive (3.7)

k∇V µ,λ k22

= nm,

F β (V

µ,λ

m )= , p −1

kV µ,λ k22,ω

µ

¶ 2p = − n m. p −1

Using (3.1) we have that (3.8)

µ2 k∇V k22 = nm, λn−2

µ2p m F β (V ) = , λn p −1

6

µ ¶ µ2 2p 2 kV k2,ω = − n m. λn p −1

Since V is in Mω , (3.6) yields γ = kV k22,ω = ν1/(p−1)−n/2

µ

¶ 2p − n m, p −1

this and (3.8) give k∇V k22

γ =n 0 2p − n µ

¶1+

2(p−1) 2−n(p−1)

µ

1 m

¶

2(p−1) 2−n(p−1)

,

µ ¶1+ 2(p−1) µ ¶ 2(p−1) 2−n(p−1) 1 γ 1 2−n(p−1) F β (V ) = . 0 p − 1 2p − n m All the above calculations imply that, if c is a negative constrained critical value of E on Mω and m is the corresponding critical value of I , than c is given by (3.4). In order to show that T −1 is surjective let us take m in K I and the corresponding U that satisfies the conditions in (3.3). For any ν > 0 we can define λ = ν1/2 ,

µ = ν1/2(p−1)

and consider U µ,λ . Using (3.7) and (3.1) and requiring that U µ,λ ∈ Mω imply that ν is related to γ by the expression · ¸ 1 γ . ν1/(p−1)−n/2 = m 2p 0 − n Moreover, since U is a free critical point of I we obtain that U µ,λ is a constrained critical point of E with Lagrange multipliers equal to ν. In order to conclude the proof we have to impose that E (U µ,λ ) = c. From conditions (3.7), (3.1) and from the definition of K I it follows that c, m and ν satisfy · ¸ 0 n 1 c =m − νp −n/2 2 p −1 and substituting the value of ν in dependence of γ implies that m = T −1 (c). Corollary 3.3 There exists a bijective correspondence between the sets G and A . Proof. Let V ∈ G and take T µ,λ (V ); Theorem 3.2 implies that T µ,λ (V ) is a critical point of I , we only have to show that I (T µ,λ (V )) = m = m ω . Indeed, suppose by contradiction that m > m ω . In [23] it is proved that m ω is achieved by a vector U , then U 1/µ,1/λ , with µ, λ as in (3.6), belongs to Mω and gives a negative critical value c given by (3.4). Since m ω < m we get c < c ω which is a contradiction, so that the claim is true. Using the preceding result and Theorem 2.1 in [23] we can prove the following statement. Theorem 3.4 Assume (2.5). For any β, ω1 , ω2 > 0, there exists a solution of the minimization problem (2.6). Proof. By using a Gagliardo-Nirenberg type inequality for systems (see [13] equation (9)), we get that the following inequality holds for any U ∈ Mω · ¸ 1 n(p−1) 2−n(p−1) C ω,β p−(p−1)n/2 E (U ) ≥ k∇U k2 k∇U k2 − γ , 2 p 7

so that E is bounded from below if and only if (2.5) holds. Moreover, note that the ³ infimum ´ cω µ,λ

µ,λ

in (2.6) is negative. Indeed, we impose λn = µ2 so that, for any U ∈ Mω , U µ,λ = u 1 , u 2 belongs to Mω . By (3.1) we derive the real function h defined by h(λ) = E (U µ,λ ) =

still

λ2 k∇U k22 − λn(p−1) F β (U ), 2

and from condition (2.5) it follows that there exists a λ0 = λ0 (U ) > 0 such that for any λ ∈ (0, λ0 ) h(λ) is negative and this shows the claim. Then, Theorem 3.2 implies that there exists m ∈ K I such that c ω = T (m). Finally, since in [23] it is proved that m ω is achieved, using Corollary 3.3 we obtain the conclusion. In order to prove the instability result Theorem 2.9 another variational characterization of a ground state solution will be useful. Let us define the functional R(U ) = k∇U k22 − n(p − 1)F β (U )

(3.9) and the infimum

d ω = inf I where P = {U ∈ 1 : R(U ) = 0} P

then the following result holds. Proposition 3.5 It holds: a) P is a natural constraint for I ; b) d ω = m ω Proof. In order to prove a) let us consider U a constrained critical point of I on P , then there exists λ ∈ such that the following identities are satisfied (3.10) (3.11)

(1 − 2λ)k∇U k22 + kU k22,ω = 2p[λn(1 − p) + 1]F β (U ) (1 − 2λ)

´ n − 1 k∇U k22 + kU k22,ω = n[λn(1 − p) + 1]F β (U ) 2 2

³n

k∇U k22 = n(p − 1)F β (U ).

(3.12) Hence, using (3.12) in (3.10) we get

· kU k22,ω = 2λ(1 − p) +

¸ 2p − 1 k∇U k22 , n(p − 1)

and using this and (3.12) in (3.11), and taking into account that p > 1 + 2/n, we obtain that λ = 0, so that U is a free critical point of I . In order to prove b) take a minimum point U of I in P ; from a) it follows that then U belongs to N so that (3.13)

dω ≥ mω ;

viceversa if V is a minimum point of I in N then V is a free critical point of I and Pohozaev identity implies that V ∈ P so that (3.14)

dω ≤ mω

yielding the conclusion.

8

³ n/2 ´ n/2 n/2 Lemma 3.6 For any U , (0, 0) let us consider U λ ,λ = u 1λ ,λ , u 2λ ,λ , for u µ,λ defined in Lemma 3.1. The following conclusions hold: n/2 a) there exists a unique λ³∗ = λ∗ (U´ ) such that U λ∗ ,λ∗ belong to P , n/2

b) the function g (λ) = I U λ ,λ has its unique point of maximum in λ = λ∗ , c) λ∗ < 1 if and only if R(U ) < 0 and λ∗ = 1 if and only if R(U ) = 0, d ) the function g (λ) is concave on (λ∗ , +∞). Proof. a): for any λ > 0 it holds ³ n/2 ´ R U λ ,λ = λ2 k∇U k22 − λn(p−1) n(p − 1)F β (U ), then there is a unique λ∗ = λ∗ (U ) given by " λ∗ =

k∇U k22

#1/[n(p−1)−2]

n(p − 1)F β (U )

³ n/2 ´ such that R U λ∗ ,λ∗ = 0. Computing the first derivative of g (λ) b) is proved. Since p > 1 + 2/n, c) easily follows. d ) immediately follows from writing the second derivative of the function g .

Lemma 3.7 For any U ∈ 1 with R(U ) < 0 it results R(U ) ≤ I (U ) − m ω .

(3.15) Proof. Lemma 3.6 implies that

g (1) ≥ g (λ∗ ) + g 0 (1)(1 − λ∗ ). Thus ³ n/2 ´ I (U ) = g (1) ≥ g (λ∗ ) + g 0 (1)(1 − λ∗ ) = g (λ∗ ) + R(U )(1 − λ∗ ) ≥ I U λ∗ ,λ∗ + R(U ). n/2 ,λ ∗

Recalling that U λ∗

∈ P and applying conclusion b) of Proposition 3.5 complete the proof.

4 Proofs of the main results In this section we will prove the main results concerning the stability (or instability) of the standing waves. In particular in the following subsection we show the orbital stability of the ground state solution, provided 1 < p < 1+2/n. For the same interval of p, in subsection 4.2 we show that every constrained critical point of E on Mω , such as a bound state, which is not a local minimum is not orbitally stable. Finally in subsection 4.3 we prove that for p ≥ 1 + 2/n every bound state solution is unstable. The proofs of Theorems 2.3 and 2.9 follow the arguments of [9, 8, 37] for the single equation, while the proof of Theorem 2.5 follows the arguments of [16, 33].

9

4.1 Stability of the ground state solutions Proof of Theorem 2.3. Let us argue by contradiction, and suppose that there exist ε0 > 0, {t k } ⊂ and a sequence of initial data {Φ0k } ⊂ 1 such that lim inf kΦ0k −U k1 = 0

(4.1)

k→0 U ∈G

and the corresponding sequence of solution {Φk } of Problem (1.1) satisfies ³ ´ (4.2) inf kΦk (·, t k ) − e i ω1 tk u 1 (·), e i ω2 tk u 2 (·) k ≥ ε0 U =(u 1 ,u 2 )∈G

Let us denote Ψk (x) = Φk (x, t k ). Conditon (4.1), definition 2.1 and the continuity properties of the functional E yield kΦ0k k22,ω = γ + o(1) E (Φ0k ) → c ω . Then, conservation laws (2.1), (2.2) imply that E (Ψk (t )) → c ω ,

kΨk k22,ω = γ + o(1)

and via concentration compactness arguments we obtain Φ such that Ψk → Φ. Moreover, Φ satisfies E (Φ) = c ω , kΦk22,ω = γ which is an evident contradiction with (4.2).

4.2 Instability of the bound state solutions In this section we will prove Theorem 2.5. In order to do so we take V = (v 1 , v 2 ) ∈ Mω is a bound state such that I 00 (V ) has two negative eingenvalues, then, there exists at least one direction in the tangent space TV (Mω ) along which I 00 (V ) is negative defined. This gives us a smooth curve Ψ : [−ρ, ρ] → Mω such that (4.3)

Ψ(0) = V,

E (Ψ(r )) < E (Ψ(0)) = E (V ),

kΨ(r ) − V k1 < δ,

and Y = (y 1 , y 2 ) = Ψ0 (0) the tangent vector to the curve in V satisfies (I 00 (V )Y |Y ) < 0. Let (4.4)

K = {Vϑ = (e i ω1 ϑ v 1 , e i ω2 ϑ v 2 ), ϑ ∈ }

the orbit generated by the bound state. Note that Vϑ is a bound state of the action I for any ϑ ∈ , and I 00 (Vϑ ) = I 00 (V ), this implies that the smooth curve Ψϑ (r ) = (e i ω1 ϑ ψ1 (r ), e i ω2 ϑ ψ2 (r )), satisfies Ψϑ (0) = Vϑ and possesses the same properties of Ψ. We point out, in particular, that the tangent direction is Yϑ = (e i ω1 ϑ y 1 , e i ω2 ϑ y 2 ), for any ϑ. In order to find out the instability regions for the dynamics, let us introduce © ª (4.5) L = W ∈ 1 : (W |iV ) = 0 , L δ = L ∩ B δ (V ) , U = ∪σ e i σ L δ , where B δ (V ) is the open ball of 1 centered in V of radius δ. Moreover, for any W ∈ U (for δ sufficiently small), we can define α(W ) = arctan

(W |iV ) . (W |V )

Following [33] it is possible to prove the following lemma. 10

Lemma 4.1 There exists a positive δ such that: a) e i σ L δ ∩ L δ = ; for σ ∈ (0, 2π); b) U is an open subset of 1 . Now we can define the operator G : U → L δ and the functionals A, P : U → by (4.6)

G(W ) = e i α(W ) W,

A(W ) = −(i Y |G(W )),

P (W ) = (E 0 (W )|i A 0 (W )).

Remark 4.2 Let us only notice that the property G(W ) ∈ L δ is a direct consequence of the definition of α(W ). Let us first prove the following preliminary result. Proposition 4.3 For any ϑ ∈ and for any r ∈ [−ρ, ρ] it holds E 0 − E (Ψϑ (r )) < r P (Vϑ ).

(4.7)

Proof. Using Taylor expansion and (2.1),(2.2), it results ¤ r 2 £ 00 (E (Vϑ )Yϑ , Yϑ ) + (E 0 (Vϑ )|Ψ00ϑ (0)) + o(r 2 ), 2 ¤ r 2 £ 00 0 0 = Q(Vϑ ) −Q(Ψϑ (r )) =r (Q (Vϑ )|Yϑ ) + (Q (Vϑ )Yϑ , Yϑ ) + (Q 0 (Vϑ )|Ψ00ϑ (0)) + o(r 2 ). 2

E 0 − E (Ψϑ (r )) = E (Vϑ ) − E (Ψϑ (r )) =r (E 0 (Vϑ )|Yϑ ) +

Note that Q(Ψϑ (r )) is constant with respect to r so that (4.8)

< Q 0 (Ψϑ (r )), Ψ0ϑ (r ) >= 0,

for any r . Then, adding the above equalities and taking into account that Vϑ is a critical point of I we have r2 E 0 − E (Ψϑ (r )) = r (E 0 (Vϑ )|Yϑ )) + (I 00 (Vϑ )Yϑ , Yϑ ) + o(r 2 ). 2 Moreover, from (4.6) and since (Vϑ |iVϑ ) = 0, we derive £ ¤ A 0 (Vϑ ) = e i α(ϑ) −i Yϑ + (Yϑ |Vϑ )α0 (Vϑ ) = −i Yϑ + α0 (Vϑ )(Yϑ |Vϑ ) then, taking into account (4.8) with r = 0, we obtain A 0 (Vϑ ) = −i Yϑ + α0 (Vϑ ) < Q 0 (Vϑ ), Yϑ >= −i Yϑ , yielding, by (4.6) E 0 − E (Ψϑ (r )) = r P (Vϑ ) +

r 2 00 (I (Vϑ )Yϑ , Yϑ ) + o(r 2 ) < r P (Vϑ ) 2

where the last inequality follows by the choice of Yϑ . Proof of Theorem 2.5. Let O a δ-tubular neighborhood of K (see (4.4)) and take W 0 ∈ N = O \K . Let W (t ) be the solution of the Cauchy problem (2.4) with initial datum W 0 ∈ N . Let us define the sets S 1 = {W ∈ N ∩ Mω : E (W ) < E 0 , P (W ) > 0}, S 2 = {W ∈ N ∩ Mω : E (W ) < E 0 , P (W ) < 0}. 11

Note that, for δ sufficiently small, P (W ) is well defined. Let us first note that S 1 ∪ S 2 is nonempty because Ψϑ (r ) ∈ S 1 ∪ S 2 for r , 0. Let us choose W 0 ∈ S 1 , an analogous argument will apply to initial datum in S 2 . Let us first show that S 1 is an invariant set under the flow. First of all, note that by continuity, inequality (4.7) holds for any W in N for δ sufficiently small, so we have (4.9)

0 < E 0 − E (W ) < r P (W ).

Therefore, P (W ) , 0 and, by continuity, it has to be positive proving that S 1 is invariant. Let us define T0 = sup{t : W (s) ∈ N , ∀s ∈ [0, t )}, the exit time for the orbit from the tubular neighborhood O ; the theorem will be proved showing that T0 < +∞. Note that there exists ε0 > 0 such that P (W (t )) > ε0 for any t < T0 . Indeed, let ε0 = E 0 − E (W 0 ) > 0 as W 0 belongs to S 1 . From (4.9) we derive ε0 < r P (W (t )), where r depends on t . Since W (t ) lies in N we can assume that r ≤ ρ < 1, in addition from (4.9) we get that there exists a positive constant σ such that 1 > r > σ hence we obtain P (W (t )) >

(4.10)

ε0 > ε0 . r

Multiplying the equation in (2.4) by −i A 0 (W (t )) and integrating we obtain ¿ À ® ∂W (t ) i , i A 0 (W (t )) = E 0 (W (t )), i A 0 (W (t )) = P (W (t )), ∂t this and (4.6) imply

∂A(W (t )) = ∂t

¿

A 0 (W (t )),

∂W (t ) ∂t

À = P (W (t )).

Thus, (4.10) yields ¯ ¡ ¢¯ ¯ A(W (t )) − A W 0 ¯ > ε0 t , as long as W (t ) ∈ N . Since N is a bounded set and A is bounded on N the solution goes out from the tubular neighborhood in finite time. Remark 4.4 The choice of the subspace L and of the open set U (see (4.5)) is crucial in finding out the regions S 1,2 where the functional P is different from zero. These subsets provide the existence of initial data, arbitrarily close to V , the trajectory of which goes far away from the orbit of the bound state.

4.3 Instability in the critical or supercritical case Proof of Theorem 2.9. Let us first consider the case 1 + 2/n < p < n/(n − 2). Let U be a solution n/2 of (1.2) and fix U s = U s ,s with s > 1. Then R(U s ) < 0 and I (U s ) < m ω = I (U ). Let Φs the solution generated by U s . By (2.2) we have I (Φs ) = I (U s ), when the solution exists. By continuity R(Φs (t )) < 0 for t small; moreover, Lemma 3.7 implies that R(Φs (t )) ≤ I (Φs (t )) − m ω = −σ < 0

12

showing that R(Φs (t )) < 0 when the solution exists. Defining the variance function V (t ) = k|x|Φs (t )k22 00

it follows that V (t ) = 8R(Φs (t )) ≤ −8σ. Thus, there exists T ∗ such that V (T ∗ ) = 0 showing, by using Hardy’s inequality, that Φs blows up in T ∗ (see [13]). Now consider p = 1+2/n. From Proposition 3.5 we get that any U solution of (1.2) satisfies (3.12), then we get R(U ) = 0 so that R(λU ) < 0 for any λ > 1. Let Uλ = λU be the initial datum of (2.4) and Φλ the corresponding solution. (2.2) implies 0 > R(Uλ ) = 2E (Uλ ) = 2E (Φλ ) = R(Φλ ), so that, also in this case, the variance is concave and the solution Φλ blows up in finite time.

5 Conclusions We have studied the problem of the orbital stability of standing waves in two weakly coupled nonlinear Schrödinger equations. Theorems 2.3 and 2.5 clarify the relation between orbital stability and critical points of E on Mω . Indeed, if we want a solution to enjoy orbital stability property then we have to look for minimum points fo the energy E on the constraint Mω or, equivalently, for solution of (1.2) with Morse index equal to one. Therefore, according to the results of [3, 23], we deduce that the stable solutions are scalar function (i.e. with one trivial component) for small value of the coupling parameter β, while are vectorial (i.e. with both nontrivial components) for β sufficiently large. In other words, there exists positive vectorial standing waves for β large or small, but these are stable solitons only for β large, in dependence of (ω1 , ω2 ). This result explains also the remark at the end of section 2 in [22]. Perhaps the surprising issue is the instability result, because we have obtained an example of positive, radial, vectorial solutions (with exponential decay, as |x| → +∞) not orbitally stable. This shows that a definition of ground state solution, which implies stability properties, has to be strictly related to the energy level of the vector solution and not to some qualitative properties of the function (see also the papers [3, 4, 23]). Finally we want to point out that all the preceding results agree with the literature for a single nonlinear Schrödinger equation (see [8, 37]). Acknowledgement. Part of this work was developed while the second autor was visiting the Universidade de Brasília: he is very grateful to all the Departmento de Matemática for the warm hospitality. Moreover he wishes to thank also professor Gustavo Gilardoni for many stimulating and friendly discussions.

References [1] G.P. Agrawal, Nonlinear fiber optics, Academic Press, San Diego 1989. [2] N.N. Akhmediev, A. Ankiewicz, Solitons, nonlinear pulses and beams, Chapman & Hall, London 1997.

13

[3] A. Ambrosetti, E. Colorado, Standing waves of some coupled nonlinear Schrödinger equations, J. London Math. Soc 75 (2007) 67-82. [4] T. Bartsch, Z.Q. Wang, Note on ground state of nonlinear Schrödinger systems, J. Partial Differential Equations 19 (2006) 200-207. [5] J. Bellazzini, V. Benci, C. Bonanno, A.M. Micheletti, Solitons for the nonlinear Klein-Gordon equation, preprint (2007) arXiv:0712.1103v1. [6] D.J. Benney, A.C. Newell, The propagation of nonlinear wave envelopes, J. Math. Phys 46 (1967) 133-139. [7] J. Busca, B. Sirakov, Symmetry results for semilinear elliptic systems in the whole space, J. Diff. Equations 163 (2000) 41-56. [8] T. Cazenave, An introduction to nonlinear Schrödinger equations, Textos de Métodos Matemáticos 26, Universidade Federal do Rio de Janeiro 1996. [9] T. Cazenave, P.L. Lions, Orbital stability of standing waves for some nonlinear Schrödinger equations, Comm. Math. Phys. 85 (1982) 549-561. [10] Z. Chen, M. Acks, E.A. Ostrovskaya, Y.S. Kivshar, Observation of bound states of interacting vector solitons, Opt. Lett. 25 (2000) 417-419. [11] D.N. Christodoulides, S.R. Singh, M.I. Carvalho, M. Segev, Incoherently coupled soliton pairs in biased photorefractive crystals, Appl. Phys. Lett. 68 (1996) 1763-1765. [12] D. de Figueiredo, O. Lopes, Solitary Waves for Some Nonlinear Schrodinger Systems, Ann. I.H. Poincaré AN 25 (2008) 149-161. [13] L. Fanelli, E. Montefusco, On the blow–up threshold for two coupled nonlinear Schrödinger equations, J. Phys. A: Math. Theor. 40 (2007) 14139-14150. [14] J. Ginibre, G. Velo, On a Class of Nonlinear Schrödinger Equations. I. The Cauchy Problem, General Case, J. Funct. Anal. 32 (1979) 1-32. [15] R.T. Glassey, On the blowing up of solutions to the Cauchy problem for nonlinear Sch¨rodinger equations, J. Math. Phys. 18 (1977) 1794-1797. [16] M. Grillakis, J. Shatah, W. Strauss, Stability theory of solitary waves in the presence of symmetry. I, J. Funct. Anal. 74 (1987) 160-197. [17] M. Haelterman, A.P. Sheppard, Bifurcation phenomena and multiple soliton-bound states in isotropic Kerr media, Phys. Rev. E 49 (1994) 3376-3381. [18] A. Hasegawa, Y. Kodama, Solitons in optical communications, Clarendon, Oxford 1995. [19] Y.S. Kivshar, G.P. Agrawal, Optical solitons: from fibers to photonic crystals, Academic press, San Diego 2003. [20] M.K. Kwong, Uniqueness of positive solutions of ∆u − u + u p = 0 in n , Arch. Rational Mech. Anal. 105 (1989) 243-266.

14

[21] T.C. Lin, J. Wei, Ground State of N Coupled Nonlinear Schrödinger equations in n , n ≤ 3, Comm. Math. Phys. 255 (2005) 629-653. [22] O. Lopes, Stability of solitary waves of some coupled systems, Nonlinearity 19 (2006) 95-113. [23] L.A. Maia, E. Montefusco, B. Pellacci, Positive solutions for a weakly coupled nonlinear Schrödinger system, J. Diff. Eqs. 229 (2006), 743-767. [24] L.A. Maia, E. Montefusco, B. Pellacci, Infinitely many nodal solutions for a weakly coupled nonlinear Schrödinger system, to appear on Comm. Cont. Math. [25] S.V. Manakov, On the theory of two-dimensional stationary self-focusing of electromagnetic waves, Sov. Phys. JETP 38 (1974) 248-253. [26] C.R. Menyuk, Nonlinear pulse propagation in birefringent optical fibers, IEEE J. Quantum Electron. 23 (1987) 174-176. [27] M. Mitchell, M. Segel, D.N. Christodoulides, Observation of multihump multimode solitons, Phys. Rev. Lett. 80 (1998) 4657-4660. [28] Z. Nehari, Characteristic values associated with a class of nonlinear second order differential equations, Acta Math. 105 (1961) 141-175. [29] T. Ozawa, Remarks on proofs of conservation laws for nonlinear Schrödinger equations, Calc. Var. P.D.E. 25 (2006) 403-408. [30] E.A. Ostrovskaya, Y.S. Kivshar, D.V. Skryabin, W.J. Firth, Stability of multihump optical solitons, J. Opt. B 1 (1999) 77-83. [31] D.E. Pelinovsky, J. Yang, Instabilities of multihump vector solitons in coupled nonlinear Schrödinger equations, Stud. Appl. Math. 115 (2005) 109-137. [32] P.H. Rabinowitz, On a class of nonlinear Schrödinger equations, Z. Angew. Math. Phys. 43 (1992) 270-291. [33] J. Shatah, W. Strauss, Instability of nonlinear bound states, Comm. Math. Phys. 100 (1985) 173-190. [34] B. Sirakov, Least Energy Solitary Waves for a System of Nonlinear Schrödinger Equations in n , Comm. Math. Phys. 271 (2007) 199-221. [35] A.W. Snyder, S.J. Hewlett, D.J. Mitchell, Dynamic spatial solitons, Phys. Rev. Lett. 72 (1994) 1012. [36] J. Stubbe, Linear stability theory of solitary waves arising from Hamiltonian systems with symmetry, Portugal. Math. 46 (1989) 17-32. [37] C. Sulem, P. L. Sulem, The nonlinear Schrödinger Equation. Self-Focusing and Wave Collapse., Springer-Verlag, New York, 1999. [38] M.V. Tratnik, J.E. Sipe, Bound solitary waves in optical fibers, Phys. Rev. A 38 (1988) 20112017.

15

[39] J. Yang, Classification of the solitary waves in coupled nonlinear Schrödinger equations, Phys. D 108 (1997) 92-112. [40] J. Yang, Vector solitons and their internal oscillations in birefringent nonlinear optical fibers, Stud. Appl. Math 98 (1997) 61-97. [41] J. Yang, Interaction of vector solitons, Phys. Rev. E 64 (2001) 026607. Liliane de Almeida Maia, Departmento de Matemática, Universidade de Brasília, 70.910 Brasilia, Brazil. E-mail address: [email protected] Eugenio Montefusco, Dipartimento di Matematica, Sapienza Università di Roma, piazzale A. Moro 5, 00185 Roma, Italy. E-mail address: [email protected] Benedetta Pellacci, Dipartimento di Scienze Applicate, Università degli Studi di Napoli Parthenope, via A. De Gasperi, 80133 Napoli, Italy. E-mail address: [email protected]

16

Abstract In this paper orbital stability of solutions of weakly coupled nonlinear Schrödinger equations is studied. It is proved that ground state solutions-scalar or vector ones-are orbitally stable, while bound states with Morse index strictly greater than one are not stable. Moreover, an instability result for large exponent in the nonlinearity is presented.

1 Introduction We consider the following Cauchy problem for two coupled nonlinear Schrödinger equations

(1.1)

¡ ¢ 2p−2 + β|φ2 |p |φ1 |p−2 φ1 = 0 i ∂t φ1 + ∆φ1 + ¡|φ1 | ¢ i ∂t φ2 + ∆φ2 + |φ2 |2p−2 + β|φ1 |p |φ2 |p−2 φ2 = 0 φ1 (0, x) = φ01 (x), φ2 (0, x) = φ02 (x)

,

where φi : × n → , φ0i : n → , p > 1 and β is a real positive constant. In the following we will denote Φ = (φ1 , φ2 ). Coupled nonlinear Schrödinger equations appear in the study of many physical processes. For instance, such equations with cubic nonlinearity model the nonlinear interaction of two wave packets (see [6]), optical pulse propagation in birefringent fibers (see [25, 26]) or wavelengthdivision-multiplexed optical systems [1, 18]. A soliton or standing wave solution is a solution of the form Φ(x, t ) = (u 1 (x)e i ω1 t , u 2 (x)e i ω2 t ) where U (x) = (u 1 (x), u 2 (x)) is a solution of the elliptic system (1.2)

( ¡ ¢ −∆u 1 + ω1 u 1 = |u 1 |2p−2 + β|u 1 |p−2 |u 2 |p u 1 ¡ ¢ −∆u 2 + ω2 u 2 = |u 2 |2p−2 + β|u 2 |p−2 |u 1 |p u 2

.

Among all the standing waves we can distinguish between ground states and bound states. A ground state corresponds to a least energy solution U of (1.2); while all the critical points of the action functional give rise to bound states (or excited states) of (1.1). The existence of ground and bound states have been investigated by many authors using different methods. In [2, 11, 17, 19, 30, 35, 38, 39] by numerical arguments or analytical expansions ∗ Research partially supported by Projeto Universal/CNP

q and FAPDF. † Research supported by MIUR project Metodi Variazionali ed Equazioni Differenziali non lineari.

1

different families of vector and scalar solitons are found. In [3, 4, 21, 23, 24, 34] the mathematical analysis using variational methods has been pursued to prove the existence of ground and bound states with both nontrivial components. Moreover, in [3] it is clarified the difference between ground and bound states in dependence to different geometrical properties of the action functional. A ground state can be viewed as a one-hump soliton of (1.1) because it is nonnegative, radially symmetric and decades exponentially at infinity ([7]). On the other hand, vector multihump solitons are of much interest in the applications, for example they have been observed in photorefractive crystals [10, 27] and are considered of much interest in applications. The stability of a soliton is an important issue both from the mathematical point of view or applications. Moreover, since the problem is invariant with respect of the action of the map θ → e i θt , it is natural to investigate stability properties that take into account this rotation invariance and this is done by the orbital stability property. In the case of a single equation it has been proved (see [8, 9, 20, 37]) that the ground state is unique and orbitally stable, that is, roughly speaking, that if an initial datum Φ0 is close to the ground state U then all the orbit generated by Φ0 remains close to the soliton generated by U up to translations or phase rotations. Moreover, in [16, 33] it has been showed that a solution V is orbitally stable if and only if the charge of V viewed as a function of the parameter ω is convex, and this holds if and only if V is a ground state solution. Finally in [36] it is proved that under the same condition of convexity, orbital stability and linear stability are equivalent, so that the task is completely understood for the single Schrödinger equation. For the coupled nonlinear Schrödinger equations, the linear stability (or instability) of one-hump and multi-hump solitons have been recently studied in some interesting papers using numerical and analytical methods. In particular, for the cubic NLS equations single-hump vector solitons are known to be linearly stable [30, 40], while in [39] it is conjectured, based on numerical evidence, that multi-hump vector solitons are all linearly unstable and this is proved by numerical and analytical arguments in [31, 41] for p = 2 in (1.1) and for any p for special families of multihump vector solitons. Moreover, in [22] it has been studied orbital stability for different systems assuming a convexity condition similar to the one of [16]. On the other hand, to our knowledge the orbital stability has not been investigated for coupled nonlinear Schrödinger equations. In this paper we prove that there exists a strict relation between orbital stability and geometrical properties of the energy functional associated to (1.2). More precisely, we will show that if p < 1 + 2/n, every minimum point of the energy functional ´ 1 1 ³ 1 2p 2p p ku 1 k2p + ku 2 k2p + 2βku 1 u 2 kp . E (u 1 , u 2 ) = k∇u 1 k22 + k∇u 2 k22 − 2 2 2p on the manifold © ª Mω = (u 1 , u 2 ) ∈ H 1 × H 1 : ω1 ku 1 k2L 2 + ω2 ku 2 k2L 2 = γ gives rise to a orbitally stable solution; while every critical point of E on Mω on which E 00 has at least one negative direction in the tangent space, generates orbital instability. Our results add to the results in [3, 23] implying that for small values of β the orbitally stable solutions are scalar, while for β sufficiently large the stable solitons have both nontrivial components. Note that, in [5], this kind of arguments is used to study the orbital stability of the solitary waves of the nonlinear Klein-Gordon equation. Finally, we will also show an instability result in dependence of the exponent p as a consequence of blowing up in finite time. The paper is organized as follows. In Section 2 we state our main results. The definitions and preliminary results, preparatory to the proofs, are presented in section 3. In section 4 we give the proofs of our main results. A conclusion comments the results obtained. 2

2 Setting of the problem and main results In order to study the orbital stability of the ground state solutions of (1.1) let us consider the functional spaces 2 = L 2 (n , ) × L 2 (n , ) and 1 = H 1 (n , ) × H 1 (n , ). We recall that the ¯ = 1/2(u v¯ + v u). ¯ Then for ω = (ω1 , ω2 ), inner product between u, v ∈ is given by u · v = ℜ(u v) ωi ∈ , ωi > 0, we can define an equivalent inner product in 2 given by Z ¤ £ ∀ Φ = (φ1 , φ2 ), Ψ = (ψ1 , ψ2 ) (Φ|Ψ) = ℜ ω1 φ1 ψ1 + ω2 φ2 ψ2 , and an equivalent norm kΦk22,ω

= kφ1 k22,ω1

+ kφ2 k22,ω2 ,

where

kφi k22,ωi

= ωi kφi k22

= ωi

Z

φi φi

for i = 1, 2. It is known (see [8, 37]) that (1.1) is well locally posed in time, for p < n/(n − 2) when n > 2 and for any p for n = 1, 2, in the space 1 endowed with the norm kΦk21 = k∇Φk22 + kΦk22,ω p

p

p

for every Φ = (φ1 , φ2 ) ∈ 1 . Moreover we set the p norm as kΦkp = kφ1 kp +kφ2 kp , for p ∈ [1, +∞). It is well known that the mass of a solution and its total energy are preserved in time, that is the following conservation laws hold (see [13, 29]): kφ1 k22 = kφ01 k22 ,

(2.1)

kφ2 k22 = kφ02 k22 ,

and the total energy of the system E (Φ(t )) =

(2.2)

°2 ¡ ¢ 1° 1 k∇Φ(t )k22 − F (Φ(t )) = °∇Φ0 °2 − F Φ0 = E (0), 2 2

where F (Φ) =

(2.3)

´ 1 ³ 2p p kΦk2p + 2βkφ1 φ2 kp . 2p

We will denote with ( f 1 , f 2 ) the vector whose components are the partial derivatives of the function F . Note that it is possibile to write problem (1.1) in a vectorial form as follows i ∂Φ = E 0 (Φ) . (2.4) ∂t Φ(0) = Φ0 In [13] it is proved that, under the assumption p < 1+

(2.5)

2 , n

the solution of the Cauchy problem (1.1) exists globally in time. Our main result is concerned with the orbital stability of a ground state solution. In order to prove this, we will use the functional energy (see (2.2)) and action 1 I (U ) = kU k21 − F (U ). 2 Definition 2.1 Given ω = (ω1 , ω2 ) with ωi > 0 for i = 1, 2, we will say that a ground state solution U of (1.2) is a solution obtained as a scaling of the solution of the following minimization problem © ª (2.6) min E (U ) = c ω , where Mω = U ∈ 1 : kU k22,ω = γ , for some γ > 0. Mω

Moreover, we will denote with G the set of the ground state solutions. 3

The orbital stability property of a ground state solution is defined in the following Definition 2.2 The set G of the ground state solutions is orbitally stable if for any ε > 0 there exists δ = δ(ε) > 0 such that if inf kΨ0 −U k1 < δ, U ∈G

then

sup inf kΨ(t , ·) − V k1 < ε, t ≥0 V ∈G

where Ψ is a solution of (1.1) with initial datum Ψ0 . Roughly speaking the set of the ground state solution is orbitally stable if any other orbit generated from an initial datum Ψ0 close to U remains close to G uniformly with respect to time. Up to now the uniqueness of the ground state solution is an open problem for system (1.1), so that in the definition of orbital stability we have to take into account the possibility of a solution Ψ to go from a ground state close to Ψ0 to a different ground state solution V . Our main result is the following. Theorem 2.3 Assume (2.5). For any β, ω1 , ω2 > 0 a ground state solution U ∈ G is orbitally stable. Remark 2.4 This result and the conservation laws (2.1) imply that solutions that starts from initial data close to ground states with both nontrivial components remain close to orbits generated by ground states with both nontrivial components. While, solutions whose initial data is close to a ground state with one trivial component live close to orbits associated to ground states with one trivial component. Theorem 2.5 Assume (2.5). For any β, ω1 , ω2 > 0 a bound state V of (1.1), such that I 00 (V ) has at least two negative eigenvalues, is not orbitally stable. Remark 2.6 The preceding statement reads as follows: the instability of a solution is generated by the existence of a direction in the space TV (Mω ) on which the energy E (or, equivalently, the action I ) is negative definite. Since the manifold Mω has codimension 1 in 1 , the negative eigenspace of I 00 (V ) has a nonempty intersection with the tangent space. Remark 2.7 The result above means that if we take an initial datum arbitrarily near to a bound state, which is not a local minimum of the functional, the solution of the system does not remain close to the orbit generated by the excited state. In particular this implies that, for β ¿ 1, positive vector solutions are not orbitally stable since in [3, 23] it is proved that the scalar solution are local minimima while the positive solution is a mountain pass on the Nehari manifold. The local minima of I on N are scalar solution, i.e. with only one component different from zero. Remark 2.8 In [16, 33] it is proved that, for the single Schrödinger equation i φt + ∆φ + |φ|2p−2 φ = 0, the existence of a direction in the tangent space T v (Mω ) generating instability of a standing wave e i ωt v, it is equivalent to the concavity of the function of one real variable ω 7−→ c ω . In our case ω = (ω1 , ω2 ) ∈ 2 and the cited papers do not apply, so whether there exists a link between the geometrical properties of the map ω 7−→ c ω and the instability of the bound state is an open question. Moreover, it is an open problem also if the orbital stability property of a solution of (1.2) is equivalent to the linear stability property. 4

We will also prove an instability result in dependence of the exponent p. More precisely, we will show the following result. Theorem 2.9 Assume 1 + 2/n¡≤ p < n/(n − 2). Let U ¢be a solution of the system (1.2). Then, for any ω1 , ω2 , β > 0 the solution Φ = e i ω1 t u 1 (x), e i ω2 t u 2 (x) is unstable in the following sense: For any ε > 0 there exists Uε0 with kUε0 −U k1 ≤ ε such that the solution Φε satisfying Φε (0) = Uε0 blows up in a finite time in 1 .

3 Preliminary results In this section we will present some general results which will be useful in proving Theorems 2.3, 2.5 and 2.9. In proving many of the results of this section we will make use of the following lemma which proof is straightforward. Lemma 3.1 For any u ∈ H 1 and for any positive numbers λ, µ, we can define the scaling u µ,λ = µu(λx) such that the following equalities hold. (3.1)

ku µ,λ k22 = µ2 λ−n kuk22 ,

k∇u µ,λ k22 = µ2 λ2−n k∇uk22 ,

ku µ,λ k2p = µ2p λ−n kuk2p . 2p

2p

Proof. The proof is an immediate consequence of a change of variables. In [3, 23] the existence of a ground state solution is proved by solving the following minimization problem (3.2)

© ª min I (U ) = m ω , where N = U , (0, 0) : 〈I 0 (U ),U 〉 = 0 ,

(u,v)∈N

is called in the literature Nehari manifold (see [28, 23, 32]). Here we want to show the equivalence between this problem and problem (2.6). In order to do this, we denote by A the set of the solution of Problem (3.2) and define the sets © ª K E = c ∈ : ∃U ∈ Mω : E (U ) = c, ∇Mω E (U ) = 0 , (3.3)

© ª K˜E = U ∈ Mω : ∇Mω E (U ) = 0 ,

© ª K I = m ∈ : ∃U ∈ N : I (U ) = m, I 0 (U ) = 0 ,

© ª K˜I = U ∈ N : I 0 (U ) = 0

where we have denoted with ∇Mω E the tangential derivative of E on Mω . The following result holds. Theorem 3.2 Assume (2.5). For any β, ω1 , ω2 > 0 the following conclusions hold: a) there exists a bijective correspondence between the sets K˜E and K˜I , b) there exists a bijective map T : K I → K E ∩ − given by (3.4)

n 1 − T (m) = − p −1 2 ·

¸·

γ 2p 0 − n

¸

2p 0 −n 2/(p−1)−n

where p 0 = p/(p − 1) stands for the conjugate exponent of p.

5

·

1 m

¸

2 2/(p−1)−n

.

Proof. In order to prove assertion a), take V = (v 1 , v 2 ) ∈ Mω such that V satisfies 〈E 0 (V ),V 〉 = −νγ,

(3.5)

E (V ) = c < 0.

Then, using that f 1 (V )v 1 + f 2 (V )v 2 = 2pF (V ), it follows 〈E 0 (V ),V 〉 − 2E (V ) = −2pF (V ) +

1 F (V ) + 2c = −2νγ, p

then ν is a positive real number, and we can define the map T˜ : Mω → N by T µ,λ (V ) = V µ,λ where µ, λ, are given by µ = ν−1/2(p−1) ,

(3.6)

λ = ν−1/2 .

Using this and (3.5) one obtains that V µ,λ solves (1.2), so that T µ,λ (V ) belongs to K˜I . Vice-versa if U ∈ K˜I let us take ν > 0 such that ν1/(p−1)−n/2 = and λ, µ > 0 given by

λ = ν1/2

γ kU k22,ω

µ = ν1/2(p−1)

so that U µ,λ belongs to Mω , ∇Mω E (U ) = 0. This shows that (T µ,λ )−1 = T 1/µ,1/λ . In order to prove assertion b), note first that any m ∈ K I is positive. Indeed, since there exists U ∈ N such that I (U ) = m and I 0 (U ) = 0, it follows µ ¶ 1 1 1 1− kU k21 > 0, 〈I 0 (U ),U 〉 = m = I (U ) − 2p 2 p so that T is a well defined and injective map. Let us first show that if c ∈ K E ∩ − , then c = T (m). Indeed take V ∈ Mω corresponding to such c, and take T µ,λ (V ) = V µ,λ . Recalling Pohozaev identity (see (5.9) in [23]) and since V µ,λ ∈ N we get µ ¶ ´ 1 1 ³ − k∇V µ,λ k22 + kV µ,λ k22,ω = m, 2 2p (n − 2)k∇V µ,λ k22 + nkV µ,λ k22,ω =

´ n³ k∇V µ,λ k22 + kV µ,λ k22,ω , p

where m = I (V µ,λ ). We derive (3.7)

k∇V µ,λ k22

= nm,

F β (V

µ,λ

m )= , p −1

kV µ,λ k22,ω

µ

¶ 2p = − n m. p −1

Using (3.1) we have that (3.8)

µ2 k∇V k22 = nm, λn−2

µ2p m F β (V ) = , λn p −1

6

µ ¶ µ2 2p 2 kV k2,ω = − n m. λn p −1

Since V is in Mω , (3.6) yields γ = kV k22,ω = ν1/(p−1)−n/2

µ

¶ 2p − n m, p −1

this and (3.8) give k∇V k22

γ =n 0 2p − n µ

¶1+

2(p−1) 2−n(p−1)

µ

1 m

¶

2(p−1) 2−n(p−1)

,

µ ¶1+ 2(p−1) µ ¶ 2(p−1) 2−n(p−1) 1 γ 1 2−n(p−1) F β (V ) = . 0 p − 1 2p − n m All the above calculations imply that, if c is a negative constrained critical value of E on Mω and m is the corresponding critical value of I , than c is given by (3.4). In order to show that T −1 is surjective let us take m in K I and the corresponding U that satisfies the conditions in (3.3). For any ν > 0 we can define λ = ν1/2 ,

µ = ν1/2(p−1)

and consider U µ,λ . Using (3.7) and (3.1) and requiring that U µ,λ ∈ Mω imply that ν is related to γ by the expression · ¸ 1 γ . ν1/(p−1)−n/2 = m 2p 0 − n Moreover, since U is a free critical point of I we obtain that U µ,λ is a constrained critical point of E with Lagrange multipliers equal to ν. In order to conclude the proof we have to impose that E (U µ,λ ) = c. From conditions (3.7), (3.1) and from the definition of K I it follows that c, m and ν satisfy · ¸ 0 n 1 c =m − νp −n/2 2 p −1 and substituting the value of ν in dependence of γ implies that m = T −1 (c). Corollary 3.3 There exists a bijective correspondence between the sets G and A . Proof. Let V ∈ G and take T µ,λ (V ); Theorem 3.2 implies that T µ,λ (V ) is a critical point of I , we only have to show that I (T µ,λ (V )) = m = m ω . Indeed, suppose by contradiction that m > m ω . In [23] it is proved that m ω is achieved by a vector U , then U 1/µ,1/λ , with µ, λ as in (3.6), belongs to Mω and gives a negative critical value c given by (3.4). Since m ω < m we get c < c ω which is a contradiction, so that the claim is true. Using the preceding result and Theorem 2.1 in [23] we can prove the following statement. Theorem 3.4 Assume (2.5). For any β, ω1 , ω2 > 0, there exists a solution of the minimization problem (2.6). Proof. By using a Gagliardo-Nirenberg type inequality for systems (see [13] equation (9)), we get that the following inequality holds for any U ∈ Mω · ¸ 1 n(p−1) 2−n(p−1) C ω,β p−(p−1)n/2 E (U ) ≥ k∇U k2 k∇U k2 − γ , 2 p 7

so that E is bounded from below if and only if (2.5) holds. Moreover, note that the ³ infimum ´ cω µ,λ

µ,λ

in (2.6) is negative. Indeed, we impose λn = µ2 so that, for any U ∈ Mω , U µ,λ = u 1 , u 2 belongs to Mω . By (3.1) we derive the real function h defined by h(λ) = E (U µ,λ ) =

still

λ2 k∇U k22 − λn(p−1) F β (U ), 2

and from condition (2.5) it follows that there exists a λ0 = λ0 (U ) > 0 such that for any λ ∈ (0, λ0 ) h(λ) is negative and this shows the claim. Then, Theorem 3.2 implies that there exists m ∈ K I such that c ω = T (m). Finally, since in [23] it is proved that m ω is achieved, using Corollary 3.3 we obtain the conclusion. In order to prove the instability result Theorem 2.9 another variational characterization of a ground state solution will be useful. Let us define the functional R(U ) = k∇U k22 − n(p − 1)F β (U )

(3.9) and the infimum

d ω = inf I where P = {U ∈ 1 : R(U ) = 0} P

then the following result holds. Proposition 3.5 It holds: a) P is a natural constraint for I ; b) d ω = m ω Proof. In order to prove a) let us consider U a constrained critical point of I on P , then there exists λ ∈ such that the following identities are satisfied (3.10) (3.11)

(1 − 2λ)k∇U k22 + kU k22,ω = 2p[λn(1 − p) + 1]F β (U ) (1 − 2λ)

´ n − 1 k∇U k22 + kU k22,ω = n[λn(1 − p) + 1]F β (U ) 2 2

³n

k∇U k22 = n(p − 1)F β (U ).

(3.12) Hence, using (3.12) in (3.10) we get

· kU k22,ω = 2λ(1 − p) +

¸ 2p − 1 k∇U k22 , n(p − 1)

and using this and (3.12) in (3.11), and taking into account that p > 1 + 2/n, we obtain that λ = 0, so that U is a free critical point of I . In order to prove b) take a minimum point U of I in P ; from a) it follows that then U belongs to N so that (3.13)

dω ≥ mω ;

viceversa if V is a minimum point of I in N then V is a free critical point of I and Pohozaev identity implies that V ∈ P so that (3.14)

dω ≤ mω

yielding the conclusion.

8

³ n/2 ´ n/2 n/2 Lemma 3.6 For any U , (0, 0) let us consider U λ ,λ = u 1λ ,λ , u 2λ ,λ , for u µ,λ defined in Lemma 3.1. The following conclusions hold: n/2 a) there exists a unique λ³∗ = λ∗ (U´ ) such that U λ∗ ,λ∗ belong to P , n/2

b) the function g (λ) = I U λ ,λ has its unique point of maximum in λ = λ∗ , c) λ∗ < 1 if and only if R(U ) < 0 and λ∗ = 1 if and only if R(U ) = 0, d ) the function g (λ) is concave on (λ∗ , +∞). Proof. a): for any λ > 0 it holds ³ n/2 ´ R U λ ,λ = λ2 k∇U k22 − λn(p−1) n(p − 1)F β (U ), then there is a unique λ∗ = λ∗ (U ) given by " λ∗ =

k∇U k22

#1/[n(p−1)−2]

n(p − 1)F β (U )

³ n/2 ´ such that R U λ∗ ,λ∗ = 0. Computing the first derivative of g (λ) b) is proved. Since p > 1 + 2/n, c) easily follows. d ) immediately follows from writing the second derivative of the function g .

Lemma 3.7 For any U ∈ 1 with R(U ) < 0 it results R(U ) ≤ I (U ) − m ω .

(3.15) Proof. Lemma 3.6 implies that

g (1) ≥ g (λ∗ ) + g 0 (1)(1 − λ∗ ). Thus ³ n/2 ´ I (U ) = g (1) ≥ g (λ∗ ) + g 0 (1)(1 − λ∗ ) = g (λ∗ ) + R(U )(1 − λ∗ ) ≥ I U λ∗ ,λ∗ + R(U ). n/2 ,λ ∗

Recalling that U λ∗

∈ P and applying conclusion b) of Proposition 3.5 complete the proof.

4 Proofs of the main results In this section we will prove the main results concerning the stability (or instability) of the standing waves. In particular in the following subsection we show the orbital stability of the ground state solution, provided 1 < p < 1+2/n. For the same interval of p, in subsection 4.2 we show that every constrained critical point of E on Mω , such as a bound state, which is not a local minimum is not orbitally stable. Finally in subsection 4.3 we prove that for p ≥ 1 + 2/n every bound state solution is unstable. The proofs of Theorems 2.3 and 2.9 follow the arguments of [9, 8, 37] for the single equation, while the proof of Theorem 2.5 follows the arguments of [16, 33].

9

4.1 Stability of the ground state solutions Proof of Theorem 2.3. Let us argue by contradiction, and suppose that there exist ε0 > 0, {t k } ⊂ and a sequence of initial data {Φ0k } ⊂ 1 such that lim inf kΦ0k −U k1 = 0

(4.1)

k→0 U ∈G

and the corresponding sequence of solution {Φk } of Problem (1.1) satisfies ³ ´ (4.2) inf kΦk (·, t k ) − e i ω1 tk u 1 (·), e i ω2 tk u 2 (·) k ≥ ε0 U =(u 1 ,u 2 )∈G

Let us denote Ψk (x) = Φk (x, t k ). Conditon (4.1), definition 2.1 and the continuity properties of the functional E yield kΦ0k k22,ω = γ + o(1) E (Φ0k ) → c ω . Then, conservation laws (2.1), (2.2) imply that E (Ψk (t )) → c ω ,

kΨk k22,ω = γ + o(1)

and via concentration compactness arguments we obtain Φ such that Ψk → Φ. Moreover, Φ satisfies E (Φ) = c ω , kΦk22,ω = γ which is an evident contradiction with (4.2).

4.2 Instability of the bound state solutions In this section we will prove Theorem 2.5. In order to do so we take V = (v 1 , v 2 ) ∈ Mω is a bound state such that I 00 (V ) has two negative eingenvalues, then, there exists at least one direction in the tangent space TV (Mω ) along which I 00 (V ) is negative defined. This gives us a smooth curve Ψ : [−ρ, ρ] → Mω such that (4.3)

Ψ(0) = V,

E (Ψ(r )) < E (Ψ(0)) = E (V ),

kΨ(r ) − V k1 < δ,

and Y = (y 1 , y 2 ) = Ψ0 (0) the tangent vector to the curve in V satisfies (I 00 (V )Y |Y ) < 0. Let (4.4)

K = {Vϑ = (e i ω1 ϑ v 1 , e i ω2 ϑ v 2 ), ϑ ∈ }

the orbit generated by the bound state. Note that Vϑ is a bound state of the action I for any ϑ ∈ , and I 00 (Vϑ ) = I 00 (V ), this implies that the smooth curve Ψϑ (r ) = (e i ω1 ϑ ψ1 (r ), e i ω2 ϑ ψ2 (r )), satisfies Ψϑ (0) = Vϑ and possesses the same properties of Ψ. We point out, in particular, that the tangent direction is Yϑ = (e i ω1 ϑ y 1 , e i ω2 ϑ y 2 ), for any ϑ. In order to find out the instability regions for the dynamics, let us introduce © ª (4.5) L = W ∈ 1 : (W |iV ) = 0 , L δ = L ∩ B δ (V ) , U = ∪σ e i σ L δ , where B δ (V ) is the open ball of 1 centered in V of radius δ. Moreover, for any W ∈ U (for δ sufficiently small), we can define α(W ) = arctan

(W |iV ) . (W |V )

Following [33] it is possible to prove the following lemma. 10

Lemma 4.1 There exists a positive δ such that: a) e i σ L δ ∩ L δ = ; for σ ∈ (0, 2π); b) U is an open subset of 1 . Now we can define the operator G : U → L δ and the functionals A, P : U → by (4.6)

G(W ) = e i α(W ) W,

A(W ) = −(i Y |G(W )),

P (W ) = (E 0 (W )|i A 0 (W )).

Remark 4.2 Let us only notice that the property G(W ) ∈ L δ is a direct consequence of the definition of α(W ). Let us first prove the following preliminary result. Proposition 4.3 For any ϑ ∈ and for any r ∈ [−ρ, ρ] it holds E 0 − E (Ψϑ (r )) < r P (Vϑ ).

(4.7)

Proof. Using Taylor expansion and (2.1),(2.2), it results ¤ r 2 £ 00 (E (Vϑ )Yϑ , Yϑ ) + (E 0 (Vϑ )|Ψ00ϑ (0)) + o(r 2 ), 2 ¤ r 2 £ 00 0 0 = Q(Vϑ ) −Q(Ψϑ (r )) =r (Q (Vϑ )|Yϑ ) + (Q (Vϑ )Yϑ , Yϑ ) + (Q 0 (Vϑ )|Ψ00ϑ (0)) + o(r 2 ). 2

E 0 − E (Ψϑ (r )) = E (Vϑ ) − E (Ψϑ (r )) =r (E 0 (Vϑ )|Yϑ ) +

Note that Q(Ψϑ (r )) is constant with respect to r so that (4.8)

< Q 0 (Ψϑ (r )), Ψ0ϑ (r ) >= 0,

for any r . Then, adding the above equalities and taking into account that Vϑ is a critical point of I we have r2 E 0 − E (Ψϑ (r )) = r (E 0 (Vϑ )|Yϑ )) + (I 00 (Vϑ )Yϑ , Yϑ ) + o(r 2 ). 2 Moreover, from (4.6) and since (Vϑ |iVϑ ) = 0, we derive £ ¤ A 0 (Vϑ ) = e i α(ϑ) −i Yϑ + (Yϑ |Vϑ )α0 (Vϑ ) = −i Yϑ + α0 (Vϑ )(Yϑ |Vϑ ) then, taking into account (4.8) with r = 0, we obtain A 0 (Vϑ ) = −i Yϑ + α0 (Vϑ ) < Q 0 (Vϑ ), Yϑ >= −i Yϑ , yielding, by (4.6) E 0 − E (Ψϑ (r )) = r P (Vϑ ) +

r 2 00 (I (Vϑ )Yϑ , Yϑ ) + o(r 2 ) < r P (Vϑ ) 2

where the last inequality follows by the choice of Yϑ . Proof of Theorem 2.5. Let O a δ-tubular neighborhood of K (see (4.4)) and take W 0 ∈ N = O \K . Let W (t ) be the solution of the Cauchy problem (2.4) with initial datum W 0 ∈ N . Let us define the sets S 1 = {W ∈ N ∩ Mω : E (W ) < E 0 , P (W ) > 0}, S 2 = {W ∈ N ∩ Mω : E (W ) < E 0 , P (W ) < 0}. 11

Note that, for δ sufficiently small, P (W ) is well defined. Let us first note that S 1 ∪ S 2 is nonempty because Ψϑ (r ) ∈ S 1 ∪ S 2 for r , 0. Let us choose W 0 ∈ S 1 , an analogous argument will apply to initial datum in S 2 . Let us first show that S 1 is an invariant set under the flow. First of all, note that by continuity, inequality (4.7) holds for any W in N for δ sufficiently small, so we have (4.9)

0 < E 0 − E (W ) < r P (W ).

Therefore, P (W ) , 0 and, by continuity, it has to be positive proving that S 1 is invariant. Let us define T0 = sup{t : W (s) ∈ N , ∀s ∈ [0, t )}, the exit time for the orbit from the tubular neighborhood O ; the theorem will be proved showing that T0 < +∞. Note that there exists ε0 > 0 such that P (W (t )) > ε0 for any t < T0 . Indeed, let ε0 = E 0 − E (W 0 ) > 0 as W 0 belongs to S 1 . From (4.9) we derive ε0 < r P (W (t )), where r depends on t . Since W (t ) lies in N we can assume that r ≤ ρ < 1, in addition from (4.9) we get that there exists a positive constant σ such that 1 > r > σ hence we obtain P (W (t )) >

(4.10)

ε0 > ε0 . r

Multiplying the equation in (2.4) by −i A 0 (W (t )) and integrating we obtain ¿ À ® ∂W (t ) i , i A 0 (W (t )) = E 0 (W (t )), i A 0 (W (t )) = P (W (t )), ∂t this and (4.6) imply

∂A(W (t )) = ∂t

¿

A 0 (W (t )),

∂W (t ) ∂t

À = P (W (t )).

Thus, (4.10) yields ¯ ¡ ¢¯ ¯ A(W (t )) − A W 0 ¯ > ε0 t , as long as W (t ) ∈ N . Since N is a bounded set and A is bounded on N the solution goes out from the tubular neighborhood in finite time. Remark 4.4 The choice of the subspace L and of the open set U (see (4.5)) is crucial in finding out the regions S 1,2 where the functional P is different from zero. These subsets provide the existence of initial data, arbitrarily close to V , the trajectory of which goes far away from the orbit of the bound state.

4.3 Instability in the critical or supercritical case Proof of Theorem 2.9. Let us first consider the case 1 + 2/n < p < n/(n − 2). Let U be a solution n/2 of (1.2) and fix U s = U s ,s with s > 1. Then R(U s ) < 0 and I (U s ) < m ω = I (U ). Let Φs the solution generated by U s . By (2.2) we have I (Φs ) = I (U s ), when the solution exists. By continuity R(Φs (t )) < 0 for t small; moreover, Lemma 3.7 implies that R(Φs (t )) ≤ I (Φs (t )) − m ω = −σ < 0

12

showing that R(Φs (t )) < 0 when the solution exists. Defining the variance function V (t ) = k|x|Φs (t )k22 00

it follows that V (t ) = 8R(Φs (t )) ≤ −8σ. Thus, there exists T ∗ such that V (T ∗ ) = 0 showing, by using Hardy’s inequality, that Φs blows up in T ∗ (see [13]). Now consider p = 1+2/n. From Proposition 3.5 we get that any U solution of (1.2) satisfies (3.12), then we get R(U ) = 0 so that R(λU ) < 0 for any λ > 1. Let Uλ = λU be the initial datum of (2.4) and Φλ the corresponding solution. (2.2) implies 0 > R(Uλ ) = 2E (Uλ ) = 2E (Φλ ) = R(Φλ ), so that, also in this case, the variance is concave and the solution Φλ blows up in finite time.

5 Conclusions We have studied the problem of the orbital stability of standing waves in two weakly coupled nonlinear Schrödinger equations. Theorems 2.3 and 2.5 clarify the relation between orbital stability and critical points of E on Mω . Indeed, if we want a solution to enjoy orbital stability property then we have to look for minimum points fo the energy E on the constraint Mω or, equivalently, for solution of (1.2) with Morse index equal to one. Therefore, according to the results of [3, 23], we deduce that the stable solutions are scalar function (i.e. with one trivial component) for small value of the coupling parameter β, while are vectorial (i.e. with both nontrivial components) for β sufficiently large. In other words, there exists positive vectorial standing waves for β large or small, but these are stable solitons only for β large, in dependence of (ω1 , ω2 ). This result explains also the remark at the end of section 2 in [22]. Perhaps the surprising issue is the instability result, because we have obtained an example of positive, radial, vectorial solutions (with exponential decay, as |x| → +∞) not orbitally stable. This shows that a definition of ground state solution, which implies stability properties, has to be strictly related to the energy level of the vector solution and not to some qualitative properties of the function (see also the papers [3, 4, 23]). Finally we want to point out that all the preceding results agree with the literature for a single nonlinear Schrödinger equation (see [8, 37]). Acknowledgement. Part of this work was developed while the second autor was visiting the Universidade de Brasília: he is very grateful to all the Departmento de Matemática for the warm hospitality. Moreover he wishes to thank also professor Gustavo Gilardoni for many stimulating and friendly discussions.

References [1] G.P. Agrawal, Nonlinear fiber optics, Academic Press, San Diego 1989. [2] N.N. Akhmediev, A. Ankiewicz, Solitons, nonlinear pulses and beams, Chapman & Hall, London 1997.

13

[3] A. Ambrosetti, E. Colorado, Standing waves of some coupled nonlinear Schrödinger equations, J. London Math. Soc 75 (2007) 67-82. [4] T. Bartsch, Z.Q. Wang, Note on ground state of nonlinear Schrödinger systems, J. Partial Differential Equations 19 (2006) 200-207. [5] J. Bellazzini, V. Benci, C. Bonanno, A.M. Micheletti, Solitons for the nonlinear Klein-Gordon equation, preprint (2007) arXiv:0712.1103v1. [6] D.J. Benney, A.C. Newell, The propagation of nonlinear wave envelopes, J. Math. Phys 46 (1967) 133-139. [7] J. Busca, B. Sirakov, Symmetry results for semilinear elliptic systems in the whole space, J. Diff. Equations 163 (2000) 41-56. [8] T. Cazenave, An introduction to nonlinear Schrödinger equations, Textos de Métodos Matemáticos 26, Universidade Federal do Rio de Janeiro 1996. [9] T. Cazenave, P.L. Lions, Orbital stability of standing waves for some nonlinear Schrödinger equations, Comm. Math. Phys. 85 (1982) 549-561. [10] Z. Chen, M. Acks, E.A. Ostrovskaya, Y.S. Kivshar, Observation of bound states of interacting vector solitons, Opt. Lett. 25 (2000) 417-419. [11] D.N. Christodoulides, S.R. Singh, M.I. Carvalho, M. Segev, Incoherently coupled soliton pairs in biased photorefractive crystals, Appl. Phys. Lett. 68 (1996) 1763-1765. [12] D. de Figueiredo, O. Lopes, Solitary Waves for Some Nonlinear Schrodinger Systems, Ann. I.H. Poincaré AN 25 (2008) 149-161. [13] L. Fanelli, E. Montefusco, On the blow–up threshold for two coupled nonlinear Schrödinger equations, J. Phys. A: Math. Theor. 40 (2007) 14139-14150. [14] J. Ginibre, G. Velo, On a Class of Nonlinear Schrödinger Equations. I. The Cauchy Problem, General Case, J. Funct. Anal. 32 (1979) 1-32. [15] R.T. Glassey, On the blowing up of solutions to the Cauchy problem for nonlinear Sch¨rodinger equations, J. Math. Phys. 18 (1977) 1794-1797. [16] M. Grillakis, J. Shatah, W. Strauss, Stability theory of solitary waves in the presence of symmetry. I, J. Funct. Anal. 74 (1987) 160-197. [17] M. Haelterman, A.P. Sheppard, Bifurcation phenomena and multiple soliton-bound states in isotropic Kerr media, Phys. Rev. E 49 (1994) 3376-3381. [18] A. Hasegawa, Y. Kodama, Solitons in optical communications, Clarendon, Oxford 1995. [19] Y.S. Kivshar, G.P. Agrawal, Optical solitons: from fibers to photonic crystals, Academic press, San Diego 2003. [20] M.K. Kwong, Uniqueness of positive solutions of ∆u − u + u p = 0 in n , Arch. Rational Mech. Anal. 105 (1989) 243-266.

14

[21] T.C. Lin, J. Wei, Ground State of N Coupled Nonlinear Schrödinger equations in n , n ≤ 3, Comm. Math. Phys. 255 (2005) 629-653. [22] O. Lopes, Stability of solitary waves of some coupled systems, Nonlinearity 19 (2006) 95-113. [23] L.A. Maia, E. Montefusco, B. Pellacci, Positive solutions for a weakly coupled nonlinear Schrödinger system, J. Diff. Eqs. 229 (2006), 743-767. [24] L.A. Maia, E. Montefusco, B. Pellacci, Infinitely many nodal solutions for a weakly coupled nonlinear Schrödinger system, to appear on Comm. Cont. Math. [25] S.V. Manakov, On the theory of two-dimensional stationary self-focusing of electromagnetic waves, Sov. Phys. JETP 38 (1974) 248-253. [26] C.R. Menyuk, Nonlinear pulse propagation in birefringent optical fibers, IEEE J. Quantum Electron. 23 (1987) 174-176. [27] M. Mitchell, M. Segel, D.N. Christodoulides, Observation of multihump multimode solitons, Phys. Rev. Lett. 80 (1998) 4657-4660. [28] Z. Nehari, Characteristic values associated with a class of nonlinear second order differential equations, Acta Math. 105 (1961) 141-175. [29] T. Ozawa, Remarks on proofs of conservation laws for nonlinear Schrödinger equations, Calc. Var. P.D.E. 25 (2006) 403-408. [30] E.A. Ostrovskaya, Y.S. Kivshar, D.V. Skryabin, W.J. Firth, Stability of multihump optical solitons, J. Opt. B 1 (1999) 77-83. [31] D.E. Pelinovsky, J. Yang, Instabilities of multihump vector solitons in coupled nonlinear Schrödinger equations, Stud. Appl. Math. 115 (2005) 109-137. [32] P.H. Rabinowitz, On a class of nonlinear Schrödinger equations, Z. Angew. Math. Phys. 43 (1992) 270-291. [33] J. Shatah, W. Strauss, Instability of nonlinear bound states, Comm. Math. Phys. 100 (1985) 173-190. [34] B. Sirakov, Least Energy Solitary Waves for a System of Nonlinear Schrödinger Equations in n , Comm. Math. Phys. 271 (2007) 199-221. [35] A.W. Snyder, S.J. Hewlett, D.J. Mitchell, Dynamic spatial solitons, Phys. Rev. Lett. 72 (1994) 1012. [36] J. Stubbe, Linear stability theory of solitary waves arising from Hamiltonian systems with symmetry, Portugal. Math. 46 (1989) 17-32. [37] C. Sulem, P. L. Sulem, The nonlinear Schrödinger Equation. Self-Focusing and Wave Collapse., Springer-Verlag, New York, 1999. [38] M.V. Tratnik, J.E. Sipe, Bound solitary waves in optical fibers, Phys. Rev. A 38 (1988) 20112017.

15

[39] J. Yang, Classification of the solitary waves in coupled nonlinear Schrödinger equations, Phys. D 108 (1997) 92-112. [40] J. Yang, Vector solitons and their internal oscillations in birefringent nonlinear optical fibers, Stud. Appl. Math 98 (1997) 61-97. [41] J. Yang, Interaction of vector solitons, Phys. Rev. E 64 (2001) 026607. Liliane de Almeida Maia, Departmento de Matemática, Universidade de Brasília, 70.910 Brasilia, Brazil. E-mail address: [email protected] Eugenio Montefusco, Dipartimento di Matematica, Sapienza Università di Roma, piazzale A. Moro 5, 00185 Roma, Italy. E-mail address: [email protected] Benedetta Pellacci, Dipartimento di Scienze Applicate, Università degli Studi di Napoli Parthenope, via A. De Gasperi, 80133 Napoli, Italy. E-mail address: [email protected]

16