RECURRENCE RELATIONS FOR PATTERNS OF

RECURRENCE RELATIONS FOR PATTERNS OF TYPE (2, 1) IN FLATTENED PERMUTATIONS TOUFIK MANSOUR, MARK SHATTUCK, AND DAVID G.L. WANG* Abstract. We consider the problem of counting the occurrences of patterns of the form xy-z within flattened permutations of a given length. Using symmetric functions, we find recurrence relations satisfied by the distributions on Sn for the patterns 12-3, 21-3, 23-1 and 32-1, and develop a unified approach to obtain explicit formulas. By these recurrences, we are able to determine simple closed form expressions for the number of permutations that, when flattened, avoid one of these patterns as well as expressions for the average number of occurrences. In particular, we find that the average number of 23-1 patterns and the average number of 32-1 patterns in Flatten(π), taken over all permutations π of the same length, are equal, as are the number of permutations avoiding either of these patterns. We also find that the average number of 21-3 patterns in Flatten(π) over all π is the same as it is for 31-2 patterns.

1. Introduction The pattern counting problem for permutations has been studied extensively from various perspectives in both enumerative and algebraic combinatorics; see, e.g., [7]. The comparable problem has also been considered on other discrete structures such as k-ary words [2], compositions [11], and set partitions [10] (see also [6] and the references contained therein). In his recent study of finite set partitions, Callan [3] introduced the notion of flattened partitions. In a previous paper [9], the authors considered permutations in the same sense and obtained formulas for the generating functions which count the flattened permutations of size n according to the number of peaks and valleys. Here, we continue this work for some related statistics and derive recurrences for the distributions. Let [n] = {1, 2, . . . , n} if n ≥ 1, and [0] = ∅. Denote the set of permutations of [n] by Sn . Let π = π1 π2 · · · πn ∈ Sn . A pattern is any permutation σ of shorter length, and an occurrence of σ in π is a subsequence of π that is order-isomorphic to σ. If r denotes the number of occurrences of a pattern σ within a permutation π, then the case that has been studied most often previously is r = 0, i.e., the avoidance of σ by π. Relatively little work has been done concerning the case when r > 0, and in what has been done, the patterns were usually of length three. For example, simple algebraic maps show that the six patterns of length three are divided into two classes with respect to pattern containment, a class with a representative pattern σ = 123 (see [14, 15]) and another with a representative pattern σ = 132 (see [12] and references contained therein). By requiring that some of the letters within an occurrence of a pattern be adjacent, Babson and Steingr´ımsson [1] generalized the concept of pattern avoidance. Claesson and Mansour [4] considered the further notion of a pattern σ = σ1 -σ2 - · · · -σk , said to be of type (ℓ1 , ℓ2 , . . . , ℓk ), wherein the subword σi has length ℓi for each i and the letters of each σi are required to be adjacent within an occurrence of σ. In this notation, a classical pattern of length k is of type (1, 1, . . . , 1). In particular, 1991 Mathematics Subject Classification. 11B37, 05A15, 05A05. Key words and phrases. pattern avoidance; recurrence relation; symmetric function; permutation. 1

2

TOUFIK MANSOUR, MARK SHATTUCK, AND DAVID G.L. WANG*

the permutation π is said to contain a pattern τ = xy-z of type (2, 1) if there exist indices 2 ≤ i < j ≤ n such that πi−1 πi πj is order-isomorphic to xyz, where xyz is some permutation of {1, 2, 3}. Otherwise, we say that π avoids τ . Suppose π ∈ Sn is represented in standard cycle form, that is, cycles arranged from left to right in ascending order according to the size of the smallest elements, where the smallest element is written first within each cycle. Let Flatten(π) be the permutation of length n obtained by erasing the parentheses enclosing the cycles of π and considering the resulting word. For example, if π = 71564328 ∈ S8 , then the standard cycle form of π is (172)(3546)(8) and Flatten(π) = 17235468. One can combine the ideas of the previous two paragraphs and say that a permutation π contains a pattern τ in the flattened sense if and only if Flatten(π) contains τ in the usual sense and avoids τ otherwise. Here, we will use this definition of pattern containment and consider the case when τ is a pattern of type (2, 1). For example, the permutation π = 71564328 avoids 23-1 but has four occurrences of 31-2 in the flattened sense as Flatten(π) = 17235468 avoids 23-1 but has four occurrences of 31-2. Patterns where some of the elements are required to be adjacent are called generalized patterns in the literature. There is a trend in the combinatorics community to refer to such patterns now as vincular patterns. It is also becoming increasingly common to represent vincular patterns by overlining elements that are to be adjacent. In this way, the classical patterns would be denoted, for example, by 123, as they have been traditionally, instead of by 1-2-3. A pattern xy-z of type (2, 1) would be represented by xyz. In this paper, though, we will make use of the former notation (in accordance with [1]) when discussing vincular patterns not only for what we feel are aesthetic reasons but also to avoid possible confusion with the notion of barred patterns (see, for example, A201497 in [17]). Let st denote a statistic on Sn which records the number of occurrences in the flattened sense of one of five patterns under consideration in this paper. In accordance with a previous paper [9], we will use the notation X gnst (a1 a2 · · · ak ) = q st(Flatten(π)) , π

where π ranges over all permutations of length n such that Flatten(π) starts with a1 a2 · · · ak . It is easy to see that gnst (a1 a2 · · · ak ) = 0 if a1 6= 1. We will often denote gnst (1) more simply by gnst . In this paper, we use symmetric functions to develop recurrences for the generating functions gnst in the cases when st is the statistic recording the number of occurrences of τ , where τ is any pattern of type (2, 1) (except for 13-2). As a consequence, we obtain simple closed formulas for the number of permutations avoiding a pattern of type (2, 1) in the flattened sense as well as for the average number of occurrences of a pattern taken over all permutations of a given size. We provide algebraic proofs of these results, as well as combinatorial proofs in all but one case. We remark that our results can be expressedP in terms of harmonic numbers, Stirling numbers, and two types of Bell numbers. Recall that Hn = nk=1 k1 denotes the n-th harmonic number; see, e.g., Graham, Knuth and Patashnik [5]. Denote the Stirling numbers of the second kind by S(n, k), the nth ˜ Bell number by Bn , and the nth complementary Bell number by Bn (also called the Rao Uppuluri˜n contains both positive and negative integers. More Carpenter number). Note that the sequence B ˜ information about Bn may be found in [16] and in the entry A000587 of OEIS [17]. We summarize our results in Table 1 below and obtain as a consequence the following result. Theorem 1.1. For any pattern p of type (2, 1), the average number avr(n) of occurrences of p in Flatten(π) over all permutations π of length n satisfies lim

n→∞

avr(n) 1 = . n2 12

RECURRENCE RELATIONS FOR PATTERNS OF TYPE (2, 1) IN FLATTENED PERMUTATIONS

3

Table 1. The number of avoiding and the average number of occurrences for patterns of type (2, 1) over flattened permutations of length n. pattern

number of avoiding

13-2

2n−1

average number n2 + 3n + 8 − Hn 12

2n − 2 n−1 n−1 X 2 kS(n − 1, k)

31-2 21-3

reference [13] Corollary 2.2

3

2

n − 3n + 26n − 12 − Hn 12n

Corollary 2.9

k=1

32-1 23-1 12-3

−2

n−2 X i=0

n−1 X k=1

Corollary 2.5

n2 − 9n − 4 + Hn 12

2k S(n − 1, k)

n−2 ˜n−i−3 (Bi + Bi+1 )B i

Corollary 2.7

n3 + 3n2 − 40n + 24 + Hn 12n

Corollary 2.13

We will need the following notation. Let X = {x1 , x2 , . . . , xm } be an ordered set. Define X ej (X) = xi1 xi2 · · · xij , 1≤i1 1 and π ∈ Am is expressed as π = U1 U2 · · · Ut for some t ≥ 1, where the blocks of Ui are arranged as the Li above and members of some subset of the right-to-left minima of π are marked (not including the right-most one). If 1 ∈ / U1 , then U1 contains no right-to-left minima and π ′ = U2 U3 · · · Ut has the same structure as π, up to relabeling the elements of π ′ . In this case, set fm (π) = U1 ∪ fp (π ′ ),

where p = |π ′ |. Note that in this case the elements 1 and m would lie in different blocks of fm (π), with the block containing m not the distinguished block. e1 = U1 − {1} and π e1 U2 U3 · · · Ut ). If 1 is marked, then Now suppose 1 ∈ U1 . Let U e = fm−1 (U add 1 to the block containing m within π e to obtain fm (π) in this case. If 1 is not marked and the distinguished block of π e does not contain m, then add 1 to the distinguished block of π e and designate the new distinguished block to be the one that contains m to obtain fm (π). If 1 is not marked and the distinguished block of π e does contain m, then fm (π) is obtained by adding the singleton block {1} and keeping the distinguished block the same. This defines fm in all cases. Note that the last two cases, taken together, correspond to when the elements 1 and m belong to different blocks of fm (π), with the block containing m distinguished. It may be verified that fm is a bijection, which gives (3.1) and completes the proof. 3.2. The average numbers. We first introduce some notation which we will use in this subsection. Given a pattern τ of type (2, 1), we will refer to an occurrence of τ (in the flattened sense) in which the 2 corresponds to the actual letter i as an i-occurrence of τ and an occurrence in which the 2 and 3 correspond to the letters i and j, respectively, as an (i, j)-occurrence. In the proofs that follow, tot(τ ) will denote the total number of occurrences of the pattern τ under consideration. 3.2.1. The average number of occurrences for 23-1 and 32-1. We first treat the case 32-1 and argue that the total number of i-occurrences of 32-1 (in the flattened sense) within all of the permutations (n−1)! of size n is given by (n − i) i−1 for i ∈ [3, n − 1]. Summing over i would then give the total 2 i number of occurrences of 32-1.


15

Note that within an i-occurrence of 32-1, the letter i cannot start a cycle since there is a letter to the right of it in the flattened form which is smaller. Note further that the position of j is determined by that of i’s within an (i, j)-occurrence of 32-1, where i + 1 ≤ j ≤ n. Given i and j, we count the permutations of size n for which there are exactly r (i, j)-occurrences of 32-1, respectively, where 1 ≤ r ≤ i − 2. Note that the position of i is determined within such permutations once the positions of the elements of [i − 1] have been, which also determines the position of j (note that i must be placed within a current cycle so that there are exactly r members of [i − 1] to its right within the flattened form). Thus, there are (n−1)! such permutations for each r, which implies that the total number of (i, j)i occurrences of 32-1 is given by i−2 X (n − 1)! i − 1 (n − 1)! = . r i 2 i r=1 Since there are n − i choices for j, given i, with each choice yielding the same number of (i, j) (n−1)! occurrences of 32-1, it follows that there are (n − i) i−1 i-occurrences of 32-1, as desired. 2 i Summing over 3 ≤ i ≤ n − 1, and simplifying, then gives n−1 X i − 3 1 n−1 i − 1 (n − 1)! tot(32-1) = (n − i) = n! + − (n − 1)! i 2 i 3 2 i=3 i=3 n! n − 3 3 n−1 = + n! Hn−1 − − (n − 1)! 2 2 2 3 n! 2 = (n − 9n − 4) + n!Hn . 12 n−1 X

Dividing by n! yields the average value formula given in Corollary 2.5. Writing the letter corresponding to 3 directly after (instead of directly before) the letter corresponding to 2 shows that the total number of (i, j)-occurrences of 23-1 is the same as the total number of (i, j)-occurrences of 32-1 for each i and j, which yields the result for 23-1. 3.2.2. The average number of occurrences for 31-2. Similar reasoning as in the prior proof shows that the total number of i-occurrences of 31-2 in the flattened sense within all of the permutations of [n] is for i ∈ [3, n − 1]. To see this, first note that there are n − i choices for given by (n − i) 2i − 1 (n−1)! i the letter j to play the role of the 3, given i, within an occurrence of 31-2. Let σ ∈ Sn . Note that for each r, 1 ≤ r ≤ i − 3, there are (n−1)! r permutations that have an (i, j)-occurrence of 31-2 in which i the letter i comes somewhere between the (r + 1)-st and (r + 2)-nd members of [i − 1] from the left within Flatten(σ), and 2 (n−1)! (i − 2) permutations in which the letter i occurs to the right of all the i members of [i − 1] within Flatten(σ). Observe that in the latter case, the letter i would either occur within a cycle whose first letter is a member of [i − 1] or as the first letter of a cycle. Note that in all cases, the possible positions for j are determined by the value of r and is independent of the value of i. Summing over r, the total number of (i, j)-occurrences of 31-2 is thus given by (1 + 2 + · · · + (i − 3) + 2(i − 2))

(n − 1)! = i

i (n − 1)! −1 . 2 i

16


Summing over i, and simplifying, then implies i (n − 1)! tot(31-2) = (n − i) −1 2 i i=3 n−1 n! X 2 n = i−1− − (n − 1)! − (n − 2) 2 i=2 i 3 n−1 X

=

n! 3 (n − 3n2 + 26n − 12) − n!Hn . 12n

Dividing by n! yields the average value formula given in Corollary 2.2. Remark: A similar proof may also be given for the entry in Table 1 above for the average number of occurrences of 13-2. 3.2.3. The average number of occurrences for 12-3 and 21-3. We first treat the case 21-3. To handle this case, we will simultaneously consider occurrences of the pattern 3-21. If i ∈ [3, n−1], first note that there are (i−2)(n−1)! permutations σ of size n in which the letter i directly precedes a member of [i−1] in Flatten(σ). To show this, first insert i directly before some member of [2, i−1] within a permutation of [i − 1] expressed in standardQcycle form. Upon treating i and the letter directly thereafter as a single object, we see that there are ns=i+1 (s − 1) choices for the positions of the elements of [i + 1, n] and Q thus the total number of such permutations is (i − 2)(i − 1)! ns=i+1 (s − 1) = (i − 2)(n − 1)!, as claimed. Within each of these permutations σ, every letter of [i + 1, n] contributes either an i-occurrence of 3-21 or of 21-3 depending on whether the letter goes somewhere before or somewhere after i within Flatten(σ). This implies tot(i-occurrences of 21-3) + tot(i-occurrences of 3-21) = (n − i)(i − 2)(n − 1)!,

3 ≤ i ≤ n − 1,

and summing over i gives tot(21-3) + tot(3-21) = (n − 1)!

(3.2)

n−1 X i=3

(n − i)(i − 2).

We now count the total number of occurrences of 3-21 within permutations of size n, which is apparently easier. We first count the number of permutations having an (i, j)-occurrence of 3-21, where 3 ≤ i < j ≤ n are given. To do so, we first create permutations of the set [i] ∪ {j} by writing some permutation of [i − 1] in standard cycle form and then deciding on the positions of the letters i and j. Either i and j can directly precede different members of [2, i −1] or can precede the same member (in which case j would come before i), whence there are i−2 + (i − 2) = i−1 choices 2 2 regarding the placement of i and j. Upon treating i and the letter directly thereafter as a single object and adding the remaining members r of [i + 1, n] − {j}, we see that the number of permutations of length n having an (i, j)-occurrence of 3-21 is

j−1 n Y Y i−1 i − 1 (n − 1)! (i − 1)! . r (r − 1) = 2 i 2 r=i+1 r=j+1


17

Since there are n − i choices for j, given i, the total number of i-occurrence of 3-21 is then given (n−1)! . Summing over 3 ≤ i ≤ n − 1 gives by (n − i) i−1 i 2 (3.3)

tot(3-21) = (n − 1)!

n−1 X i=3

n−i i−1 . i 2

Subtracting (3.3) from (3.2) yields 1 i−1 (n − i) i − 2 − i 2 i=3 n−1 n−1 n! X 2 (n − 1)! X 2 = i−1− − (i − i − 2) 2 i=2 i 2 i=2

tot(21-3) = (n − 1)!

=

n−1 X

n! 3 (n − 3n2 + 26n − 12) − n!Hn , 12n

which completes the proof in the case 21-3. A similar proof may be given for the case 12-3. In fact, there are the comparable formulas tot(12-3) + tot(3-12) = (n − 1)! and tot(3-12) = (n − 1)!

n−1 X i=2

n−i i

n−1 X i=2

(n − i)i

i −1 . 2

Subtracting, simplifying, and dividing by n! then gives the average value formula found in Corollary 2.13. 4. Further Results In Section 2, we presented a recurrence in several cases for the generating function gnst for the statistic st recording the number of occurrences of a given pattern of type (2,1) in Flatten(π), where π is a permutation of length n. In this section, we consider the problem of trying to find an explicit st st formula for the coefficient of q r in gnst , which we will denote by gn,r . Note that gn,r is the number of permutations π of length n such that Flatten(π) contains exactly r occurrences of the given pattern. st We will denote the generating function for the sequence {gn,r }n≥3 , where r is fixed, by Gst r (x). In 13-2 [13], it was shown that the generating function Gr (x) is rational for all r ≥ 0. For the five patterns discussed in this paper, however, the comparable problem is apparently more difficult. st st Let gn,r (1i) denote the coefficient of q r in gnst (1i). Note that gn,r (1i) is the number of permutations π of length n such that Flatten(π) starts with the letters 1i and contains exactly r occurrences of the associated pattern. Let X st gn,r (1i)v i−2 , Gst n,r (v) = (4.1)

i≥2

Gst r (x, v)

=

X

n≥3

n Gst n,r (v)x =

XX

n≥3 i≥2

st gn,r (1i)v i−2 xn .

18


st It follows that Gst n,r (1) = gn,r if n ≥ 2 and st Gst r (x) = Gr (x, 1) =

X

st gn,r xn ,

n≥3

r ≥ 0.

We now describe a general approach for finding the generating function Gst r (x). First, we will need a recurrence for gnst (1i). For the patterns of type (2, 1) featured, we have already obtained such a recurrence in each case; see Section 2 above. We then extract the coefficient of q r in these st recurrences for gnst (1i) and obtain comparable recurrences for the numbers gn,r (1i), which we convert into generating functional form. Using this form of the recurrence, we then derive explicit expressions for Gst r (x) in a recursive manner, starting with r = 0. To do so, we make use of various generating function techniques, including the kernel method. We will illustrate how this approach can be applied to find Gst r (x) in the cases 31-2 and 32-1. Similar arguments apply to the other three (2, 1) patterns discussed.

4.1. The pattern 31-2. We illustrate the approach described with the pattern 31-2. We often will omit the superscript 31-2 and write, for example, Gr (x, v) in place of G31-2 (x, v). We first obtain a r recurrence for the generating function Gr (x, v). Theorem 4.1. For any integer r ≥ 0, we have (4.2)

1+

r−1 X v2 x 2−v Gr (x, v) = x v r−j+1 Gj (x, v) + xGr (x, 1) + 2(2 + v)x3 δr,0 + Hr (x, v), 1−v 1 − v j=0

where Hr (x, v) = −x

Pr−1 P s=0

j≥2

Pj+r−s−1 n=3

gn,s (1j)v j+r−s−1 xn .

Proof. Let r ≥ 0. Extracting the coefficient of q r in (2.1) gives (4.3)

gn,r (1i) − gn−1,r +

i−2 X j=2

gn−1,r (1j) −

i−2 X

gn−1,r−i+j+1 (1j) = 0,

j=2

3 ≤ i ≤ n,

with g2,r = g2,r (12) = 2δr,0 . Multiplying each of the four terms on the left-hand side of (4.3) by v i−2 xn , and summing over n ≥ 3 and 3 ≤ i ≤ n, yields n XX

n≥3 i=3 n XX

gn,r (1i)v i−2 xn = Gr (x, v) − 2xGr (x, 1) − 4x3 δr,0 , gn−1,r v i−2 xn =

n≥3 i=3

n X i−2 XX

n≥3 i=3 j=2

vx x Gr (x, 1) − Gr (vx, 1) + 2vx3 δr,0 , 1−v 1−v

gn−1,r (1j)v i−2 xn =

x 2 v Gr (x, v) − Gr (vx, 1) , 1−v


19

and n XX

i−2 X

gn−1,r−i+j+1 (1j)v i−2 xn

n≥3 i=3 j=i−1−r

=x

X n+1 X

i−2 X

gn,r−i+j+1 (1j)v i−2 xn

n≥3 i=4 j=i−1−r

=x

X

i−2 X

X

gn,r−i+j+1 (1j)v i−2 xn

i≥4 j=i−1−r n≥i−1

X X

=x

X

gn,s (1j)v r+j−s−1 xn ,

s≤r−1 j≥2 n≥r+j−s

which combine to give (4.2).

Taking r = 0 in recurrence (4.2) gives v2 x 2−v 1+ G0 (x, v) = xG0 (x) + 2(2 + v)x3 . 1−v 1−v

√

1−2x To solve this equation, we use the kernel method and substitute v = C(x) = 1− 2x to obtain x − x − 2x2 G0 (x) = √ 1 − 4x and √ x2 ( 1 − 4x + 8x − 1) (4.4) G0 (x, C(x)) = lim G0 (x, v) = . v→C(x) 1 − 4x P 4 3 v Taking r = 1 in (4.2) and using the fact that H1 (x, v) = −x j≥3 gj,0 (1j)v j xj = − 2x 1−xv , we obtain v2 x 2−v x4 v 3 (4.5) 1+ G1 (x, v) = xv 2 G0 (x, v) + xG1 (x) − 2 . 1−v 1−v 1 − xv √ 1− 1−2x 2x

into this equation, and using (4.4), yields √ (3x − 1)(1 − 5x + 2x2 ) + (1 − 6x + 7x2 ) 1 − 4x p G1 (x) = , x (1 − 4x)3

Substituting v = C(x) =

and thus G1 (x, C(x)) =

lim G1 (x, v)

v→C(x)

√ (1 − 4x)(5x4 − x3 − 16x2 + 8x − 1) − (17x4 + 19x3 − 30x2 + 10x − 1) 1 − 4x p = . x (1 − 4x)5

Continuing in this way for r = 2, 3, we obtain the following result. Corollary 4.2. For 0 ≤ r ≤ 3, we have G31-2 (x) r where (i) a0 (x) = x, b0 (x) = −x − 2x2 ;

√ ar (x) + br (x) 1 − 4x p = , (1 − 4x)2r+1

20


(ii) a1 (x) = x1 (3x − 1)(1 − 5x + 2x2 ), b1 (x) = x1 (1 − 6x + 7x2 ); (iii) a2 (x) = x1 (1 − 12x + 50x2 − 76x3 + 22x4 ), b2 (x) = x1 (−1 + 10x − 32x2 + 28x3 ); (iv) a3 (x) = x12 (2 − 37x + 270x2 − 972x3 + 1748x4 − 1346x5 + 220x6 ), b3 (x) = x12 (−2 + 33x − 208x2 + 614x3 − 824x4 + 368x5 ). 4.2. The pattern 32-1. In this subsection, we provide a second example of our approach using the pattern 32-1. We often will omit the superscript 32-1 when discussing the various generating functions. We first establish a recurrence for Gr (x, v). Theorem 4.3. For any integer r ≥ 0, we have (4.6)

(1 − v + vx)Gr (x, v) =x(2 − v + 2vx)Gr (x, 1) − 2vx2 Gr (vx, 1)

where Hr (x, v) =

P

n≥3

+ 2x3 (1 − v)(2 + v + 2vx + 2v 2 x)δr,0 + Hr (x, v),

Pn

n+1 j−1 (v j=3 gn,r−j+2 (1j)x

− v n ).

Proof. Let r ≥ 0. Extracting the coefficient of q r in (2.6) gives X X (4.7) gn,r (1i) − gn−1,r + gn−1,r (1j) − gn−1,r−j+2 (1j) = 0, j≤i−1

j≤i−1

3 ≤ i ≤ n,

with g2,r = g2,r (12) = 2δr,0 . Multiplying each of the four summands on the left-hand side of (4.7) by v i−2 xn , and summing over n ≥ 3 and 3 ≤ i ≤ n, yields n XX

n≥3 i=3 n XX

gn,r (1i)v i−2 xn = Gr (x, v) − 2xGr (x, 1) − 4x3 δr,0 , gn−1,r v i−2 xn =

n≥3 i=3

n X i−1 XX

x vx Gr (x, 1) − Gr (vx, 1) + 2vx3 δr,0 , 1−v 1−v

gn−1,r (1j)v i−2 xn =

n≥3 i=3 j=2

vx x Gr (x, v) − Gr (vx, 1) + 2vx3 δr,0 , 1−v 1−v

and n X i−1 XX

gn−1,r−j+2 (1j)v i−2 xn

n≥3 i=3 j=2

=x =

i−1 X n+1 XX

gn,r−j+2 (1j)v i−2 xn

n≥2 i=3 j=2 n XX

x 1−v

n≥2 j=2

gn,r−j+2 (1j)xn (v j−1 − v n )

=2vx3 (1 + 2x + 2vx)δr,0 + which combine to give (4.6).

2vx2 Gr (x, 1) − Gr (vx, 1) + Hr (x, v), 1−v

Taking r = 0 in recurrence (4.6) gives (1 − v + vx)G0 (x, v) = x(2 − v + 2vx)G0 (x) − 2vx2 G0 (vx) + 2x3 (1 − v)(2 + v + 2vx + 2v 2 x).


Substituting v = 1/(1 − x), we obtain G0 (x) = 2xG0 (x/(1 − x)) +

(4.8)

21

2x3 (3 − x) . (1 − x)2

Iterating this recurrence an infinite number of times yields X 2i xi+2 (3 − (3i − 2)x) G32-1 (x) = . Qi 0 j=0 (1 − jx) i≥1 (1 − (i − 1)x)(1 − ix) Taking r = 1 in (4.6) gives

where

(1 − v + vx)G1 (x, v) = x(2 − v + 2vx)G1 (x) − 2vx2 G1 (vx) + H1 (x, v), H1 (x, v) = x

X

n≥3

gn,0 (13)xn+1 (v 2 − v n ) = x2

X

n≥3

gn−1,0 xn−1 (v 2 − v n )

= x2 v 2 (G0 (x) + 2x2 ) − x2 v(G0 (xv) + 2x2 v 2 ). Substituting v = 1/(1 − x) and using (4.8), we obtain G1 (x) = 2xG1 (x/(1 − x)) +

1 − 3x 3x3 G0 (x) − . 2(1 − x) 1−x

Iterating this recurrence yields X 2i−1 xi x 6x3 32-1 32-1 G1 (x) = (1 − (3 + i)x)(1 − ix)G0 − . Qi+1 1 − ix 1 − ix j=0 (1 − jx) i≥0

4.3. Remarks. We have found explicit formulas for the generating functions G31-2 (x) when 0 ≤ r ≤ 3 r as well as expressions for G32-1 (x) when r = 0, 1. We see that the procedure discussed for finding these r formulas may be extended to other small r. Our formulas for G31-2 (x) lead one to ask the following r question: For any r ≥ 0, does the generating function G31-2 (x) have the form r √ Ar (x) + Br (x) 1 − 4x p , xr (1 − 4x)2r+1

where Ar (x) and Br (x) are polynomials with integer coefficients? For the other patterns τ ∈ {12-3, 21-3, 23-1} featured in this paper, one can apply similar arguments and obtain in each case comparable explicit formulas for the generating function Gτr (x) when r is small, the details of which we leave to the interested reader. Acknowledgements. The authors are grateful to the anonymous referee for his careful reading. The third author was supported by the National Natural Science Foundation of China (Grant No. 11101010). References [1] E. Babson and E. Steingr´ımsson, Generalized permutation patterns and a classification of Mahonian statistics, S´ em. Lothar. Combin. 44 (2000) B44b. [2] A. Burstein and T. Mansour, Counting occurrences of some subword patterns, Discrete Math. Theor. Comput. Sci. 6 (2003) 1–11. [3] D. Callan, Pattern avoidance in “flattened” partitions, Discrete Math. 309 (2009) 4187–4191. [4] A. Claesson and T. Mansour, Counting occurrences of a pattern of type (1, 2) or (2, 1) in permutations, Adv. in Appl. Math. 29 (2002) 293–310.

22


[5] R. Graham, D. Knuth and O. Patashnik, Concrete Mathematics: A Foundation for Computer Science, 2nd ed., Addison-Wesley, 1994. [6] S. Heubach and T. Mansour, Combinatorics of Compositions and Words, CRC Press, Boca Raton, 2009. [7] S. Kitaev, Patterns in Permutations and Words, Springer-Verlag, Heidelberg, 2011. [8] T. Mansour and M. Shattuck, Pattern avoidance in flattened permutations, Pure Math. Appl. (PU.M.A.) 22(1) (2011) 75–86. [9] T. Mansour, M. Shattuck and D. G.L. Wang, Counting subwords in flattened permutations, submitted. [10] T. Mansour, M. Shattuck and S. H.F. Yan, Counting subwords in a partition of a set, Electron. J. Combin. 17 (2010) #R19. [11] T. Mansour and B. Sirhan, Counting ℓ-letter subwords in compositions, Discrete Math. Theor. Comput. Sci. 8 (2006) 285–298. [12] T. Mansour and A. Vainshtein, Counting occurrences of 132 in a permutation, Adv. in Appl. Math. 28 (2002) 185–195. [13] T. Mansour and D. G.L. Wang, On the 13-2 patterns in flattened permutations, submitted. [14] B. Nakamura and D. Zeilberger, Using Noonan-Zeilberger functional equations to enumerate (in polynomial time!) generalized Wilf classes, Adv. in Appl. Math. (2012), http://dx.doi.org/10.1016/j.aam.2012.10.003. [15] J. Noonan and D. Zeilberger, The enumeration of permutations with a prescribed number of forbidden patterns, Adv. in Appl. Math. 17 (1996) 381–407. [16] V. R. Rao Uppuluri and J. A. Carpenter, Numbers generated by the function exp(1 − ex ), Fibonacci Quart. 7 (1969) 437–448. [17] N. J. A. Sloane, The On-Line Encyclopedia of Integer Sequences, http://oeis.org. Department of Mathematics, University of Haifa, 31905 Haifa, Israel E-mail address: [email protected] Department of Mathematics, University of Tennessee, Knoxville, TN 37996 E-mail address: [email protected] Department of Mathematics, University of Haifa, 31905 Haifa, Israel E-mail address: [email protected], [email protected]