Continued fractions and Catalan problems

Mahendra Jani William Paterson University

               janim@wpunj.edu

Robert G. Rieper William Paterson University

                                                 jrieper@cybernex.net

Submitted: December 21, 1999; Accepted: July 13, 2000.

Abstract

We find a generating function expressed as a continued fraction that enumerates ordered trees by the number of vertices at different levels. Several Catalan problems are mapped to an ordered-tree problem and their generating functions also expressed as a continued fraction. Among these problems is the enumeration of (132)-pattern avoiding permutations that have a given number of increasing patterns of length k. This extends and illuminates a result of Robertson, Wilf and Zeilberger for the case k = 3.

1 Introduction

A Catalan problem is any enumerative problem that produces the Catalan sequence of numbers or one of its many q-analogs. Stanley [3] provides a catalog of 66 Catalan problems. Interestingly, many of the generating functions that arise from these problems can be given as a continued fraction with a simple yet elegant form. Two of these generating functions are reproduced below and a third we derive anew. Our intent is to show that the first two continued fractions are special instances of the third with the implication that many others are as well. We begin with the three Catalan problems and their corresponding generating functions.

Problem 1. A (132) pattern (respectively, a (123) pattern) in a permutation π of length n is a triple 1 ≤ i < j < k ≤ n of indices for which π(i) < π(k) < π(j) (respectively, π(i) < π(j) < π(k)). Let f_{r}(n) denote the number of permutations π of length n that have no (132) patterns and exactly r (123) patterns. Recently, Robertson, Wilf and Zeilberger ([2]) derived the generating function

∑_{r,n≥0} f_{r}(n)z^{n}q^{r} = \frac{1}{1 − \frac{z}{1 − \frac{z}{1 − \frac{zq}{1 − \frac{zq^{3}}{1 − \frac{zq^{6}}{...}}}}}}  (1)

in which the lth numerator is zq^{\binom{l−1}{2}} (l is for level in anticipation of Problem 3).

This result has subsequently been extended to increasing patterns of arbitrary length by Mansour and Vainshtein [1]. Their approach uses properties of the Chebyshev polynomials to analyze the generating function directly. In contrast, we attack this same problem with an interesting bijection between labeled trees and (132)-avoiding permutations.

Problem 2. The number of lattice paths from (0, 0) to (n, n) with steps (1, 0) and (0, 1) that never rise above the line y = x is a Catalan number. Let P be such a path, A(P) the area under the path (and above the x-axis), and let C_{n}(q) = ∑_{P} q^{A(P)}. Then a generating function is given by (see Exercise 6.34 in Stanley [3] and replace the x therein with zq )

∑_{n≥0} q^{\binom{n+1}{2}}C_{n}(1/q)z^{n} = \frac{1}{1 − \frac{zq}{1 − \frac{zq^{2}}{1 − \frac{zq^{3}}{1 − \frac{zq^{4}}{...}}}}}  (2)

in which the lth numerator is zq^{l}.

Problem 3. The number of ordered trees (also known as plane trees) on n edges is a Catalan number. The level of a vertex is the number of edges on the unique path from the root to the vertex. Thus, the root is the unique vertex at level zero and the vertices at level one are adjacent to the root. Let T_{l_{1},l_{2},...} be the number of ordered trees that have l_{k} vertices at level k > 0 and let v_{k}, k > 0, be indeterminates. The generating function T that enumerates ordered trees by the number of vertices at each level is defined as

T (v_{1}, v_{2}, …) = ∑_{l_{1},l_{2},...≥0} T_{l_{1},l_{2},...}v_{1}^{l_{1}}v_{2}^{l_{2}} · · · .

The first few terms (number of edges n ≤ 3) of T are

T (v_{1}, v_{2}, …) = 1 + v_{1} + v_{1}v_{2} + v_{1}^{2} + v_{1}v_{2}v_{3} + 2v_{1}^{2}v_{2} + v_{1}v_{2}^{2} + v_{1}^{3} + · · · .

That T can be written as a continued fraction that subsumes the previous continued fractions is our main result. It is simple, yet has some interesting applications.

Theorem 1. The generating function that enumerates ordered trees by the number of vertices at each level is

T (v_{1}, v_{2}, …) = \frac{1}{1 − \frac{v_{1}}{1 − \frac{v_{2}}{1 − \frac{v_{3}}{...}}}}

Proof. We exploit the natural recursive property of ordered trees to obtain a recursion for T. The recursion immediately leads to the continued fraction. Any ordered tree on more than one vertex can be constructed from a collection of others (the subtrees) by joining the roots of these subtrees to a new vertex. The new vertex becomes the root of the tree under construction. Note that the level of a vertex in a subtree increases by one after the new root is inserted. The function T (v_{1}, v_{2}, …) enumerates the choices for a subtree and each of these choices contributes a factor of v_{1}T (v_{2}, v_{3}, …) because of the level changes. The factor of v_{1} is present because the root of the subtree becomes a vertex at level one. Thus, the trees with k subtrees (of the root) are enumerated by v_{1}^{k}T^{k}(v_{2}, v_{3}, …). The generating function satisfies

T (v_{1}, v_{2}, …) = 1 + v_{1}T (v_{2}, v_{3}, …) + v_{1}^{2}T^{2}(v_{2}, v_{3}, …) + …
= \frac{1}{1 − v_{1}T (v_{2}, v_{3}, …)}.

Iteration of the last functional recursion produces the continued fraction.

An immediate application is obtained by replacing each indeterminate v_{k} with z denoted simply as T (z). The resulting function enumerates ordered trees by the number of edges and is

T (z) = \frac{1}{1 − \frac{z}{1 − \frac{z}{1 − \frac{z}{...}}}}

= \frac{1}{1 − zT (z)}.

The well-known solution of the above generates the Catalan numbers and is T (z) = (1 − \sqrt{1 − 4z})/2z.

The more challenging applications are the evaluations needed to produce the continued fractions of permutations (Equation 1 of Problem 1) and lattice paths (Equation 2 of Problem 2). Both applications require that we map their respective problems to an ordered-tree problem. These mappings are of interest in their own right and we explore them now. We begin with the lattice path problem because it is simpler and the mapping is already known.

2 Lattice paths and ordered trees

We draw our ordered trees with the root at the top and proceed downward to the leaves. The first leaf is the leftmost leaf in the drawing and the remaining leaves are referred to by their positions in a left-to-right order. A preorder (depth-first) traversal of the ordered tree provides a well-known correspondence with a lattice path. When an edge of the tree is traversed downward away from the root we take a (1, 0) (east) step in the lattice, otherwise we take a (0, 1) (north) step. In this manner, an ordered tree with n edges corresponds to a unique lattice path from (0, 0) to (n, n).

If the path P corresponds to the tree T, then we need to determine what statistic of the tree corresponds to the area A(P) under the path. We let A(T) = A(P) be this statistic of the tree and claim that it depends only on the vertex levels.

Lemma 2. If T is an ordered tree on n edges, then

A(T) = \binom{n + 1}{2} − ∑_{vertices v} level(v),

where level(v) is the level of vertex v.

Proof. Let w be the rightmost leaf of the tree T. Our immediate interest is to calculate the area under the east step that arises in the lattice path by traversing the last edge to this leaf downward away from the root. This area is equal to the height that the east step has in the lattice path and is equal to the number of north steps that have occurred prior to the east step. There are a total of n north steps in the lattice path (one for each edge of the tree) and level(w) remaining north steps after the east step. Hence, the east step is at height n−level(w) in the lattice. If we now delete the leaf w from the tree T, then the resulting tree has a lattice path with area n − level(w) less than that of T. A formal inductive argument on the number of edges provides the result.

We use the lemma to prove the following continued-fraction result. The result is the same as that given in Equation 2.

Theorem 3. If C_{n}(q) = ∑_{T} q^{A(T)} enumerates the set of ordered trees on n edges by the area under their corresponding lattice paths, then

∑_{n≥0} q^{\binom{n+1}{2}}C_{n}(1/q)z^{n} = \frac{1}{1 − \frac{zq}{1 − \frac{zq^{2}}{1 − \frac{zq^{3}}{1 − \frac{zq^{4}}{...}}}}}

in which the lth numerator is zq^{l}.

Proof. Let T be an ordered tree on n edges and assign to every nonroot vertex at level l > 0 the value zq^{l}. The product of all these values is then z^{n}q^{∑_{v} level(v)}. Summing over all ordered trees on n edges we have by the lemma

z^{n} ∑_{T} q^{∑_{v} level(v)} = z^{n} ∑_{T} q^{\binom{n+1}{2}−A(T)} = z^{n}q^{\binom{n+1}{2}}C_{n}(1/q).

The sum of these over all n ≥ 0 then enumerates ordered trees and the generating function is given by the continued fraction of Theorem 1 with v_{l} = zq^{l}.

3 Permutations and ordered trees

The previous problem used an existing bijection between the set of ordered trees and the set of lattice paths to get the desired result. We seek a similar approach for ordered trees and permutations. There are many ways to map a permutation onto a tree (often an unordered tree) but none of these serve our needs. The mapping we introduce appears to be new.

Let T be an ordered tree on n edges. We use a preorder traversal of T to label the nonroot vertices in decreasing order with the integers n, n − 1, …, 1. Thus, the first vertex visited gets the label n and the last receives 1. We now construct a permutation written as a word by reading the labeled tree in postorder. We again traverse the tree from left to right and record the label of a vertex when we last visit it. In Catalan fashion, the five ordered trees and their corresponding permutations are shown in Figure 1. Note that the only permutation missing from those of length three is 132. The (132) pattern has been avoided. Also note that there is exactly one permutation with a (123) pattern (the first permutation shown). Thus, recalling the definition of f_{r}(n) given in the introduction we have f_{0}(3) = 4, f_{1}(3) = 1, and f_{r}(3) = 0, r > 1. We generalize these observations after introducing some useful notation.

If T is an ordered tree on n edges, then we let π(T) be its corresponding permutation written as a word on the numbers 1, 2, …, n. We let π(T, k) be the same permutation except we use the corresponding numbers 1 + k, 2 + k, …, n + k. For example, the first tree shown in the figure has π(T) = π(T, 0) = 123 and π(T, 1) = 234. For emphasis, we denote the concatenation of two words π and π′ as π∧π′ (instead of the usual ππ′). The following lemma describes how the permutation of a tree can be constructed from those of its subtrees.

Lemma 4. Let T be an ordered tree on n edges with subtrees T_{1}, T_{2}, …, T_{s} on n_{1}, n_{2}, …, n_{s} edges, respectively. Let N_{0} = n and N_{k} = N_{k−1} − n_{k} − 1, k = 1, 2, …, s, then

π(T) = π(T_{1}, N_{1}) ∧ N_{0} ∧ π(T_{2}, N_{2}) ∧ N_{1} ∧ … ∧ π(T_{s}, N_{s}) ∧ N_{s−1}.

Proof. Note that N_{0} = n is the total number of vertices to be labeled, that N_{1} is the total number of vertices to be labeled after those of the subtree T_{1}, and so on. Since the vertices of T are labeled in decreasing order using a preorder traversal, i < j implies that all vertices of T_{i} receive labels greater than those of T_{j}. Since π(T) is constructed by reading these labels

Figure 1: The five ordered trees on 3 edges and their permutations.

in postorder, i < j implies that the vertex labels in T_{i} appear in π(T) before any of those in T_{j}. Thus, π(T) begins as π(T_{1}) with each number incremented by N_{1}, i.e., as π(T_{1}, N_{1}). The root of this subtree is the first vertex visited in the preorder traversal and receives the label n = N_{0}. It is read last among the vertices of T_{1} in postorder, however, so that π(T) begins as π(T_{1}, N_{1}) ∧ N_{0}. The general case follows similarly.

Note that the lemma also provides the means to prove that the mapping T → π(T) is injective. That it is onto the set of all (132)-pattern avoiding permutations is proved in the next theorem. Before proceeding to this theorem we present another lemma which enables us to count (123) patterns and their generalization. An increasing pattern of length k, k > 0, in a permutation π of length n is a k-tuple 1 ≤ i_{1} < i_{2} < … < i_{k} ≤ n of indices for which π(i_{1}) < π(i_{2}) < … < π(i_{k}).

Lemma 5. Let T be an ordered tree on n edges and V_{k} a subset of k vertices, 0 < k ≤ n, none of which are the root, then the labels of these vertices provide an increasing pattern of length k in π(T) if and only if they lie along a path from the root to some leaf.

Proof. We induct on n. If n = 1, then k = 1 and the lemma is obvious. In fact, this is the case for all n whenever k = 1. We assume that the lemma is true for any ordered tree on n or fewer edges, n > 0, and let T be a tree on n + 1 edges. Suppose that π(T) contains an increasing pattern of length k > 1. We let v_{i} be the vertex in T that provides the i in such a pattern, i = 1, 2, …, k. Then v_{k} must receive a larger label than v_{1} implies that the subtree containing v_{k} precedes or is the same as that of v_{1} (see the proof of Lemma 4). However, the label of v_{k} must be read after that of v_{1} implies that its subtree must follow or be the same as that of v_{1}. The conclusion is that they are in the same subtree. A similar argument applies to v_{i} and v_{1}, i > 1, so that all the vertices must be in the same subtree, say T_{i}. Thus, the increasing pattern of length k lies entirely within π(T_{i}, N_{i}) ∧ N_{i−1}.

Recall that it is the root of the subtree that receives the label N_{i−1}. If the root of the subtree is one of the vertices providing the increasing pattern, then it must be v_{k}. We must consider two cases depending on whether the subtree root is v_{k} or not.

If it is not, then the pattern lies entirely within π(T_{i}, N_{i}) and corresponds uniquely to an increasing pattern of length k in π(T_{i}). By the inductive hypothesis, the vertices v_{1}, v_{2}, …, v_{k} must lie along some path from the root of the subtree to a leaf. Necessarily, this is also a path from the root of T to a leaf as required by this lemma.

If the root of the subtree is v_{k}, then v_{1}, v_{2}, …, v_{k−1} provide the labels for an increasing pattern of length k − 1 in π(T_{i}, N_{i}). This pattern corresponds to a unique increasing pattern of length k − 1 in π(T_{i}) and again by hypothesis the vertices v_{1}, v_{2}, …, v_{k−1} must lie along a path from the root of the subtree to some leaf of the subtree. This path together with the root of the subtree provides the needed path of this lemma.

Conversely, if V_{k} = {v_{1}, v_{2}, …, v_{k}} is a subset of nonroot vertices of T, where we may assume the label of v_{i} is less than that of v_{j} whenever i < j, and the vertices lie along a path from the root to a leaf, then v_{i} lies below v_{j} on this path whenever i < j. Thus, when the labels are read in postorder, the label of v_{i} is read prior to that of v_{j}, i < j. The vertices then provide an increasing pattern of length k in π(T).

The two previous lemmas enable us to prove the following interesting combinatorial theorem. Its corollary establishes a continued fraction as the generating function of (132)- avoiding permutations by number of increasing patterns of length k.

Theorem 6. A permutation π avoids the (132) pattern if and only if π = π(T) for some tree T. If this is the case, then the number of increasing patterns of length k depends only on the levels of the vertices in the tree and is given by ∑_{v} \binom{level(v)−1}{k−1}.

Proof. Suppose that π(T) contains a (132) pattern and that T is among the smallest such trees. We let v_{3} be a vertex in T that provides the 3 in such a pattern and let v_{1}, v_{2} be the vertices that provide the corresponding 1 and 2, respectively. Then v_{3} must receive a larger label than v_{1} implies that the subtree containing v_{3} precedes or is the same as that of v_{1} (see the proof of Lemma 4). However, the label of v_{3} must be read after that of v_{1} implies that its subtree must follow or be the same as that of v_{1}. The conclusion is that they are in the same subtree. A similar argument applies to v_{2} and v_{1} so that all three vertices must be in the same subtree, say T_{i}.

Also note that none of them can be the root of the subtree since the root receives the largest label among the vertices of the tree and, hence, can not be v_{1} or v_{2}. Its label appears later in π(T) than the others in the subtree implies that the root can not be v_{3}. Thus, the (132) pattern lies entirely within π(T_{i}, N_{i}) which implies that π(T_{i}) itself must have a (132) pattern contradicting our choice of T. Since the number of (132)-pattern avoiding permutations of length n and the number of ordered trees on n edges are the same Catalan number, the mapping T → π(T) is a bijection between these sets. It is an instructive exercise to construct T from a (132)-pattern avoiding permutation.

It remains to determine the number of increasing patterns of length k in π(T). As a result of Lemma 5 it is only necessary to count the number of vertex subsets of size k, none of which are the root, such that the vertices lie along a path from the root to a leaf. We claim this number is ∑_{v} \binom{level(v)−1}{k−1} as stated. To see this, let v be any vertex of T and choose v_{k} = v. There are level(v) − 1 nonroot vertices other than v along the unique path from the root to v. From these we select any k − 1 of them, which together with v, form the required subset. It is clear that every subset with the required properties arises this way and we are done.

We now use the theorem to write a generating function as a continued fraction. We let f_{r}^{(k)}(n) denote the number of permutations of length n that have no (132) pattern and exactly r increasing patterns of length k. The case k = 3 is that considered by Robertson, Wilf and Zeilberger [2].

Corollary 7. The generating function that enumerates (132)-pattern avoiding permutations of length n by number of increasing patterns of length k is

∑_{r,n≥0} f_{r}^{(k)}(n)z^{n}q^{r} = \frac{1}{1 − \frac{N_{1}}{1 − \frac{N_{2}}{1 − \frac{N_{3}}{1 − \frac{N_{4}}{...}}}}}  (3)

in which the lth numerator N_{l} is zq^{\binom{l−1}{k−1}}.

Proof. Let T be an ordered tree on n edges and assign to every nonroot vertex at level l > 0 the value zq^{\binom{l−1}{k−1}}. The product of all these values is then z^{n}q∑^{v} ^{\binom{level(v)−1}{k−1}} and the result follows from the previous theorem and Theorem 1.

References

[1] Tofik Mansour and Alek Vainshtein, Restricted permutations, continued fractions and chebyshev polynomials, The Electronic Journal of Combinatorics 7 (2000), no. R17.

[2] Aaron Robertson, Herbert S. Wilf, and Doron Zeilberger, Permutation patterns and continued fractions, The Electronic Journal of Combinatorics 6 (1999), no. R38.

[3] Richard P. Stanley, Enumerative combinatorics, vol. 2, Cambridge University Press, 1999.