CSCI 341 Theory of Computation

Parse Trees

Last time, we saw how to use context-free grammars to derive least solutions to polynomial systems of equations. One of the exercises we did was a derivation of the grammatical correctness of a few different english sentences. This was the grammar: \[\begin{aligned} S &\to N_P V_P N_P \\ N_P &\to BAN \mid BN \\ A &\to \texttt{nice} \mid \texttt{mean} \\ B &\to \texttt{the} \mid \texttt{a} \\ N &\to \texttt{boy} \mid \texttt{girl} \mid \texttt{dog} \\ V_P &\to \texttt{likes} \mid \texttt{hates} \end{aligned}\] In this context-free grammar, the variables all mean something: \(S\) stands for "sentence", \(N_P\) stands for "noun phrase", \(A\) stands for "adjective", \(B\) for "article", \(N\) for "noun", and \(V_P\) for "verb". This makes the derivations a bit opaque, and not really how we build sentences in our heads. The reason they are opaque is that the don't build the sentence in a way that is informed by the semantics of the sentence (what the sentence means). This is more clearly seen in arithmetic expressions, which have their own grammar and meanings. This becomes a little clearer when a notion of evaluation is introduced, like in arithmetic.

(Arithmetic Expressions) In the following grammar, the variable \(E\) derives the language \(\mathit{ArExp}\) of arithmetic expressions. \[\begin{aligned} E &\to N \mid (E + E) \mid (E \times E) \mid (E - E) \mid (E / E) \\ N &\to 0 \mid 1 \mid 2 \mid 3 \mid 4 \mid 5 \mid 6 \mid 7 \mid 8 \mid 9 \mid NN \end{aligned}\] Let's call this grammar \(\mathcal G_{\mathbb N} = (X, A, R)\). Above, \[ A = \big\{ (, ), +, \times, -, /, 0,1,2,3,4,5,6,7,8,9 \big\} \] because these are all the symbols we needed to form the monomial expressions above. For a first example of an arithmetic expression, let's derive: \[\begin{aligned} E &\Rightarrow (E + E) \\ &\Rightarrow (N + E) \\ &\Rightarrow (N + (E \times E)) \\ &\Rightarrow (N + (E \times N)) \\ &\Rightarrow (N + (N \times N)) \\ &\Rightarrow (NN + (N \times N)) \\ &\Rightarrow (9N + (N \times N)) \\ &\Rightarrow (90 + (N \times N)) \\ &\Rightarrow (90 + (NN \times N)) \\ &\Rightarrow (90 + (1N \times N)) \\ &\Rightarrow (90 + (10 \times N)) \\ &\Rightarrow (90 + (10 \times 2)) \end{aligned}\] This tells us that \((90 + (10 \times 2)) \in \mathit{ArExp}\). But of course, this derivation could have happened in many different ways; we could have changed the first \(N\) to an \(NN\) much earlier, or changed the second \(E\) in the first step to an \((E \times E)\) right away. The following diagram condenses all of the different ways this expression could be formed into a single tree:

A parse tree for \((90 + (10 \times 2))\).

This is called a parse tree for the arithmetic expression \((90 + (10 \times 2))\), the word "parse" here being used in the literal sense. This tree might seem opaque at first, but if you read all of the leaves of the tree left-to-right, then you recover the expression that formed the tree to begin with.

How to read the parse tree for \((90 + (10 \times 2))\).

Here is the formal definition of what a parse tree consists of.

Agonizing over the technical definition of parse tree is not so educational, so I encourage you to forget it for now and focus on the pictures, since those really convey what the technical definition is trying to capture: in the example, the root of the tree, labelled \(E\), branched into \((\), \(E\), \(+\), \(E\), and \()\), in that order. The reason this was a valid part of the tree for the grammar and variable deriving arithmetic expressions was because of the rewrite rule \[ E \to (E + E) \] in that grammar.

Parse trees, thankfully, are enough to determine derivability.

A proof can be found at the end of the page.

Coding Languages via Grammars

Grammars are widely used in the design of programming languages, although they usually appear in Backus-Naur Form, which is a special format of context-free grammar (we won't be convering it, but it's interesting to read about). You will recognize the next two examples, and we might see the third one later on.

The following example is an important historical one, and comes from early papers of Alonzo Church. It deals with blocks of code in a language called the \(\lambda\)-calculus. Most programming languages support \(\lambda\) terms as function abstractions, so you might even be familiar with this kind of program. If not, and if it seems like the example is old, then it's worth noting that it is indeed very old. But don't underestimate it! The \(\lambda\)-calculus is the foundation on which all functional programming languages are built. This example walked so that Haskell could run! Pun intended!

Proof of Parsing is Deriving Theorem

The proof uses an important idea, left-first derivations, which are a kind of procedural way of going between parse trees and derivations in an unambiguous manner. This is important because not every word derived from a grammar has a unique parse tree.

Intuitively, in a left-most derivation, only the left-most variable gets rewritten in each step.

Left-most derivations are not necessarily unique, for kind of dumb reasons, but they are close enough.

It is worth taking a moment to convince yourself of the following statement.

(of the Parsing is Deriving Theorem) Let's start by showing that \(\mathcal L(\mathcal G, x) \subseteq \mathcal L_{yield}(\mathcal G, x)\). Let \(w \in \mathcal L(\mathcal G, x)\). Then by the Left-most Derivations lemma there is a left-most derivation \[ x \Rightarrow \mu_1 \Rightarrow \mu_2 \Rightarrow \cdots \Rightarrow \mu_m \Rightarrow w \hspace{4em}(*) \] We proceed by induction on the length of the derivation \(m\) above. In the base case, if \(m = 0\), then \(x \Rightarrow w\), meaning that there is a rewrite rule \(x \to w\) in \(\mathcal G\). Then a parse tree for \(w\) in \(\mathcal G\) at \(x\) is where \(w = a_1\cdots a_k\). This handles the base case.

In the induction step, we assume that every left-most derivation of length strictly less than \(m\) has a corresponding parse tree. Now let \(\mu_1 = u_0 x_1 u_1 x_2 \cdots u_k x_k\) in \((*)\) for some words \(u_i \in A^*\) and variables \(x_i \in X\). We obtain the first level of our parse tree this way. Our goal is to obtain the whole parse tree from the following picture: Since the derivation is a left-most derivation, the derivation that picks up after the initial step of \((*)\) is obtained from a left-most derivation \(x_1 \Rightarrow^* v_1\). By the induction hypothesis, we obtain the subtree of the parse tree above rooted at \(x_1\). Now move from left to right in the tree depicted, and use the induction hypothesis to construct the rest of the subtrees representing parse trees from each of \(x_2, \dots x_k\). Above, \(w = u_0v_1 u_1 \cdots v_nu_n\), so we have just built a parse tree for \(w\) at \(x\). This shows that \(w \in \mathcal L_{yield}(\mathcal G, x)\), like we wanted.

Now let's show that \(\mathcal L_{yield}(\mathcal G, x) \subseteq \mathcal L(\mathcal G, x)\). We aren't going to start this in the standard way, just because it makes things a lot easier if we have the following fact up our sleeve:

Claim 1. Let \(w \in A^*\), and let \((T, \downarrow, d_0, l)\) be a parse tree for \(w\) in \(\mathcal G\) at \(x\). Then \(w\) is derivable from \(x\) in \(\mathcal G\).

(of Claim 1) We proceed via a strong induction on the height of the tree \((T, \downarrow, d_0)\); that is, the number of nodes in the longest chain of descendent relations \(d_0 \downarrow d_1 \downarrow d_2 \downarrow \cdots \downarrow d_n\) (how many layers the tree has). It is worth noting that there are two base cases: in the forst base case, the height of \((T, \downarrow, d_0)\) is \(1\), because we are not allowing for the empty tree (because \(d_0 \in T\)). But this would mean that \(l(d_0) = x\), which is not a word. In this case, the claim is vaccuously true. In the second base case, the height of the tree is \(2\), which means that the tree consists of two layers: With its labelling, \(l(d_0) = x\) (by definition of parse tree) and \(l(d_i) = a_i \in A\) for each \(i = 1, \dots, m\), because the leaves of the parse tree must be letters. Furthermore, because the tree yields \(w\), we must have \(w = a_1\cdots a_m\). By definition of parse tree, there is a rewrite rule \(x \to a_1\cdots a_m\) in \(\mathcal G\), and \(w = a_1 \cdots a_m\). Therefore, \(x\) derives \(w\) in \(\mathcal G\), i.e., \(w \in \mathcal L(\mathcal G, x)\).

Now let's do the induction step. Given a number \(n > 2\), assume that Claim 1 is true for any tree with height \(k < n\), and let the height of \((T, \downarrow, d_0, l)\) be \(n\). Since \(n > 2\), there are nodes \(d_i\) with \(d_0 \downarrow (d_1, \dots, d_m)\) corresponding to a derivation rule \(x \to l(d_1)~l(d_2)~\cdots~l(d_m)\). This allows us to consider \(m\) differen subtrees of height strictly less than \(n\): the subtrees rooted at \(d_1\), \(d_2\), up to \(d_n\), respectively. Since the whole tree yields \(w\), each of these subtrees must yield subwords of \(w\), altogether adding up to \(w\) (note that we are allowing \(d_i\) to be a leaf here, in which case it represents a single letter of \(w\)). That is, if we let \(u_i\) be the subword of \(w\) yielded by the subtree rooted at \(d_i\) for each \(i\), then \(w = u_1 \cdots u_m\). This is where we use the induction hypothesis. Let \(l(d_i) = y_i\) for each \(i\). Since the height of the tree rooted at \(d_i\) is strictly less than \(n\) (it doesn't include the root layer), there is a derivation \[ y_i \Rightarrow \mu_{i1} \Rightarrow \cdots \Rightarrow \mu_{ik_i} \Rightarrow u_{i} \] in \(\mathcal G\) for each \(i\). Putting all of this together for each \(i\), we obtain the following derivation: \[\begin{aligned} x &\Rightarrow y_1 y_2\cdots y_m && \text{derive \(u_1\) from \(y_1\)}\\ &\Rightarrow \mu_{11} y_2\cdots y_m \\ &\hspace{2em}\vdots \\ &\Rightarrow \mu_{1k_1} y_2\cdots y_m \\ &\Rightarrow u_1 y_2\cdots y_m && \text{now derive \(u_2\) from \(y_2\)}\\ &\Rightarrow u_1 \mu_{21} \cdots y_m \\ &\Rightarrow u_1 \mu_{22} \cdots y_m \\ &\hspace{2em}\vdots \\ &\Rightarrow u_1 u_2 \cdots y_m && \text{finally, derive \(u_m\) from \(y_m\)}\\ &\Rightarrow u_1 u_2 \cdots \mu_{m1} \\ &\hspace{2em}\vdots \\ &\Rightarrow u_1 u_2 \cdots u_m \\ &= w\\ \end{aligned}\] We have just derived \(w\) from \(x\), which is what we were trying to do.

Now let's use Claim 1 to prove that \(\mathcal L_{yield}(\mathcal G, x) \subseteq \mathcal L(\mathcal G, x)\). Let \(w \in \mathcal L_{yield}(\mathcal G, x)\). Then there is a parse tree for \(w\) in \(\mathcal G\) from \(x\) that yields \(w\). By Claim 1, there must also be a derivation of \(w\) from \(x\). Therefore, \(w \in \mathcal L(\mathcal G, x)\).