Kleene's Theorem
We have already seen that \(\mathsf{Reg} \subseteq \mathsf{Fin}\). The proof of this inclusion entailed turning a regular expression \(r\) into a state of an automaton \(\mathcal A_{Ant}\) such that \(\mathcal L(\mathcal A_{Ant}, r) = \mathcal L(r)\). Today we're going to do the opposite: we are going to use a procedure (called Kleene's algeorithm) for turning a state in a finite automaton into a regular expression whose language semantics is the language accepted by the state. This procedure requires a bit of the algebra of regular expressions, which we explored last time, to solve certain systems of equations.
To get at the "systems of equations" part, let's do a little analysis of a couple automata.
Two two-state automata, \(\mathcal A_1\) and \(\mathcal A_2\).
The one on the left, \(\mathcal A_1\), has two states: \(x_1\) and \(x_2\). We are going to calculate the language that \(x_1\) accepts. Whatever it is, call it \(L_1 = \mathcal L(\mathcal A_1, x_1)\). We also need a name for the language \(x_2\) accepts, so let's call that \(L_2 = \mathcal L(\mathcal A_2, x_2)\).
Observing the transitions in \(\mathcal A_1\) out of \(x_1\), one sees that for any word \(w\) accepted by \(x_2\), we find the word \(aw \in L_1\). This is simply because \(x_1 \xrightarrow{a} x_2\). This ties the two languages together: What we have just deduced is that \(\{a\} \cdot L_2 \subseteq L_1\). Similarly, for any word \(w\) accepted by \(x_1\), \(bw\) is also accepted by \(x_1\), again because \(x_1 \xrightarrow{b} x_1\). So also, \(\{b\} \cdot L_1 \subseteq L_1\).
Since we have exhausted the outgoing transitions from \(x_1\), there are no other words in \(L_1\), so we have arrived at the following equation: \[ L_1 = (\{a\} \cdot L_2) \cup (\{b\} \cdot L_1) \] In terms of regular expressions, if \(s_1\) is a regular expression whose semantics is \(L_1\) and \(s_2\) is a regular expression whose semantics is \(L_2\), then the equation above can be written \[ s_1 =_{\mathcal L} as_2 + bs_1 \] Now, check it out! Since \(b\) does not have the empty word property, we can use Arden's rule to simplify this equation: \[\begin{aligned} \overbrace{\framebox{\(s_1\)}}^s &=_{\mathcal L} \overbrace{(as_2)}^t + \overbrace{(b)}^r \overbrace{\framebox{\(s_1\)}}^s \\ &=_{\mathcal L} \overbrace{b^*}^{r^*}\overbrace{(a s_2)}^t && \text{(Arden's Rule)}\\ &=_{\mathcal L} b^*as_2 \end{aligned}\] So it remains to figure out what \(s_2\) is.
The situation for \(s_2\) is only slightly different. It's true that \(\{a\} \cdot L_1 \subseteq L_2\), and similarly that \(\{a\} \cdot L_2 \subseteq L_2\) and \(\{b\} \cdot L_2 \subseteq L_2\). But this isn't quite all of \(L_2\): we also know that \(\varepsilon \in L_2\). Altogether, that makes \[ L_2 = (\{\varepsilon\}) \cup (\{a\}\cdot L_1) \cup (\{a\}\cdot L_2) \cup (\{b\} \cdot L_2) \] In terms of regular expressions, this gives the equation \[ s_2 =_{\mathcal L} \varepsilon + as_2 + bs_2 \] Let's collect like terms. \[\begin{aligned} s_2 &=_{\mathcal L} \varepsilon + as_2 + bs_2 \\ &=_{\mathcal L} \varepsilon + (a + b)s_2 && \text{(collect like terms)} \end{aligned}\]
Neither \(a\) nor \(b\) have the empty word property, so neither does \(a + b\). This allows us to use Arden's rule again: \[\begin{aligned} \overbrace{\framebox{\(s_2\)}}^s &=_{\mathcal L} \overbrace{\varepsilon}^t + \overbrace{(a + b)}^r \overbrace{\framebox{\(s_2\)}}^s \\ &=_{\mathcal L} \overbrace{(a + b)^*}^{r^*} \overbrace{\varepsilon}^t && \text{(Arden's Rule)}\\ &=_{\mathcal L} (a + b)^* && \text{(\(\varepsilon\) sequential composition)} \end{aligned}\] This has no indeterminate variables, so we have just derived a regular expression for \(s_2\)!
But what about \(s_1\)? Well, now all you have to do is plug this back into the equation we derived for \(s_1\): \[ s_1 =_{\mathcal L} b^*as_2 =_{\mathcal L} b^*a(a + b)^* \] Using algebraic reasoning, we have come to the following conclusion: that the language accepted by \(x_1\) is the semantics of the regular expression... \[ \mathcal L(\mathcal A_1, x_1) = \mathcal L(s_1) = \mathcal L( b^*a(a + b)^* ) \] This regular expression should not come as too much of a surprise: its language semantics includes words that start with some number of \(b\)s, then an \(a\), and then anything that comes after is ignored. This is precisely the language that \(x_1\) accepts. Neat, huh?
In the example above, we used Arden's rule a bunch to fish out a regular expression from a system of equations that we obtained from an automaton. Altogether, the system of equations in the example was (and I have moved terms around a little bit) \[\begin{aligned} s_1 &= bs_1 + as_2 \\ s_2 &=_{\mathcal L} \varepsilon + as_1 + (a + b)s_2 \end{aligned}\] and to find the regular expression that corresponds to \(s_1\) we needed to eliminate the indeterminate variables, \(s_1\) and \(s_2\), from the right-hand side of each equation.
It turns out that every system of equations of this form has a solution.
A solution to the left-affine system of equations above is a sequence of regular expressions \(s_1, s_2, \dots, s_n\) such that \[\begin{aligned} s_1 &=_{\mathcal L} b_1 + r_{11} s_1 + r_{12} s_2 + \cdots + r_{1n} s_n \\ s_2 &=_{\mathcal L} b_2 + r_{21} s_1 + r_{22} s_2 + \cdots + r_{2n} s_n \\ &\hspace{2em} \vdots \\ s_n &=_{\mathcal L} b_n + r_{n1} s_1 + r_{n2} s_2 + \cdots + r_{nn} s_n \end{aligned}\]
Let \(\mathcal A = (Q, A, \delta, F)\) be a finite automaton, and write \(Q = \{x_1, x_2, \dots, x_n\}\). The left-affine system from \(\mathcal A\), \(\mathcal S(\mathcal A)\), is given by \[\begin{aligned} x_1 &= b_1 + r_{11} x_1 + r_{12} x_2 + \cdots + r_{1n} x_n \\ x_2 &= b_2 + r_{21} x_1 + r_{22} x_2 + \cdots + r_{2n} x_n \\ &\hspace{2em} \vdots \\ x_n &= b_n + r_{n1} x_1 + r_{n2} x_2 + \cdots + r_{nn} x_n \end{aligned}\] where \[ b_i = \begin{cases} \varepsilon &\text{if \(x_i \in F\)} \\ \emptyset &\text{if \(x_i \notin F\)} \end{cases} \] and \[ r_{ij} = \sum_{(x_i, a, x_j) \in \delta} a \]
That last bit of notation might be confusing. There, I am trying to generally say that if the transitions between states \(x_i\) and \(x_j\) are \(x_i \xrightarrow{a_1} x_j\), \(x_i \xrightarrow{a_2} x_j\), \(\dots\), \(x_i \xrightarrow{a_k} x_j\), then \(r_{ij} = a_1 + a_2 + \dots + a_k\). If there are no such transitions, then \(r_{ij} = \emptyset\).
- The all-accepting automaton, \(\mathcal A_\checkmark = (\{s_0\}, \{a, b\}, \{(s_0, a, s_0), (s_0, b, s_0)\}, \{s_0\})\)
- The sink automaton, \(\mathcal A_\bullet = (\{s_0\}, \{a, b\}, \{\}, \{\})\)
- The diverging loop, \(\mathcal A_\circlearrowright = (\{s_0\}, \{a, b\}, \{(s_0, a, s_0), (s_0, b, s_0)\}, \{\})\)
- The automaton that only accepts, \(\mathcal A_\varepsilon = (\{s_0\}, \{a, b\}, \{\}, \{s_0\})\)
The next theorem states that solving the left-affine system of equations for an automaton finds you all the regular expressions that correspond to its states. The reason is a kind of it-works-by-design type deal, as we saw in the Finding a Regular Expression Example.
Kleene's Algorithm and Kleene's Theorem
It turns out that every left-affine system of equations has a (in fact, unique) solution. The most important part of this section is Kleene's algorithm, which finds that solution always.
Kleene's algorithm is recursive, which means that it has a base case and a recursive step. Basically, you recurse on the number of indeterminate variables in the left-affine system: if the number of variables is zero, then you are done. There are no indeterminate variables, so all of the regular expressions on the right-hand side are explicit regular expressions with no \(x_i\)s in them.
In the recursive step, you will have a left-affine system of the form \[\begin{aligned} x_1 &= b_1 + r_{11} x_1 + r_{12} x_2 + \cdots + r_{1n} x_n + r_{1(n+1)}x_{n+1} \\ x_2 &= b_2 + r_{21} x_1 + r_{22} x_2 + \cdots + r_{2n} x_n + r_{2(n+1)}x_{n+1} \\ &\hspace{2em} \vdots \\ x_n &= b_n + r_{n1} x_1 + r_{n2} x_2 + \cdots + r_{nn} x_n + r_{n(n+1)}x_{n+1} \\ x_{n+1} &= b_{n+1} + r_{(n+1)1} x_1 + r_{(n+1)2} x_2 + \cdots + r_{(n+1)n} x_n + r_{(n+1)(n+1)}x_{n+1} \end{aligned}\] The idea is to start as follows: observe that the \((n+1)\)th equation is an instance of Arden's rule, because \(r_{(n+1)(n+1)}\) does not have the empty word property. \[\begin{aligned} x_{n+1} &= \overbrace{b_{n+1} + r_{(n+1)1} x_1 + \cdots + r_{(n+1)n} x_n}^t + \overbrace{r_{(n+1)(n+1)}}^r \overbrace{x_{n+1}}^s \\ &= \overbrace{(r_{(n+1)(n+1)})^*}^{r^*}\overbrace{(b_{n+1} + r_{(n+1)1} x_1 + \cdots + r_{(n+1)n} x_n)}^t &&\text{(Arden's Rule)} \\ \end{aligned}\] Now we can distribute the \((r_{(n+1)(n+1)})^*\) to get \[ x_{n+1} = (r_{(n+1)(n+1)})^* b_{n+1} + (r_{(n+1)(n+1)})^*r_{(n+1)1} x_1 + \cdots + (r_{(n+1)(n+1)})^*r_{(n+1)n} x_n \] This has one fewer indeterminate variable! Now we can plug this back into the rest of the left-affine system to get a system of equations with only \(n\) indeterminate variables. \[\begin{aligned} x_1 &= b_1 + r_{11} x_1 + \cdots + r_{1n} x_n \\ & \hspace{3em} + r_{1(n+1)}((r_{(n+1)(n+1)})^* b_{n+1} + (r_{(n+1)(n+1)})^*r_{(n+1)1} x_1 + \cdots + (r_{(n+1)(n+1)})^*r_{(n+1)n} x_n) \\ x_2 &= b_2 + r_{21} x_1 + \cdots + r_{2n} x_n \\ & \hspace{3em} + r_{2(n+1)}((r_{(n+1)(n+1)})^* b_{n+1} + (r_{(n+1)(n+1)})^*r_{(n+1)1} x_1 + \cdots + (r_{(n+1)(n+1)})^*r_{(n+1)n} x_n) \\ &\hspace{2em} \vdots \\ x_n &= b_n + r_{n1} x_1 + \cdots + r_{nn} x_n \\ & \hspace{3em} + r_{n(n+1)}((r_{(n+1)(n+1)})^* b_{n+1} + (r_{(n+1)(n+1)})^*r_{(n+1)1} x_1 + \cdots + (r_{(n+1)(n+1)})^*r_{(n+1)n} x_n) \\ \end{aligned}\] Using distributivity and collecting like terms turns the above system of equations into a left-affine system of equations. For example, we can turn the first equation into \[\begin{aligned} x_1 &= (b_1 + r_{1(n+1)}(r_{(n+1)(n+1)})^* b_{n+1}) + (r_{11} + r_{1(n+1)}(r_{(n+1)(n+1)})^*r_{(n+1)1}) x_1 \\ &\hspace{3em} + \cdots + (r_{1n} + r_{1(n+1)}(r_{(n+1)(n+1)})^*r_{(n+1)n}) x_n \\ \end{aligned}\]
This is how to find a regular expression \(s_1\) for \(x_1\). To find the regular expressions for the other \(x_i\), plug \(s_1\) into the equation for \(x_2\) to find its regular expression \(s_2\). Then plug \(s_1\) and \(s_2\) into the equation for \(x_3\), and so on. Is it pretty? No, but it does work!
Kleene's algorithm tells us that we need to deal with the last equation first. Since \(\mathcal L(\emptyset)\) does not have the empty word property, we use Arden's rule to eliminate the \(y_2\) on the right-hand side. We get the following calculation out of this, where I've included some simplifying (using the Using the Basic Equations exercise from earlier). \[\begin{aligned} y_2 &= \varepsilon + (a + b)y_1 + \emptyset y_2 \\ &= \emptyset^*(\varepsilon + (a + b)y_1) &&\text{(Arden's Rule)} \\ &= \varepsilon(\varepsilon + (a + b)y_1) &&\text{(Using the Basic Equations)} \\ &= \varepsilon + (a + b)y_1 &&\text{(\(\varepsilon\) sequential composition)} \end{aligned}\] Now Kleene's algorithm tells us we want to plug this back into the first equation and collect like terms: \[\begin{aligned} y_1 &= \emptyset + ay_1 + by_2 \\ &= \emptyset + ay_1 + b(\varepsilon + (a + b)y_1) \\ &= \emptyset + ay_1 + b\varepsilon + b(a + b)y_1 &&\text{(distribute)} \\ &= \emptyset + ay_1 + b + b(a + b)y_1 &&\text{(\(\varepsilon\) seq comp)} \\ &= \emptyset + b + ay_1 + b(a + b)y_1 &&\text{(union rules)} \\ &= (\emptyset + b) + (a + b(a + b))y_1 &&\text{(collect like terms)} \end{aligned}\] Kleene's algorithm now tells us to use Arden's rule to finish the calculation. \[\begin{aligned} y_1 &= (\emptyset + b) + (a + b(a + b))y_1 \\ &= (a + b(a + b))^*(\emptyset + b) &&\text{(Arden's Rule)} \\ \end{aligned}\] And viola! One regular expression whose semantics is the language accepted by \(y_1\) is \((a + b(a + b))^*(\emptyset + b)\)! Of course, we can simplify using the \(\emptyset\) union rule to eliminate \(\emptyset\) above, and as a result we get \[ \mathcal L(\mathcal A_2, y_1) = \mathcal L((a + b(a + b))^*(\emptyset + b)) = \mathcal L((a + b(a + b))^*b) \] This means that we proceed to the next step, which is to plug our solution to the first equation into the second: \[\begin{aligned} y_2 &= \varepsilon + (a + b)y_1 \\ &= \varepsilon + (a + b)((a + b(a + b))^*b) &&\text{(plug \(y_1\) into \(y_2\))} \end{aligned}\]
Because it guarantees a solution always, Kleene's algorithm is a good one to practice a couple times. It's not hard to understand that it works, I think, but using it can be a bit tricky.
The significance of Kleene's algorithm, left-affine systems and their solutions, and Antimirov derivatives, is that together they land us at the following theorem.
- \((01^*)^*0\)
- \((0 + 1^*)0(1^* + 0)\)
- \(((ab^*)^* + c)^*\)
An automaton \(\mathcal A\) with a state \(x_0\) that accepts a string of \(0\)s and \(1\)s if and only if every instance of \(01\) is eventually followed by \(0\).
- Use Kleene's algorithm to derive a regular expression \(s_0 \in \mathit{RExp}\) such that \(\mathcal L(\mathcal A, x_0) = \mathcal L(s_0)\).
- Draw the portion of the Antimirov automaton generated by \(s_0\), \(\mathcal A' = \langle s_0\rangle_{\mathcal A_{Ant}}\). There is a state \(y_0\) is the Antimirov automaton such that \(\mathcal A = \langle y_0\rangle_{\mathcal A'}\). Find \(y_0\).