CSCI 341 Theory of Computation

Fall 2025, with Schmid

\(\looparrowleft\) finite and infinite automata determinization \(\leadsto\)

Finitely Recognizable Languages

Brzozowski's Fixed Point Theorem tells us that every language is accepted by some state of some automaton. In particular, the language \(L\) is accepted by the state \(L\) in the Brzozowski automaton \(\mathcal A_{Brz}\). One issue with this theorem is that it doesn't tell you how big that automaton might be. In fact, it could be infinite for all we know!

But here's the thing: our goal in this course is to find a mathematical model of computers as they exist IRL. Physical computers are finite: the memory is finite, so the number of programs they can run is finite. If you'll recall, states are programs in our models of computers, so physical computers are like automata but with finitely many states!

This brings us to the question: which languages can be accepted by a state in a finite automaton?

(Finitely Recognizable) A language \(L\subseteq A^*\) is finitely recognizable if there is a finite automaton \(\mathcal A = (Q, A, \delta, F)\) (i.e., \(Q\) is finite) and a state \(x \in Q\) such that \(L = \mathcal L(\mathcal A, x)\).

In this first part of the course, the Automata and Languages part, we are going to focus our attention on the finitely recognizable languages, their properties, and how to program them, in the following sense: if you want to show that a language is finitely recognizable, then you need to find a finite automaton with a state that recognizes the language. We have already done this for a ton of different languages.

(A First Go) Consider the alphabet \(A = \{0, 1\}\). Show that the language of all strings, \(L = A^*\), is finitely recognizable.

(Languages as Trees) Let \(A = \{0, 1\}\), and let \(L \subseteq A^*\) be a language from \(A\). Prove that if \(L\) is finite, i.e., \(L = \{w_1, \dots, w_n\}\) for some \(n \in \mathbb N\), then \(L\) is finitely recognizable.

Draw a tree with \(n\) different branches, where the nodes of the tree are states. Alternatively, you could give an upper bound on the number of states reachable from \(L\) in the Brzozowski automaton; see Reachability and Local Finiteness below.

We have seen all sorts of finitely recognizable languages so far, because we have seen all sorts of finite automata. But we have also seen different types of automata: automata with properties like determinism and totality. We can also ask the question, which languages are accepted by total/deterministic automata? After all, physical computers are total and deterministic (think about this for a second). Nondeterminism probably came as a surprise when we allowed for it in our model of computation! So far, we've just used partiality and nondeterminism as a convenient tool for drawing smaller automata than the total deterministic ones. The languages recognized by each of these types of automata form families of languages, and it would be worthwhile to know which is which.

(Families of Languages, Finitely Recognizable) A family of languages is a set of languages \(\mathsf{Fam} \subseteq 2^{A^*}\) over a fixed alphabet of input letters \(A\).

A word is a string of input letters. A language is a set of words. A family of languages is a set of sets of words. If this is a bit much, the following exercise may help to put families of languages into perspective.

(Family Unit) For each of the families of languages below, find an automaton \(\mathcal A = (\{x, y, z\}, \{a, b, c\}, \delta, F)\) such that \(\mathsf{Fam}_i = \{\mathcal L(\mathcal A, x), \mathcal L(\mathcal A, y), \mathcal L(\mathcal A, z)\}\).

\(\mathsf{Fam}_1 = \big\{~ \{\}, \{\varepsilon\}, \{a\} ~\big\}\)
\(\mathsf{Fam}_3 = \big\{~ \{b(ab)^n \mid n \in \mathbb N\}, \{(ab)^n \mid n \in \mathbb N\}~\big\}\)
A bit more challenging: \[\begin{aligned} \mathsf{Fam}_3 &= \big\{ \quad \{\varepsilon\}, \\ &\hspace{3em} \{(ab)^nc \mid n \in \mathbb N\} \cup \{a(ba)^nc \mid n \in \mathbb N\}, \\ &\hspace{3em} \{(ba)^nc \mid n \in \mathbb N\} \cup \{b(ab)^nc \mid n \in \mathbb N\} \quad\big\} \end{aligned} \]

Remember that \((ab)^0 = \varepsilon\), \((ab)^1 = ab\), \((ab)^2 = abab\), \((ab)^3 = ababab\), and in general, \((ab)^{n+1} = ab(ab)^n\) for any \(n \in \mathbb N\). So the words in \(\{b(ab)^n \mid n \in \mathbb N\}\) are \(b, bab, babab, bababab, \dots\)

Altogether, we can split the languages we care about into three families.

(Finitely Recognizable) Let \(A\) be an alphabet, and \(L \subseteq A^*\) a language.

\(L\) is finitely recognizable if there is a finite automaton \(\mathcal A = (Q, A, \delta, F)\) (i.e., \(Q\) is finite) and a state \(x \in Q\) such that \(L = \mathcal L(\mathcal A, x)\). The family of finitely recognizable languages is \[ \mathsf{Fin} = \big\{ L \subseteq A^* \mid \text{\(L\) is finitely recognizable} \big\} \]
\(L\) is deterministic finitely recognizable if there is a deterministic finite automaton \(\mathcal A = (Q, A, \delta, F)\) and a state \(x \in Q\) such that \(L = \mathcal L(\mathcal A, x)\). The family of deterministic finitely recognizable languages is \[ \mathsf{DFin} = \big\{ L \subseteq A^* \mid \text{\(L\) is deterministic finitely recognizable} \big\} \]
\(L\) is total deterministic finitely recognizable if there is a total deterministic finite automaton \(\mathcal A = (Q, A, \delta, F)\) and a state \(x \in Q\) such that \(L = \mathcal L(\mathcal A, x)\). The family of total deterministic finitely recognizable languages is \[ \mathsf{TDFin} = \big\{ L \subseteq A^* \mid \text{\(L\) is total deterministic finitely recognizable} \big\} \]

Something you should notice immediately: every total deterministic automaton is a deterministic automaton, and every deterministic automaton is an automaton. This means implies that \[ \mathsf{TDFin} \subseteq \mathsf{DFin} \subseteq \mathsf{Fin} \] right off the bat! But what about the reverse inclusions?

(CJ's Pergatory) For each of the deterministic partial automata below, find a total deterministic automaton with a state that accepts the same language as \(x\). In each example, the input alphabet is \(A = \{a, b, c\}\).

(1)

(2)

(3)

(4)

You only ever need to add one state!

(Total vs Partial) Prove that \(\mathsf{DFin} = \mathsf{TDFin}\) by describing how to turn a deterministic automaton into a total deterministic automaton without changing the languages accepted by the states.

What did you do in the CJ's Pergatory exercise every time?

In the next lecture, we are going to see that in fact, all three families of languages are equal!

Reachability and Local Finiteness

The Brzozowski automaton was most definitely an infinite automaton. But something very special happened in the Unravelling a Language problem: the states reachable from that particular language were finite in number, and using that fact we were able to show that that language is finitely recognizable. We just had to draw the finite automaton consisting of the states reachable from \(L = \{\varepsilon, aa, ba, cab, c, acab\}\), of which there are finitely many. The states reachable from a given state in an automaton form a subautomaton generated by the state.

(Generated, Locally Finite) Let \(\mathcal A = (Q, A, \delta, F)\) be an automaton and let \(x \in Q\) be a state. The automaton generated by \(x\) is the automaton \(\langle x \rangle_{\mathcal A} = (Q_x, A, \delta_x, F_x)\) defined by \[\begin{aligned} Q_x &= \bigcup_{w \in A^*} \delta(x, w) \\ \delta_x &= \delta \cap (Q_x \times A \times Q_x) \\ F_x &= F \cap Q_x \\ \end{aligned}\] The automaton \(\mathcal A\) is called locally finite if \(\langle x \rangle_{\mathcal A}\) is a finite automaton for every state \(x \in Q\).

Above, the notation \(\bigcup_{w \in A^*} \delta(x, w)\) is shorthand for the set \[ \bigcup_{w \in A^*} \delta(x, w) = \{y \mid \text{there is a word \(w \in A^*\) such that \(y \in \delta(x, w)\)}\} \] There is a lot of notation going on in that definition, but the idea is pretty simple: \(\langle x\rangle_{\mathcal A}\) is the automaton that you get by deleting all of the states that are not reachable from \(x\) by some path (of length \(0\) or more).

(Computing Reachable Parts) Consider the automaton \(\mathcal A = (Q, A, \delta, F)\) given by the state diagram below.

The automaton \(\mathcal A = (Q, A, \delta, F)\). Here, \(A = \{0, 1\}\).

Circle the automata \(\langle x_0 \rangle_{\mathcal A}\), \(\langle x_1 \rangle_{\mathcal A}\), \(\langle x_2 \rangle_{\mathcal A}\), \(\langle x_3 \rangle_{\mathcal A}\), and \(\langle x_4 \rangle_{\mathcal A}\), generated by each of the states of \(\mathcal A\).

The definition of \(\delta_x\) just says that it has all the same transitions as \(\delta\) but restricted to \(Q_x\). Note that \(\langle x \rangle_{\mathcal A}\) has the same alphabet as \(\mathcal A\).

(Reverse Naturals Again) Remember that the Reverse Naturals automaton was defined by \(Q = \mathbb N\), \(A = \{l\}\), and \(n + 1 \xrightarrow{l} n\) for all \(n \in Q\).

Is the Reverse Naturals automaton locally finite?
What if we replace \(l\) with the the input letter \(s\) that adds one instead of subtracts: \(n \xrightarrow{s} n + 1\)? What does the state diagram look like now? Is it locally finite?

We are not going to prove the theorem below (Brzozowski is Minimal) in this course. The proof would be a detour through several other concepts like bisimilarity and quotient automata, which would take us too far off course. It would be more appropriate in a course just about automata theory, but we have got places to be! But in any case, it is helpful to keep in mind, because it can guide you through the process of determining whther a language is total deterministic finitely recognizable or not.

(Minimal) A total deterministic automaton \(\mathcal A = (Q, A, \delta, F)\) is minimal if for any states \(x,y \in Q\), if \(\mathcal L(\mathcal A, x) = \mathcal L(\mathcal A, y)\), then \(x = y\). In other words, there are no two distinct states that accept the same language in \(\mathcal A\).

Given a language \(L\) and a state \(x \in Q\) in a minimal total deterministic automaton \(\mathcal A\) such that \(L = \mathcal L(\mathcal A, x)\), \(\langle x \rangle_{\mathcal A}\) is the smallest automaton with a state that accepts \(L\). This is where the following theorem comes from.

(Brzozowski is Minimal) The Brzozowski automaton \(\mathcal A = (2^{A^*}, A, \delta, F)\) is minimal. In particular, for any language \(L\), \(\langle L \rangle_{\mathcal A_{Brz}}\) is the smallest automaton that accepts \(L\). Thus, \(L\) is total deterministic finitely recognizable if and only if \(\langle L \rangle_{\mathcal A_{Brz}}\) is finite.

(A=B) Recall that in the Brzozowski automaton \(\mathcal A_{Brz}\), the states are languages, the transitions are given by derivatives, \(L \xrightarrow{a} a^{-1}L\), and the accepting states are the languages \(L\subseteq A^*\) that contain the empty word \(\varepsilon \in L\). The following language is a classic example of a language that generates an infinite subautomaton of the Brzozowski automaton \(\mathcal A_{Brz}\), and therefore is not totally deterministic finitely recognizable: \[ L_{a=b} = \big\{ a^nb^n \mid n \in \mathbb N\big\} \] After \(k\) \(a\)-derivatives, the language becomes \[ a^{-k}L_{a=b} = \big\{ a^{n-k}b^n \mid n \in \mathbb N, n \ge k\big\} \] If \(k < k'\), then \(a^{-k}L_{a=b} \neq a^{-k'}L_{a=b}\), so there is a distinct state in \(\langle L_{a=b}\rangle_{\mathcal A_{Brz}}\) for each \(k \in \mathbb N\). This means infinitely many!

(Distinct Derivatives of A=B) Show that \(a^{-1}L_{a=b} \neq a^{-2}L_{a=b}\) by finding a word in one of the languages that is not in the other. How would you generalize this to a proof that if \(k < k'\), then \(a^{-k}L_{a=b} \neq a^{-k'}L_{a=b}\)?

It turns out that \(L_{a=b}\) is not accepted by any state in any finite automaton. We don't quite have the tools to prove this yet, though. (We will see those later when we talk about the Pumping Lemma for finite automata.)

(Reachable = Generated) Let \(\mathcal A = (Q, A, \delta, F)\) be an automaton and let \(x \in Q\) be a state. Prove the following three statements using the definition of \(\langle x \rangle_{\mathcal A}\) and \(\delta(x, w)\):

\(Q_x = \{ y \in Q \mid \text{there is a path \(x \xrightarrow{a_1} x_1 \xrightarrow{a_2} \cdots \xrightarrow{a_n} y\) in \(\mathcal A\)} \}\)
for any \(y,z \in Q_x\), \(y \xrightarrow{a} z\) in \(\mathcal A\) if and only if \(y \xrightarrow{a} z\) in \(\langle x \rangle_{\mathcal A}\)
for any \(y \in Q_x\), \(y\in F\) if and only if \(y \in F_x\)

Explain why this implies that \(\mathcal L(\mathcal A, x) = \mathcal L(\langle x \rangle_{\mathcal A}, x)\).

Remember that to prove a statement like "\(P\) if and only if \(Q\)", you need to write two separate proofs: you need to prove (1) that "if \(P\), then \(Q\)", and (2) that "if \(Q\), then \(P\)".

Top