Extreme Value Theorem

In this document we will study the extreme value theorem, which is an important result regarding the existence of maximum and minimum values of a function defined in a closed and bounded interval. Before stating and proving the actual theorem, we will need to introduce a couple of intermediary results first.

Subsequences

Let $$ (x_n)_{x \in \mathbb{N}} $$

be a sequence of real numbers. We define $$ (x_{n_k})_{k \in \mathbb{N}} $$

to be the subsequence of $(x_n)$ where $(n_k)_{k \in \mathbb{N}}$ is a strictly increasing sequence of natural numbers. That is, a sequence whose terms satisfy the following inequality:

$$ n_1 < n_2 < \dotsc < n_k $$

As an example, consider the following alternating sequence: $$ a_n = (-1)^n $$

If we try to enumerate some of its terms, we get: $-1, 1, -1, 1$. Clearly, this sequence does not converge. Let’s now consider the sequence of even terms defined by $b_k = 2k$, we get the following subsequence:

$$ a_{b_n} = (-1)^{2k} = 1 $$

Therefore, the subsequence $(a_{2k})$ converges to $1$.

Bolzano-Weierstrass Theorem

The second prerequisite to prove the extreme value theorem is the Bolzano-Weierstrass theorem; it states the following:

Every bounded sequence has at least one convergent subsequence.

From the hypothesis, a bounded sequence $x_n$ admits two bounds $a,b \in \mathbb{R}$ such that: $$ a \leq x_n \leq b \ \ \forall n \in \mathbb{N} $$

In other words, we have an interval $I = [a,b]$ such that $x_n \in I$ $\forall n \in \mathbb{N}$. Let’s now apply the bisection method on $I$ by splitting it in two parts using the following midpoint:

$$ c = \frac{a + b}{2} $$

At least one of the two intervals $[a, c]$ or $[c, b]$ must contain infinitely many terms of the sequence $(x_n)$. Let’s choose such interval and call it $[a_1, b_1]$. We therefore have:

$$ b_1 - a_1 = \frac{b - a}{2} $$

Let’s now choose an index $n_1 \in \mathbb{N}$ of the $(x_n)$ sequence such that $(x_{n_1}) \in [a_1, b_1]$ and let’s split this interval using its midpoint:

$$ c_1 = \frac{a_1 + b_1}{2} $$

Just as before, at least on of the two intervals $[a_1, c_1]$ or $[c_1, b_1]$ must contain infinitely many terms of the $(x_n)$ sequence; let’s choose such interval and call it $[a_2, b_2]$. Since the interval $[a_2, b_2]$ contains infinitely many terms of the sequence, we can choose an index $n_2 > n_1$ such that $x_{n_2} \in [a_2, b_2]$.

If we iterate this process $\forall k \in \mathbb{N}$, we obtain three sequences $a_k, b_k$ and $x_{n_k}$ such that ($\clubsuit$):

$$ a_k \leq x_{n_k} \leq b_k \ \ \forall k \in \mathbb{N} $$

And ($\spadesuit$):

$$ b_k - a_k = \frac{b - a}{2^k} \ \ \forall k \in \mathbb{N} $$

Moreover, $(a_k)$ is a non-decreasing sequence while $b_k$ is a non-increasing sequence. From $\clubsuit$, we deduce that both $(a_k)$ and $(b_k)$ are monotone and bounded above by $b$ and below by $a$. Therefore, both sequences converge:

$$ \lim_{k \to +\infty} a_k = l_1, \ \ \lim_{k \to +\infty} b_k = l_2, \ \ l_1, l_2 \in \mathbb{R} $$

Now we need to prove that these two limits are equal. From $\spadesuit$, we notice that:

$$ \lim_{k \to +\infty} (b_k - a_k) = \lim_{k \to +\infty} \underbrace{\frac{b - a}{2^k}}_{\to 0} $$

Which implies that:

$$ l_2 - l_1 = 0 \iff l_2 = l_1 $$

Thus:

$$ \lim_{k \to +\infty} a_k = \lim_{k \to +\infty} b_k = l \in \mathbb{R} $$

Finally, since $\clubsuit$ holds and both the lower and upper bounds converge to $l$ as $k \to +\infty$, by the squeeze theorem, we conclude that:

$$ \lim_{k \to +\infty} x_{n_k} = l\\ \blacksquare $$

Extreme Value Theorem

We have now all the prerequisites needed to prove the extreme value theorem. Let’s begin with the statement:

Let $f: [a, b] \to \mathbb{R}$ be a continuous function defined on the closed and bounded interval $[a,b]$. We wish to show that $f$ attains a global maximum and a global minimum on $[a,b]$. That is: $$ \exists x_{\min}, x_{\max} \in [a,b], \\ f(x_{\min}) \leq f(x) \leq f(x_{\max}) \ \ \forall x \in [a, b] $$

Let’s first prove the existence of the maximum. Let $$ M = \sup \{ f(x) : x \in [a,b] \} $$

Since continuous functions on compact sets are bounded, $M$ is a finite real number, that is $M < +\infty$. By the definition of the supremum, for every $n \in \mathbb{N}$ there exists an $x_n \in [a, b]$ such that:

$$ M - \frac{1}{n} < f(x_n) \leq M $$

By taking the limit as $n \to +\infty$ and by the squeeze theorem, we get:

$$ \lim_{n \to +\infty} f(x_n) = M $$

Since the sequence $(x_n)$ is contained in the closed and bounded interval $[a, b]$, by the Bolzano-Weierstrass theorem, we can extract a convergent subsequence $(x_{n_k})$ such that:

$$ \lim_{k \to +\infty} x_{n_k} = x_0 \in [a, b] $$

and since $f$ is continuous at $x_0$, we obtain:

$$ f(x_0) = \lim_{k \to +\infty} f(x_{n_k}) = M $$

Thus, $f(x_0) = M$, meaning $M$ is an upper bound for $f([a, b])$. Therefore, $f(x_0)$ is the global maximum of $f$ on $[a, b]$. Let’s now apply the same reasoning to prove the existence of the global minimum. Let:

$$ m = \inf \{ f(x) : x \in [a, b] \} $$

Just like before, $m$ is a finite real number. By the definition of the infimum, for every $n \in \mathbb{N}$, there exists a sequence $x_n \in [a, b]$ such that:

$$ m \leq f(x_n) < m + \frac{1}{n} $$

If we take the limit as $n \to +\infty$, we get that:

$$ \lim_{n \to +\infty} f(x_n) = m $$

Again, by the Bolzano-Weierstrass theorem, we extract a subsequence $(x_{n_k})$ converging to some real value $x_1 \in [a, b]$. In other words:

$$ \lim_{k \to +\infty} x_{n_k} = x_1 $$

By the continuity of $f$, we obtain:

$$ f(x_1) = \lim_{k \to +\infty} f(x_{n_k}) = m $$

Thus, $f(x_1) = m$, meaning $m$ is a lower bound for $f([a, b])$. Therefore, $f(x_1)$ is the global minimum of $f$ on $[a, b] \ \blacksquare$.