align fix; theorem style; condition number

This commit is contained in:
jverzani
2024-10-31 14:22:21 -04:00
parent 3e7e3a9727
commit 18aae2aa93
61 changed files with 1705 additions and 819 deletions

View File

@@ -36,12 +36,13 @@ nothing
Consider a function $f: R^n \rightarrow R$. It has multiple arguments for its input (an $x_1, x_2, \dots, x_n$) and only one, *scalar*, value for an output. Some simple examples might be:
$$
\begin{align*}
f(x,y) &= x^2 + y^2\\
g(x,y) &= x \cdot y\\
h(x,y) &= \sin(x) \cdot \sin(y)
\end{align*}
$$
For two examples from real life consider the elevation Point Query Service (of the [USGS](https://nationalmap.gov/epqs/)) returns the elevation in international feet or meters for a specific latitude/longitude within the United States. The longitude can be associated to an $x$ coordinate, the latitude to a $y$ coordinate, and the elevation a $z$ coordinate, and as long as the region is small enough, the $x$-$y$ coordinates can be thought to lie on a plane. (A flat earth assumption.)
@@ -631,23 +632,25 @@ Before answering this, we discuss *directional* derivatives along the simplified
If we compose $f \circ \vec\gamma_x$, we can visualize this as a curve on the surface from $f$ that moves in the $x$-$y$ plane along the line $y=c$. The derivative of this curve will satisfy:
$$
\begin{align*}
(f \circ \vec\gamma_x)'(x) &=
\lim_{t \rightarrow x} \frac{(f\circ\vec\gamma_x)(t) - (f\circ\vec\gamma_x)(x)}{t-x}\\
&= \lim_{t\rightarrow x} \frac{f(t, c) - f(x,c)}{t-x}\\
&= \lim_{h \rightarrow 0} \frac{f(x+h, c) - f(x, c)}{h}.
\end{align*}
$$
The latter expresses this to be the derivative of the function that holds the $y$ value fixed, but lets the $x$ value vary. It is the rate of change in the $x$ direction. There is special notation for this:
$$
\begin{align*}
\frac{\partial f(x,y)}{\partial x} &=
\lim_{h \rightarrow 0} \frac{f(x+h, y) - f(x, y)}{h},\quad\text{and analogously}\\
\frac{\partial f(x,y)}{\partial y} &=
\lim_{h \rightarrow 0} \frac{f(x, y+h) - f(x, y)}{h}.
\end{align*}
$$
These are called the *partial* derivatives of $f$. The symbol $\partial$, read as "partial", is reminiscent of "$d$", but indicates the derivative is only in a given direction. Other notations exist for this:
@@ -685,11 +688,12 @@ Let $f(x,y) = x^2 - 2xy$, then to compute the partials, we just treat the other
Then
$$
\begin{align*}
\frac{\partial (x^2 - 2xy)}{\partial x} &= 2x - 2y\\
\frac{\partial (x^2 - 2xy)}{\partial y} &= 0 - 2x = -2x.
\end{align*}
$$
Combining, gives $\nabla{f} = \langle 2x -2y, -2x \rangle$.
@@ -697,12 +701,13 @@ Combining, gives $\nabla{f} = \langle 2x -2y, -2x \rangle$.
If $g(x,y,z) = \sin(x) + z\cos(y)$, then
$$
\begin{align*}
\frac{\partial g }{\partial x} &= \cos(x) + 0 = \cos(x),\\
\frac{\partial g }{\partial y} &= 0 + z(-\sin(y)) = -z\sin(y),\\
\frac{\partial g }{\partial z} &= 0 + \cos(y) = \cos(y).
\end{align*}
$$
Combining, gives $\nabla{g} = \langle \cos(x), -z\sin(y), \cos(y) \rangle$.
@@ -938,12 +943,17 @@ where $\epsilon(h) \rightarrow 0$ as $h \rightarrow 0$.
It is this characterization of differentiable that is generalized to define when a scalar function is *differentiable*.
::: {.callout-note icon=false}
## Differentiable
> *Differentiable*: Let $f$ be a scalar function. Then $f$ is [differentiable](https://tinyurl.com/qj8qcbb) at a point $C$ **if** the first order partial derivatives exist at $C$ **and** for $\vec{h}$ going to $\vec{0}$:
>
> $\|f(C + \vec{h}) - f(C) - \nabla{f}(C) \cdot \vec{h}\| = \mathcal{o}(\|\vec{h}\|),$
>
> where $\mathcal{o}(\|\vec{h}\|)$ means that dividing the left hand side by $\|\vec{h}\|$ and taking a limit as $\vec{h}\rightarrow 0$ the limit will be $0$.
Let $f$ be a scalar function. Then $f$ is [differentiable](https://tinyurl.com/qj8qcbb) at a point $C$ **if** the first order partial derivatives exist at $C$ **and** for $\vec{h}$ going to $\vec{0}$:
$$
\|f(C + \vec{h}) - f(C) - \nabla{f}(C) \cdot \vec{h}\| = \mathcal{o}(\|\vec{h}\|),
$$
where $\mathcal{o}(\|\vec{h}\|)$ means that dividing the left hand side by $\|\vec{h}\|$ and taking a limit as $\vec{h}\rightarrow 0$ the limit will be $0$.
:::
@@ -962,8 +972,12 @@ Later we will see how Taylor's theorem generalizes for scalar functions and inte
In finding a partial derivative, we restricted the surface along a curve in the $x$-$y$ plane, in this case the curve $\vec{\gamma}(t)=\langle t, c\rangle$. In general if we have a curve in the $x$-$y$ plane, $\vec{\gamma}(t)$, we can compose the scalar function $f$ with $\vec{\gamma}$ to create a univariate function. If the functions are "smooth" then this composed function should have a derivative, and some version of a "chain rule" should provide a means to compute the derivative in terms of the "derivative" of $f$ (the gradient) and the derivative of $\vec{\gamma}$ ($\vec{\gamma}'$).
> *Chain rule*: Suppose $f$ is *differentiable* at $C$, and $\vec{\gamma}(t)$ is differentiable at $c$ with $\vec{\gamma}(c) = C$. Then $f\circ\vec{\gamma}$ is differentiable at $c$ with derivative $\nabla f(\vec{\gamma}(c)) \cdot \vec{\gamma}'(c)$.
::: {.callout-note icon=false}
## Chain rule
Suppose $f$ is *differentiable* at $C$, and $\vec{\gamma}(t)$ is differentiable at $c$ with $\vec{\gamma}(c) = C$. Then $f\circ\vec{\gamma}$ is differentiable at $c$ with derivative $\nabla f(\vec{\gamma}(c)) \cdot \vec{\gamma}'(c)$.
:::
This is similar to the chain rule for univariate functions $(f\circ g)'(u) = f'(g(u)) g'(u)$ or $df/dx = df/du \cdot du/dx$. However, when we write out in components there are more terms. For example, for $n=2$ we have with $\vec{\gamma} = \langle x(t), y(t) \rangle$:
@@ -1217,7 +1231,10 @@ Let $f(x,y) = \sin(x+2y)$ and $\vec{v} = \langle 2, 1\rangle$. The directional d
$$
\nabla{f}\cdot \frac{\vec{v}}{\|\vec{v}\|} = \langle \cos(x + 2y), 2\cos(x + 2y)\rangle \cdot \frac{\langle 2, 1 \rangle}{\sqrt{5}} = \frac{4}{\sqrt{5}} \cos(x + 2y).
\nabla{f}\cdot \frac{\vec{v}}{\|\vec{v}\|} =
\langle \cos(x + 2y), 2\cos(x + 2y)\rangle \cdot
\frac{(\langle 2, 1 \rangle)}{\sqrt{5}} =
\frac{4}{\sqrt{5}} \cos(x + 2y).
$$
##### Example
@@ -1408,17 +1425,18 @@ Let $f(x,y) = x^2 + y^2$ be a scalar function. We have if $G(r, \theta) = \langl
Were this computed through the chain rule, we have:
$$
\begin{align*}
\nabla G_1 &= \langle \frac{\partial r\cos(\theta)}{\partial r}, \frac{\partial r\cos(\theta)}{\partial \theta} \rangle=
\langle \cos(\theta), -r \sin(\theta) \rangle,\\
\nabla G_2 &= \langle \frac{\partial r\sin(\theta)}{\partial r}, \frac{\partial r\sin(\theta)}{\partial \theta} \rangle=
\langle \sin(\theta), r \cos(\theta) \rangle.
\end{align*}
$$
We have $\partial f/\partial x = 2x$ and $\partial f/\partial y = 2y$, which at $G$ are $2r\cos(\theta)$ and $2r\sin(\theta)$, so by the chain rule, we should have
$$
\begin{align*}
\frac{\partial (f\circ G)}{\partial r} &=
\frac{\partial{f}}{\partial{x}}\frac{\partial G_1}{\partial r} +
@@ -1430,6 +1448,7 @@ We have $\partial f/\partial x = 2x$ and $\partial f/\partial y = 2y$, which at
\frac{\partial f}{\partial y}\frac{\partial G_2}{\partial \theta} =
2r\cos(\theta)(-r\sin(\theta)) + 2r\sin(\theta)(r\cos(\theta)) = 0.
\end{align*}
$$
## Higher order partial derivatives
@@ -1467,9 +1486,11 @@ In `SymPy` the variable to differentiate by is taken from left to right, so `dif
We see that `diff(ex, x, y)` and `diff(ex, y, x)` are identical. This is not a coincidence, as by [Schwarz's Theorem](https://tinyurl.com/y7sfw9sx) (also known as Clairaut's theorem) this will always be the case under typical assumptions:
::: {.callout-note icon=false}
## Theorem on mixed partials
> Theorem on mixed partials. If the mixed partials $\partial^2 f/\partial x \partial y$ and $\partial^2 f/\partial y \partial x$ exist and are continuous, then they are equal.
If the mixed partials $\partial^2 f/\partial x \partial y$ and $\partial^2 f/\partial y \partial x$ exist and are continuous, then they are equal.
:::
For higher order mixed partials, something similar to Schwarz's theorem still holds. Say $f:R^n \rightarrow R$ is $C^k$ if $f$ is continuous and all partial derivatives of order $j \leq k$ are continuous. If $f$ is $C^k$, and $k=k_1+k_2+\cdots+k_n$ ($k_i \geq 0$) then