align fix; theorem style; condition number

quarto/differentiable_vector_calculus/scalar_functions_applications.qmd

@@ -341,11 +341,12 @@ The level curve $f(x,y)=0$ and the level curve $g(x,y)=0$ may intersect. Solving
 To elaborate, consider two linear equations written in a general form:
 
 
-
+$$
 \begin{align*}
 ax + by &= u\\
 cx + dy &= v
 \end{align*}
+$$
 
 
 A method to solve this by hand would be to solve for $y$ from one equation, replace this expression into the second equation and then solve for $x$. From there, $y$ can be found. A more advanced method expresses the problem in a matrix formulation of the form $Mx=b$ and solves that equation. This form of solving is implemented in `Julia`, through the "backslash" operator. Here is the general solution:
@@ -422,21 +423,23 @@ We look to find the intersection point near $(1,1)$ using Newton's method
 We have by linearization:
 
 
-
+$$
 \begin{align*}
 f(x,y) &\approx f(x_n, y_n)  + \frac{\partial f}{\partial x}\Delta x + \frac{\partial f}{\partial y}\Delta y \\
 g(x,y) &\approx g(x_n, y_n)  + \frac{\partial g}{\partial x}\Delta x + \frac{\partial g}{\partial y}\Delta y,
 \end{align*}
+$$
 
 
 where $\Delta x = x- x_n$ and $\Delta y = y-y_n$. Setting $f(x,y)=0$ and $g(x,y)=0$, leaves these two linear equations in $\Delta x$ and $\Delta y$:
 
 
-
+$$
 \begin{align*}
 \frac{\partial f}{\partial x} \Delta x + \frac{\partial f}{\partial y} \Delta y &= -f(x_n, y_n)\\
 \frac{\partial g}{\partial x} \Delta x + \frac{\partial g}{\partial y} \Delta y &= -g(x_n, y_n).
 \end{align*}
+$$
 
 
 One step of Newton's method defines $(x_{n+1}, y_{n+1})$ to be the values $(x,y)$ that make the linearized functions about $(x_n, y_n)$ both equal to $\vec{0}$.
@@ -679,14 +682,18 @@ An *absolute* maximum over $U$, should it exist, would be $f(\vec{a})$ if there
 
 The difference is the same as the one-dimensional case: local is a statement about nearby points only, absolute a statement about all the points in the specified set.
 
+::: {.callout-note icon=false}
+## The [Extreme Value Theorem](https://tinyurl.com/yyhgxu8y)
 
-> The [Extreme Value Theorem](https://tinyurl.com/yyhgxu8y) Let $f:R^n \rightarrow R$ be continuous and defined on *closed* set $V$. Then $f$ has a minimum value $m$ and maximum value $M$ over $V$ and there exists at least two points $\vec{a}$ and $\vec{b}$ with $m = f(\vec{a})$ and $M = f(\vec{b})$.
+Let $f:R^n \rightarrow R$ be continuous and defined on *closed* set $V$. Then $f$ has a minimum value $m$ and maximum value $M$ over $V$ and there exists at least two points $\vec{a}$ and $\vec{b}$ with $m = f(\vec{a})$ and $M = f(\vec{b})$.
+:::
 
+::: {.callout-note icon=false}
+## [Fermat](https://tinyurl.com/nfgz8fz)'s theorem on critical points
 
+Let $f:R^n \rightarrow R$ be a continuous function defined on an *open* set $U$. If $x \in U$ is a point where $f$ has a local extrema *and* $f$ is differentiable, then the gradient of $f$ at $x$ is $\vec{0}$.
 
-> [Fermat](https://tinyurl.com/nfgz8fz)'s theorem on critical points. Let $f:R^n \rightarrow R$ be a continuous function defined on an *open* set $U$. If $x \in U$ is a point where $f$ has a local extrema *and* $f$ is differentiable, then the gradient of $f$ at $x$ is $\vec{0}$.
-
-
+:::
 
 Call a point in the domain of $f$ where the function is differentiable and the  gradient is zero a *stationary point* and a point in the domain where the function is either not differentiable or is a stationary point a *critical point*. The local extrema can only happen at critical points by Fermat.
 
@@ -735,16 +742,16 @@ To identify these through formulas, and not graphically, we could try and use th
 
 The generalization of the *second* derivative test is more concrete though. Recall, the second derivative test is about the concavity of the function at the critical point. When the concavity can be determined as non-zero, the test is conclusive; when the concavity is zero, the test is not conclusive. Similarly here:
 
+::: {.callout-note icon=false}
+## The [second](https://en.wikipedia.org/wiki/Second_partial_derivative_test) Partial Derivative Test for $f:R^2 \rightarrow R$.
 
-> The [second](https://en.wikipedia.org/wiki/Second_partial_derivative_test) Partial Derivative Test for $f:R^2 \rightarrow R$.
->
-> Assume the first and second partial derivatives of $f$ are defined and continuous; $\vec{a}$ be a critical point of $f$; $H$ is the hessian matrix, $[f_{xx}\quad f_{xy};f_{xy}\quad f_{yy}]$, and $d = \det(H) = f_{xx} f_{yy} - f_{xy}^2$ is the determinant of the Hessian matrix. Then:
->
->   * The function $f$ has a local minimum at $\vec{a}$ if $f_{xx} > 0$ *and* $d>0$,
->   * The function $f$ has a local maximum at $\vec{a}$ if $f_{xx} < 0$ *and* $d>0$,
->   * The function $f$ has a saddle point at $\vec{a}$ if $d < 0$,
->   * Nothing can be said if $d=0$.
+Assume the first and second partial derivatives of $f$ are defined and continuous; $\vec{a}$ be a critical point of $f$; $H$ is the hessian matrix, $[f_{xx}\quad f_{xy};f_{xy}\quad f_{yy}]$, and $d = \det(H) = f_{xx} f_{yy} - f_{xy}^2$ is the determinant of the Hessian matrix. Then:
 
+  * The function $f$ has a local minimum at $\vec{a}$ if $f_{xx} > 0$ *and* $d>0$,
+  * The function $f$ has a local maximum at $\vec{a}$ if $f_{xx} < 0$ *and* $d>0$,
+  * The function $f$ has a saddle point at $\vec{a}$ if $d < 0$,
+  * Nothing can be said if $d=0$.
+:::
 
 
 ---
@@ -1069,11 +1076,12 @@ $$
 
 Another might be the vertical squared distance to the line:
 
-
+$$
 \begin{align*}
 d2(\alpha, \beta) &= (y_1 - l(x_1))^2 + (y_2 - l(x_2))^2 + (y_3 - l(x_3))^2 \\
 &= (y1 - (\alpha + \beta x_1))^2 + (y2 - (\alpha + \beta x_2))^2 + (y3 - (\alpha + \beta x_3))^2
 \end{align*}
+$$
 
 Another might be the *shortest* distance to the line:
 
@@ -1407,18 +1415,22 @@ contour!(xs, ys, f,  levels = [.7, .85, 1, 1.15, 1.3])
 
 We can still identify the tangent and normal directions. What is different about this point is that local movement on the constraint curve is also local movement on the contour line of $f$, so $f$ doesn't increase or decrease here, as it would if this point were an extrema along the constraint. The key to seeing this is the contour lines of $f$ are *tangent* to the constraint. The respective gradients are *orthogonal* to their tangent lines, and in dimension $2$, this implies they are parallel to each other.
 
+::: {.callout-note icon=false}
+## The method of Lagrange multipliers
 
-> *The method of Lagrange multipliers*: To optimize $f(x,y)$ subject to a constraint $g(x,y) = k$ we solve for all *simultaneous* solutions to
->
->
-> \begin{align*}
-> \nabla{f}(x,y) &= \lambda \nabla{g}(x,y), \text{and}\\
-> g(x,y) &= k.
-> \end{align*}
->
->
-> These *possible* points are evaluated to see if they are maxima or minima.
+To optimize $f(x,y)$ subject to a constraint $g(x,y) = k$ we solve for all *simultaneous* solutions to
 
+$$
+\begin{align*}
+\nabla{f}(x,y) &= \lambda \nabla{g}(x,y), \text{and}\\
+g(x,y) &= k.
+\end{align*}
+$$
+
+
+These *possible* points are evaluated to see if they are maxima or minima.
+
+:::
 
 
 The method will not work if $\nabla{g} = \vec{0}$ or if $f$ and $g$ are not differentiable.
@@ -1472,12 +1484,13 @@ $$
 The we have
 
 
-
+$$
 \begin{align*}
 \frac{\partial L}{\partial{x}} &= \frac{\partial{f}}{\partial{x}} - \lambda \frac{\partial{g}}{\partial{x}}\\
 \frac{\partial L}{\partial{y}} &= \frac{\partial{f}}{\partial{y}} - \lambda \frac{\partial{g}}{\partial{y}}\\
 \frac{\partial L}{\partial{\lambda}} &= 0 + (g(x,y)  -  k).
 \end{align*}
+$$
 
 
 But if the Lagrange condition holds, each term is $0$, so Lagrange's method can be seen as solving for point $\nabla{L} = \vec{0}$. The optimization problem in two variables with a constraint becomes a problem of finding and classifying  zeros of a function with *three* variables.
@@ -1556,13 +1569,14 @@ The starting point is a *perturbation*: $\hat{y}(x) = y(x) + \epsilon_1 \eta_1(x
 With this notation, and fixing $y$ we can re-express the equations in terms of $\epsilon_1$ and $\epsilon_2$:
 
 
-
+$$
 \begin{align*}
 F(\epsilon_1, \epsilon_2) &= \int f(x, \hat{y}, \hat{y}') dx =
 \int f(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsilon_2 \eta_2') dx,\\
 G(\epsilon_1, \epsilon_2) &= \int g(x, \hat{y}, \hat{y}') dx =
 \int g(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsilon_2 \eta_2') dx.
 \end{align*}
+$$
 
 
 Then our problem is restated as:
@@ -1590,7 +1604,7 @@ $$
 Computing just the first one, we have using the chain rule and assuming interchanging the derivative and integral is possible:
 
 
-
+$$
 \begin{align*}
 \frac{\partial{F}}{\partial{\epsilon_1}}
 &= \int \frac{\partial}{\partial{\epsilon_1}}(
@@ -1598,6 +1612,7 @@ f(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsil
 &= \int \left(\frac{\partial{f}}{\partial{y}} \eta_1 + \frac{\partial{f}}{\partial{y'}} \eta_1'\right) dx\quad\quad(\text{from }\nabla{f} \cdot \langle 0, \eta_1, \eta_1'\rangle)\\
 &=\int \eta_1 \left(\frac{\partial{f}}{\partial{y}} - \frac{d}{dx}\frac{\partial{f}}{\partial{y'}}\right) dx.
 \end{align*}
+$$
 
 
 The last line by integration by parts:
@@ -1664,11 +1679,12 @@ ex2 = Eq(ex1.lhs()^2 - 1, simplify(ex1.rhs()^2) - 1)
 Now $y'$ can be integrated using the substitution $y - C = \lambda \cos\theta$ to give: $-\lambda\int\cos\theta d\theta = x + D$, $D$ some constant. That is:
 
 
-
+$$
 \begin{align*}
 x + D &=  - \lambda \sin\theta\\
 y - C &= \lambda\cos\theta.
 \end{align*}
+$$
 
 
 Squaring gives the equation of a circle: $(x +D)^2 + (y-C)^2 = \lambda^2$.
@@ -1680,11 +1696,12 @@ We center and *rescale* the problem so that $x_0 = -1, x_1 = 1$. Then $L > 2$ as
 We have $y=0$ at $x=1$ and $-1$ giving:
 
 
-
+$$
 \begin{align*}
 (-1 + D)^2 + (0 - C)^2 &= \lambda^2\\
 (+1 + D)^2 + (0 - C)^2 &= \lambda^2.
 \end{align*}
+$$
 
 
 Squaring out and solving gives $D=0$, $1 + C^2 = \lambda^2$. That is, an arc of circle with radius $\sqrt{1+C^2}$ and centered at $(0, C)$.
@@ -1776,7 +1793,7 @@ where $R_k(x) = f^{k+1}(\xi)/(k+1)!(x-a)^{k+1}$ for some $\xi$ between $a$ and $
 This theorem can be generalized to scalar functions, but the notation can be cumbersome. Following [Folland](https://sites.math.washington.edu/~folland/Math425/taylor2.pdf) we use *multi-index* notation. Suppose $f:R^n \rightarrow R$, and let $\alpha=(\alpha_1, \alpha_2, \dots, \alpha_n)$. Then define the following notation:
 
 
-
+$$
 \begin{align*}
 |\alpha| &= \alpha_1 + \cdots + \alpha_n, \\
 \alpha! &= \alpha_1!\alpha_2!\cdot\cdots\cdot\alpha_n!, \\
@@ -1784,6 +1801,7 @@ This theorem can be generalized to scalar functions, but the notation can be cum
 \partial^\alpha f &= \partial_1^{\alpha_1}\partial_2^{\alpha_2}\cdots \partial_n^{\alpha_n} f \\
 & = \frac{\partial^{|\alpha|}f}{\partial x_1^{\alpha_1} \partial x_2^{\alpha_2} \cdots \partial x_n^{\alpha_n}}.
 \end{align*}
+$$
 
 
 This notation makes many formulas from one dimension carry over to higher dimensions. For example, the binomial theorem says:
@@ -1800,8 +1818,8 @@ $$
 (x_1 + x_2 + \cdots + x_n)^n = \sum_{|\alpha|=k} \frac{k!}{\alpha!} \vec{x}^\alpha.
 $$
 
-Taylor's theorem then becomes:
-
+::: {.callout-note icon=false}
+## Taylor's theorem using multi-index
 
 If $f: R^n \rightarrow R$ is sufficiently smooth ($C^{k+1}$) on an open convex set $S$ about $\vec{a}$ then if $\vec{a}$ and $\vec{a}+\vec{h}$ are in $S$,
 
@@ -1812,18 +1830,20 @@ $$
 
 where $R_{\vec{a},k} = \sum_{|\alpha|=k+1}\partial^\alpha \frac{f(\vec{a} + c\vec{h})}{\alpha!} \vec{h}^\alpha$ for some $c$ in $(0,1)$.
 
+:::
 
 ##### Example
 
 
 The elegant notation masks what can be complicated expressions. Consider the simple case $f:R^2 \rightarrow R$ and $k=2$. Then this says:
 
-
+$$
 \begin{align*}
 f(x + dx, y+dy) &= f(x, y) + \frac{\partial f}{\partial x} dx + \frac{\partial f}{\partial y} dy \\
 &+ \frac{\partial^2 f}{\partial x^2} \frac{dx^2}{2} +  2\frac{\partial^2 f}{\partial x\partial y} \frac{dx dy}{2}\\
 &+ \frac{\partial^2 f}{\partial y^2} \frac{dy^2}{2} + R_{\langle x, y \rangle, k}(\langle dx, dy \rangle).
 \end{align*}
+$$
 
 Using $\nabla$ and $H$ for the Hessian and $\vec{x} = \langle x, y \rangle$ and $d\vec{x} = \langle dx, dy \rangle$, this can be expressed as: