daviddoji/CalculusWithJuliaNotes.jl
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

lots of cleanup

Browse Source
This commit is contained in:
jverzani committed 2026-08-11 17:17:08 -04:00
1 parent ae461659e0
commit 253295ff6e
91 files changed
+18268 -7856
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

quarto/derivatives/curve_sketching.qmd
+205 -69
View File
@@ -19,21 +19,26 @@ import Polynomials: variable, Polynomial, RationalFunction # avoid name clash
---
The figure illustrates a means to *sketch* a sine curve - identify as many of the following values as you can:
Drawing a sketch of a function can be enhanced by identifying as many of the following as possible before filling in a curve representing the function:
* asymptotic behaviour (as $x \rightarrow \pm \infty$),
* periodic behaviour,
* vertical asymptotes,
* the $y$ intercept,
* any $x$ intercept(s),
* local peaks and valleys (relative extrema).
* concavity
* asymptotic behaviour (as $x \rightarrow \pm \infty$),
* periodic behaviour,
* vertical asymptotes,
* the $y$ intercept,
* any $x$ intercept(s),
* local peaks and valleys (relative extrema).
* concavity
With these, a sketch fills in between the points/lines associated with these values.
@fig-animation-sketch-sin-plot and @fig-animation-sketch-rational-function-plot illustrate this approach for two different types of functions.
::: {#fig-animation-sketch-sin-plot}
```{julia}
#| hold: true
#| echo: false
@@ -69,15 +74,7 @@ function sketch_sin_plot_graph(i)
end
caption = L"""
After identifying asymptotic behaviours,
a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the periodic function $f(x) = 10\cdot\sin(\pi/2\cdot x)$ is sketched over $[0,4]$.
"""
caption = ""
n = 8
anim = @animate for i=1:n
sketch_sin_plot_graph(i)
@@ -89,6 +86,101 @@ plotly()
ImageFile(imgfile, caption)
```
After identifying asymptotic behaviors, a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the periodic function $f(x) = 10\cdot\sin(\pi/2\cdot x)$ is sketched over $[0,4]$.
:::
::: {#fig-animation-sketch-rational-function-plot}
```{julia}
#| echo: false
#| cache: true
### {{{ sketch_sin_plot }}}
let
gr()
function sketch_rational_plot_graph(i)
f(x) = 1/2 + (x-1)^2/((x-2)^2 * (x+1))
plt = plot(; xlim=(-4, 5), ylim=(-5, 5), xticks = -4:5, legend=false, framestyle=:origin)
title = "Sketch a rational function"
# (1) asymptotic behaviour (as $x \rightarrow \pm \infty$),
i == 1 && (title = L"A horizontal asymptote $y=1/2$")
if i >= 1
hline!(plt, [1/2]; line=(:dash, :gray50))
end
# (2) periodic behaviour,
i == 2 && (title = "No periodic behavious")
if i >= 2
end
# (3) vertical asymptotes,
i == 3 && (title = L"Vertical asyptotes at $x=-1$ and $x=2$")
if i >= 3
vline!(plt, [-1, 2]; line=(:dash, :gray50))
end
# (4) the $y$ intercept,
i == 4 && (title = L"f(0) = 3/4")
if i >= 4
scatter!(plt, [(0, f(0))]; marker=(5, :green))
end
# (5) any $x$ intercept(s)
i == 5 && (title = L"Zero at $x=-2.1527576\cdots$")
if i >= 5
z = -2.1527576
scatter!(plt, [(z, 0)]; marker=(5, :red))
end
# (6) local peaks and valleys (relative extrema).
i == 6 && (title = L"Relative minimum at $(1, f(1))$")
if i >= 6
scatter!(plt, [(1,f(1))]; marker = (5, :blue))
end
# (7) concavity
i == 7 && (title = L"Concave up on $(-1, 2)$, $(2, 3)$" )
if i == 7
xs = range(-0.4, 0.4, 100)
ys = 1 .+ sec.(xs.* pi)
plot!(xs .- 2.5, -1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right)
plot!(xs .+ 1.0, 1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right)
plot!(xs .+ 3.5, 1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right)
end
# connect the dots
if i >= 8
title = "Sketch"
plot!(plt, rangeclamp(f, 5); line = (1, :black))
end
title!(plt, title)
plt
end
caption = ""
n = 10
anim = @animate for i=1:n
sketch_rational_plot_graph(i)
end
imgfile = tempname() * ".gif"
gif(anim, imgfile, fps = 1)
plotly()
ImageFile(imgfile, caption)
end
```
After identifying asymptotic behaviors, a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the rational function $f(x) = 1/2 + (x-1)^2/((x-2)^2 \cdot (x+1))$ is sketched.
:::
Though this approach is most useful for hand-sketches, the underlying concepts are important for properly framing graphs made with the computer.
@@ -148,42 +240,52 @@ We finally check that if we were to just use $[0,7]$ as a domain to plot over th
f.([0, cps..., 7])
```
The values at $0$ and at $7$ are a bit large, as compared to the relative extrema, and since we know the graph is eventually $U$-shaped, this offers no insight. So we narrow the range a bit for the graph:
The values at $0$ and at $7$ are a bit large, as compared to the relative extrema, and since we know the graph is eventually $U$-shaped, this offers no insight. So we narrow the range a bit for the graph of @fig-plot-poly-x-4-minus-13-x-3-plus-56-x-2-minus92-x-plus-48.
::: {#fig-plot-poly-x-4-minus-13-x-3-plus-56-x-2-minus92-x-plus-48}
```{julia}
plot(f, 0.5, 6.5)
```
Plot of $f(x) = x^4 - 13x^3 + 56x^2 -92x + 48$ over $[0.5, 6.5]$
:::
---
This sort of analysis can be automated. The plot "recipe" for polynomials from the `Polynomials` package does similar considerations to choose a viewing window:
::: {#fig-plot-polynomial-f-of-x-not-f}
```{julia}
𝐱 = variable(Polynomial)
plot(f(𝐱)) # f(𝐱) of Polynomial type
```
Plot of polynomial `f(x)` produced by a plot recipe
:::
##### Example
Graph the function
Graph the rational function
$$
f(x) = \frac{(x-1)\cdot(x-3)^2}{x \cdot (x-2)}.
$$
Not much to do here if you are satisfied with a graph that only gives insight into the asymptotes of this rational function:
Not much to do here if you are satisfied with a graph that only gives insight into the asymptotes of this rational function (@fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-bad-one).
::: {#fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-bad-one}
```{julia}
f(x) = ( (x-1)*(x-3)^2 ) / (x * (x-2) )
plot(f, -50, 50)
```
Plot of rational function over $[-50, 50] shows some features, but not all that are commonly of interest
:::
We can see the slant asymptote and hints of vertical asymptotes, but, we'd like to see more of the basic features of the graph.
@@ -204,12 +306,16 @@ So a range over $[-5,5]$ should display the key features including the slant asy
Previously we used the `rangeclamp` function defined in `CalculusWithJulia` to avoid the distortion that vertical asymptotes can have:
::: {#fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-after-settling-on-viewing-window}
```{julia}
plot(rangeclamp(f), -5, 5)
#| echo: false
plot(rangeclamp(f), -5, 5; legend=false)
```
With this graphic, we can now clearly see in the graph the two zeros at $x=1$ and $x=3$, the vertical asymptotes at $x=0$ and $x=2$, and the slant asymptote.
Plot of rational function over identified domain $[-5, 5]$ and passing through `rangeclamp` to avoid spurious lines
:::
With the graph in @fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-after-settling-on-viewing-window, we can now clearly see in the graph the two zeros at $x=1$ and $x=3$, the vertical asymptotes at $x=0$ and $x=2$, and the slant asymptote.
---
@@ -217,33 +323,42 @@ With this graphic, we can now clearly see in the graph the two zeros at $x=1$ an
Again, this sort of analysis can be systematized. The rational function type in the `Polynomials` package takes a stab at that, but isn't quite so good at capturing the slant asymptote:
::: {#fig-plot-of-rational-function-using-plot-recipe}
```{julia}
𝐱 = variable(RationalFunction)
plot(f(𝐱)) # f(x) of RationalFunction type
x = variable(RationalFunction)
plot(f(x)) # f(x) of RationalFunction type
```
Plot of rational function using a plot recipe
:::
##### Example
Consider the function $V(t) = 170 \sin(2\pi\cdot 60 \cdot t)$, a model for the alternating current waveform for an outlet in the United States. Create a graph.
Blindly trying to graph this, we will see immediate issues:
Blindly trying to graph this (@fig-plot-of-170-sin-2-pi-60-t-oscillates), we will see immediate issues.
::: {#fig-plot-of-170-sin-2-pi-60-t-oscillates}
```{julia}
V(t) = 170 * sin(2*pi*60*t)
plot(V, -2pi, 2pi)
```
Plot of current function, $V(t)$ showing it is highly oscillatory
:::
Ahh, this periodic function is *too* rapidly oscillating to be plotted without care. We recognize this as being of the form $V(t) = a\cdot\sin(c\cdot t)$, so where the sine function has a period of $2\pi$, this will have a period of $2\pi/c$, or $1/60$. So instead of using $(-2\pi, 2\pi)$ as the interval to plot over, we need something much smaller:
::: {#fig-plot-of-170-sin-2-pi-60-t-two-periods}
```{julia}
plot(V, -1/60, 1/60)
```
Plot of two periods of highly oscillatory function, $V(t)$
:::
##### Example
@@ -262,10 +377,13 @@ limit(ex, x=>0, dir="+"), limit(ex, x=>oo)
The $\ln(x/100)$ part of $f$ goes $-\infty$ as $x \rightarrow 0+$; yet $f(x)$ is eventually positive as $x \rightarrow \infty$. So a graph should
* not show too much of the vertical asymptote
* capture the point where $f(x)$ must cross $0$
* capture the point where $f(x)$ has a relative maximum
* show enough past this maximum to indicate to the reader the eventual horizontal asymptote.
* not show too much of the vertical asymptote
* capture the point where $f(x)$ must cross $0$
* capture the point where $f(x)$ has a relative maximum
* show enough past this maximum to indicate to the reader the eventual horizontal asymptote.
For that, we need to get the $x$ intercepts and the critical points. The $x/100$ means this graph has some scaling to it, so we first look between $0$ and $200$:
@@ -282,36 +400,44 @@ Trying the same for the critical points comes up empty. We know there is one, bu
find_zeros(diff(ex,x), 0, 500)
```
So maybe graphing over $[50, 300]$ will be a good start:
So maybe graphing over $[50, 300]$ will be a good start. A graph is shown in @fig-plot-log-x-over-100-over-x-take-1.
::: {#fig-plot-log-x-over-100-over-x-take-1}
```{julia}
#| echo: false
plot(ex, 50, 300)
```
But it isn't! The function takes its time getting back towards $0$. We know that there must be a change of concavity as $x \rightarrow \infty$, as there is a horizontal asymptote. We looks for the anticipated inflection point to ensure our graph includes that:
Plot of $f(x) = \ln(x/100)/x$ over $[50, 200]$
:::
But it isn't! The function takes its time getting back towards $0$. We know that there must be a change of concavity as $x \rightarrow \infty$, as there is a horizontal asymptote. We look for the anticipated inflection point to ensure our graph includes that:
```{julia}
find_zeros(diff(ex, x, x), 1, 5000)
```
So a better plot is found by going well beyond that inflection point:
So a better plot is found by going well beyond that inflection point. In @fig-plot-log-x-over-100-over-x-take-1 we plot over $[75, 1500]$.
::: {#fig-plot-log-x-over-100-over-x-take-2}
```{julia}
#| echo: false
plot(ex, 75, 1500)
```
Plot of $f(x) = \ln(x/100)/x$ over $[75, 1500]$
:::
## Questions
###### Question
Consider this graph
Consider the graph of a rational function in @fig-rational-function-x-minus-2-times-x-2point5-times-x-minus-3-over-x-minus-1-x-plus-1.
::: {#fig-rational-function-x-minus-2-times-x-2point5-times-x-minus-3-over-x-minus-1-x-plus-1}
```{julia}
#| hold: true
#| echo: false
@@ -321,6 +447,9 @@ plot!(p, f, -1 + .2, 1 - .02, color=:blue)
plot!(p, f, 1 + .05, 20, color=:blue)
```
Graph of rational function
:::
What kind of *asymptotes* does it appear to have?
@@ -333,8 +462,8 @@ L"Just vertical asymptotes at $x=-1$ and $x=1$",
L"Vertical asymptotes at $x=-1$ and $x=1$ and a horizontal asymptote $y=1$",
L"Vertical asymptotes at $x=-1$ and $x=1$ and a slant asymptote"
]
answ = 4
radioq(choices, answ)
answer = 4
radioq(choices, answer)
```
###### Question
@@ -448,15 +577,16 @@ radioq(qchoices, 7, keep_order=true)
A function $f$ has
* zeros of $\{-0.7548\dots, 2.0\}$,
* zeros of $\{-0.7548\dots, 2.0\}$,
* critical points at $\{-0.17539\dots, 1.0, 1.42539\dots\}$,
* inflection points at $\{0.2712\dots,1.2287\}$.
* critical points at $\{-0.17539\dots, 1.0, 1.42539\dots\}$,
* inflection points at $\{0.2712\dots,1.2287\}$.
Is this a possible graph of $f$?
Does @fig-function-x-4-minus-3-x-3-plus-2-x-2-plus-x-minus-2 show a possible graph of $f$?
::: {#fig-function-x-4-minus-3-x-3-plus-2-x-2-plus-x-minus-2}
```{julia}
#| hold: true
#| echo: false
@@ -464,6 +594,9 @@ f(x) = x^4 - 3x^3 + 2x^2 + x - 2
plot(f, -1, 2.5, legend=false)
```
Plot of $f(x)$ over $[-1, 1.25]$
:::
```{julia}
#| hold: true
#| echo: false
@@ -476,19 +609,22 @@ yesnoq("yes")
Two models for population growth are *exponential* growth: $P(t) = P_0 a^t$ and [logistic growth](https://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth): $P(t) = K P_0 a^t / (K + P_0(a^t - 1))$. The exponential growth model has growth rate proportional to the current population. The logistic model has growth rate depending on the current population *and* the available resources (which can limit growth).
Letting $K=50$, $P_0=5$, and $a= e^{1/4}$. A plot over $[0,5]$ shows somewhat similar behaviour:
Letting $K=50$, $P_0=5$, and $a= e^{1/4}$, the plots in @fig-plot-exponential-and-logistic-growth-over-0-5 $ shows somewhat similar behaviour over $[0,5].
::: {#fig-plot-exponential-and-logistic-growth-over-0-5}
```{julia}
K, P0, a = 50, 5, exp(1/4)
exponential_growth(t) = P0 * a^t
logistic_growth(t) = K * P0 * a^t / (K + P0*(a^t-1))
plot(exponential_growth, 0, 5)
plot!(logistic_growth)
plot(exponential_growth, 0, 5; label="exponential growth")
plot!(logistic_growth; label="logistic growth")
```
Does a plot over $[0,50]$ show qualitatively similar behaviour?
Plot of exponential growth model and logisitc growth model over $[0,5]$
:::
Does a plot over $[0,50]$ show qualitatively similar behaviour?
```{julia}
@@ -526,8 +662,8 @@ choices = [
"The exponential growth model",
"The limit does not exist",
"The limit is ``P_0``"]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -547,8 +683,8 @@ choices = [
"The function will have more curvature when the second derivative is large, so there needs to be more points to capture the shape",
"The function will be much larger (in absolute value) when the second derivative is large, so there needs to be more points to capture the shape",
]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
###### Question
@@ -567,8 +703,8 @@ choices = [
"An informative graph only needs to show one or two periods, as others can be inferred.",
"An informative graph need only show a part of the period, as the rest can be inferred.",
L"An informative graph needs to show several periods, as that will allow proper computation for the $y$ axis range."]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
Why should asymptotics matter?
@@ -582,8 +718,8 @@ L"A vertical asymptote can distort the $y$ range, so it is important to avoid to
L"A horizontal asymptote must be plotted from $-\infty$ to $\infty$",
"A slant asymptote must be plotted over a very wide domain so that it can be identified."
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
Monotonicity means increasing or decreasing. This is important for what reason?
@@ -597,6 +733,6 @@ choices = [
"For monotonic regions, a function is basically a straight line",
"For monotonic regions, the function will have a vertical asymptote, so the region should not be plotted"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
quarto/derivatives/derivatives.qmd
+276 -229
View File
@@ -22,7 +22,11 @@ nothing
---
![Device to measure units of distance by units of time](figures/galileo-ramp.png){width=60%}
::: {#fig-picture-of-galileo-ramp}
![](figures/galileo-ramp.png){width=60%}
Device to measure units of distance by units of time
:::
Before defining the derivative of a function, let's begin with two motivating examples.
@@ -47,14 +51,17 @@ If the rate is a constant $60$ miles/hour, then in one hour the distance travele
Of course, the odometer isn't just incrementing once per hour, it is incrementing once every $1/10$th of a mile. How much time does that take? Well, we would need to solve $1/10=60 \cdot t$ which means $t=1/600$ hours, better known as once every $6$ seconds.
Using some mathematical notation, would give $x(t) = v\cdot t$, where $x$ is position at time $t$, $v$ is the *constant* velocity and $t$ the time traveled in hours. A simple graph of the first three hours of travel would show:
Using some mathematical notation, would give $x(t) = v\cdot t$, where $x$ is position at time $t$, $v$ is the *constant* velocity and $t$ the time traveled in hours. The simple graph of @fig-plot-position-time-over-0-3 shows the first three hours of travel.
::: {#fig-plot-position-time-over-0-3}
```{julia}
#| hold: true
position(t) = 60 * t
plot(position, 0, 3)
```
Plot of position versus time for a constant speed
:::
Oh no, we hit traffic. In the next $30$ minutes we only traveled $15$ miles. We were so busy looking out for traffic, the speedometer was not checked. What would the average speed have been? Though in the $30$ minutes of stop-and-go traffic, the displayed speed may have varied, the *average speed* would simply be the change in distance over the change in time, or $\Delta x / \Delta t$. That is
@@ -63,9 +70,9 @@ Oh no, we hit traffic. In the next $30$ minutes we only traveled $15$ miles. We
15/(1/2)
```
Now suppose that after $6$ hours of travel the GPS in the car gives us a readout of distance traveled as a function of time. The graph looks like this:
Now suppose that after $6$ hours of travel the GPS in the car gives us a readout of distance traveled as a function of time. @fig-plot-complicated-position-versus-time-from-0-to-6 shows the graph.
::: {#fig-plot-complicated-position-versus-time-from-0-to-6}
```{julia}
#| hold: true
#| echo: false
@@ -78,6 +85,9 @@ end
plot(position, 0, 6)
```
Plot of position versus time for a certain non constant speed
:::
We can see with some effort that the slope is steady for the first three hours, is slightly less between $3$ and $3.5$ hours, then is a bit steeper for the next half hour. After that, it is flat for the about half an hour, then the slope continues on with same value as in the first $3$ hours. What does that say about our speed during our trip?
@@ -108,9 +118,9 @@ Okay, so there was some speeding involved.
The next half hour the car did not move. What was the average speed? Well the change in position was $0$, but the time was $1/2$ hour, so the average was $0$.
Perhaps a graph of the speed is a bit more clear. We can do this based on the above:
Perhaps a graph of the speed is a bit more clear. We can do this based on the above in @fig-plot-non-constant-speed-over-0-6.
::: {#fig-plot-non-constant-speed-over-0-6}
```{julia}
function speed(t)
0 < t <= 3 ? 60 :
@@ -121,6 +131,10 @@ end
plot(speed, 0, 6)
```
Plot of non-constant speed over $[0,6]$
:::
The jumps, as discussed before, are artifacts of the graphing algorithm. What is interesting, is we could have derived the graph of `speed` from that of `x` by just finding the slopes of the line segments, and we could have derived the graph of `x` from that of `speed`, just using the simple formula relating distance, rate, and time.
@@ -133,42 +147,46 @@ We were pretty loose with some key terms. There is a distinction between "speed"
##### Example: Galileo's ball and ramp experiment
One of history's most famous experiments was performed by [Galileo](http://en.wikipedia.org/wiki/History_of_experiments) where he rolled balls down inclined ramps, making note of distance traveled with respect to time. As Galileo had no ultra-accurate measuring device, he needed to slow movement down by controlling the angle of the ramp. With this, he could measure units of distance per units of time. (Click through to *Galileo and Perspective* [Dauben](http://www.mcm.edu/academic/galileo/ars/arshtml/mathofmotion1.html).)
One of history's most famous experiments was performed by [Galileo](http://en.wikipedia.org/wiki/History_of_experiments) where he rolled balls down inclined ramps, like that in @fig-picture-of-galileo-ramp, making note of distance traveled with respect to time. As Galileo had no ultra-accurate measuring device, he needed to slow movement down by controlling the angle of the ramp. With this, he could measure units of distance per units of time.
Suppose that no matter what the incline was, Galileo observed that in units of the distance traveled in the first second that the distance traveled between subsequent seconds was $3$ times, then $5$ times, then $7$ times, ... @tbl-galileo-distance-time-delta summarizes.
Suppose that no matter what the incline was, Galileo observed that in units of the distance traveled in the first second that the distance traveled between subsequent seconds was $3$ times, then $5$ times, then $7$ times, ... This table summarizes.
::: {#tbl-galileo-distance-time-delta .striped .hover}
| t | distance | delta |
|:------|:--------|:---------|
| 0 | 0 | . |
| 1 | 1 | 1 |
| 2 | 4 | 3 |
| 3 | 9 | 5 |
| 4 | 16 | 7 |
| 5 | 25 | 9 |
Distance traveled by time $t$ with differences computed
:::
```{julia}
#| hold: true
#| echo: false
ts = [0,1,2,3,4,5]
dxs = [0,1,3, 5, 7, 9]
ds = [0,1,4,9,16,25]
d = DataFrame(t=ts, delta=dxs, distance=ds)
table(d)
```
A graph of distance versus time could be found by interpolating between the measured points:
The graph of distance versus time in @fig-plot-of-distance-versus-time-galileo-simple-data is found by interpolating between the measured points.
::: {#fig-plot-of-distance-versus-time-galileo-simple-data}
```{julia}
ts = [0,1,2,3,4, 5]
xs = [0,1,4,9,16,25]
plot(ts, xs)
```
Plot of distance traveled versus time
:::
The graph looks almost quadratic. What would the following questions have yielded?
* What is the average speed between $0$ and $3$?
* What is the average speed between $0$ and $3$?
```{julia}
(9-0) / (3-0) # (xs[4] - xs[1]) / (ts[4] - ts[1])
```
* What is the average speed between $2$ and $3$?
* What is the average speed between $2$ and $3$?
```{julia}
@@ -188,7 +206,7 @@ xs[2]-xs[1], xs[3] - xs[2], xs[4] - xs[3], xs[5] - xs[4]
We see it increments by $2$. The acceleration is the rate of change of speed. We see the rate of change of speed is constant, as the speed increments by $2$ each time unit.
Based on this - and given Galileo's insight - it appears the acceleration for a falling body subject to gravity will be **constant** and the position as a function of time will be quadratic.
Based on this---and given Galileo's insight---it appears the acceleration for a falling body subject to gravity will be **constant** and the position as a function of time will be quadratic.
## The slope of the secant line
@@ -196,27 +214,35 @@ Based on this - and given Galileo's insight - it appears the acceleration for a
In the above examples, we see that the average speed is computed using the slope formula. This can be generalized for any univariate function $f(x)$:
::: {.definition title="Average rate of change"}
> The average rate of change between $a$ and $b$ is $(f(b) - f(a)) / (b - a)$. It is typical to express this as $\Delta y/ \Delta x$, where $\Delta$ means "change".
Geometrically, this is the slope of the line connecting the points $(a, f(a))$ and $(b, f(b))$. This line is called a [secant](http://en.wikipedia.org/wiki/Secant_line) line, which is just a line intersecting two specified points on a curve.
Rather than parameterize this problem using $a$ and $b$, we let $c$ and $c+h$ represent the two values for $x$, then the secant-line-slope formula becomes
For a continuous function $f$ the average rate of change between $a$ and $b$ is defined by
$$
m = \frac{f(c+h) - f(c)}{h}.
\frac{f(b) - f(a)}{b - a}.
$$
It is typical to express this as $\Delta y/ \Delta x$, where $\Delta$ means "change".
:::
Geometrically, the average rate of change is the slope of the line connecting the points $(a, f(a))$ and $(b, f(b))$. This line is called a [secant](http://en.wikipedia.org/wiki/Secant_line) line---a line intersecting two specified points on a curve.
## The slope of the tangent line
Before continuing, we re-parameterize with $a=c$ and $b=c+h$ thinking of $c$ as fixed, and $h$ as varying. The secant-line-slope formula becomes:
$$
\text{slope (of a secant line)} = \frac{f(c+h) - f(c)}{h}.
$$
The slope of the secant line represents the average rate of change over a given period, $h$. What if this rate is so variable, that it makes sense to take smaller and smaller periods $h$? In fact, what if $h$ goes to $0$?
::: {#fig-secant-line-tangent-line-animation}
```{julia}
#| hold: true
#| echo: false
@@ -261,14 +287,7 @@ function secant_line_tangent_line_graph(n)
plt
end
caption = L"""
The slope of each secant line represents the *average* rate of change between $c$ and $c+h$. As $h$ goes towards $0$, we recover the slope of the tangent line, which represents the *instantatneous* rate of change.
"""
caption = ""
n = 6
anim = @animate for i=0:n
secant_line_tangent_line_graph(i)
@@ -281,21 +300,41 @@ plotly()
ImageFile(imgfile, caption)
```
The graphic suggests that the slopes of the secant line converge to the slope of a "tangent" line. That is, for a given $c$, this limit exists:
The slope of each secant line represents the *average* rate of change between $c$ and $c+h$. As $h$ goes towards $0$, we recover the slope of the tangent line, which represents the *instantatneous* rate of change.
:::
@fig-secant-line-tangent-line-animation suggests that for this function and at the point $x=c$ the slopes of the secant line converge to the slope of a "tangent" line. That is this limit exists:
$$
\lim_{h \rightarrow 0} \frac{f(c+h) - f(c)}{h}.
$$
We will define the tangent line at $(c, f(c))$ to be the line through the point with the slope from the limit above - provided that limit exists. Informally, the tangent line is the line through the point that best approximates the function.
With this, we define
::: {.definition title="Tangent line at c"}
When the following limit exists, the tangent line to the graph of $f(x)$ at $x=c$ is the line through the point $(c, f(c))$ with slope:
$$
m = \lim_{h \rightarrow 0} \frac{f(c+h) - f(c)}{h}.
$$
In point-slope form, the line is described by: $y = f(c) + m \cdot (x-c)$.
Later we will write $f'(c)$ for $m$.
:::
Informally, the tangent line is the line through the point that best approximates the function, as in @fig-tangent_line_approx_graph. The tangent line is not just a line that intersects the graph in one point, nor does it need only intersect the line in just one point.
::: {#fig-tangent_line_approx_graph}
```{julia}
#| echo: false
gr()
let
gr()
function make_plot(Δ)
f(x) = 1 + sin(x-c)
df(x) = cos(x-c)
@@ -322,61 +361,16 @@ let
end
ps = make_plot.((1.5, 1.0, 0.5, 0.1))
plotly()
plot(ps...)
end
```
Illustration that the tangent line is the best linear approximation *near* $c$.
Illustration that the tangent line is the best linear approximation to $f(x)$ *near* $c$.
:::
```{julia}
#| echo: false
plotly()
nothing
```
```{julia}
#| hold: true
#| echo: false
#| cache: true
#| eval: false
gr()
function line_approx_fn_graph(n)
f(x) = sin(x)
c = pi/3
h = round(2.0^(-n) * pi/2, digits=2)
m = cos(c)
Delta = max(f(c) - f(c-h), f(min(c+h, pi/2)) - f(c))
p = plot(f, c-h, c+h, legend=false, xlims=(c-h,c+h), ylims=(f(c)-Delta,f(c)+Delta ))
plot!(p, x -> f(c) + m*(x-c))
scatter!(p, [c], [f(c)])
p
end
caption = L"""
The tangent line is the best linear approximation to the function at the point $(c, f(c))$. As the viewing window zooms in on $(c,f(c))$ we
can see how the graph and its tangent line get more similar.
"""
n = 6
anim = @animate for i=1:n
line_approx_fn_graph(i)
end
imgfile = tempname() * ".gif"
gif(anim, imgfile, fps = 1)
plotly()
ImageFile(imgfile, caption)
```
The tangent line is not just a line that intersects the graph in one point, nor does it need only intersect the line in just one point.
:::{.callout-note}
@@ -391,41 +385,47 @@ This last point was certainly not obvious at first. [Barrow](http://www.maa.org/
What is the slope of the tangent line to $f(x) = \sin(x)$ at $c=0$?
We need to compute the limit $(\sin(c+h) - \sin(c))/h$ which is the limit as $h$ goes to $0$ of $\sin(h)/h.$ We know this to be $1.$
We need to compute the limit $(\sin(c+h) - \sin(c))/h$ which is the limit as $h$ goes to $0$ of $\sin(h)/h.$ We know this to be $1$. See @fig-plot-sin-tangent-line for a graph of the tangent line to the function.
::: {#fig-plot-sin-tangent-line}
```{julia}
#| hold: true
#| echo: false
f(x) = sin(x)
c = 0
tl(x) = f(c) + 1 * (x - c)
plot(f, -pi/2, pi/2)
plot!(tl, -pi/2, pi/2)
plot(f, -pi/2, pi/2; label="sin(x)")
plot!(tl; label="Tangent line")
```
Plot of $f(x) = \sin(x)$ with its tangent line ($y = 0 + 1 \cdot (x-0)$) at the origin
:::
## The derivative
The limit of the slope of the secant line gives an operation: for each $c$ in the domain of $f$ there is a number (the slope of the tangent line) or it does not exist. That is, there is a derived function from $f$. Call this function the *derivative* of $f$.
The limit of the slopes of the secant lines for a fixed $c$ gives an operation: for each $c$ in the domain of $f$ when the limit exists
associate the slope of the tangent line.
::: {.definition title="The derivative"}
There are many notations for the derivative, mostly we use the "prime" notation:
For each $x$ in the domain of $f(x)$, let $f'(x)$ define a function whose domain is all $x$ for which the following limit exists and whose value is given by:
$$
f'(x) = \lim_{h \rightarrow 0} \frac{f(x+h) - f(x)}{h}.
f'(x) = \lim_{h\rightarrow 0} \frac{f(x+h) - f(x)}{h}.
$$
The limit above is identical, only it uses $x$ instead of $c$ to emphasize that we are thinking of a function now, and not just a value at a point.
The function $f'(x)$ is called the *derivative* of $f(x)$.
:::
The derivative is related to a function, but at times it is more convenient to write only the expression defining the rule of the function. In that case, we use this notation for the derivative $[\text{expression}]'$.
There are many notations for the derivative, mostly we use the "prime" notation in the definition, but at times it is more convenient to write only the expression defining the rule of the function. In that case, we use this notation for the derivative $[\text{expression}]'$.
### Some basic derivatives
* **The constant rule**. If $f(x) = c$, some constant, then the graph has zero slope, as $f(x+h) - f(x) = 0$ for all $x$ and $h$. That is the derivative is constantly $0$, or $[c]' = 0$.
* **The power rule**. What is the derivative of the monomial $f(x) = x^n$? We need to look at $(x+h)^n - x^n$ for positive, integer-value $n$. Let's look at a case, $n=5$
* **The power rule**. What is the derivative of the monomial $f(x) = x^n$? We need to look at $(x+h)^n - x^n$ for positive, integer-value $n$. Let's look at a case, $n=5$
```{julia}
@@ -451,10 +451,10 @@ $$
[x^n]' = nx^{n-1}.
$$
It isn't a special case, but when $n=0$, we also have the above formula applies, as $x^0$ is the constant $1$, and all constant functions will have a derivative of $0$ at all $x$. We will see that in general, the power rule applies for any $n$ where $x^n$ is defined.
It isn't a special case, but when $n=0$, we also have the above formula applies, as $x^0$ is the constant $1$. We will see later that in general, the power rule applies for any $n$ where $x^n$ is defined.
* What is the derivative of $f(x) = \sin(x)$? We know that $f'(0)= 1$ by the earlier example with $(\sin(0+h)-\sin(0))/h = \sin(h)/h$, here we solve in general.
* What is the derivative of $f(x) = \sin(x)$? We know that $f'(0)= 1$ by the earlier example with $(\sin(0+h)-\sin(0))/h = \sin(h)/h$, here we solve in general.
We need to consider the difference $\sin(x+h) - \sin(x)$:
@@ -464,17 +464,14 @@ We need to consider the difference $\sin(x+h) - \sin(x)$:
sympy.expand_trig(sin(x+h) - sin(x)) # expand_trig is not exposed in `SymPy`
```
That used the formula $\sin(x+h) = \sin(x)\cos(h) + \sin(h)\cos(x)$.
We could then rearrange the secant line slope formula to become:
$$
\cos(x) \cdot \frac{\sin(h)}{h} + \sin(x) \cdot \frac{\cos(h) - 1}{h}
\cos(x) \cdot \frac{\sin(h)}{h} + \sin(x) \cdot \frac{\cos(h) - 1}{h}.
$$
and take a limit. If the answer isn't clear, we can let `SymPy` do this work:
We then take a limit as $h \rightarrow 0$. If the answer isn't clear, we can let `SymPy` do this work:
```{julia}
@@ -484,7 +481,7 @@ limit((sin(x+h) - sin(x))/ h, h => 0)
From the formula $[\sin(x)]' = \cos(x)$ we can easily get the *slope* of the tangent line to $f(x) = \sin(x)$ at $x=0$ by simply evaluating $\cos(0) = 1$.
* Let's see what the derivative of $\ln(x) = \log(x)$ is (using base $e$ for $\log$ unless otherwise indicated). We have
* Let's see what the derivative of $\ln(x) = \log(x)$ is (using base $e$ for $\log$ unless otherwise indicated). We have
$$
@@ -508,37 +505,39 @@ $$
\frac{e^{x+h} - e^x}{h} = \frac{e^x \cdot(e^h -1)}{h}.
$$
Earlier, we saw that $\lim_{h \rightarrow 0}(e^h - 1)/h = 1$. With this, we get $[e^x]' = e^x$, that is it is a function satisfying $f'=f$.
Earlier, we saw that $\lim_{h \rightarrow 0}(e^h - 1)/h = 1$. With this, we get $[e^x]' = e^x$, that is it is a function satisfying $f'(x)=f(x)$.
---
There are several different [notations](http://en.wikipedia.org/wiki/Notation_for_differentiation) for derivatives. Some are historical, some just add flexibility. We use the prime notation of Lagrange: $f'(x)$, $u'$ and $[\text{expr}]'$, where the first emphasizes that the derivative is a function with a value at $x$, the second emphasizes the derivative operates on functions, the last emphasizes that we are taking the derivative of some expression.
There are several [notations](http://en.wikipedia.org/wiki/Notation_for_differentiation) for derivatives. Some are historical, some just add flexibility. We use the prime notation of Lagrange: $f'(x)$, $u'$ and $[\text{expr}]'$, where the first emphasizes that the derivative is a function with a value at $x$, the second emphasizes the derivative operates on functions, the last emphasizes that we are taking the derivative of some expression.
There are many other notations:
Some other notations include:
* The Leibniz notation uses the infinitesimals: $dy/dx$ to relate to $\Delta y/\Delta x$. This notation is very common, and especially useful when more than one variable is involved. `SymPy` uses Leibniz notation in some of its output, expressing somethings such as:
* The Leibniz notation which uses the infinitesimals, $dy/dx$, to relate to $\Delta y/\Delta x$. This notation is very common, and especially useful when more than one variable is involved. `SymPy` uses Leibniz notation in some of its output, expressing somethings such as:
$$
f'(x) = \frac{d}{d\xi}(f(\xi)) \big|_{\xi=x}.
$$
The notation - $\big|$ - on the right-hand side separates the tasks of finding the derivative and evaluating the derivative at a specific value.
The notation--$\big|$---on the right-hand side separates the tasks of finding the derivative and evaluating the derivative at a specific value.
* Euler used `D` for the operator `D(f)`. This was initially used by [Argobast](http://jeff560.tripod.com/calculus.html). The notation `D(f)(c)` would be needed to evaluate the derivative at a point.
* Newton used a "dot" above the variable, $\dot{x}(t)$, which is still widely used in physics to indicate a derivative in time. This indicates first taking the derivative and then plugging in $t$.
* The notation $[expr]'(c)$ or $[expr]'\big|_{x=c}$would similarly mean, take the derivative of the expression and **then** evaluate at $c$.
* Euler used `D` for the operator `D(f)`. This was initially used by [Argobast](http://jeff560.tripod.com/calculus.html). The notation `D(f)(c)` would be needed to evaluate the derivative at a point.
* Newton used a "dot" above the variable, $\dot{x}(t)$, which is still widely used in physics to indicate a derivative in time. This indicates first taking the derivative and then plugging in $t$.
* The notation $[expr]'(c)$ or $[expr]'\big|_{x=c}$would similarly mean, take the derivative of the expression and **then** evaluate at $c$.
## Rules of derivatives
We could proceed in a similar manner – using limits to find other derivatives, but let's not. If we have a function $f(x) = x^5 \sin(x)$, it would be nice to leverage our previous work on the derivatives of $f(x) =x^5$ and $g(x) = \sin(x)$, rather than derive an answer from scratch.
We could proceed in a similar manner---using limits to find other derivatives, but let's not. If we have a function $f(x) = x^5 \sin(x)$, it would be nice to leverage our previous work on the derivatives of $f(x) =x^5$ and $g(x) = \sin(x)$, rather than derive an answer from scratch.
As with limits and continuity, it proves very useful to consider rules that make the process of finding derivatives of combinations of functions a matter of combining derivatives of the individual functions in some manner.
@@ -547,20 +546,20 @@ As with limits and continuity, it proves very useful to consider rules that make
We already have one such rule:
### Power rule
::: {.relationship title="Power rule"}
We have seen for integer $n \geq 0$ the formula:
For integer $n \geq 0$ the power rule is:
$$
[x^n]' = n x^{n-1}.
$$
This will be shown true for all real exponents.
The power rule will be shown true for all real exponents.
:::
### Sum rule
We now discuss rules which express derivatives of compound expressions in terms of derivatives of different pieces of an expression.
Let's consider $k(x) = a\cdot f(x) + b\cdot g(x)$, what is its derivative? That is, in terms of $f$, $g$ and their derivatives, can we express $k'(x)$?
@@ -586,8 +585,17 @@ $$
That is $[a\cdot f(x) + b \cdot g(x)]' = a\cdot f'(x) + b\cdot g'(x)$.
::: {.relationship title="Constant multiple and sum rule"}
The sum rule applies to constant multiple and sums of functions:
$$
[a f(x) + b g(x)]' = a f'(x) + b g'(x)
$$
This holds two rules: the derivative of a constant times a function is the constant times the derivative of the function; and the derivative of a sum of functions is the sum of the derivative of the functions.
:::
This example shows a useful template:
@@ -601,21 +609,20 @@ $$
\end{align*}
$$
----
### Product rule
Other rules can be similarly derived. We simply state the rules for products and quotients.
::: {.relationship title="Product rule"}
The derivative of a product of functions is given by:
$$
[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x)
$$
Other rules can be similarly derived. `SymPy` can give us them as well. Here we define two symbolic functions `u` and `v` and let `SymPy` derive a formula for the derivative of a product of functions:
```{julia}
#| hold: true
@syms u() v()
f(x) = u(x) * v(x)
limit((f(x+h) - f(x))/h, h => 0)
```
The output uses the Leibniz notation to represent that the derivative of $u(x) \cdot v(x)$ is the $u$ times the derivative of $v$ evaluated at $x$ plus $v$ times the derivative of $u$ evaluated at $x$. A common shorthand is $[uv]' = u'v + uv'$.
A common shorthand is $[uv]' = u'v + uv'$.
:::
This example shows a useful template for the product rule:
@@ -629,19 +636,21 @@ $$
\end{align*}
$$
----
### Quotient rule
The derivative of $f(x) = u(x)/v(x)$---a ratio of functions---can be similarly computed.
::: {.relationship title="Quotient rule"}
The derivative of a ratio of functions can be computed by the quotient rule:
$$
\left[\frac{f(x)}{g(x)}\right]' = \frac{f'(x)\cdot g(x) - f(x) \cdot g'(x)}{g(x)^2}.
$$
This is often written as $[u/v]' = (u'v - uv')/v^2$.
:::
The derivative of $f(x) = u(x)/v(x)$ - a ratio of functions - can be similarly computed. The result will be $[u/v]' = (u'v - uv')/v^2$:
```{julia}
#| hold: true
@syms u() v()
f(x) = u(x) / v(x)
limit((f(x+h) - f(x))/h, h => 0)
```
This example shows a useful template for the quotient rule:
@@ -770,7 +779,11 @@ Find the derivative of $x\sin(x)$ evaluated at $\pi$.
$$
[x\sin(x)]'\big|_{x=\pi} = (1\sin(x) + x\cos(x))\big|_{x=\pi} = (\sin(\pi) + \pi \cdot \cos(\pi)) = -\pi.
\begin{align*}
[x\sin(x)]'\big|_{x=\pi} &= (1\sin(x) + x\cos(x))\big|_{x=\pi} \\
&= (\sin(\pi) + \pi \cdot \cos(\pi)) \\
&= -\pi.
\end{align*}
$$
### Chain rule
@@ -813,17 +826,24 @@ The left-hand side will converge to the derivative of $u(x)$ or $[f(g(x))]'$.
The right-most part of the right-hand side would have a limit $g'(x)$, were we to let $h$ go to $0$.
It isn't obvious, but the left part of the right-hand side has the limit $f'(g(x))$. This would be clear if *only* $g(x+h) = g(x) + h$, for then the expression would be exactly the limit expression with $c=g(x)$. But, alas, except to some hopeful students and some special cases, it is definitely not the case in general that $g(x+h) = g(x) + h$ - that right parentheses actually means something. However, it is *nearly* the case that $g(x+h) = g(x) + kh$ for some $k$ and this can be used to formulate a proof (one of the two detailed [here](http://en.wikipedia.org/wiki/Chain_rule#Proofs) and [here](http://kruel.co/math/chainrule.pdf)).
It isn't obvious, but the left part of the right-hand side has the limit $f'(g(x))$. This would be clear if *only* $g(x+h) = g(x) + h$, for then the expression would be exactly the limit expression with $c=g(x)$. But, alas, except to some hopeful students and some special cases, it is definitely not the case in general that $g(x+h) = g(x) + h$---that right parentheses actually means something. However, it is *nearly* the case that $g(x+h) = g(x) + kh$ for some $k$ and this can be used to formulate a proof, like the one sketched below.
Combined, we would end up with:
Combined, we would end up with the chain rule
::: {.relationship title="Chain rule"}
The *chain rule* is used to find derivatives of compositions:
> *The chain rule*: $[f(g(x))]' = f'(g(x)) \cdot g'(x)$. That is the derivative of the outer function evaluated at the inner function times the derivative of the inner function.
$$
[f(g(x))]' = f'(g(x)) \cdot g'(x).
$$
That is the derivative of the outer function evaluated at the inner function times the derivative of the inner function.
:::
To see that this works in our specific case, we assume the general power rule that $[x^n]' = n x^{n-1}$ to get:
To see that the chain rule works in our specific case, we assume the general power rule that $[x^n]' = n x^{n-1}$ (for all real $n$) to get:
$$
@@ -848,7 +868,7 @@ $$
$$
This is the same as the derivative of $x$ found by first evaluating the composition. For this problem, the chain rule is not necessary, but typically it is a needed rule to fully differentiate a function.
This is the same as the derivative of $x$ found by first evaluating the composition. For this problem, the chain rule is not necessary, but typically it is the most important rule to fully differentiate a function.
##### Examples
@@ -875,7 +895,7 @@ $$
---
Find the derivative of $\log(2 + \sin(x))$. This is a composition $\log(x)$ – with derivative $1/x$ and $2 + \sin(x)$ – with derivative $\cos(x)$. We get $(1/(2 + \sin(x))) \cos(x)$.
Find the derivative of $\log(2 + \sin(x))$. This is a composition $\log(x)$---with derivative $1/x$ and $2 + \sin(x)$---with derivative $\cos(x)$. We get $(1/(2 + \sin(x))) \cos(x)$.
In general,
@@ -914,23 +934,29 @@ $$
A function is *differentiable* at $a$ if the following limit exists $\lim_{h \rightarrow 0}(f(a+h)-f(a))/h$.
This is reexpressed as: $f(a+h) - f(a) - f'(a)h = \epsilon_f(h) h$ where as $h\rightarrow 0$, $\epsilon_f(h) \rightarrow 0$.
This is reexpressed as $f$ is differentitable at $a$ is there exists $\epsilon_f(h) \rightarrow 0$ such that:
$$
f(a+h) - f(a) - f'(a)h = \epsilon_f(h) h.
$$
With that in mind, we have:
As $g$ is differentiable, we have:
$$
g(a+h) = g(a) + g'(a)h + \epsilon_g(h) h = g(a) + h',
$$
Where $h' = (g'(a) + \epsilon_g(h))h \rightarrow 0$ as $h \rightarrow 0$ will be used to simplify the following:
Where $h' = g'(a)h + \epsilon_g(h)h \rightarrow 0$ as $h \rightarrow 0$ will be used to simplify the following:
$$
\begin{align*}
f(g(a+h)) - f(g(a)) &=
f(g(a) + g'(a)h + \epsilon_g(h)h) - f(g(a)) \\
&= f(g(a)) + f'(g(a)) (g'(a)h + \epsilon_g(h)h) + \epsilon_f(h')(h') - f(g(a))\\
&= f'(g(a)) g'(a)h + f'(g(a))(\epsilon_g(h)h) + \epsilon_f(h')(h').
&= \textcolor{red}{f(g(a))} + f'(g(a)) (g'(a)h + \epsilon_g(h)h) + \epsilon_f(h')(h') \\
&\qquad- \textcolor{red}{f(g(a))}\\
&= f'(g(a)) g'(a)h + \textcolor{blue}{f'(g(a))(\epsilon_g(h)h) + \epsilon_f(h')(h')}.
\end{align*}
$$
@@ -939,13 +965,13 @@ Rearranging:
$$
\begin{align*}
f(g(a+h)) &- f(g(a)) - f'(g(a)) g'(a) h\\
&= f'(g(a))\epsilon_g(h)h + \epsilon_f(h')(h')\\
&= \textcolor{blue}{f'(g(a))\epsilon_g(h)h + \epsilon_f(h')(h')}\\
&=(f'(g(a)) \epsilon_g(h) + \epsilon_f(h') (g'(a) + \epsilon_g(h)))h \\
&=\epsilon(h)h,
\end{align*}
$$
where $\epsilon(h)$ combines the above terms which go to zero as $h\rightarrow 0$ into one. This is the alternative definition of the derivative, showing $(f\circ g)'(a) = f'(g(a)) g'(a)$ when $g$ is differentiable at $a$ and $f$ is differentiable at $g(a)$.
where $\epsilon(h)$ combines the above terms which go to zero as $h\rightarrow 0$ into one. This is the alternative definition of differentiable, showing $(f\circ g)'(a) = f'(g(a)) g'(a)$ when $g$ is differentiable at $a$ and $f$ is differentiable at $g(a)$.
##### The "chain" rule
@@ -1025,47 +1051,30 @@ $$
---
We can verify these with `SymPy`. Rather than take a limit, we will use `SymPy`'s `diff` function to compute derivatives.
We can verify these with `SymPy`.
```{julia}
diff(x^5 * sin(x))
diff(x^5 * sin(x), x)
```
```{julia}
diff(x^5/sin(x))
diff(x^5/sin(x), x)
```
```{julia}
diff(sin(x^5))
diff(sin(x^5), x)
```
and finally,
```{julia}
diff(sin(x)^5)
diff(sin(x)^5, x)
```
:::{.callout-note}
## Note
The `diff` function can be called as `diff(ex)` when there is just one free variable, as in the above examples; as `diff(ex, var)` when there are parameters in the expression.
:::
---
The general product rule: For any $n$ - not just integer values - we can re-express $x^n$ using $e$: $x^n = e^{n \log(x)}$. Now the chain rule can be applied:
$$
[x^n]' = [e^{n\log(x)}]' = e^{n\log(x)} \cdot (n \frac{1}{x}) = n x^n \cdot \frac{1}{x} = n x^{n-1}.
$$
---
Find the derivative of $f(x) = x^3 (1-x)^2$ using either the power rule or the sum rule.
@@ -1081,7 +1090,6 @@ the last by the chain rule. Combining with $u' v + u v'$ we get: $f'(x) = (3x^2)
Otherwise, the polynomial can be expanded to give $f(x)=x^5-2x^4+x^3$ which has derivative $f'(x) = 5x^4 - 8x^3 + 3x^2$.
---
@@ -1131,6 +1139,23 @@ $$
e^{-1/2} \cdot (-1) = -e^{-1/2}.
$$
##### Example: the general product rule
The general product rule: For any real $n$---not just integer values---we can re-express $x^n$ using $e$:
$$
x^n = e^{\log(x^n)} = e^{n \log(x)}.
$$
Now the chain rule can be applied:
$$
[x^n]' = [e^{n\log(x)}]' = e^{n\log(x)} \cdot (n \frac{1}{x}) = n x^n \cdot \frac{1}{x} = n x^{n-1}.
$$
##### Example: derivative of inverse functions
@@ -1141,15 +1166,15 @@ $$
1 = (\frac{1}{e^x}) \cdot [e^x]'.
$$
Or solving, $[e^x]' = e^x$. This is a general strategy to find the derivative of an *inverse* function.
Or solving, $[e^x]' = e^x$. This is a general strategy to find the derivative of an *inverse* function in terms of the derivative of the function.
The graph of an inverse function is related to the graph of the function through the symmetry $y=x$.
For example, the graph of $e^x$ and $\log(x)$ have this symmetry, emphasized below:
For example, the graph of $e^x$ and $\log(x)$ have this symmetry, emphasized in @fig-plot-f-inverse-several-points-showing-derivative.
::: {#fig-plot-f-inverse-several-points-showing-derivative}
```{julia}
#| hold: true
#| echo: false
@@ -1172,6 +1197,9 @@ xs′ = @. x₀ + g(y₀) * (ys - y₀)
plot!(ys, xs′, linestyle=:dash)
```
Plot of $f(x) = e^x$ with its inverse function showing the relationship between tangent lines
:::
The point $(1, e)$ on the graph of $e^x$ matches the point $(e, 1)$ on the graph of the inverse function, $\log(x)$. The slope of the tangent line at $x=1$ to $e^x$ is given by $e$ as well. What is the slope of the tangent line to $\log(x)$ at $x=e$?
@@ -1203,40 +1231,33 @@ So the reciprocal of the slope of the tangent line of $f$ at the mirror image po
This table summarizes the rules of derivatives that allow derivatives of more complicated expressions to be computed with the derivatives of their pieces.
::: {#tbl-rules-of-derivatives .striped .hover}
| Name | Rule |
| --------------:| ---------------------------------------------------------------:|
| Power rule | $[x^n]' = n\cdot x^{n-1}$ |
| constant | $[cf(x)]' = c \cdot f'(x)$ |
| sum/difference | $[f(x) \pm g(x)]' = f'(x) \pm g'(x)$ |
| product | $[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x)$ |
| quotient | $[f(x)/g(x)]' = (f'(x) \cdot g(x) - f(x) \cdot g'(x)) / g(x)^2$ |
| chain | $[f(g(x))]' = f'(g(x)) \cdot g'(x)$ |
```{julia}
#| hold: true
#| echo: false
nm = ["Power rule", "constant", "sum/difference", "product", "quotient", "chain"]
rule = [L"[x^n]' = n\cdot x^{n-1}",
L"[cf(x)]' = c \cdot f'(x)",
L"[f(x) \pm g(x)]' = f'(x) \pm g'(x)",
L"[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x)",
L"[f(x)/g(x)]' = (f'(x) \cdot g(x) - f(x) \cdot g'(x)) / g(x)^2",
L"[f(g(x))]' = f'(g(x)) \cdot g'(x)"]
d = DataFrame(Name=nm, Rule=rule)
table(d)
```
: Table summarizing different rules for derivatives
:::
This table gives some useful derivatives:
@tbl-derivatives-of-key-functions shows the derivative of a few key functions.
::: {#tbl-derivatives-of-key-functions .striped .hover}
```{julia}
#| hold: true
#| echo: false
fn = [L"x^n (\text{ all } n)",
L"e^x",
L"\log(x)",
L"\sin(x)",
L"\cos(x)"]
a = [L"nx^{n-1}",
L"e^x",
L"1/x",
L"\cos(x)",
L"-\sin(x)"]
d = DataFrame(Function=fn, Derivative=a)
table(d)
```
| Function | Derivative |
| ----------------------:| ----------:|
| $x^n (\text{ all } n)$ | $nx^{n-1}$ |
| $e^x$ | $e^x$ |
| $\log(x)$ | $1/x$ |
| $\sin(x)$ | $\cos(x)$ |
| $\cos(x)$ | $-\sin(x)$ |
Derivatives of some functions
:::
## Higher-order derivatives
@@ -1251,6 +1272,7 @@ Find the first $3$ derivatives of $f(x) = ax^3 + bx^2 + cx + d$.
Differentiating a polynomial is done with the sum rule, here we repeat three times:
$$
\begin{align*}
f(x) &= ax^3 + bx^2 + cx + d\\
@@ -1260,7 +1282,7 @@ f'''(x) &= 6a
\end{align*}
$$
We can see, the fourth derivative – and all higher order ones – would be identically $0$. This is part of a general phenomenon: an $n$th degree polynomial has only $n$ non-zero derivatives.
We can see, the fourth derivative---and all higher order ones---would be identically $0$. This is part of a general phenomenon: an $n$th degree polynomial has only $n$ non-zero derivatives.
---
@@ -1294,8 +1316,12 @@ We need the chain rule *and* the product rule:
$$
[e^{-x^2}]'' = [e^{-x^2} \cdot (-2x)]' = \left(e^{-x^2} \cdot (-2x)\right) \cdot(-2x) + e^{-x^2} \cdot (-2) =
e^{-x^2}(4x^2 - 2).
\begin{align*}
[e^{-x^2}]''
&= [e^{-x^2} \cdot (-2x)]' \\
&= \left(e^{-x^2} \cdot (-2x)\right) \cdot(-2x) + e^{-x^2} \cdot (-2) \\
&= e^{-x^2}(4x^2 - 2).
\end{align*}
$$
This can be verified:
@@ -1305,15 +1331,24 @@ This can be verified:
diff(diff(exp(-x^2))) |> simplify
```
Having to iterate the use of `diff` is cumbersome. An alternate notation is either specifying the variable twice: `diff(ex, x, x)` or using a number after the variable: `diff(ex, x, 2)`:
::: {.callout-note}
## higher order derivatives with `diff`
Having to iterate the use of `diff` is cumbersome. An alternate notation is either
* specifying the variable twice: `diff(ex, x, x)` or
* using a number after the variable: `diff(ex, x, 2)`:
For example,
```{julia}
diff(exp(-x^2), x, x) |> simplify
diff(exp(-x^2), x, x)
```
Higher-order derivatives can become involved when the product or quotient rules becomes involved.
:::
## Questions
@@ -1321,14 +1356,18 @@ Higher-order derivatives can become involved when the product or quotient rules
###### Question
The derivative at $c$ is the slope of the tangent line at $x=c$. Answer the following based on this graph:
The derivative at $c$ is the slope of the tangent line at $x=c$. Answer the following based on @fig-x-expx-sin-pix-over-0-2.
::: {#fig-x-expx-sin-pix-over-0-2}
```{julia}
#| echo: false
fn = x -> -x*exp(x)*sin(pi*x)
plot(fn, 0, 2)
```
Plot of function $f(x)$ over $[0,2]$
:::
At which of these points $c= 1/2, 1, 3/2$ is the derivative negative?
@@ -1363,15 +1402,18 @@ numericq(0, 1e-2)
###### Question
Consider the graph of the `airyai` function (from `SpecialFunctions`) over $[-5, 5]$.
Consider the graph of the `airyai` function (from `SpecialFunctions`) over $[-5, 5]$ in @fig-plot-airyai-over-minus5-5-identify-sign-of-derivative.
::: {#fig-plot-airyai-over-minus5-5-identify-sign-of-derivative}
```{julia}
#| hold: true
#| echo: false
plot(airyai, -5, 5)
```
Plot of `airyai` over $[-5,5]$
:::
At $x = -2.5$ the derivative is positive or negative?
@@ -1669,8 +1711,9 @@ radioq(trig_choices, 4)
###### Question
Consider this picture of composition:
Consider @fig-composition-of-functions-image, a picture of composition.
::: {#fig-composition-of-functions-image}
```{julia}
#| hold: true
@@ -1699,6 +1742,10 @@ l = @layout [a b]
plot(pf, pg, layout=l)
```
Plot of function $f(x)$ rotated so $x$ axis is pointing to the left; plot of $g(x)$ on right. This picture allows graphical composition to be computed.
:::
The right graph is of $g(x) = \exp(x)$ at $x=1$, the left graph of $f(x) = \sin(x)$ *rotated* $90$ degrees counter-clockwise. Chasing the arrows shows graphically how $f(g(1))$ can be computed. The nearby values $f(g(1+h))$ are – using the tangent line of $g$ at $x-1$ – approximated by $f(g(1) + g'(1)\cdot h)$, as shown in the graph segment on the left.
quarto/derivatives/first_second_derivatives.qmd
+483 -204
View File
File diff suppressed because it is too large. Load diff
quarto/derivatives/implicit_differentiation.qmd
+100 -56
View File
@@ -1,6 +1,5 @@
# Implicit differentiation
{{< include ../_common_code.qmd >}}
This section uses these add-on packages:
@@ -39,23 +38,26 @@ There are a few options for plotting equations in `Julia`. We will use a functi
To plot an implicit equation using `implicit_plot` requires expressing the relationship in terms of a function, and then plotting the equation `f(x,y) = 0`. In practice this simply requires all the terms be moved to one side of an equals sign.
To plot the circle of radius $2$, or the equations $x^2 + y^2 = 2^2$ we would move all terms to one side $x^2 + y^2 - 2^2 = 0$ and then express the left hand side through a function:
To plot the circle of radius $2$, or the equations $x^2 + y^2 = 2^2$ we would move all terms to one side $x^2 + y^2 - 2^2 = 0$ and then express the left hand side through a function with two arguments:
```{julia}
f(x,y) = x^2 + y^2 - 2^2
```
This function is then is passed to the `implicit_plot` function, which works with `Plots` to render the graphic:
This function is then is passed to the `implicit_plot` function, which works with `Plots` to render the graphic in @fig-implicit-plot-of-ellipse-with-radius-2.
::: {#fig-implicit-plot-of-ellipse-with-radius-2}
```{julia}
implicit_plot(f)
```
Plot of equation involving $f(x,y)$ created with `implicit_plot`.
:::
:::{.callout-note}
## Note
The `f` is a function of *two* variables, used here to express one side of an equation. `Julia` makes this easy to do - just make sure two variables are in the signature of `f` when it is defined. Using functions like this, we can express our equation in the form $f(x,y) = c$ or, more generally, as $f(x,y) = g(x,y)$. The latter of which can be expressed as $h(x,y) = f(x,y) - g(x,y) = 0$. That is, only the form $f(x,y)=0$ is needed to represent an equation.
The `f` is a function of *two* variables, used here to express one side of an equation. `Julia` makes this easy to do---just make sure two variables are in the signature of `f` when it is defined. Using functions like this, we can express our equation in the form $f(x,y) = c$ or, more generally, as $f(x,y) = g(x,y)$. The latter of which can be expressed as $h(x,y) = f(x,y) - g(x,y) = 0$. That is, only the form $f(x,y)=0$ is needed to represent an equation.
:::
@@ -65,7 +67,7 @@ There are two different styles in `Julia` to add simple plot recipes. `ImplicitP
:::
Of course, more complicated equations are possible and the steps are similar - only the function definition is more involved. For example, the [Devils curve](http://www-groups.dcs.st-and.ac.uk/~history/Curves/Devils.html) has the form
Of course, more complicated equations are possible and the steps are similar---only the function definition is more involved. For example, the [Devil's curve](http://www-groups.dcs.st-and.ac.uk/~history/Curves/Devils.html) has the form
$$
@@ -74,7 +76,7 @@ $$
Here we draw the curve for a particular choice of $a$ and $b$. For illustration purposes, a narrower viewing window is specified below using `xlims` and `ylims`:
::: {#fig-implicit-plot-of-devils-curve}
```{julia}
#| hold: true
a,b = -1,2
@@ -82,6 +84,9 @@ f(x,y) = y^4 - x^4 + a*y^2 + b*x^2
implicit_plot(f; xlims=(-3,3), ylims=(-3,3), legend=false)
```
Plot of Devil's curve for a certain choice of parameter
:::
## Tangent lines, implicit differentiation
@@ -91,7 +96,7 @@ The graph $x^2 + y^2 = 1$ has well-defined tangent lines at all points except $(
In general though, we may not be able to solve for $y$ in terms of $x$. What then?
The idea is to *assume* that $y$ is representable by some function of $x$. This makes sense, moving on the curve from $(x,y)$ to some nearby point, means changing $x$ will cause some change in $y$. This assumption is only made *locally* - basically meaning a complicated graph is reduced to just a small, well-behaved, section of its graph.
The idea is to *assume* that $y$ is representable by some function of $x$. This makes sense, moving on the curve from $(x,y)$ to some nearby point, means changing $x$ will cause some change in $y$. This assumption is only made *locally*---basically meaning a complicated graph is reduced to just a small, well-behaved, section of its graph.
::: {#fig-well-behaved-section}
@@ -153,10 +158,10 @@ $$
This says the slope of the tangent line depends on the point $(x,y)$ through the formula $-x/y$.
As a check, we compare to what we would have found had we solved for $y= \sqrt{1 - x^2}$ (for $(x,y)$ with $y \geq 0$). We would have found: $dy/dx = 1/2 \cdot 1/\sqrt{1 - x^2} \cdot (-2x)$. Which can be simplified to $-x/y$. This should show that the method above - assuming $y$ is a function of $x$ and differentiating - is not only more general, but can even be easier.
As a check, we compare to what we would have found had we solved for $y= \sqrt{1 - x^2}$ (for $(x,y)$ with $y \geq 0$). We would have found: $dy/dx = 1/2 \cdot 1/\sqrt{1 - x^2} \cdot (-2x)$. Which can be simplified to $-x/y$. This should show that the method above---assuming $y$ is a function of $x$ and differentiating---is not only more general, but can even be easier.
The name - *implicit differentiation* - comes from the assumption that $y$ is implicitly defined in terms of $x$. According to the [Implicit Function Theorem](http://en.wikipedia.org/wiki/Implicit_function_theorem) the above method will work provided the curve has sufficient smoothness near the point $(x,y)$. (Continuously differentiable and non vanishing derivative in $y$.)
The name---*implicit differentiation*---comes from the assumption that $y$ is implicitly defined in terms of $x$. According to the [Implicit Function Theorem](http://en.wikipedia.org/wiki/Implicit_function_theorem) the above method will work provided the curve has sufficient smoothness near the point $(x,y)$. (Continuously differentiable and non vanishing derivative in $y$.)
##### Examples
@@ -169,20 +174,23 @@ $$
x^2y + a\cdot b \cdot y - a^2 \cdot x = 0, \quad a\cdot b > 0.
$$
For $a = 2, b=1$ we have the graph:
For $a = 2, b=1$ we have the graph in @fig-serpentine-a-2-b-1.
::: {#fig-serpentine-a-2-b-1}
```{julia}
#| hold: true
a, b = 2, 1
f(x,y) = x^2*y + a * b * y - a^2 * x
implicit_plot(f; legend=false)
x₀, y₀ = 0, 0
m = (a^2 - 2x₀*y₀) / (a*b + x₀^2)
plot!(x -> y₀ + m*(x - x₀), -1, 1)
x0, y0 = 0, 0
m = (a^2 - 2x0*y0) / (a*b + x0^2)
plot!(x -> y0 + m*(x - x0), -1, 1)
```
Plot of serpentine equation $x^2y + 2 \cdot 1 \cdot y - 2^2 \cdot x$ along with a tangent line.
:::
To the plot we added a tangent line at $(0,0)$.
We can see that at each point in the viewing window the tangent line exists due to the smoothness of the curve. To find the slope of the tangent line at a point $(x,y)$ the tangent line will have slope $dy/dx$ satisfying:
@@ -214,7 +222,7 @@ $$
A graph for $a=3$ shows why it has the name it does:
::: {#fig-plot-eight-curve{
```{julia}
#| hold: true
a = 3
@@ -222,6 +230,9 @@ f(x,y) = x^4 - a^2*(x^2 - y^2)
implicit_plot(f; xticks=-5:5)
```
Plot of eight curve when $a=3$
:::
The tangent line at $(x,y)$ will have slope, $dy/dx$ satisfying:
@@ -277,7 +288,7 @@ Let's see how to add a graph of a tangent line to the graph of an equation. Tang
Returning to the equation for a circle, $x^2 + y^2 = 1$, let's look at $(\sqrt{2}/2, - \sqrt{2}/2)$. The derivative is $-x/y$, so the slope at this point is $1$. The line itself has equation $y = b + m \cdot (x-a)$. The following represents this in `Julia`:
::: {#fig-adding-tangent-line-as-layer}
```{julia}
#| hold: true
F(x,y) = x^2 + y^2 - 1
@@ -291,7 +302,10 @@ implicit_plot(F, xlims=(-2, 2), ylims=(-2, 2), aspect_ratio=:equal)
plot!(tl)
```
We added *both* the implicit plot of $F$ and the tangent line to the graph at the given point.
Implicit plot of $F(x,y)$ along with a tangent line
:::
We added *both* the implicit plot of $F$ and the tangent line to @fig-adding-tangent-line-as-layer at the given point.
##### Example
@@ -310,19 +324,27 @@ $$
Setting $a=1$ we have the graph:
::: {#fig-folium-of-descartes-plotted-implicitly}
```{julia}
a = 1
G(x,y) = x^3 + y^3 - 3*a*x*y
implicit_plot(G)
```
Plot of folium of Descartes when $a=1$
:::
We can solve for the lower curve, $y$, as a function of $x$, as follows:
::: {#fig-folium-of-descartes-plotted-by-finding-minimum}
```{julia}
y1(x) = minimum(find_zeros(y->G(x,y), -10, 10)) # find_zeros from `Roots`
```
Numerically finding a function for a part of an equation
:::
This gives the lower part of the curve, which we can plot with:
@@ -407,7 +429,7 @@ end
Let $a = b = c = d = 1$, then $(1,4)$ is a point on the curve. We can draw a tangent line to this point with these commands:
::: {#fig-implicit-plot-of-trident-of-newton}
```{julia}
H = ex(a=>1, b=>1, c=>1, d=>1)
x0, y0 = 1, 4
@@ -416,7 +438,10 @@ implicit_plot(lambdify(H); xlims=(-5,5), ylims=(-5,5), legend=false)
plot!(y0 + m * (x-x0))
```
Basically this includes all the same steps as if done "by hand." Some effort could have been saved in plotting, had values for the parameters been substituted initially, but not doing so shows their dependence in the derivative.
Plot of trident of Newton along with a tangent line for a given point
:::
Basically this includes all the same steps as if done "by hand". Some effort could have been saved in plotting, had values for the parameters been substituted initially, but not doing so shows their dependence in the derivative.
:::{.callout-warning}
@@ -444,7 +469,7 @@ $$
3x^2 - (3y^2 \frac{dy}{dx}) = 0.
$$
We could solve for $dy/dx$ at this point - it always appears as a linear factor - to get:
We could solve for $dy/dx$ at this point---it always appears as a linear factor---to get:
$$
@@ -475,18 +500,21 @@ $$
It isn't so pretty, but that's all it takes.
To visualize, we plot implicitly and notice that:
To visualize, we plot implicitly in @fig-implicit-plot-K and notice that:
* as we change quadrants from the third to the fourth to the first the concavity changes from down to up to down, as the sign of the second derivative changes from negative to positive to negative;
* and that at these inflection points, the "tangent" line is vertical when $y=0$ and flat when $x=0$.
::: {#fig-implicit-plot-K}
```{julia}
K(x,y) = x^3 - y^3 - 3
implicit_plot(K, xlims=(-3, 3), ylims=(-3, 3))
```
Implicit plot of cubic equation, $K$
:::
The same problem can be done symbolically. The steps are similar, though the last step (replacing $x^3 - y^3$ with $3$) isn't done without explicitly asking.
@@ -496,8 +524,8 @@ The same problem can be done symbolically. The steps are similar, though the las
eqn = K(x,y)
eqn1 = eqn(y => u(x))
dydx = solve(diff(eqn1,x), diff(u(x), x))[1] # 1 solution
d2ydx2 = solve(diff(eqn1, x, 2), diff(u(x),x, 2))[1] # 1 solution
dydx = only(solve(diff(eqn1,x), diff(u(x), x))) # 1 solution
d2ydx2 = only(solve(diff(eqn1, x, 2), diff(u(x),x, 2))) # 1 solution
eqn2 = subs(d2ydx2, diff(u(x), x) => dydx, u(x) => y)
simplify(eqn2)
```
@@ -514,7 +542,7 @@ In short, both $f \circ g$ and $g \circ f$ are identify functions on their respe
The chain rule can be used to give the derivative of an inverse function when applied to $f(f^{-1}(x)) = x$. Solving gives, $[f^{-1}(x)]' = 1 / f'(f^{-1}(x))$.
This is great - if we can remember the rules. If not, sometimes implicit differentiation can also help.
This is great---if we can remember the rules. If not, sometimes implicit differentiation can also help.
Consider the inverse function for the tangent, which exists when the domain of the tangent function is restricted to $(-\pi/2, \pi/2)$. The function solves $y = \tan^{-1}(x)$ or $\tan(y) = x$. Differentiating this yields:
@@ -566,7 +594,7 @@ This problem illustrates one best done with implicit derivatives. A video showin
Well, suppose you hold the rope in two places, which we can take to be $(0,0)$ and $(a,b)$. Then let $(x,y)$ be all the possible positions of the ring that hold the rope taught. Then we have this picture:
::: {#fig-string-with-a-weight}
```{julia}
#| hold: true
#| echo: false
@@ -594,18 +622,17 @@ let
(1/2, 0, text(L"x",:top)),
(5/2, 1, text(L"a-x", :top)),
(1, -1, text(L"|y|",:right)),
(1+Δ, -1, text(L"b-y",:left)),
(1+Δ, -1/2, text(L"b-y",:left)),
(1+2a, -3, text(L"(x,y)",:left))
])
plotly()
current()
end
```
```{julia}
#| echo: false
plotly()
nothing
```
Picture of string with fixed end points weighted down
:::
Since the length of the rope does not change, we must have for any admissible $(x,y)$ that:
@@ -638,11 +665,14 @@ F₀(x, y) = F₀(x, y, a, b)
Our values $(x,y)$ must satisfy $f(x,y) = L$. Let's graph:
::: {#fig-implicit-plot-of-Fzero-is-an-ellipse}
```{julia}
implicit_plot((x,y) -> F₀(x,y) - L, xlims=(-5, 7), ylims=(-5, 7))
```
Implicit plot of $F_0 -L$ is an ellipse
:::
The graph is an ellipse, though slightly tilted.
@@ -680,7 +710,7 @@ The values of `dydx` depend on any pair $(x,y)$, but our solution must also sati
```{julia}
eqn1 = eqn(x => cps[2])
eqn1 = eqn(x => last(cps))
```
We would try to solve `eqn1` for `y` with `solve(eqn1, y)`, but `SymPy` can't complete this problem. Instead, we will approach this numerically using `find_zero` from the `Roots` package. We make the above a function of `y` alone
@@ -700,7 +730,7 @@ xstar = N(cps[2](y => ystar, a =>3, b => 3))
Our minimum is at `(xstar, ystar)`, as this graphic shows:
::: {#fig-implicit-plot-F0-10}
```{julia}
tl(x) = ystar + 0 * (x- xstar)
@@ -708,6 +738,10 @@ implicit_plot((x,y) -> F₀(x,y,3,3) - 10, xlims=(-4, 7), ylims=(-10, 10))
plot!(tl)
```
Implicit plot of $F_0 - L$ for a sest of values.
:::
If you watch the video linked to above, you will see that the surprising fact here is the resting point is such that the angles formed by the rope are the same. Basically this makes the tension in both parts of the rope equal, so there is a static position (if not static, the ring would move and not end in the final position). We can verify this fact numerically by showing the arctangents of the two triangles are the same up to a sign:
@@ -731,12 +765,15 @@ Here we try the same problem numerically, using a zero-finding approach to ident
Starting with $F(x,y) = \sqrt{x^2 + y^2} + \sqrt{(x-1)^2 + (y-2)^2}$ and $L=3$, we have:
::: {@fig-implicit-plot-of-F0-L-with-values-found-numerically}
```{julia}
F₁(x,y) = F₀(x,y, 1, 2) - 3 # a,b,L = 1,2,3
implicit_plot(F₁)
```
Plot of $F - L$
:::
Trying to find the lowest $y$ value we have from the graph it is near $x=0.1$. We can do better.
@@ -762,12 +799,15 @@ This shows the smallest value is around $-0.414$ and occurs in the $33$rd positi
With this slight modification, we have:
::: {#fig-plot-of-inverse-F1-numerically-found}
```{julia}
f₁(x) = find_zero(y -> F₁(x, y), zero(x))
plot(f₁', -0.5, 1.5)
```
Plot of $f_1$ found by solving $F_1(x,y(x)) = 0$.
:::
The zero of `f'` is a bit to the right of $0$, say $0.2$; we use `find_zero` again to find it:
@@ -891,8 +931,8 @@ choices = [
"``b \\cdot (1 - (x/a)^n)^{1/n}``",
"``-(x/a)^n / (y/b)^n``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1013,8 +1053,8 @@ choices = [
"``2xy / (x^2 + a^2)``",
"``a^3/(x^2 + a^2)``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
In @fig-witch-agnesi for a given $\theta$ the point $P = (x,y)$ where $x$ is the $x$ value of the intersection of the drawn line with the line $y=a$ and $y$ is the $y$ value of the intersection of the drawn line with the circle $x^2 + y^2 = a^2$.
@@ -1026,8 +1066,9 @@ Suppose $O=(0,0)$ and $A=(u,v)$. Which of these formulas is true:
choices = [
L"(v+a)/u = 2a/u = \tan(\theta)",
L"v/u = a/u = \tan(\theta)"
],
radioq(choices, 1)
]
answer = 1
radioq(choices, answer)
```
Suppose $B=(u,v)$. Which of these is true:
@@ -1055,10 +1096,13 @@ Image number 35 from L'Hospitals calculus book (the first). Given a description
#ImageFile(:derivatives, imgfile, caption)
nothing
```
::: {#fig-image-35-lhospital}
![](figures/fcarc-may2016-fig35-350.png){fig-alt="image 35 of L'Hospital's book"}
![Image number 35 from L'Hospitals calculus book (the first). Given a description of the curve, identify the point ``E`` which maximizes the height.](figures/fcarc-may2016-fig35-350.png)
Image number 35 from L'Hospitals calculus book (the first). Given a description of the curve, identify the point ``E`` which maximizes the height.
:::
The figure above shows a problem appearing in L'Hospital's first calculus book. Given a function defined implicitly by $x^3 + y^3 = axy$ (with $AP=x$, $PM=y$ and $AB=a$) find the point $E$ that maximizes the height. In the [AMS feature column](http://www.ams.org/samplings/feature-column/fc-2016-05) this problem is illustrated and solved in the historical manner, with the comment that the concept of implicit differentiation wouldn't have occurred to L'Hospital.
@fig-image-35-lhospital shows a problem appearing in L'Hospital's first calculus book. Given a function defined implicitly by $x^3 + y^3 = axy$ (with $AP=x$, $PM=y$ and $AB=a$) find the point $E$ that maximizes the height. In the [AMS feature column](http://www.ams.org/samplings/feature-column/fc-2016-05) this problem is illustrated and solved in the historical manner, with the comment that the concept of implicit differentiation wouldn't have occurred to L'Hospital.
Using Implicit differentiation, find when $dy/dx = 0$.
@@ -1068,8 +1112,8 @@ Using Implicit differentiation, find when $dy/dx = 0$.
#| hold: true
#| echo: false
choices = ["``y^2 = 3x/a``", "``y=3x^2/a``", "``y=a/(3x^2)``", "``y^2=a/(3x)``"]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
Substituting the correct value of $y$, above, into the defining equation gives what value for $x$:
@@ -1084,8 +1128,8 @@ choices=[
"``x=(1/2) a^3 3^{1/3}``",
"``x=(1/3) a^2 2^{1/2}``"
]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
###### Question
@@ -1118,8 +1162,8 @@ If $y>0$ is the sign positive or negative?
#| hold: true
#| echo: false
choices = ["positive", "negative", "Can be both"]
answ = 2
radioq(choices, answ, keep_order=true)
answer = 2
radioq(choices, answer, keep_order=true)
```
If $x>0$ is the sign positive or negative?
@@ -1129,8 +1173,8 @@ If $x>0$ is the sign positive or negative?
#| hold: true
#| echo: false
choices = ["positive", "negative", "Can be both"]
answ = 3
radioq(choices, answ, keep_order=true)
answer = 3
radioq(choices, answer, keep_order=true)
```
When $x>0$, the graph of the equation is...
@@ -1140,6 +1184,6 @@ When $x>0$, the graph of the equation is...
#| hold: true
#| echo: false
choices = ["concave up", "concave down", "both concave up and down"]
answ = 3
radioq(choices, answ, keep_order=true)
answer = 3
radioq(choices, answer, keep_order=true)
```
quarto/derivatives/jsxgraph-newton.qmd
+95
View File
@@ -0,0 +1,95 @@
XXX do we include this? XXX
---
This interactive graphic (built using [JSXGraph](https://jsxgraph.uni-bayreuth.de/wp/index.html)) allows the adjustment of the point `x0`, initially at $0.85$. Five iterations of Newton's method are illustrated. Different positions of `x0` clearly converge, others will not.
::: {#fig-jsxgraph-newtons-method}
```{=html}
<div id="jsxgraph" style="width: 500px; height: 500px;"></div>
```
```{ojs}
//| echo: false
//| output: false
JXG = require("jsxgraph");
// newton's method
b = JXG.JSXGraph.initBoard('jsxgraph', {
boundingbox: [-3,5,3,-5], axis:true
});
f = function(x) {return x*x*x*x*x - x - 1};
fp = function(x) { return 4*x*x*x*x - 1};
x0 = 0.85;
nm = function(x) { return x - f(x)/fp(x);};
l = b.create('point', [-1.5,0], {name:'', size:0});
r = b.create('point', [1.5,0], {name:'', size:0});
xaxis = b.create('line', [l,r])
P0 = b.create('glider', [x0,0,xaxis], {name:'x0'});
P0a = b.create('point', [function() {return P0.X();},
function() {return f(P0.X());}], {name:''});
P1 = b.create('point', [function() {return nm(P0.X());},
0], {name:''});
P1a = b.create('point', [function() {return P1.X();},
function() {return f(P1.X());}], {name:''});
P2 = b.create('point', [function() {return nm(P1.X());},
0], {name:''});
P2a = b.create('point', [function() {return P2.X();},
function() {return f(P2.X());}], {name:''});
P3 = b.create('point', [function() {return nm(P2.X());},
0], {name:''});
P3a = b.create('point', [function() {return P3.X();},
function() {return f(P3.X());}], {name:''});
P4 = b.create('point', [function() {return nm(P3.X());},
0], {name:''});
P4a = b.create('point', [function() {return P4.X();},
function() {return f(P4.X());}], {name:''});
P5 = b.create('point', [function() {return nm(P4.X());},
0], {name:'x5', strokeColor:'black'});
P0a.setAttribute({fixed:true});
P1.setAttribute({fixed:true});
P1a.setAttribute({fixed:true});
P2.setAttribute({fixed:true});
P2a.setAttribute({fixed:true});
P3.setAttribute({fixed:true});
P3a.setAttribute({fixed:true});
P4.setAttribute({fixed:true});
P4a.setAttribute({fixed:true});
P5.setAttribute({fixed:true});
sc = '#000000';
b.create('segment', [P0,P0a], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P0a, P1], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P1,P1a], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P1a, P2], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P2,P2a], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P2a, P3], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P3,P3a], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P3a, P4], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P4,P4a], {strokeColor:sc, strokeWidth:1});
b.create('segment', [P4a, P5], {strokeColor:sc, strokeWidth:1});
b.create('functiongraph', [f, -1.5, 1.5])
```
Interactive means to explore Newton's method for varying starting points
:::
quarto/derivatives/lhospitals_rule.qmd
+35 -28
View File
@@ -17,7 +17,7 @@ using SymPy
---
Let's return to limits of the form $\lim_{x \rightarrow c}f(x)/g(x)$ which have an indeterminate form of $0/0$ if both are evaluated at $c$. The typical example being the limit considered by Euler:
Let's return to limits of the form $\lim_{x \rightarrow c}f(x)/g(x)$ which have an indeterminate form of $0/0$ when are evaluated at $c$. The typical example being the limit considered by Euler:
$$
@@ -41,7 +41,7 @@ $$
This is because we know $\sin(\xi) \frac{x}{2}$ has a limit of $0$, when $|\xi| \leq |x|$.
That doesn't look any easier, as we worried about the error term, but if just mentally replaced $\sin(x)$ with $x$ - which it basically is near $0$ - then we can see that the limit should be the same as $x/x$ which we know is $1$ without thinking.
That doesn't look any easier, as we worried about the error term, but if just mentally replaced $\sin(x)$ with $x$---which it basically is near $0$---then we can see that the limit should be the same as $x/x$ which we know is $1$ without thinking.
Basically, we found that in terms of limits, if both $f(x)$ and $g(x)$ are $0$ at $c$, that we *might* be able to just take this limit: $(f(c) + f'(c) \cdot(x-c)) / (g(c) + g'(c) \cdot (x-c))$ which is just $f'(c)/g'(c)$.
@@ -50,16 +50,17 @@ Basically, we found that in terms of limits, if both $f(x)$ and $g(x)$ are $0$ a
Wouldn't that be nice? We could find difficult limits just by differentiating the top and the bottom at $c$ (and not use the messy quotient rule).
Well, in fact that is more or less true, a fact that dates back to [L'Hospital](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) - who wrote the first textbook on differential calculus - though this result is likely due to one of the Bernoulli brothers.
Well, in fact that is more or less true, a fact that dates back to [L'Hospital](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) - who wrote the first textbook on differential calculus---though this result is likely due to one of the Bernoulli brothers.
::: {.callout-note icon=false}
## L'Hospital's rule
::: {.definition title="L'Hospital's rule"}
Suppose:
Suppose for $f$ and $g$ that:
* that $\lim_{x\rightarrow c+} f(c) =0$ and $\lim_{x\rightarrow c+} g(c) =0$,
* that $f$ and $g$ are differentiable in $(c,b)$, and
* that $g(x)$ exists and is non-zero for *all* $x$ in $(c,b)$,
* $\lim_{x\rightarrow c+} f(c) =0$ and $\lim_{x\rightarrow c+} g(c) =0$,
* $f$ and $g$ are differentiable in $(c,b)$, and
* $g(x)$ exists and is non-zero for *all* $x$ in $(c,b)$,
then **if** the following limit exists: $\lim_{x\rightarrow c+}f'(x)/g'(x)=L$ it follows that $\lim_{x \rightarrow c+}f(x)/g(x) = L$.
@@ -94,7 +95,7 @@ In [Gruntz](http://www.cybertester.com/data/gruntz.pdf), in a reference attribut
##### Examples
* Consider this limit at $0$: $(a^x - 1)/x$. We have $f(x) =a^x-1$ has $f(0) = 0$, so this limit is indeterminate of the form $0/0$. The derivative of $f(x)$ is $f'(x) = a^x \log(a)$ which has $f'(0) = \log(a)$. The derivative of the bottom is also $1$ at $0$, so we have:
* Consider this limit at $0$: $(a^x - 1)/x$. We have $f(x) =a^x-1$ has $f(0) = 0$, so this limit is indeterminate of the form $0/0$. The derivative of $f(x)$ is $f'(x) = a^x \log(a)$ which has $f'(0) = \log(a)$. The derivative of the bottom is also $1$ at $0$, so we have:
$$
@@ -104,11 +105,11 @@ $$
:::{.callout-note}
## Note
Why rewrite in the "opposite" direction? Because the theorem's result – $L$ is the limit – is only true if the related limit involving the derivative exists. We don't do this in the following, but did so here to emphasize the need for the limit of the ratio of the derivatives to exist.
Why rewrite in the "opposite" direction? Because the theorem's result---$L$ is the limit---is only true if the related limit involving the derivative exists. We don't do this in the following, but did so here to emphasize the need for the limit of the ratio of the derivatives to exist to apply the theorem.
:::
* Consider this limit:
* Consider this limit:
$$
@@ -118,7 +119,7 @@ $$
It too is of the indeterminate form $0/0$. The derivative of the top is $e^x + e^{-x}$, which is $2$ when $x=0$, so the ratio of $f'(0)/g'(0)$ is seen to be $2$. By continuity, the limit of the ratio of the derivatives is $2$. Then by L'Hospital's rule, the limit above is $2$.
* Sometimes, L'Hospital's rule must be applied twice. Consider this limit:
* Sometimes, L'Hospital's rule must be applied twice. Consider this limit:
$$
@@ -142,7 +143,7 @@ $$
As $L = 1/2$ for this related limit, it must also be the limit of the original problem, by L'Hospital's rule.
* Our "intuitive" limits can bump into issues. Take for example the limit of $(\sin(x)-x)/x^2$ as $x$ goes to $0$. Using $\sin(x) \approx x$ makes this look like $0/x^2$ which is still indeterminate. (Because the difference is higher order than $x$.) Using L'Hospitals, says this limit will exist (and be equal) if the following one does:
* Our "intuitive" limits can bump into issues. Take for example the limit of $(\sin(x)-x)/x^2$ as $x$ goes to $0$. Using $\sin(x) \approx x$ makes this look like $0/x^2$ which is still indeterminate. (Because the difference is higher order than $x$.) Using L'Hospitals, says this limit will exist (and be equal) if the following one does:
$$
@@ -159,7 +160,7 @@ $$
So as this limit exists, working backwards, the original limit in question will also be $0$.
* This example comes from the Wikipedia page. It "proves" a discrete approximation for the second derivative.
* This example comes from the Wikipedia page. It "proves" a discrete approximation for the second derivative.
Show if $f''(x)$ exists at $c$ and is continuous at $c$, then
@@ -180,8 +181,11 @@ We have to be careful, as we differentiate in the $h$ variable, not the $c$ one,
$$
\lim_{h \rightarrow 0} \frac{f''(c+h) - 0 - (-f''(c-h))}{2} =
\lim_{c \rightarrow 0}\frac{f''(c+h) + f''(c-h)}{2} = f''(c).
\begin{align*}
\lim_{h \rightarrow 0} \frac{f''(c+h) - 0 - (-f''(c-h))}{2}
&= \lim_{c \rightarrow 0}\frac{f''(c+h) + f''(c-h)}{2}\\
&= f''(c).
\end{align*}
$$
That last equality follows, as it is assumed that $f''(x)$ exists at $c$ and is continuous, that is, $f''(c \pm h) \rightarrow f''(c)$.
@@ -190,7 +194,7 @@ That last equality follows, as it is assumed that $f''(x)$ exists at $c$ and is
The expression above finds use when second derivatives are numerically approximated. (The middle expression is the basis of the central-finite difference approximation to the derivative.)
* L'Hospital himself was interested in this limit for $a > 0$ ([math overflow](http://mathoverflow.net/questions/51685/how-did-bernoulli-prove-lh%C3%B4pitals-rule))
* L'Hospital himself was interested in this limit for $a > 0$ ([math overflow](http://mathoverflow.net/questions/51685/how-did-bernoulli-prove-lh%C3%B4pitals-rule))
$$
@@ -241,9 +245,9 @@ limit(f(x)/g(x), x => a)
A first proof of L'Hospital's rule takes advantage of Cauchy's [generalization](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) of the mean value theorem to two functions. Suppose $f(x)$ and $g(x)$ are continuous on $[c,b]$ and differentiable on $(c,b)$. On $(c,x)$, $c < x < b$ there exists a $\xi$ with $f'(\xi) \cdot (g(x) - g(c)) = g'(\xi) \cdot (f(x) - f(c))$. In our formulation, both $f(c)$ and $g(c)$ are zero, so we have, provided we know that $g(x)$ is non zero, that $f(x)/g(x) = f'(\xi)/g'(\xi)$ for some $\xi$, $c < \xi < x$. That the right-hand side has a limit as $x \rightarrow c+$ is true by the assumption that the limit of the ratio of the derivatives exists. (The $\xi$ part can be removed by considering it as a composition of a function going to $c$.) Thus the right limit of the ratio $f/g$ is known.
---
::: {#fig-lhospitals-picture-graph-animation}
```{julia}
#| echo: false
#| cache: true
@@ -277,14 +281,6 @@ function lhopitals_picture_graph(n)
end
caption = raw"""
Geometric interpretation of
``L=\lim_{x \rightarrow 0} x^2 / (\sqrt{1 + x} - 1 - x^2)``.
At ``0`` this limit is indeterminate of the form
``0/0``. The value for a fixed ``x`` can be seen as the slope of a secant
line of a parametric plot of the two functions, plotted as
``(g, f)``. In this figure, the limiting "tangent" line has ``0`` slope,
corresponding to the limit ``L``. In general, L'Hospital's rule is
nothing more than a statement about slopes of tangent lines.
"""
@@ -302,7 +298,18 @@ gif(anim, imgfile, fps = 1)
end
```
## Generalizations
Geometric interpretation of
$L=\lim_{x \rightarrow 0} x^2 / (\sqrt{1 + x} - 1 - x^2)$.
At $0$ this limit is indeterminate of the form
$0/0$. The value for a fixed $x$ can be seen as the slope of a secant
line of a parametric plot of the two functions, plotted as
$(g, f)$. In this figure, the limiting "tangent" line has $0$ slope,
corresponding to the limit $L$. In general, L'Hospital's rule is
nothing more than a statement about slopes of tangent lines.
:::
## Direct generalizations to the $\infty/\infty$ case and the $c=\infty$ case
L'Hospital's rule generalizes to other indeterminate forms, in particular the indeterminate form $\infty/\infty$ can be proved at the same time as $0/0$ with a more careful [proof](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule#General_proof).
quarto/derivatives/linearization.qmd
+90 -215
View File
@@ -20,7 +20,7 @@ using DualNumbers
---
The derivative of $f(x)$ has the interpretation as the slope of the tangent line. The tangent line is the line that best approximates the function at the point.
The derivative of $f(x)$, when it exists, is the slope of the tangent line; the tangent line being the line that best approximates the function at the point.
Using the point-slope form of a line, we see that the tangent line to the graph of $f(x)$ at $(c,f(c))$ is given by:
@@ -48,17 +48,20 @@ tangent(sin, pi/4)
```
We make some graphs with tangent lines:
In @fig-plot-x-squared-two-tangent-lines-at-minus-1-2 we make some graphs with tangent lines.
::: {#fig-plot-x-squared-two-tangent-lines-at-minus-1-2}
```{julia}
#| hold: true
f(x) = x^2
plot(f, -3, 3)
plot!(tangent(f, -1))
plot!(tangent(f, 2))
plot(f, -3, 3; label="f", ylims=(-10, 10))
plot!(tangent(f, -1); label="tangent line at -1")
plot!(tangent(f, 2); label="tangent line at 2")
```
Plot of $f(x) = x^2$ over $[-3,3]$ with tangent lines drawn at $x=2$ and $x=-1$
:::
The graph shows that near the point, the line and function are close, but this need not be the case away from the point. We can express this informally as
@@ -69,10 +72,13 @@ $$
with the understanding this applies for $x$ "close" to $c$.
Usually for the applications herein, instead of $x$ and $c$ the two points are $x+\Delta_x$ and $x$. This gives:
Usually for the applications herein, instead of $x$ and $c$ the two points are labeled $x+\Delta_x$ and $x$. This gives:
::: {.definition title="Linearization"}
> *Linearization*: $\Delta_y = f(x +\Delta_x) - f(x) \approx f'(x) \Delta_x$, for small $\Delta_x$.
$\Delta_y = f(x +\Delta_x) - f(x) \approx f'(x) \Delta_x$, for small $\Delta_x$.
:::
@@ -85,15 +91,18 @@ This section gives some implications of this fact and quantifies what "close" ca
There are several approximations that are well known in physics, due to their widespread usage:
* That $\sin(x) \approx x$ around $x=0$:
* That $\sin(x) \approx x$ around $x=0$:
::: {#fig-plot-sin-tangent-line-at-0}
```{julia}
#| hold: true
plot(sin, -pi/2, pi/2)
plot!(tangent(sin, 0))
plot(sin, -pi/2, pi/2; label="sin")
plot!(tangent(sin, 0); label="tangent line at 0")
```
Plot of $\sin(x)$ and its tangent line at $0$
:::
Symbolically:
@@ -105,16 +114,19 @@ f(x) = sin(x)
f(c) + diff(f(x),x)(c) * (x - c)
```
* That $\log(1 + x) \approx x$ around $x=0$:
* That $\log(1 + x) \approx x$ around $x=0$:
::: {#fig-plot-log1p-x-and-tangent-at-0}
```{julia}
#| hold: true
f(x) = log(1 + x)
plot(f, -1/2, 1/2)
plot!(tangent(f, 0))
plot(f, -1/2, 1/2; label="sin")
plot!(tangent(f, 0); label="tangent line at 0")
```
Plot of $f(x) = log(1 + x)$ and its tangent line at $x=0$
:::
Symbolically:
@@ -129,16 +141,19 @@ f(c) + diff(f(x),x)(c) * (x - c)
(The `log1p` function implements a more accurate version of this function when numeric values are needed.)
* That $1/(1-x) \approx 1+x$ around $x=0$:
* That $1/(1-x) \approx 1+x$ around $x=0$:
::: {#fig-plot-1-over-1-minus-x-over-minus-one-half-one-half}
```{julia}
#| hold: true
f(x) = 1/(1-x)
plot(f, -1/2, 1/2)
plot!(tangent(f, 0))
plot(f, -1/2, 1/2; label="f")
plot!(tangent(f, 0); label="tangent line at 0")
```
Plot of $1/(1-x)$ and its tangent line at $x=0$
:::
Symbolically:
@@ -150,17 +165,21 @@ f(x) = 1 / (1 - x)
f(c) + diff(f(x),x)(c) * (x - c)
```
* That $(1+x)^n \approx 1 + nx$ around $x = 0$. For example, with $n=5$
* That $(1+x)^n \approx 1 + nx$ around $x = 0$. For example, with $n=5$
::: {#fig-plot-1-plus-x-to-the-n-and-tangent-line}
```{julia}
#| hold: true
n = 5
f(x) = (1+x)^n # f'(0) = n = n(1+x)^(n-1) at x=0
plot(f, -1/2, 1/2)
plot!(tangent(f, 0))
plot(f, -1/2, 1/2; label="f")
plot!(tangent(f, 0); label="tangent line at 0")
```
Plot of $f(x) = (1 + x)^5$ and its tangent line at $0$
:::
Symbolically:
@@ -227,7 +246,7 @@ plotly()
nothing
```
The plot in @fig-tangent-dy-dx shows a tangent line with slope $dy/dx$ and the actual change in $y$, $\Delta y$, for some specified $\Delta x$ at a point $(c,f(c))$. The small gap above the sine curve is the error were the value of the sine approximated using the drawn tangent line. We can see that approximating the value of $\Delta y = \sin(c+\Delta x) - \sin(c)$ with the often easier to compute $(dy/dx) \cdot \Delta x = f'(c)\Delta x$ - for small enough values of $\Delta x$ - is not going to be too far off provided $\Delta x$ is not too large.
The plot in @fig-tangent-dy-dx shows a tangent line with slope $dy/dx$ and the actual change in $y$, $\Delta y$, for some specified $\Delta x$ at a point $(c,f(c))$. The small gap above the sine curve is the error were the value of the sine approximated using the drawn tangent line. We can see that approximating the value of $\Delta y = \sin(c+\Delta x) - \sin(c)$ with the often easier to compute $(dy/dx) \cdot \Delta x = f'(c)\Delta x$---for small enough values of $\Delta x$---is not going to be too far off provided $\Delta x$ is not too large.
This approximation is known as linearization. It can be used both in theoretical computations and in practical applications. To see how effective it is, we look at some examples.
@@ -389,7 +408,7 @@ Vₛ(r) = 4/3 * pi * r^3
A *simple* pendulum is comprised of a massless "bob" on a rigid "rod" of length $l$. The rod swings back and forth making an angle $\theta$ with the perpendicular. At rest $\theta=0$, here we have $\theta$ swinging with $\lvert\theta\rvert \leq \theta_0$ for some $\theta_0$.
According to [Wikipedia](http://tinyurl.com/yz5sz7e) - and many introductory physics book - while swinging, the angle $\theta$ varies with time following this equation:
According to [Wikipedia](http://tinyurl.com/yz5sz7e)---and many introductory physics book---while swinging, the angle $\theta$ varies with time following this equation:
$$
@@ -405,7 +424,7 @@ This would be much easier if the second derivative were proportional to the angl
[Huygens](http://en.wikipedia.org/wiki/Christiaan_Huygens) used the approximation of $\sin(x) \approx x$, noted above, to say that when the angle is not too big, we have the pendulum's swing obeying $\theta''(t) = -g/l \cdot \theta(t)$. Without getting too involved in why, we can verify by taking two derivatives that $\theta_0\sin(\sqrt{g/l}\cdot t)$ will be a solution to this modified equation.
With this solution, the motion is periodic with constant amplitude (assuming frictionless behaviour), as the sine function is. More surprisingly, the period is found from $T = 2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$ - longer "rods" take more time to swing back and forth - but does not depend on the how wide the pendulum is swinging between (provided $\theta_0$ is not so big the approximation of $\sin(x) \approx x$ fails). This latter fact may be surprising, though not to Galileo who discovered it.
With this solution, the motion is periodic with constant amplitude (assuming frictionless behaviour), as the sine function is. More surprisingly, the period is found from $T = 2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$---longer "rods" take more time to swing back and forth---but does not depend on the how wide the pendulum is swinging between (provided $\theta_0$ is not so big the approximation of $\sin(x) \approx x$ fails). This latter fact may be surprising, though not to Galileo who discovered it.
## Differentials
@@ -414,11 +433,11 @@ With this solution, the motion is periodic with constant amplitude (assuming fri
The Leibniz notation for a derivative is $dy/dx$ indicating the change in $y$ as $x$ changes. It proves convenient to decouple this using *differentials* $dx$ and $dy$. What do these notations mean? They measure change along the tangent line in same way $\Delta_x$ and $\Delta_y$ measure change for the function. The differential $dy$ depends on both $x$ and $dx$, it being defined by $dy=f'(x)dx$. As tangent lines locally represent a function, $dy$ and $dx$ are often associated with an *infinitesimal* difference.
Taking $dx = \Delta_x$, as in the previous graphic, we can compare $dy$ – the change along the tangent line given by $dy/dx \cdot dx$ – and $\Delta_y$ – the change along the function given by $f(x + \Delta_x) - f(x)$. The linear approximation, $f(x + \Delta_x) - f(x)\approx f'(x)dx$, says that
Taking $dx = \Delta_x$, as in the previous graphic, we can compare $dy$---the change along the tangent line given by $dy/dx \cdot dx$---and $\Delta_y$---the change along the function given by $f(x + \Delta_x) - f(x)$. The linear approximation, $f(x + \Delta_x) - f(x)\approx f'(x)dx$, says that
$$
\Delta_y \approx dy; \quad \text{ when } \Delta_x = dx
\Delta_y \approx dy; \quad \text{ when } \Delta_x = dx.
$$
## The error in approximation
@@ -426,8 +445,15 @@ $$
How good is the approximation? Graphically we can see it is pretty good for the graphs we choose, but are there graphs out there for which the approximation is not so good? Of course. However, we can say this (the [Lagrange](http://en.wikipedia.org/wiki/Taylor%27s_theorem) form of a more general Taylor remainder theorem):
::: {.definition title="Lagrange remainder form"}
> Let $f(x)$ be twice differentiable on $I=(a,b)$, $f$ is continuous on $[a,b]$, and $a < c < b$. Then for any $x$ in $I$, there exists some value $\xi$ between $c$ and $x$ such that $f(x) = f(c) + f'(c)(x-c) + (f''(\xi)/2)\cdot(x-c)^2$.
Let $f(x)$ be twice differentiable on $I=(a,b)$, $f$ is continuous on $[a,b]$, and $a < c < b$. Then for any $x$ in $I$, there exists some value $\xi$ between $c$ and $x$ such that
$$
f(x) = f(c) + f'(c)(x-c) + \frac{f''(\xi)}{2}\cdot(x-c)^2.
$$
:::
@@ -437,9 +463,9 @@ That is, the error is basically a constant depending on the concavity of $f$ tim
For $\sin(x)$ at $c=0$ we get $\lvert\sin(x) - x\rvert = \lvert-\sin(\xi)\cdot x^2/2\rvert$. Since $\lvert\sin(\xi)\rvert \leq 1$, we must have this bound: $\lvert\sin(x) - x\rvert \leq x^2/2$.
Can we verify? Let's do so graphically:
Can we verify? We do so graphically in @fig-graph-abs-sin-x-minus-x-and-x-squared-over-2.
::: {#fig-graph-abs-sin-x-minus-x-and-x-squared-over-2}
```{julia}
#| hold: true
h(x) = abs(sin(x) - x)
@@ -448,6 +474,9 @@ plot(h, -2, 2, label="h")
plot!(g, -2, 2, label="f")
```
Plot of $h(x) = \lvert \sin(x) - x \vert$ and $g(x) = x^2/2$ over $[-2, 2]$
:::
The graph shows a tight bound near $0$ and then a bound over this viewing window.
@@ -479,9 +508,9 @@ $$
\lvert f(x) - x\rvert \leq \lvert f''(0)\rvert \cdot \frac{x^2}{2} = x^2/2.
$$
Plotting we verify the bound on $|\log(1+x)-x|$:
In @fig-abs-log1p-minus-x-and-bound we verify the bound on $|\log(1+x)-x|$.
::: {#fig-abs-log1p-minus-x-and-bound}
```{julia}
#| hold: true
h(x) = abs(log(1+x) - x)
@@ -490,6 +519,9 @@ plot(h, -0.5, 2, label="h")
plot!(g, -0.5, 2, label="g")
```
Plot of $f(x) = \lvert \log(1 + x) - x \rvert$ and a derived upper bound
:::
Again, we see the very close bound near $0$, which widens at the edges of the viewing window.
@@ -500,15 +532,17 @@ To see formally why the remainder is as it is, we recall the mean value theorem
$$
\text{error} = h(x) - h(0) = (g(x) - g(0)) \frac{h'(e)}{g'(e)} =
(x^2 - 0) \cdot \frac{f'(e) - f'(0)}{2e} =
x^2 \cdot \frac{1}{2} \cdot f''(\xi).
\begin{align*}
\text{error} &= h(x) - h(0) = (g(x) - g(0)) \frac{h'(e)}{g'(e)} \\
&= (x^2 - 0) \cdot \frac{f'(e) - f'(0)}{2e} \\
&= x^2 \cdot \frac{1}{2} \cdot f''(\xi).
\end{align*}
$$
The value of $\xi$, from the mean value theorem applied to $f'(x)$, satisfies $0 < \xi < e < x$, so is in $[0,x].$
### The big (and small) "oh"
### The big---and small--""oh"
`SymPy` can find the tangent line expression as a special case of its `series` function (which implements [Taylor series](../taylor_series_polynomials.html)). The `series` function needs an expression to approximate; a variable specified, as there may be parameters in the expression; a value $c$ for *where* the expansion is taken, with default $0$; and a number of terms, for this example $2$ for a constant and linear term. (There is also an optional `dir` argument for one-sided expansions.)
@@ -681,165 +715,6 @@ x = Dual(1, 1)
We again see `log(x)` being evaluated in line `%6`. The derivative evaluated at `x` is done in line `%11` and this is multiplied by `xp` in line `%12`.
## Curvature
The curvature of a function will be a topic in a later section on differentiable vector calculus, but the concept of linearization can be used to give an earlier introduction.
The tangent line linearizes the function, it being the best linear approximation to the graph of the function at the point. The slope of the tangent line is the limit of the slopes of different secant lines. Consider now, the orthogonal concept, the *normal line* at a point. This is a line perpendicular to the tangent line that goes through the point on the curve.
At a point $(c,f(c))$ the slope of the normal line is $-1/f'(c)$.
Following [Kirby C. Smith](https://doi.org/10.2307/2687102), consider two nearby points on the curve of $f$ and suppose we take the two normal lines at $x=c$ and $x=c+h$. These two curves will intersect if the lines are not parallel. To ensure this, assueme that in some neighborhood of $c$, $f'(c)$ is increasing.
The two normal lines are:
$$
\begin{align*}
y &= f(c) - \frac{1}{f'(c)}(x-c)\\
y &= f(c+h) - \frac{1}{f'(c+h)}(x-(c+h))\\
\end{align*}
$$
Rearranging, we have
$$
\begin{align*}
-f'(c)(y-f(c)) &= x-c\\
-f'(c+h)(y-f(c+h)) &= x-(c+h)
\end{align*}
$$
Call $R$ the intersection point of the two normal lines:
```{julia}
#| echo: false
using Roots
let
gr()
f(x) = x^4
fp(x) = 4x^3
c = 1/4
h = 1/4
nlc(x) = f(c) - 1/fp(c) * (x - c)
nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h))
canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal)
canvas()
plot!(f, 0, 3/4; line=(3,))
plot!(nlc; ylim=(-1/4, 1))
plot!(nlch; ylim=(-1/4, 1))
Rx = find_zero(x -> nlc(x) - nlch(x), (-10, 10))
scatter!([c,c+h], f.([c, c+h]))
scatter!([Rx], [nlc(Rx)])
annotate!([(c, f(c), L"(c,f(c))",:top),
(c+h, f(c+h), L"(c+h, f(c+h))",:bottom),
(Rx, nlc(Rx), L"R",:left)])
end
```
```{julia}
#| echo: false
plotly()
nothing
```
What happens to $R$ as $h \rightarrow 0$?
We can symbolically solve to see:
```{julia}
@syms 𝑓() 𝑓p() 𝑓pp() x y c ℎ
n1 = -𝑓p(c)*(y-𝑓(c)) ~ x - c
n2 = -𝑓p(c+ℎ)*(y-𝑓(c+ℎ)) ~ x - (c+ℎ)
R = solve((n1, n2), (x, y))
```
Taking limits of each term as $h$ goes to zero we have after some notation-simplifying substitution:
```{julia}
R = Dict(k => limit(R[k], ℎ=>0) for k in (x,y))
Rx = R[x](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓p(c),
limit((𝑓p(c+ℎ)-𝑓p(c))/ℎ, ℎ=>0) => 𝑓pp(c))
```
and
```{julia}
Ry = R[y](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓p(c),
limit((𝑓p(c+ℎ)-𝑓p(c))/ℎ, ℎ=>0) => 𝑓pp(c))
```
The squared distance, $r^2$, of $R$ to $(c,f(c))$ is then:
```{julia}
simplify((Rx-c)^2 + (Ry-𝑓(c))^2)
```
Or
$$
r^2 = \frac{(f'(c)^2 + 1)^3}{f''(c)^2}.
$$
This formula for $r$ is known as the radius of curvature of $f$ -- the radius of the *circle* that best approximates the function at the point. That is, this value reflects the curvature of $f$ supplementing the tangent line or best *linear* approximation to the graph of $f$ at the point.
```{julia}
#| echo: false
let
gr()
f(x) = x^4
fp(x) = 4x^3
fpp(x) = 12x^2
c = 1/4
h = 1/4
nlc(x) = f(c) - 1/fp(c) * (x - c)
nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h))
canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal)
canvas()
plot!(f, -1/4, 3/4; line=(3,))
tl(x) = f(c) + f'(c)*(x-c)
plot!(tl, ylim=(-1/4, 3/2); line=(2, :dot))
Rx, Ry = c - fp(c)^3 / fpp(c) - fp(c)/fpp(c), f(c) + (fp(c)^2+1)/fpp(c)
r = (fp(c)^2 + 1)^(3/2) / abs(fpp(c))
scatter!([c], f.([c]))
scatter!([Rx], [nlc(Rx)])
annotate!([(c, f(c), L"(c,f(c))",:top),
(Rx, nlc(Rx), L"R",:left)])
Delta = pi/10
theta = range(3pi/2 - Delta, 2pi - 3Delta, length=100)
xs, ys = cos.(theta), sin.(theta)
plot!(Rx .+ r.*xs, Ry .+ r.*ys)
x0s, y0s = [Rx,Rx .+ r * first(xs)],[Ry,Ry .+ r * first(ys)]
xns, yns = [Rx,Rx .+ r * last(xs)],[Ry,Ry .+ r * last(ys)]
xcs, ycs = [Rx,c],[Ry,f(c)]
sty = (2, :0.25, :dash)
plot!(x0s, y0s; line=sty);
plot!(xcs, ycs; line=sty);
plot!(xns, yns; line=sty)
end
```
```{julia}
#| echo: false
plotly()
nothing
```
## Questions
@@ -857,8 +732,8 @@ choices = [
"``1 + x^{1/2}``",
"``1 + (1/2) \\cdot x``",
"``1 - (1/2) \\cdot x``"]
answ = 3
radioq(choices, answ)
answer = 3
radioq(choices, answer)
```
###### Question
@@ -875,8 +750,8 @@ choices = [
"``1 + x^k``",
"``1 + k \\cdot x``",
"``1 - k \\cdot x``"]
answ = 3
radioq(choices, answ)
answer = 3
radioq(choices, answer)
```
###### Question
@@ -894,8 +769,8 @@ choices = [
"``x``",
"``1 - x^2/2``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -913,8 +788,8 @@ choices = [
"``1 + x``",
"``1 - x``"
]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
###### Question
@@ -932,8 +807,8 @@ choices = [
"``1 + x``",
"``25``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -946,8 +821,8 @@ Let $f(x) = \sqrt{x}$. Find the actual error in approximating $f(26)$ by the val
#| hold: true
#| echo: false
tgent(x) = 5 + x/10
answ = tgent(1) - sqrt(26)
numericq(answ)
answer = tgent(1) - sqrt(26)
numericq(answer)
```
###### Question
@@ -961,8 +836,8 @@ An estimate of some quantity was $12.34$ the actual value was $12$. What was the
#| echo: false
est = 12.34
act = 12.0
answ = (est -act)/act * 100
numericq(answ)
answer = (est -act)/act * 100
numericq(answer)
```
###### Question
@@ -978,8 +853,8 @@ tl(x) = x
x0 = 5 * pi/180
est = x0
act = sin(x0)
answ = (est -act)/act * 100
numericq(answ)
answer = (est -act)/act * 100
numericq(answer)
```
###### Question
@@ -992,8 +867,8 @@ The side length of a square is measured roughly to be $2.0$ cm. The actual lengt
#| hold: true
#| echo: false
tl(x) = 4 + 4x
answ = tl(.2) - 4
numericq(abs(answ))
answer = tl(.2) - 4
numericq(abs(answer))
```
###### Question
quarto/derivatives/mean_value_theorem.qmd
+258 -135
View File
@@ -1,4 +1,4 @@
# The mean value theorem for differentiable functions
# Implications of Differentiability
{{< include ../_common_code.qmd >}}
@@ -24,7 +24,21 @@ nothing
---
A function is *continuous* at $c$ if $f(c+h) - f(c) \rightarrow 0$ as $h$ goes to $0$. We can write that as $f(c+h) - f(c) = \epsilon_h$, with $\epsilon_h$ denoting a function going to $0$ as $h \rightarrow 0$. With this notion, differentiability could be written as $f(c+h) - f(c) - f'(c)h = \epsilon_h \cdot h$. This is clearly a more demanding requirement than mere continuity at $c$.
Supposed $\epsilon_h$ is some function going to $0$ as $h \rightarrow 0$ that may be different from line to line. Then we have two somewhat similar characterizations of continuity and differentiability:
A function is *continuous* at $c$ if
$$
f(c+h) - f(c) = \epsilon_h.
$$
A function is *differentiable* at $c$ if
$$
f(c+h) - f(c) - f'(c)h = \epsilon_h \cdot h.
$$
We defined a function to be *continuous* on an interval $I=(a,b)$ if it was continuous at each point $c$ in $I$. Similarly, we define a function to be *differentiable* on the interval $I$ if it is differentiable at each point $c$ in $I$.
@@ -33,67 +47,79 @@ We defined a function to be *continuous* on an interval $I=(a,b)$ if it was cont
This section looks at properties of differentiable functions. As there is a more stringent definition, perhaps more properties are a consequence of the definition.
## Differentiable is more restrictive than continuous.
## Differentiability implies continuity
Let $f$ be a differentiable function on $I=(a,b)$. We see that $f(c+h) - f(c) = f'(c)h + \epsilon_h\cdot h = h(f'(c) + \epsilon_h)$. The right hand side will clearly go to $0$ as $h\rightarrow 0$, so $f$ will be continuous. In short:
::: {.relationship title="Differentiable implies continuous"}
> A differentiable function on $I=(a,b)$ is continuous on $I$.
A differentiable function on $I=(a,b)$ is continuous on $I$.
:::
Is it possible that all continuous functions are differentiable?
The fact that the derivative is related to the tangent line's slope might give an indication that this won't be the case - we just need a function which is continuous but has a point with no tangent line. The usual suspect is $f(x) = \lvert x\rvert$ at $0$.
The fact that the derivative is related to the tangent line's slope might give an indication that this won't be the case - we just need a function which is continuous but has a point with no tangent line. The usual suspect is $f(x) = \lvert x\rvert$ at $0$, plotted around $0$ in @fig-plot-abs-over-minus1-1-not-diff-at-0.
::: {#fig-plot-abs-over-minus1-1-not-diff-at-0}
```{julia}
#| hold: true
#| echo: false
f(x) = abs(x)
plot(f, -1,1)
```
Plot of $f(x) = \lvert x \rvert$ over $[-1, 1]$. This function does not have a tangent line at $x=0$.
:::
We can see formally that the secant line expression will not have a limit when $c=0$ (the left limit is $-1$, the right limit $1$). But more insight is gained by looking at the shape of the graph. At the origin, the graph always is vee-shaped. There is no linear function that approximates this function well. The function is just not smooth enough, as it has a kink.
There are other functions that have kinks. These are often associated with powers. For example, at $x=0$ this function will not have a derivative:
There are other functions that have kinks. These are often associated with powers. For example, at $x=0$ the function $f(x) = x^{2/3}$ (@fig-plot-x-2-thirds-over-minus1-1) will not have a derivative at $x=0$.
::: {#fig-plot-x-2-thirds-over-minus1-1}
```{julia}
#| hold: true
#| echo: false
f(x) = (x^2)^(1/3)
plot(f, -1, 1)
```
Other functions have tangent lines that become vertical. The natural slope would be $\infty$, but this isn't a limiting answer (except in the extended sense we don't apply to the definition of derivatives). A candidate for this case is the cube root function:
Plot of $f(x) = x^{2/3}$ over $[-1, 1]$. This function does not have a tangent line at $x=0$.
:::
Other functions have tangent lines that become vertical. The natural slope would be $\infty$, but this isn't a limiting answer (except in the extended sense we don't apply to the definition of derivatives). A candidate for this case is the cube root function, shown in @fig-cbrt-over-minus1-1-not-tangent-line-at-0.
::: {#fig-cbrt-over-minus1-1-not-tangent-line-at-0}
```{julia}
#| echo: false
plot(cbrt, -1, 1)
```
Plot of `cbrt` over $[-1,1]$. The "tangent" line at $x=0$ is vertical; the function is not differentiable at $0$
:::
The derivative at $0$ would need to be $+\infty$ to match the graph. This is implied by the formula for the derivative from the power rule: $f'(x) = 1/3 \cdot x^{-2/3}$, which has a vertical asymptote at $x=0$.
:::{.callout-note}
## Note
The `cbrt` function is used above, instead of `f(x) = x^(1/3)`, as the latter is not defined for negative `x`. Though it can be for the exact power `1/3`, it can't be for an exact power like `1/2`. This means the value of the argument is important in determining the type of the output - and not just the type of the argument. Having type-stable functions is part of the magic to making `Julia` run fast, so `x^c` is not defined for negative `x` and most floating point exponents.
The `cbrt` function is used to plot @fig-cbrt-over-minus1-1-not-tangent-line-at-0}, instead of `f(x) = x^(1/3)`, as the latter is not defined for negative `x`. Though it can be for the exact power `1/3`, it can't be for an exact power like `1/2`. This means the value of the argument is important in determining the type of the output - and not just the type of the argument. Having type-stable functions is part of the magic to making `Julia` run fast, so `x^c` is not defined for negative `x` and most floating point exponents.
:::
Lest you think that continuous functions always have derivatives except perhaps at exceptional points, this isn't the case. The functions used to [model](http://tinyurl.com/cpdpheb) the stock market are continuous but have no points where they are differentiable.
Lest you think that continuous functions always have derivatives except perhaps at exceptional points, this isn't the case. The functions used to [model](http://tinyurl.com/cpdpheb) the stock market are continuous but have **no** points where they are differentiable.
## Derivatives and maxima.
## Fermat's theorem
We have defined an *absolute maximum* of $f(x)$ over an interval to be a value $f(c)$ for a point $c$ in the interval that is as large as any other value in the interval. Just specifying a function and an interval does not guarantee an absolute maximum, but specifying a *continuous* function and a *closed* interval does, by the extreme value theorem.
::: {.callout-note icon=false}
## A relative maximum
::: {.definition title="A relative maximum"}
We say $f(x)$ has a *relative maximum* at $c$ if there exists *some* interval $I=(a,b)$ with $a < c < b$ for which $f(c)$ is an absolute maximum for $f$ and $I$.
@@ -104,46 +130,36 @@ The difference is a bit subtle, for an absolute maximum the interval must also b
:::{.callout-note}
## Note
A hiker can appreciate the difference. A relative maximum would be the crest of any hill, but an absolute maximum would be the summit.
A hiker can appreciate the difference. A relative maximum would be the crest of any hill, but an absolute maximum would often be the summit.
:::
What does this have to do with derivatives?
A theorem attributed to [Fermat](https://digitalcommons.ursinus.edu/cgi/viewcontent.cgi?params=/context/triumphs_calculus/article/1011/&path_info=M05_Fermats_Method_for_Finding_Maxima_and_Minima_2022_05_17.pdf) says something about where a relative or absolute maximum (or minimum) can occur under assumptions:
[Fermat](http://science.larouchepac.com/fermat/fermat-maxmin.pdf), perhaps with insight from Kepler, was interested in maxima of polynomial functions. As a warm up, he considered a line segment $AC$ and a point $E$ with the task of choosing $E$ so that $(E-A) \times (C-E)$ being a maximum. We might recognize this as finding the maximum of $f(x) = (x-A)\cdot(C-x)$ for some $A < C$. Geometrically, we know this to be at the midpoint, as the equation is a parabola, but Fermat was interested in an algebraic solution that led to more generality.
::: {.theorem title="Fermat's theorem"}
If a differentiable function on $(a,b)$ has a maximum at $c$ with $a < c < b$ then $f'(c) = 0$
He takes $b=AC$ and $a=AE$. Then the product is $a \cdot (b-a) = ab - a^2$. He then perturbs this writing $AE=a+e$, then this new product is $(a+e) \cdot (b - a - e)$. Equating the two, and canceling like terms gives $be = 2ae + e^2$. He cancels the $e$ and basically comments that this must be true for all $e$ even as $e$ goes to $0$, so $b = 2a$ and the value is at the midpoint.
In a more modern approach, this would be the same as looking at this expression:
$$
\frac{f(x+e) - f(x)}{e} = 0.
$$
Working on the left hand side, for non-zero $e$ we can cancel the common $e$ terms, and then let $e$ become $0$. This becomes a problem in solving $f'(x)=0$. Fermat could compute the derivative for any polynomial by taking a limit, a task we would do now by the power rule and the sum and difference of function rules.
This insight holds for other types of functions:
> If $f(c)$ is a relative maximum then either $f'(c) = 0$ or the derivative at $c$ does not exist.
:::
When the derivative exists, this says the tangent line is flat. (If it had a slope, then the function would increase by moving left or right, as appropriate, a point we pursue later.)
Relaxing differentibility to continuity, we have
::: {.relationship title="The derivative at a relative maximum"}
If a continuous function on $(a,b)$ has a maximum at $c$ with $a < c < b$ then $f'(c) = 0$ or the derivative of $f$ at $c$ does not exist.
:::
For a continuous function $f(x)$, call a point $c$ in the domain of $f$ where either $f'(c)=0$ or the derivative does not exist a **critical** **point**.
We can combine Bolzano's extreme value theorem with Fermat's insight to get the following:
::: {.callout-note icon=false}
## Absolute maxima characterization
::: {.relationship title="Absolute maxima characterization"}
A continuous function on $[a,b]$ has an absolute maximum that occurs at a critical point $c$, $a < c < b$, or an endpoint, $a$ or $b$.
@@ -151,10 +167,9 @@ A similar statement holds for an absolute minimum.
:::
The above gives a restricted set of places to look for absolute maximum and minimum values - all the critical points and the endpoints.
The above gives a restricted set of places to look for absolute maximum and minimum values---all the critical points and the endpoints, but no where else.
It is also the case that all relative extrema occur at a critical point, *however* not all critical points correspond to relative extrema. We will see *derivative tests* that help characterize when that occurs.
It is the case that all relative extrema occur at a critical point, *however* it is *not* the case that all critical points correspond to relative extrema. We will see *derivative tests* that help characterize when a critical point corresponds to a relative extrema.
```{julia}
@@ -163,7 +178,7 @@ It is also the case that all relative extrema occur at a critical point, *howeve
### {{{lhopital_32}}}
imgfile = "figures/lhopital-32.png"
caption = L"""
Image number ``32`` from L'Hopitals calculus book (the first) showing that
Image number ``32`` from L'Hospitals calculus book (the first) showing that
at a relative minimum, the tangent line is parallel to the
$x$-axis. This of course is true when the tangent line is well defined
by Fermat's observation.
@@ -172,23 +187,15 @@ by Fermat's observation.
nothing
```
![Image number $32$ from L'Hopitals calculus book (the first) showing that
::: {#fig-lhospital-image-number-32}
![](./figures/lhopital-32.png){fig-alt="Image number 32 of L'Hospital's book"}
Image number $32$ from L'Hospitals calculus book (the first) showing that
at a relative minimum, the tangent line is parallel to the
$x$-axis. This of course is true when the tangent line is well defined
by Fermat's observation.](./figures/lhopital-32.png)
by Fermat's observation.
:::
### Numeric derivatives
The `ForwardDiff` package provides a means to numerically compute derivatives without approximations at a point. In `CalculusWithJulia` this is extended to find derivatives of functions and the `'` notation is overloaded for function objects. Hence these two give nearly identical answers, the difference being only the type of number used:
```{julia}
#| hold: true
f(x) = 3x^3 - 2x
fp(x) = 9x^2 - 2
f'(3), fp(3)
```
##### Example
@@ -196,7 +203,7 @@ f'(3), fp(3)
For the function $f(x) = x^2 \cdot e^{-x}$ find the absolute maximum over the interval $[0, 5]$.
We have that $f(x)$ is continuous on the closed interval of the question, and in fact differentiable on $(0,5)$, so any critical point will be a zero of the derivative. We can check for these with:
We have that $f(x)$ is continuous on the closed interval of the question, and in fact differentiable on $(0,5)$. By differentiability, any critical point will be a zero of the derivative. We can check for these using `f'` to compute the derivative automatically:
```{julia}
@@ -270,16 +277,16 @@ Here the maximum occurs at an endpoint. The critical point $c=0.67\dots$ does no
Let $f(x)$ be differentiable on $(a,b)$ and continuous on $[a,b]$. Then the absolute maximum occurs at an endpoint or where the derivative is $0$ (as the derivative is always defined). This gives rise to:
::: {.callout-note icon=false}
## [Rolle's](http://en.wikipedia.org/wiki/Rolle%27s_theorem) theorem
::: {.theorem title="Rolle's theorem"}
For $f$ differentiable on $(a,b)$ and continuous on $[a,b]$, if $f(a)=f(b)$, then there exists some $c$ in $(a,b)$ with $f'(c) = 0$.
[Rolle's](http://en.wikipedia.org/wiki/Rolle%27s_theorem) theorem states that
if $f$ differentiable on $(a,b)$ and continuous on $[a,b]$ and if $f(a)=f(b)$, then there exists some $c$ in $(a,b)$ with $f'(c) = 0$.
:::
::: {#fig-l-hospital-144}
![Figure from L'Hospital's calculus book](figures/lhopital-144.png)
![](figures/lhopital-144.png){ig-alt="Figure from L'Hospital's calculus book"}
Figure from L'Hospital's calculus book showing Rolle's theorem where $c=E$ in the labeling.
:::
@@ -287,7 +294,7 @@ Figure from L'Hospital's calculus book showing Rolle's theorem where $c=E$ in th
This modest observation opens the door to many relationships between a function and its derivative, as it ties the two together in one statement.
To see why Rolle's theorem is true, we assume that $f(a)=0$, otherwise consider $g(x)=f(x)-f(a)$. By the extreme value theorem, there must be an absolute maximum and minimum. If $f(x)$ is ever positive, then the absolute maximum occurs in $(a,b)$ - not at an endpoint - so at a critical point where the derivative is $0$. Similarly if $f(x)$ is ever negative. Finally, if $f(x)$ is just $0$, then take any $c$ in $(a,b)$.
To see why Rolle's theorem is true, we assume that $f(a)=0$, otherwise consider $g(x)=f(x)-f(a)$. By the extreme value theorem, there must be an absolute maximum and minimum. If $f(x)$ is ever positive, then the absolute maximum occurs in $(a,b)$---not at an endpoint---so at a critical point where the derivative is $0$. Similarly if $f(x)$ is ever negative. Finally, if $f(x)$ is just $0$, then take any $c$ in $(a,b)$.
The statement in Rolle's theorem speaks to existence. It doesn't give a recipe to find $c$. It just guarantees that there is *one* or *more* values in the interval $(a,b)$ where the derivative is $0$ if we assume differentiability on $(a,b)$ and continuity on $[a,b]$.
@@ -296,7 +303,8 @@ The statement in Rolle's theorem speaks to existence. It doesn't give a recipe t
##### Example
Let $j(x) = e^x \cdot x \cdot (x-1)$. We know $j(0)=0$ and $j(1)=0$, so on $[0,1]$. Rolle's theorem guarantees that we can find *at* *least* one answer (unless numeric issues arise):
Let $j(x) = e^x \cdot x \cdot (x-1)$. We know $j(0)=0$ and $j(1)=0$, so on $[0,1]$. Rolle's theorem guarantees that we can find *at* *least* one answer to $j'(x) = 0$ between $0$ and $1$. We see there is only the one numerically. @fig-plot-expx-times-x-times-x-minus-1-over-0-1 also illustrates graphically the lone value for $c$ in $[a,b]$ for this problem.
```{julia}
@@ -304,21 +312,29 @@ j(x) = exp(x) * x * (x-1)
find_zeros(j', 0, 1)
```
The following graph illustrates the lone value for $c$ in $[a,b]$ for
this problem:
::: {#fig-plot-expx-times-x-times-x-minus-1-over-0-1}
```{julia}
#| echo: false
x0 = find_zero(j', (0, 1))
j₀ = j(x0)
plot([j, x->j₀ + 0*(x-x0)], 0, 1; legend=false)
scatter!([0,x0,1], [j₀, j₀, j₀])
annotate!([(0,j₀,text("a", :bottom)),
(x0, j₀, text("c", :bottom)),
(1, j₀, text("b", :bottom))])
let
gr()
x0 = find_zero(j', (0, 1))
j₀ = j(x0)
plt = plot(; legend=false, framestyle=:origin)
plot!(plt, [j, x->j₀ + 0*(x-x0)], 0, 1)
scatter!(plt, [(x0, j₀)])
annotate!(plt, [
(0,0,text(L"a", :bottom, :left)),
(x0, j₀, text(L"(c, f(c))", :bottom, :left)),
(1, 0, text(L"b", :bottom))])
plotly()
plt
end
```
Plot of $f(x) = e^x \cdot x \cdot (x-1)$ over $[0,1]$ showing a single value $c$ satisfying Rolle's theorem
:::
## The mean value theorem
@@ -327,27 +343,25 @@ We are driving south and in one hour cover 70 miles. If the speed limit is 65 mi
The mean value theorem is a direct generalization of Rolle's theorem.
::: {.callout-note icon=false}
## Mean value theorem
::: {.theorem title="Mean value theorem"}
Let $f(x)$ be differentiable on $(a,b)$ and continuous on $[a,b]$. Then there exists a value $c$ in $(a,b)$ where
$$
f'(c) = (f(b) - f(a)) / (b - a).
f'(c) = \frac{f(b) - f(a)}{b - a}.
$$
:::
This says for any secant line between $a < b$ there will be a parallel tangent line at some $c$ with $a < c < b$ (all provided $f$ is differentiable on $(a,b)$ and continuous on $[a,b]$).
This theorem says appropriate functions the secant line between $a < b$ will have at least one parallel tangent line at a value $c$ with $a < c < b$.
@fig-mean-value-theorem illustrates the theorem. The secant line between $a$ and $b$ is dashed. For this function there are two values of $c$ where the slope of the tangent line is seen to be the same as the slope of this secant line. At least one is guaranteed by the theorem.
::: {#fig-mean-value-theorem}
```{julia}
#| hold: true
#| echo: false
#| label: fig-mean-value-theorem
let
# mean value theorem
gr()
@@ -397,6 +411,9 @@ plotly()
nothing
```
Figure illustrating the mean value theorem. The secant line from $(a,f(a))$ to $(b, f(b))$ is matched by two points $c$ in $(a,b)$ with parallel tangent lines
:::
Like Rolle's theorem this is a guarantee that something exists, not a recipe to find it. In fact, the mean value theorem is just Rolle's theorem applied to:
@@ -406,7 +423,7 @@ $$
That is the function $f(x)$, minus the secant line between $(a,f(a))$ and $(b, f(b))$.
::: {#fig-jsxgraph-mvt}
```{julia}
#| hold: true
#| echo: false
@@ -456,9 +473,28 @@ board.create('tangent', [r], {strokeColor:'#ff0000'});
line = board.create('line',[p[0],p[1]],{strokeColor:'#ff0000',dash:1});
```
This interactive example can also be found at [jsxgraph](http://jsxgraph.uni-bayreuth.de/wiki/index.php?title=Mean_Value_Theorem). It shows a cubic polynomial fit to the $4$ adjustable points labeled A through D. The secant line is drawn between points A and B with a dashed line. A tangent line – with the same slope as the secant line – is identified at a point $(\alpha, f(\alpha))$ where $\alpha$ is between the points A and B. That this can always be done is a consequence of the mean value theorem.
Interactive graphic showing a parallel tangent line to $f(x) at $a < c < b$ can always be found that has the same slope as the secant line between $(a, f(a))$ and $(b, f(b))$
:::
The interactive example of @fig-jsxgraph-mvt can also be found at [jsxgraph](http://jsxgraph.uni-bayreuth.de/wiki/index.php?title=Mean_Value_Theorem). It shows a cubic polynomial fit to the $4$ adjustable points labeled A through D. The secant line is drawn between points A and B with a dashed line. A tangent line---with the same slope as the secant line---is identified at a point $(\alpha, f(\alpha))$ where $\alpha$ is between the points A and B. That this can always be done is a consequence of the mean value theorem.
##### Example
The function $f(x) = e^{-x^2/2}$ is continuously differentiable on $[0,1]$. That means the mean value theorem applies. Find a value $c$ satisfying the theorem.
The pattern is the same, we use `Roots` to solve an equation as follows:
```{julia}
f(x) = exp(-x^2/2)
a, b = 0, 1
m = (f(b) - f(a)) / (b - a) # slope of secant line
h(x) = f'(x) - m # solving f'(x) = m
find_zeros(h, (a, b))
```
The call to `find_zeros` returns just one value for $c$.
##### Example
@@ -468,7 +504,9 @@ The mean value theorem is an extremely useful tool to relate properties of a fun
For example, suppose we have a function $f(x)$ and we know that the derivative is **always** $0$. What can we say about the function?
Well, constant functions have derivatives that are constantly $0$. But do others? We will see the answer is no: If a function has a zero derivative in $(a,b)$ it must be a constant. We can readily see that if $f$ is a polynomial function this is the case, as we can differentiate a polynomial function and this will be zero only if **all** its coefficients are $0$, which would mean there is no non-constant leading term in the polynomial. But polynomials are not representative of all functions, and so a proof requires a bit more effort.
Well, constant functions have derivatives that are constantly $0$. But do others? We will see the answer is no: If a function has a zero derivative in $(a,b)$ it must be a constant.
We can readily see that if $f$ is a polynomial function this is the case, as we can differentiate a polynomial function and this will be zero only if **all** its coefficients are $0$, which would mean there is no non-constant leading term in the polynomial. But polynomials are not representative of all functions, and so a proof requires a bit more effort.
Suppose it is known that $f'(x)=0$ on some interval $I$ and we take any $a < b$ in $I$. Since $f'(x)$ always exists, $f(x)$ is always differentiable, and hence always continuous. So on $[a,b]$ the conditions of the mean value theorem apply. That is, there is a $c$ in $(a,b)$ with $(f(b) - f(a)) / (b-a) = f'(c) = 0$. But this would imply $f(b) - f(a)=0$. That is $f(x)$ is a constant, as for any $a$ and $b$, we see $f(a)=f(b)$.
@@ -477,10 +515,9 @@ Suppose it is known that $f'(x)=0$ on some interval $I$ and we take any $a < b$
### The Cauchy mean value theorem
[Cauchy](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) offered an extension to the mean value theorem above.
[Cauchy](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) offered an extension to the mean value theorem.
::: {.callout-note icon=false}
## Cauchy mean value theorem
::: {.theorem title="Cauchy mean value theorem"}
Suppose both $f$ and $g$ satisfy the conditions of the mean value theorem on $[a,b]$ with $g(b)-g(a) \neq 0$, then there exists at least one $c$ with $a < c < b$ such that
@@ -513,12 +550,12 @@ For some $c$ in $[0,x]$. If $\lim_{x \rightarrow 0} f'(x)/g'(x) = L$, then the r
This could be used to prove the limit of $\sin(x)/x$ as $x$ goes to $0$ just by showing the limit of $\cos(x)/1$ is $1$, as is known by continuity.
### Visualizing the Cauchy mean value theorem
##### Example: visualizing the Cauchy mean value theorem
The Cauchy mean value theorem can be visualized in terms of a tangent line and a *parallel* secant line in a similar manner as the mean value theorem as long as a *parametric* graph is used. A parametric graph plots the points $(g(t), f(t))$ for some range of $t$. That is, it graphs *both* functions at the same time. The following illustrates the construction of such a graph:
::: {#fig-illustrate-cauchy-mean-value-theore}
```{julia}
#| hold: true
#| echo: false
@@ -540,15 +577,7 @@ function parametric_fns_graph(n)
val = @sprintf("% 0.2f", ts[end])
annotate!(plt, [(0, 1, L"t = %$val")])
end
caption = L"""
Illustration of parametric graph of $(g(t), f(t))$ for $-\pi/2 \leq t
\leq \pi/2$ with $g(x) = \sin(x)$ and $f(x) = x$. Each point on the
graph is from some value $t$ in the interval. We can see that the
graph goes through $(0,0)$ as that is when $t=0$. As well, it must go
through $(1, \pi/2)$ as that is when $t=\pi/2$
"""
caption = ""
n = 10
@@ -562,9 +591,17 @@ plotly()
ImageFile(imgfile, caption)
```
With $g(x) = \sin(x)$ and $f(x) = x$, we can take $I=[a,b] = [0, \pi/2]$. In the figure below, the *secant line* is drawn in red which connects $(g(a), f(a))$ with the point $(g(b), f(b))$, and hence has slope $\Delta f/\Delta g$. The parallel lines drawn show the *tangent* lines with slope $f'(c)/g'(c)$. Two exist for this problem, the mean value theorem guarantees at least one will.
Illustration of parametric graph of $(g(t), f(t))$ for $-\pi/2 \leq t
\leq \pi/2$ with $g(x) = \sin(x)$ and $f(x) = x$. Each point on the
graph is from some value $t$ in the interval. We can see that the
graph goes through $(0,0)$ as that is when $t=0$. As well, it must go
through $(1, \pi/2)$ as that is when $t=\pi/2$
:::
With $g(x) = \sin(x)$ and $f(x) = x$, we can take $I=[a,b] = [0, \pi/2]$. In the @fig-mvt-two-c-exists-for-sinx, the *secant line* is drawn in red which connects $(g(a), f(a))$ with the point $(g(b), f(b))$, and hence has slope $\Delta f/\Delta g$. The parallel lines drawn show the *tangent* lines with slope $f'(c)/g'(c)$. Two exist for this problem, the mean value theorem guarantees at least one will.
::: {#fig-mvt-two-c-exists-for-sinx}
```{julia}
#| hold: true
#| echo: false
@@ -587,6 +624,9 @@ end
p
```
Illustration of the Cauchy mean value theorem
:::
## Questions
@@ -659,6 +699,44 @@ numericq(float(val))
###### Question
Let $f(x) = 1/x$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$.
```{julia}
#| hold: true
#| echo: false
choices = [
"``c = (a+b)/2``",
"``c = \\sqrt{ab}``",
"``c = 1 / (1/a + 1/b)``",
"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``"
]
answ = 2
radioq(choices, answ)
```
###### Question
Let $f(x) = x^2$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$.
```{julia}
#| hold: true
#| echo: false
choices = [
"``c = (a+b)/2``",
"``c = \\sqrt{ab}``",
"``c = 1 / (1/a + 1/b)``",
"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``"
]
answ = 1
radioq(choices, answ)
```
###### Question
Will the function $f(x) = x + 1/x$ satisfy the conditions of the mean value theorem over $[-1/2, 1/2]$?
@@ -705,43 +783,6 @@ answ = 3
radioq(choices, answ)
```
###### Question
Let $f(x) = 1/x$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$.
```{julia}
#| hold: true
#| echo: false
choices = [
"``c = (a+b)/2``",
"``c = \\sqrt{ab}``",
"``c = 1 / (1/a + 1/b)``",
"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``"
]
answ = 2
radioq(choices, answ)
```
###### Question
Let $f(x) = x^2$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$.
```{julia}
#| hold: true
#| echo: false
choices = [
"``c = (a+b)/2``",
"``c = \\sqrt{ab}``",
"``c = 1 / (1/a + 1/b)``",
"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``"
]
answ = 1
radioq(choices, answ)
```
###### Question
@@ -774,3 +815,85 @@ L"The squeeze theorem applies, as $0 < g(x) < x$",
answ = 3
radioq(choices, answ)
```
##### Question
[Fermat](https://digitalcommons.ursinus.edu/cgi/viewcontent.cgi?params=/context/triumphs_calculus/article/1011/&path_info=M05_Fermats_Method_for_Finding_Maxima_and_Minima_2022_05_17.pdf) didn't exactly prove his theorem in the language of today. Rather, following Monks, we quote
> Let a be the desired unknown, whether it be a length, a plane region or a solid, depending
> on what the given magnitude equals, and let its maximum or minimum be found in terms of
> $a$, involving whatever degree. Replace this first quantity with $a + e$, and the maximum or
> minimum will be found in terms of $a$ and $e$, with coefficients of whatever degree. These two
> representations of the maximum or minimum are adequated, to use Diophantus’ term,
> and the common terms are subtracted. Having done this, all terms from either part (affected by
> $e$ or its powers) are divided each by $e$, or by a higher power of the same, until some term of
> one or the other of the expressions is altogether freed from being affected by $e$.
>
> All terms involving $e$ or one of its powers are then eliminated and the remaining terms
> are equated; or, should one of the expressions be left as nothing, then the positive terms
> are equated with the negatives, which reduces to the same thing. The solution to this last
> equation will yield the value of $a$, which will reveal knowledge of the maximum or minimum
Huh? As an example, he considered a line segment $AC$ and a point $E$ with the task of choosing $E$ so that $(E-A) \times (C-E)$ being a maximum.
::: {#fig-fermat-line-segment}
```{julia}
#| echo: false
let
gr()
A, E, C = (0,0), (1, 0), (3, 0)
plt = plot(; empty_style...)
plot!(plt, [A,C]; line=(1, :black))
tck = (0, 0.1)
for P ∈ (A, E, C)
plot!(plt, [P, P .+ tck], line=(1, :black))
end
annotate!(plt, [
(A..., text(L"A", :top)),
(E..., text(L"E", :top)),
(C..., text(L"C", :top))])
plotly()
plt
end
```
$AC$ is a line divided at $E$ so that $AE \times EC$ is maximum
::::
Set $b=AC$ and $a = AE$ then the product is $a \cdot (b-a)$. the point was at $a + e$, then the product would be $(a+e) \cdot (b - a - e)$. The term *adequated* means approximately equal gives what?
```{julia}
#| echo: false
choices = [L"a \cdot(b-a) - a \cdot (b-a) = 0",
L"a \cdot(b-a) - (a + e) \cdot (b- (a - e)) \approx 0"]
answer = 2
buttonq(choices, answer)
```
Next we divide by $e$---or a higher power of $e$---and simplify so that some term has not $e$ in it.
For this case, does this satisfy the above?
$$
\frac{(a \cdot (b-a) - (a + e)\cdot(b - (a + e))}{e} = 2a - b + e
$$
```{julia}
#| echo: false
choices = ["Yes", "No"]
answer = 1
buttonq(choices, answer)
```
The value $b - 2a = 0$ gives $a = $b/2$. Is this true: geometrically, we know this to be at the midpoint, as the equation is a parabola.
```{julia}
#| echo: false
choices = ["Yes", "No"]
answer = 1
buttonq(choices, answer)
```
quarto/derivatives/more_zeros.qmd
+474 -292
View File
@@ -1,5 +1,4 @@
# Derivative-free alternatives to Newton's method
# Other zero-finding algorithms
{{< include ../_common_code.qmd >}}
@@ -17,379 +16,537 @@ using SymPy
---
Newton's method is not the only algorithm of its kind for identifying zeros of a function. In this section we discuss some alternatives.
There are *numerous* zero-finding methods in addition to the secant method and Newton's method. This section shows a few different directions. It then discusses the topic of when to terminate an algorithm.
This section is entirely optional, none of the algorithms discussed below are utilized in the sequel.
## Other methods
We discuss variations of both Newton's method and the bisection method.
### Estimating the derivative
[Sidi](https://www.math.nthu.edu.tw/~amen/2008/070227-1.pdf) starts with Newton's method with its update step
$$
x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)}
$$
and notes that the secant method just uses the slope of the secant line between $x_i$ and $x_{i-1}$ to estimate $f'(x_i)$. The secant line is the *linear* polynomial interpolating the two points $(x_i, f(x_i))$ and $(x_{i-1}, f(x_{i-1}))$ and the slope the derivative of this polynomial. Sidi generalizes this to approximate the function $f'(x_i)$ by using more points from the algorithm to interpolate a polynomial at $x_i, x_{i-1}, \dots, x_{i-k}$ and then using the derivative of this polynomial to estimate the derivative of $f$ at $x_i$.
## The `find_zero(f, x0)` function
Let's consider a case with three points or $k=2$. Here we use formula (10) of Sidi computing the derivative of the interpolating polynomial at $x_n$ specialized for $k=2$.
```{julia}
function pprime_n2(xn_2, xn_1, xn, yn_2, yn_1, yn)
m = (yn - yn_1) / (xn - xn_1)
m_1 = (yn_1 - yn_2) / (xn_1 - xn_2)
m + (m - m_1)/(xn - xn_2) * (xn - xn_1)
end
```
The function `find_zero` from the `Roots` packages provides several different algorithms for finding a zero of a function, including some derivative-free algorithms for finding zeros when started with a nearby initial guess. The default method is similar to Newton's method in that only a good, initial guess is needed. However, the algorithm, while possibly slower in terms of function evaluations and steps, is engineered to be a bit more robust to the choice of initial estimate than Newton's method. (If it finds a bracket, it will use a bisection algorithm which is guaranteed to converge, but can be slower to do so.) Here we see how to call the function:
Typically, we would start with two initial points and apply the secant method to produce a third, but for this example we will use an initial three points.
```{julia}
f(x) = cos(x) - x/2
xs = [0, pi/2, pi/4]
ys = f.(xs) # y₀, y₁, y₂
xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx); push!(ys, yy) # y₃
```
This method generalizes the secant method with a convergence rate of $1.83928\cdots$. For this problem we see it takes 5 iterations to converge to machine tolerance:
```{julia}
xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx); push!(ys, yy) # y₄ = 0.0004182051168989398
xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx); push!(ys, yy) # y₅ = -3.6161489780361933e-7
xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx); push!(ys, yy) # y₆ = 3.609335053056384e-13
xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx); push!(ys, yy) # y₇ = 0.0
xx
```
### Estimating the inverse function
Suppose $f^{-1}$ exists in a neighborhood of $\alpha$ and we have generated steps in our algorithm $x_0, x_1, \dots, x_n$. We can find $\alpha$ from $f^{-1}(0)$. Typically though, we wouldn't have the inverse function, but we can use facts about functions such as linearization, which *near* $0$ has:
$$
f^{-1}(y) \approx f^{-1}(0) + (f^{-1})'(0)\cdot(y) = \alpha + (f^{-1})'(0)\cdot y
$$
Solving for $\alpha$ gives for $(x_i, f(x_i))$
$$
\alpha \approx f^{-1}(f(x_i)) - f^{-1}(0) f(x_i).
$$
Replacing $f^{-1}(0) \approx (f^{-1})'(y_i) = (f^{-1})'(f(x_i)) = 1/f'(x_i)$ we get:
$$
\alpha \approx x_i - \frac{f(x_i)}{f'(x_i)}.
$$
Which is basically Newton's method.
The above uses one point, $(x_i, f(x_i))$ and the fact that $f$ is differentiable. What if two (or more) points were used, would that give some insight?
Here is some code that interpolates a *polynomial* for $f^{-1}(y)$ through $k$ points and then solves for it's value at $0$, which is $a_0$.
```{julia}
f(x) = cos(x) - x
x₀ = 1
find_zero(f, x₀)
@syms x[1:5] y[1:5] a[0:5]
a₀ = first(a)
function I(k)
eqs = Tuple(sum(a[j] * y[i]^(j-1) for j in 1:k) ~ x[i] for i in 1:k)
sols = solve(eqs, Tuple(a[1:k]))
sols[a₀] # intercept
end
```
Compare to this related call which uses the bisection method:
We use $k=2$ and see what comes:
```{julia}
find_zero(f, (0, 1)) ## [0,1] must be a bracketing interval
I(2)
```
For this example both give the same answer, but the bisection method is a bit less convenient as a bracketing interval must be pre-specified.
## The secant method
The default `find_zero` method above uses a secant-like method unless a bracketing method is found. The secant method is historic, dating back over $3000$ years. Here we discuss the secant method in a more general framework.
One way to view Newton's method is through the inverse of $f$ (assuming it exists): if $f(\alpha) = 0$ then $\alpha = f^{-1}(0)$.
If $f$ has a simple zero at $\alpha$ and is locally invertible (that is some $f^{-1}$ exists) then the update step for Newton's method can be identified with:
* fitting a polynomial to the local inverse function of $f$ going through the point $(f(x_0),x_0)$,
* and matching the slope of $f$ at the same point.
That is, we can write $g(y) = h_0 + h_1 (y-f(x_0))$. Then $g(f(x_0)) = x_0 = h_0$, so $h_0 = x_0$. From $g'(f(x_0)) = 1/f'(x_0)$, we get $h_1 = 1/f'(x_0)$. That is, $g(y) = x_0 + (y-f(x_0))/f'(x_0)$. At $y=0,$ we get the update step $x_1 = g(0) = x_0 - f(x_0)/f'(x_0)$.
A similar viewpoint can be used to create derivative-free methods.
For example, the [secant method](https://en.wikipedia.org/wiki/Secant_method) can be seen as the result of fitting a degree-$1$ polynomial approximation for $f^{-1}$ through two points $(f(x_0),x_0)$ and $(f(x_1), x_1)$.
Again, expressing this approximation as $g(y) = h_0 + h_1(y-f(x_1))$ leads to $g(f(x_1)) = x_1 = h_0$. Substituting $f(x_0)$ gives $g(f(x_0)) = x_0 = x_1 + h_1(f(x_0)-f(x_1))$. Solving for $h_1$ leads to $h_1=(x_1-x_0)/(f(x_1)-f(x_0))$. Then $x_2 = g(0) = x_1 - (x_1-x_0)/(f(x_1)-f(x_0)) \cdot f(x_1)$. This is the first step of the secant method:
$$
x_{n+1} = x_n - f(x_n) \frac{x_n - x_{n-1}}{f(x_n) - f(x_{n-1})}.
$$
That is, where the next step of Newton's method comes from the intersection of the tangent line at $x_n$ with the $x$-axis, the next step of the secant method comes from the intersection of the secant line defined by $x_n$ and $x_{n-1}$ with the $x$ axis. That is, the secant method simply replaces $f'(x_n)$ with the slope of the secant line between $x_n$ and $x_{n-1}$.
We code the update step as `λ2`:
We can see this is a rewriting of the secant method through:
```{julia}
λ2(f0,f1,x0,x1) = x1 - f1 * (x1-x0) / (f1-f0)
x1, x2 = x[1:2]; y1, y2 = y[1:2]
m = (y2 - y1) / (x2 - x1)
secant_method = x2 - (1/m) * y2
simplify(I(2) - secant_method)
```
An inverse quadratic step ($k=2$) is utilized by Brent's method, as possible, to yield a rapidly convergent bracketing algorithm implemented as a default zero finder in many software languages. `Julia`'s `Roots` package implements the method in `Roots.Brent()`.
Then we can run a few steps to identify the zero of sine starting at $3$ and $4$
To see an example of inverse quadratic, we first make a function to compute the next $x$ value, given three previous $x$ and $f(x)$ values.
```{julia}
#| hold: true
#| term: true
x0,x1 = 4,3
f0,f1 = sin.((x0,x1))
@show x1,f1
x0,x1 = x1, λ2(f0,f1,x0,x1)
f0,f1 = f1, sin(x1)
@show x1,f1
x0,x1 = x1, λ2(f0,f1,x0,x1)
f0,f1 = f1, sin(x1)
@show x1,f1
x0,x1 = x1, λ2(f0,f1,x0,x1)
f0,f1 = f1, sin(x1)
@show x1,f1
x0,x1 = x1, λ2(f0,f1,x0,x1)
f0,f1 = f1, sin(x1)
x1,f1
u = lambdify(I(3), (x[1:3]..., y[1:3]...))
```
Like Newton's method, the secant method converges quickly for this problem (though its rate is less than the quadratic rate of Newton's method).
This method is included in `Roots` as `Secant()` (or `Order1()`):
Let's try initial values $(x_0, x_1, x_2) = (0, \pi/2, \pi/4)$:
```{julia}
find_zero(sin, (4,3), Secant())
f(x) = cos(x) - x/2
xs = [0, pi/2, pi/4]
ys = f.(xs)
```
Though the derivative is related to the slope of the secant line, that is in the limit. The convergence of the secant method is not as fast as Newton's method, though at each step of the secant method, only one new function evaluation is needed, so it can be more efficient for functions that are expensive to compute or differentiate.
Now we do a step. The new values is "pushed" to the vector of values.
```{julia}
xx = u(xs[end-2:end]..., ys[end-2:end]...)
yy = f(xx)
push!(xs, xx)
push!(ys, yy)
xx, yy
```
We know do a few more steps, the value of `yy` is shown as a comment.
```{julia}
xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx)
push!(xs, xx); push!(ys, yy) # yy = 0.0011053827937966831
xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx)
push!(xs, xx); push!(ys, yy) # yy = -2.113895167021873e-6
xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx)
push!(xs, xx); push!(ys, yy) # yy = 1.1904810470753091e-11
xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx)
push!(xs, xx); push!(ys, yy) # yy = 0.0
xx
```
The convergence happens quite quickly with this well-behaved problem.
An inverse cubic interpolation is utilized by [Alefeld, Potra, and Shi](https://dl.acm.org/doi/10.1145/210089.210111) which gives an asymptotically even more rapidly convergent algorithm than Brent's (implemented in `Roots.AlefeldPotraShi()` and also `Roots.A42()`). This is used as a finishing step in many cases by the default hybrid `Order0()` method of `find_zero`.
Let $\epsilon_{n+1} = x_{n+1}-\alpha$, where $\alpha$ is assumed to be the *simple* zero of $f(x)$ that the secant method converges to. A [calculation](https://math.okstate.edu/people/binegar/4513-F98/4513-l08.pdf) shows that
$$
\begin{align*}
\epsilon_{n+1} &\approx \frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})} \frac{(1/2)f''(\alpha)(\epsilon_n-\epsilon_{n-1})}{x_n-x_{n-1}} \epsilon_n \epsilon_{n-1}\\
& \approx \frac{f''(\alpha)}{2f'(\alpha)} \epsilon_n \epsilon_{n-1}\\
&= C \epsilon_n \epsilon_{n-1}.
\end{align*}
$$
The constant `C` is similar to that for Newton's method, and reveals potential troubles for the secant method similar to those of Newton's method: a poor initial guess (the initial error is too big), the second derivative is too large, the first derivative too flat near the answer.
Assuming the error term has the form $\epsilon_{n+1} = A|\epsilon_n|^\phi$ and substituting into the above leads to the equation
$$
\frac{A^{1+1/\phi}}{C} = |\epsilon_n|^{1 - \phi +1/\phi}.
$$
The left side being a constant suggests $\phi$ solves: $1 - \phi + 1/\phi = 0$ or $\phi^2 -\phi - 1 = 0$. The solution is the golden ratio, $(1 + \sqrt{5})/2 \approx 1.618\dots$.
### Steffensen's method
Another alternative to the secant method is Steffensen's method.
Steffensen's method is a secant-like method that converges with $|\epsilon_{n+1}| \approx C |\epsilon_n|^2$. The secant is taken between the points $(x_n,f(x_n))$ and $(x_n + f(x_n), f(x_n + f(x_n))$. Like Newton's method this requires $2$ function evaluations per step. Steffensen's is implemented through `Roots.Steffensen()`. Steffensen's method is more sensitive to the initial guess than other methods, so in practice must be used with care, though it is a starting point for many higher-order derivative-free methods.
The secant method has super-linear convergence, but not quadratic convergence. It uses these points to evaluate the values $(x_i, f(x_i)$ and $(x_{i-1}, f(x_{i-1}))$. When $x_i$ converges to $\alpha$, $x_i - x_{i-1}$ will converge to $0$. The secant lines used are eventually "converging" to tangent lines.
Steffensen's method takes a different pair of points to use for a secant line, these being $(x_n,f(x_n))$ and $(x_n + f(x_n), f(x_n + f(x_n)))$. When $x_i \rightarrow \alpha$ it follows for a continuous $f(x)$ that $f(x_i) \rightarrow 0$, so the secant lines used by Steffensen's method will also be close to the tangent line.
[This note](https://fractal.math.unr.edu/~ejolson/701-12/code/hw2sol/hw2sol.pdf) shows that with $\eta_i$ and $\xi_i$ being values that *converge* to $\alpha$, that
$$
e_{i+1} = -e_{i}^2 \cdot
\left(\frac{f''(\xi_i)\left(f'(x_i) - \frac{1}{2} f''(\eta_i) e_i\right) +f''(\eta_i)}{
2f'(x_i) + f''(\xi_i)f(x_i)}\right)
$$
As the following ratio converges to something non zero under assumptions, the Steffensen method has quadratic convergence.
$$
e_{i+1}/e_i^2 \rightarrow \frac{f''(\alpha)(1 + f'(\alpha))}{2f'(\alpha)}
$$
## Inverse quadratic interpolation
Like Newton's method this method requires $2$ function evaluations per step, but unlike Newton's method is derivative free. Steffensen's is implemented in the `Roots` package through `Roots.Steffensen()`. Steffensen's method is more sensitive to the initial guess than other methods, so in practice must be used with care, though it is a starting point for many higher-order derivative-free methods.
Inverse quadratic interpolation fits a quadratic polynomial through three points, not just two like the Secant method. The third being $(f(x_2), x_2)$.
### Alternative bracketing methods
For example, here is the inverse quadratic function, $g(y)$, going through three points marked with red dots. The blue dot is found from $(g(0), 0)$.
The bisection method has several advantages, primarily it is guaranteed to converge regardless of any assumptions on the shape of the function. It's implementation in `Roots` can handle any $x$ values as long as the function value has a sign (not `NaN` and not an error). However, it is slow---linearly convergent. There can be improvements.
#### Regula falsi
One alternative is the (modified) *regula falsi* method which replaces the midpoint ($x_i/2 + x_{i-1}/2$) with the intersection point of the line between two bracketing points $(x_i, f(x_i))$ and $(x_{i-1}, f(x_{i-1}))$ given by solving the following, which comes from similar triangles:
```{julia}
#| hold: true
@syms xᵢ xᵢ₋₁ yᵢ yᵢ₋₁ x
only(solve(yᵢ / (x - xᵢ) ~ -yᵢ₋₁ / (xᵢ₋₁ - x), x))
```
As seen earlier, this formula is a single step of the secant method, but unlike the secant method, for this method the two points chosen to continue are picked to ensure $x_i, x_{i-1}$ form a bracketing interval.
Despite being related to the secant method, the convergence rate of *regula falsi* is only linear. Some function shapes preference a certain endpoint, whereas the secant method chooses the last two values.
@fig-regula-false-convex show that some function shapes result in one end point being fixed as the algorithm progresses which can lead to linear convergence.
::: {#fig-regula-false-convex}
```{julia}
#| echo: false
a,b,c = 1,2,3
fa,fb,fc = -1,1/4,1
g(y) = (y-fb)*(y-fa)/(fc-fb)/(fc-fa)*c + (y-fc)*(y-fa)/(fb-fc)/(fb-fa)*b + (y-fc)*(y-fb)/(fa-fc)/(fa-fb)*a
ys = range(-2,2, length=100)
xs = g.(ys)
plot(xs, ys, legend=false)
scatter!([a,b,c],[fa,fb,fc], color=:red, markersize=5)
scatter!([g(0)],[0], color=:blue, markersize=5)
plot!(zero, color=:blue)
```
Here we use `SymPy` to identify the degree-$2$ polynomial as a function of $y$, then evaluate it at $y=0$ to find the next step:
```{julia}
@syms y hs[0:2] xs[0:2] fs[0:2]
H(y) = sum(hᵢ*(y - fs[end])^i for (hᵢ,i) ∈ zip(hs, 0:2))
eqs = tuple((H(fᵢ) ~ xᵢ for (xᵢ, fᵢ) ∈ zip(xs, fs))...)
ϕ = solve(eqs, hs)
hy = subs(H(y), ϕ)
```
The value of `hy` at $y=0$ yields the next guess based on the past three, and is given by:
```{julia}
q⁻¹ = hy(y => 0)
```
Though the above can be simplified quite a bit when computed by hand, here we simply make this a function with `lambdify` which we will use below.
```{julia}
λ3 = lambdify(q⁻¹) # fs, then xs
```
(`SymPy`'s `lambdify` function, by default, picks the order of its argument lexicographically, in this case they will be the `f` values then the `x` values.)
An inverse quadratic step is utilized by Brent's method, as possible, to yield a rapidly convergent bracketing algorithm implemented as a default zero finder in many software languages. `Julia`'s `Roots` package implements the method in `Roots.Brent()`. An inverse cubic interpolation is utilized by [Alefeld, Potra, and Shi](https://dl.acm.org/doi/10.1145/210089.210111) which gives an asymptotically even more rapidly convergent algorithm than Brent's (implemented in `Roots.AlefeldPotraShi()` and also `Roots.A42()`). This is used as a finishing step in many cases by the default hybrid `Order0()` method of `find_zero`.
In a bracketing algorithm, the next step should reduce the size of the bracket, so the next iterate should be inside the current bracket. However, quadratic convergence does not guarantee this to happen. As such, sometimes a substitute method must be chosen.
[Chandrapatla's](https://www.google.com/books/edition/Computational_Physics/cC-8BAAAQBAJ?hl=en&gbpv=1&pg=PA95&printsec=frontcover) method, is a bracketing method utilizing an inverse quadratic step as the centerpiece. The key insight is the test to choose between this inverse quadratic step and a bisection step. This is done in the following based on values of $\xi$ and $\Phi$ defined within:
```{julia}
function chandrapatla(f, u, v, λ; verbose=false)
a,b = promote(float(u), float(v))
fa,fb = f(a),f(b)
@assert fa * fb < 0
if abs(fa) < abs(fb)
a,b,fa,fb = b,a,fb,fa
let
gr()
Δ = 0.2
function add_line!(xs, ys, f, a)
x = (xs[1]*ys[2] - xs[2]*ys[1])/(ys[2] - ys[1])
scatter!(collect(zip(xs, ys)); marker=(5, :orange))
plot!(collect(zip(xs, ys)); line=(1, :gray50))
scatter!([(x, 0)]; marker=(5, :blue))
annotate!([(x, 0, text(a, :top))])
plot!(plt, [(x, 0), (x, Δ)]; line=(1, :gray50))
xs[2] = x
ys[2] = f(x)
end
c, fc = a, fa
f(x) = 10*log(x)/x^3
plt = plot(; empty_style..., xlims=(0.7, 3.2))
plt = plot!(plt, f, 0.8, 3)
plot!([(0.8, 0), (3.2, 0)]; line=(1, :black), arrow=true, side=:right)
maxsteps = 100
for ns in 1:maxsteps
x₀ = 2.75
x₁ = 0.85
α = 1.0
xs = [x₁, x₀]
ys = f.(xs)
Δ = abs(b-a)
m, fm = (abs(fa) < abs(fb)) ? (a, fa) : (b, fb)
ϵ = eps(m)
if Δ ≤ 2ϵ
return m
end
@show m,fm
iszero(fm) && return m
add_line!(xs, ys, f, L"x_2")
add_line!(xs, ys, f, L"x_3")
add_line!(xs, ys, f, L"x_4")
add_line!(xs, ys, f, L"x_5")
add_line!(xs, ys, f, L"x_6")
add_line!(xs, ys, f, L"x_7")
ξ = (a-b)/(c-b)
Φ = (fa-fb)/(fc-fb)
plot!(plt, [(x₀, 0), (x₀, Δ)]; line=(1, :gray50))
plot!(plt, [(x₁, 0), (x₁, Δ)]; line=(1, :gray50))
annotate!(plt, [(x₀, 0, text(L"x_0", :top)),
(x₁, 0, text(L"x_1", :top)),
(α, 0, text(L"\alpha", :left, :top))])
scatter!(plt, [(α, 0)]; marker=(5, :green))
if Φ^2 < ξ < 1 - (1-Φ)^2
xt = λ(fa,fc,fb, a,c,b) # inverse quadratic
else
xt = a + (b-a)/2
end
ft = f(xt)
isnan(ft) && break
if sign(fa) == sign(ft)
c,fc = a,fa
a,fa = xt,ft
else
c,b,a = b,a,xt
fc,fb,fa = fb,fa,ft
end
verbose && @show ns, a, fa
end
error("no convergence: [a,b] = $(sort([a,b]))")
plotly()
plt
end
```
Like bisection, this method ensures that $a$ and $b$ is a bracket, but it moves $a$ to the newest estimate, so does not maintain that $a < b$ throughout.
Plot illustrating that the *regula falsi* method may have a fixed endpoint for some convex functions
:::
We can see it in action on the sine function. Here we pass in $\lambda$, but in a real implementation (as in `Roots.Chandrapatla()`) we would have programmed the algorithm to compute the inverse quadratic value.
#### Modified regula falsi
@fig-modified-regula-falsi shows a scenario where the secant line between $(x_{i-1}, f(x_{i-1}))$ and $(x_i, f(x_i))$ crosses the $x$ axis at $x_{i+1}$ which is to the *right* of the zero $\alpha$, as it always will be for this curve and these points. A modified *regula falsi* method modifies the fixed end by using $\tilde{f}(x_i)$ and not $f(x_i)$ to compute the secant line, where $\tilde{f}$ is some multiple, $\gamma$, of $f$. In the figure, $\gamma$ is shown so that the *next* choice ($x_{i+2}$ would be its label) is exactly $\alpha$. And value for the multiplier less than this $\gamma$ will shift the intersection point to the other side of $\alpha$. The value of $\gamma$ is the ratio of the secant line slopes between $x_{i+1}$ and $\alpha$ and between $x_{i-1}$ and $\alpha$. Some choices for $\gamma$ lead to super-linear convergence.
::: {#fig-modified-regula-falsi}
```{julia}
#| echo: false
let
gr()
dd(f, a, b) = (f(a) - f(b)) / (a - b)
f(x) = (x-3)^2 - 1
plt = plot(; xlims=(0.5, 3.25), empty_style...)
plot!(plt, [(1,0), (3.25,0)]; arrow=true, side=:right, line=(1, :gray))
plot!(f, 1, 3.25; line=(1, :black))
xᵢ₋₁ , xᵢ = 1.25, 2.75
xᵢ₊₁ = (xᵢ₋₁ * f(xᵢ) - xᵢ * f( xᵢ₋₁)) / (f(xᵢ) - f(xᵢ₋₁))
α = 2
γ = dd(f, xᵢ₊₁, α) / dd(f, xᵢ₋₁, α)
plot!(plt, [(xᵢ₋₁, 0), (xᵢ₋₁, f(xᵢ₋₁))]; line=(1, :dot))
plot!(plt, [(xᵢ₋₁, f(xᵢ₋₁)) , (xᵢ, f(xᵢ))]; line=(1, :dash, :blue))
plot!(plt, [(xᵢ₋₁, γ*f(xᵢ₋₁)) , (xᵢ₊₁, f(xᵢ₊₁))]; line=(1, :dash, :blue))
scatter!(plt, [(xᵢ₋₁, γ*f(xᵢ₋₁)), (xᵢ₋₁, f(xᵢ₋₁)),
(α, 0), (xᵢ₊₁, 0),
(xᵢ₊₁, f(xᵢ₊₁)), (xᵢ, f(xᵢ))
]; marker=(3, :orange))
annotate!(plt, [
(xᵢ₋₁, 0, text(L"x_{i-1}", :top)),
(xᵢ₋₁, γ*f(xᵢ₋₁), text(L"\gamma \cdot f(x_{i-1})", :right)),
(xᵢ₋₁, f(xᵢ₋₁), text(L"f(x_{i-1})", :right)),
(α, 0, text(L"\alpha", :bottom)),
(xᵢ₊₁, 0, text(L"x_{i+1}", :top)),
(xᵢ, 0, text(L"x_{i}", :top))
])
plotly()
plt
end
```
Modified *regula falsi* method illustration. When midpoint $x_{i+1}$ is on same side of zero $\alpha$ as $x_i$ the *next* step will be between $x_{i-1}$ and $x_{i+1}$. *Were* $x_{i-1}$ modified by $\gamma$ the next midpoint would be an exact zero. If multiplied by a value less, then the midpoint moves to other side of $\alpha$ and would break the repeated choice of a fixed side when keeping a bracketing interval.
:::
#### Anderson Bjork
There are numerous modifications of the *regula falsi* algorithm that
employ different scaling values, we discuss one now. The
[Anderson-Bjork](https://iopscience.iop.org/article/10.1088/1757-899X/1276/1/012010/pdf)
method is a modification of the *regula falsi* method that avoids the
linear convergence when one endpoint is always fixed.
The modification works as follows, suppose the bracketing interval is $[a,b]$ and $c$ is the point found by the secant line. Then if $f(a)$ and $f(c)$ have the same sign **and** the previous step kept the right side point ($b$) fixed, then instead of using $(c, f(c))$ and $(b, f(c))$ as the new points (as $[c,b]$ is a bracket) use $(b, \gamma \cdot f(b))$ where $\gamma = 1 - f(c)/f(a)$ if $\gamma$ is positive and $\gamma=1/2$ if not. This will modify the next step in the algorithm. The $\gamma$ factors are multiplied each time, so that eventually the $y$ value used at the fixed side should lead to a midpoint on the other side of the zero, as happens when the modified value of $f(x_1)$ and $f(x_5)$ are used to find the midpoint in @fig-anderson-bjork-trajectory.
::: {#fig-anderson-bjork-trajectory}
```{julia}
#| echo: false
let
gr()
midpt(a,b,fa,fb) = (a*fb - b*fa)/(fb-fa)
side = nothing
function ABstep!(xs, ys, side, label)
a, b = xs; fa, fb = ys
c = midpt(a,b,fa,fb)
fc = f(c)
if sign(fa) == sign(fc)
xs[1] = c
ys[1] = fc
if side == :right
m = 1 - fc/fa
m = m < 0 ? 1/2 : m
ys[2] *= m
else
side = :right
end
else
xs[2] = c
ys[2] = fc
if side == :left
m = 1 - fc/fb
m = m < 0 ? 1/2 : m
ys[1] *= m
else
side = :left
end
end
plot!([(c,0), (c,fc)]; line=(1, :dash, :gray50))
scatter!([(c,0)]; marker=(5, :blue))
scatter!(collect(zip(xs, ys)); marker=(5, :orange))
plot!(collect(zip(xs, ys)); line=(1, :gray50))
annotate!([(c, 0, text(label, :top, :left))])
side
end
f(x) = 10*log(x)/x^3
plt = plot(f, 0.8, 3; xlims=(0.7, 3.2), empty_style...)
plot!([(0.8, 0), (3.2, 0)]; line=(1, :black), arrow=true, side=:right)
α = 1
x₀, x₁ = 2.75, 0.85
xs = [x₀, x₁]
ys = f.(xs)
scatter!(plt, collect(zip(xs, ys)); marker=(5, :orange))
plot!(plt, collect(zip(xs, ys)); line=(1, :gray50))
plot!(plt, [(x₀,0), (x₀, f(x₀))]; line=(1, :dash, :gray50))
plot!(plt, [(x₁,0), (x₁, f(x₁))]; line=(1, :dash, :gray50))
side = ABstep!(xs, ys, side, L"x_2")
side = ABstep!(xs, ys, side, L"x_3")
side = ABstep!(xs, ys, side, L"x_4")
side = ABstep!(xs, ys, side, L"x_5")
side = ABstep!(xs, ys, side, L"x_6")
c = midpt(xs..., ys...)
scatter!([(c,0)]; marker=(5, :blue))
scatter!([(α,0)]; marker=(5, :green))
annotate!([(c,0, text(L"x_7", :top, :left)),
(x₀, 0, text(L"x_0", :top, )),
(x₁, 0, text(L"x_1", :top, :right)),
(α, 0, text(L"\alpha", :top, :left))
])
plotly()
current()
end
```
Illustration of Anderson-Bjork algorithm. The point $x_1$ stays as the left-hand endpoint up until $x_6$, but by modifying $f(x_0)$ the algorithm converges super-linearly towards $\alpha$, as compared to @fig-regula-false-convex.
:::
::: {.callout-note}
## Hybrid algorithms
There are a few, newer, *hybrid algorithms* where some dynamic choice is made as to what update step should be chosen. One due to Chandrapatla (implemented in `Roots.Chandrapatla`) is a bracketing algorithm which chooses between an inverse quadratic step or a bisection step using a certain inequality. We note another, a bracketing algorithm due to Ganchovski and Traykov (with improvements by some `Julia` programmers since its inclusion in the `NonLinearSolve.jl` package) that chooses between bisection or the Anderson-Bjork update based on an estimate of how "straight" the curve is. This is implemented in `Roots.ModAB`. The latter is quite efficient over a wide range of problems.
:::
##### Examples
The function $f(x) = (x^2 + 1) \cdot \sin(x) - e^{\sqrt{\lvert x\rvert}} \cdot (x - 1) \cdot (x^2 - 5)$ has a zero *between* $0$ and $1$, and *near* $0.8$. We see how to find it using various algorithms implemented in `Roots`.
```{julia}
#| term: true
chandrapatla(sin, 3, 4, λ3, verbose=true)
f(x) = (x^2 + 1) * sin(x) - exp(sqrt(abs(x))) * (x - 1) * (x^2 - 5)
x0 = 0.8
xs = (0, 1)
```
The Steffensen method needs a *nearby* estimate:
```{julia}
find_zero(f, x0, Roots.Steffensen()) # 5 iterations, 12 function evaluations
```
The `Sidi(2)` method needs a *nearby* estimate or an initial two points for a secant line which it bootstraps to get a third point. We use the bracketing interval below:
```{julia}
find_zero(f, xs, Roots.Sidi(2)) # 3 iterations, 6 function evaluations
```
For some steps, `Brent` and `Chandrapatla` use a quadratic inverse calculation, whereas `A42` uses a cubic inverse calculation:
```{julia}
find_zero(f, xs, Roots.Brent()) # 16 iterations, 18 function evaluations
find_zero(f, xs, Roots.Chandrapatla()) # 20 iterations, 22 function evaluations
find_zero(f, xs, Roots.A42()) # 4 iterations, 10 function evaluations
```
Finally, we compare *regula falsi* variants:
```{julia}
find_zero(f, xs, Roots.RegulaFalsi(:classic)) # 10 iterations, 13 function evaluations
find_zero(f, xs, Roots.RegulaFalsi(:AndersonBjork)) # 6 iterations, 9 function evaluations
find_zero(f, xs, Roots.ModAB()) # 6 iterations, 8 function evaluations
```
For this problem, all methods converge to the same zero, but from the counts of iterations and function evaluations they differ in how the get there.
## Tolerances
Iterative zero-finding algorithms may mathematically converge, but when implemented on the computer a stopping rule must be articulated. Typically these involve the following:
The `chandrapatla` algorithm typically waits until `abs(b-a) <= 2eps(m)` (where $m$ is either $b$ or $a$ depending on the size of $f(a)$ and $f(b)$) is satisfied. Informally this means the algorithm stops when the two bracketing values are no more than a small amount apart. What is a "small amount?"
* stop (and fail) if too many steps are taken
* stop when $\lvert x_i - x_{i-1} \rvert$ is quite small (as then the algorithm stops improving)
* stop when $f(x_i)$ is quite small, as it is close to being zero.
Small on the computer is a *relative* term and requires a bit of discussion.
To understand, we start with the fact that floating point numbers are an approximation to real numbers.
When $\lvert x_i - x_{i-1} \rvert$ is small, we have to recall that the gap between floating point numbers depends on the size of the number, and doubles in going from $[2^{i-1}, 2^i)$ to $[2^i, 2^{i+1})$. As such, a relative tolerance is often chosen so that *small* really means that for some $\epsilon$
Floating point numbers effectively represent a number in scientific notation in terms of
* a sign (plus or minus) ,
* a *mantissa* (a number in $[1,2)$, in binary ), and
* an exponent (to represent a power of $2$).
The mantissa is of the form `1.xxxxx...xxx` where there are $m$ different `x`s each possibly a `0` or `1`. The `i`th `x` indicates if the term `1/2^i` should be included in the value. The mantissa is the sum of `1` plus the indicated values of `1/2^i` for `i` in `1` to `m`. So the last `x` represents if `1/2^m` should be included in the sum. As such, the mantissa represents a discrete set of values, separated by `1/2^m`, as that is the smallest difference possible.
For example if `m=2` then the possible value for the mantissa are `11 => 1 + 1/2 + 1/4 = 7/4`, `10 => 1 + 1/2 = 6/4`, `01 => 1 + 1/4 = 5/4`. and `00 => 1 = 4/4`, values separated by `1/4 = 1/2^m`.
For $64$-bit floating point numbers `m=52`, so the values in the mantissa differ by `1/2^52 = 2.220446049250313e-16`. This is the value of `eps()`.
However, this "gap" between numbers is for values when the exponent is `0`. That is the numbers in `[1,2)`. For values in `[2,4)` the gap is twice, between `[1/2,1)` the gap is half. That is the gap depends on the size of the number. The gap between `x` and its next largest floating point number is given by `eps(x)` and that always satisfies `eps(x) <= eps() * abs(x)`.
One way to think about this is the difference between `x` and the next largest floating point values is *basically* `x*(1+eps()) - x` or `x*eps()`.
For the specific example, `abs(b-a) <= 2eps(m)` means that the gap between `a` and `b` is essentially 2 floating point values from the $x$ value with the smallest $f(x)$ value.
$$
\lvert x_i - x_{i-1} \rvert \leq \max(\lvert x_i \rvert, \lvert x_{i-1} \rvert) \cdot \epsilon.
$$
In code, this might be `abs(b-a) <= 2eps(m)`, which means that the "gap" between `a` and `b` is essentially no more than $2$ floating point values from the $x$ value with the smallest $f(x)$ value.
For bracketing methods that is about as good as you can get. However, once floating point values are understood, the absolute best you can get for a bracketing interval would be
* along the way, a value `f(c)` is found which evaluates *exactly* to `0.0`
* the endpoints of the bracketing interval are *adjacent* floating point values, meaning the interval can not be bisected and `f` changes sign between the two values.
* along the way, a value `f(c)` is found which evaluates *exactly* to `0.0`
* the endpoints of the bracketing interval are *adjacent* floating point values, meaning the interval can not be bisected and `f` changes sign between the two values.
There can be problems when the stopping criteria is `abs(b-a) <= 2eps(m))` and the answer is `0.0` that require engineering around. For example, the algorithm above for the function `f(x) = -40*x*exp(-x)` does not converge when started with `[-9,1]`, even though `0.0` is an obvious zero.
There can be problems when the stopping criteria is `abs(b-a) <= 2eps(m))` and the answer is `0.0` that require engineering around. As such, an *absolute* tolerance might be needed, one where $\lvert x_i - x_{i-1} \rvert \leq \delta$.
For bracketing algorithms, consideration of $\lvert x_i - x_{i-1} \rvert$ might be all that matters, but not for algorithms like Newton's or the secant algorithm. In Newton's method the update step is $f(x_{i-1})/f'(x_{i-1})$. Naturally when $f(x_i)$ is close to $0$, the update step is small and $\lvert x_{i} - x_{i-1}\rvert = \Delta$ will be close to $0$. *However*, should $f'(x_i)$ be large, then $\Delta$ can also be small and the algorithm will possibly stop, as $x_{i} \approx x_{i-1}$---but not necessarily $x_{i} \approx \alpha$. So termination on $\Delta$ alone can be off. Checking if $f(x_{i})$ is an approximate zero---as it should be if $f$ is continuous---is also useful to include in a stopping criteria.
However, there may never be a value with `f(x_i)` exactly `0.0`. (The value of `sin(1pi)` is not zero, for example, as `1pi` is an approximation to $\pi$, as well the `sin` of values adjacent to `float(pi)` do not produce `0.0` exactly.)
```{julia}
#| hold: true
#| error: true
fu(x) = -40*x*exp(-x)
chandrapatla(fu, -9, 1, λ3)
```
Here the issue is `abs(b-a)` is tiny (of the order `1e-119`) but `eps(m)` is even smaller.
> For checking if $x_n \approx x_{n+1}$ both a relative and absolute error should be used unless something else is known.
Suppose `x_i` is the closest floating point number to $\alpha$, the mathematical zero. Then the relative rounding error, $($ `x_i` $- \alpha)/\alpha$, will be a value $\delta$ with $\delta$ less than `eps()`.
For non-bracketing methods, like Newton's method or the secant method, different criteria are useful. There may not be a bracketing interval for `f` (for example `f(x) = (x-1)^2`) so the second criteria above might need to be restated in terms of the last two iterates, $x_n$ and $x_{n-1}$. Calling this difference $\Delta = |x_n - x_{n-1}|$, we might stop if $\Delta$ is small enough. As there are scenarios where this can happen, but the function is not at a zero, a check on the size of $f$ is needed.
However, there may be no floating point value where $f$ is exactly `0.0` so checking the size of `f(x_n)` requires some agreement.
First if `f(x_n)` is `0.0` then it makes sense to call `x_n` an *exact zero* of $f$, even though this may hold even if `x_n`, a floating point value, is not mathematically an *exact* zero of $f$. (Consider `f(x) = x^2 - 2x + 1`. Mathematically, this is identical to `g(x) = (x-1)^2`, but `f(1 + eps())` is zero, while `g(1+eps())` is `4.930380657631324e-32`.
However, there may never be a value with `f(x_n)` exactly `0.0`. (The value of `sin(1pi)` is not zero, for example, as `1pi` is an approximation to $\pi$, as well the `sin` of values adjacent to `float(pi)` do not produce `0.0` exactly.)
Suppose `x_n` is the closest floating point number to $\alpha$, the zero. Then the relative rounding error, $($ `x_n` $- \alpha)/\alpha$, will be a value $\delta$ with $\delta$ less than `eps()`.
How far then can `f(x_n)` be from $0 = f(\alpha)$?
How far then can `f(x_i)` be from $0 = f(\alpha)$? Consider:
$$
f(x_n) = f(x_n - \alpha + \alpha) = f(\alpha + \alpha \cdot \delta) = f(\alpha \cdot (1 + \delta)),
f(x_i) = f(x_i - \alpha + \alpha) = f(\alpha + \alpha \cdot \delta) = f(\alpha \cdot (1 + \delta)),
$$
where $\delta = x_i/\alpha - 1$ is close to $0$ if $x_i$ converges to $\alpha$.
Assuming $f$ has a derivative, the linear approximation gives:
$$
f(x_n) \approx f(\alpha) + f'(\alpha) \cdot (\alpha\delta) = f'(\alpha) \cdot \alpha \delta
f(x_n) \approx f(\alpha) + f'(\alpha) \cdot (\alpha\delta) = \alpha \cdot f'(\alpha) \cdot \delta
$$
So we should consider `f(x_n)` an *approximate zero* when it is on the scale of $f'(\alpha) \cdot \alpha \delta$. That $\alpha$ factor means we consider a *relative* tolerance for `f`.
So we should consider `f(x_i)` an *approximate zero* when it is on the scale of $\alpha \cdot f'(\alpha) \cdot \delta$. That $\alpha$ factor means we consider a *relative* tolerance, $\delta$, for $f(x_i)$ based on $\lvert x_i\rvert$.
> For checking if $f(x_n) \approx 0$ both a relative and absolute error should be used---the relative error involving the size of $x_n$.
A good condition to check if `f(x_n)` is small is
As well though, for $\alpha$ values close to $0$ this relative tolerance might be an issue, and a small absolute tolerance can be needed.
`abs(f(x_n)) <= abs(x_n) * rtol + atol`, or `abs(f(x_n)) <= max(abs(x_n) * rtol, atol)`
A good condition to check if `f(x_i)` is small is
* `abs(f(x_i)) <= abs(x_i) * rtol + atol`, or
* `abs(f(x_i)) <= max(abs(x_i) * rtol, atol)`
where the relative tolerance, `rtol`, would absorb an estimate for $f'(\alpha)$.
Now, in Newton's method the update step is $f(x_n)/f'(x_n)$. Naturally when $f(x_n)$ is close to $0$, the update step is small and $\Delta$ will be close to $0$. *However*, should $f'(x_n)$ be large, then $\Delta$ can also be small and the algorithm will possibly stop, as $x_{n+1} \approx x_n$ – but not necessarily $x_{n+1} \approx \alpha$. So termination on $\Delta$ alone can be off. Checking if $f(x_{n+1})$ is an approximate zero is also useful to include in a stopping criteria.
One thing to keep in mind is that the right-hand side of the rule `abs(f(x_i)) <= abs(x_i) * rtol + atol`, as a function of `x_i`, goes to `Inf` as `x_i` increases. So if `f` has `0` as an asymptote (like `e^(-x)`) for large enough `x_i`, the rule will be `true` and `x_i` could be counted as an approximate zero, despite it not being one.
One thing to keep in mind is that the right-hand side of the rule `abs(f(x_n)) <= abs(x_n) * rtol + atol`, as a function of `x_n`, goes to `Inf` as `x_n` increases. So if `f` has `0` as an asymptote (like `e^(-x)`) for large enough `x_n`, the rule will be `true` and `x_n` could be counted as an approximate zero, despite it not being one.
A modified criteria for convergence might look like:
So a modified criteria for convergence might look like:
* stop if $\Delta$ is small and `f` is an approximate zero with some tolerances
* stop if $\Delta$ is small and `f` is an approximate zero with some tolerances
* stop if `f` is an approximate zero with some tolerances, but be mindful that this rule can identify mathematically erroneous answers.
* stop if `f` is an approximate zero with some tolerances, but be mindful that this rule can identify mathematically erroneous answers.
It is not uncommon to assign `rtol` to have a value like `sqrt(eps())` to account for accumulated floating point errors and the factor of $f'(\alpha)$, though in the `Roots` package it is set smaller by default.
@@ -397,12 +554,17 @@ It is not uncommon to assign `rtol` to have a value like `sqrt(eps())` to accoun
### Conditioning and stability
In Part III of @doi:10.1137/1.9781611977165 we find language of numerical analysis useful to formally describe the zero-finding problem. Key concepts are errors, conditioning, and stability. These give some theoretical justification for the tolerances above.
This next part is a technical, mathematical---not practical---motivation for why we might stop when $x_i \approx x_{i-1}$ or $f(x_i) \approx 0$.
In Part III of @doi:10.1137/1.9781611977165 we find language of numerical analysis useful to formally describe the zero-finding problem. Key concepts are errors, conditioning, and stability, which can be used to give some theoretical justification for the tolerances above.
Abstractly a *problem* is a mapping, $F$, from a domain $X$ of data to a range $Y$ of solutions. Both $X$ and $Y$ have a sense of distance given by a *norm*. A norm (denoted with $\lVert\cdot\rVert$) is a generalization of the absolute value and gives quantitative meaning to terms like small and large.
::: {.definition title="Well conditioned problem"}
> A *well-conditioned* problem is one with the property that all small perturbations of $x$ lead to only small changes in $F(x)$.
A *well-conditioned* problem is one with the property that all small perturbations of $x$ lead to only small changes in $F(x)$.
:::
This sense of "small" is measured through a *condition number*.
@@ -412,25 +574,32 @@ The *forward error* is $\lVert\delta_F\rVert = \lVert F(x+\delta_x) - F(x)\rVert
The *backward error* is $\lVert\delta_x\rVert$, the *relative backward error* is $\lVert\delta_x\rVert / \lVert x\rVert$.
The *absolute condition number* $\hat{\kappa}$ is worst case of this ratio $\lVert\delta_F\rVert/ \lVert\delta_x\rVert$ as the perturbation size shrinks to $0$.
The relative condition number $\kappa$ divides $\lVert\delta_F\rVert$ by $\lVert F(x)\rVert$ and $\lVert\delta_x\rVert$ by $\lVert x\rVert$ before taking the ratio.
The *absolute condition number*, $\hat{\kappa}$, is the worst case of the forward error divided by the backward error, or this ratio $\lVert\delta_F\rVert/ \lVert\delta_x\rVert$, as the perturbation size shrinks to $0$.
The *relative condition number*, $\kappa$, divides $\lVert\delta_F\rVert$ by $\lVert F(x)\rVert$ and $\lVert\delta_x\rVert$ by $\lVert x\rVert$ before taking the ratio.
A *problem* is a mathematical concept, an *algorithm* the computational version. Algorithms may differ for many reasons, such as floating point errors, tolerances, etc. We use notation $\tilde{F}$ to indicate the algorithm.
The absolute error in the algorithm is $\lVert\tilde{F}(x) - F(x)\rVert$, the relative error divides by $\lVert F(x)\rVert$. A good algorithm would have smaller relative errors.
The *absolute error in the algorithm* is $\lVert\tilde{F}(x) - F(x)\rVert$, the relative error divides by $\lVert F(x)\rVert$. A good algorithm would have smaller relative errors.
An algorithm is called *stable* if
$$
\frac{\lVert\tilde{F}(x) - F(\tilde{x})\rVert}{\lVert F(\tilde{x})\rVert}
\frac{\lVert\tilde{F}(x) - F(\tilde{x})\rVert}{\lVert F(\tilde{x})\rVert},
$$
is *small* for *some* $\tilde{x}$ relatively near $x$, $\lVert\tilde{x}-x\rVert/\lVert x\rVert$.
> A *stable* algorithm gives nearly the right answer to nearly the right question.
::: {.definition tilte="Stable algorithm"}
A *stable* algorithm gives nearly the right answer to nearly the right question.
:::
The right answer is $F(x)$, the nearly right answer is $F(\tilde{x})$, the nearly right question is $\tilde{F}(x)$.
(The answer it gives is $\tilde{F}(x)$, the nearly right question: what is $F(\tilde{x})$?)
A related concept is an algorithm $\tilde{F}$ for a problem $F$ is *backward stable* if for each $x \in X$,
@@ -438,9 +607,15 @@ $$
\tilde{F}(x) = F(\tilde{x})
$$
for some $\tilde{x}$ where $\lVert\tilde{x} - x\rVert/\lVert x\rVert$ is small.
for *some* $\tilde{x}$ where $\lVert\tilde{x} - x\rVert/\lVert x\rVert$ is small.
> "A backward stable algorithm gives exactly the right answer to nearly the right question."
::: {.definition tilte="Backward stable algorithm"}
"A backward stable algorithm gives exactly the right answer to nearly the right question."
:::
The nearly right question is $\tilde{F}(x)$, the exactly right answer to this is $F(\tilde{x})$.
The concepts are related by Trefethen and Bao's Theorem 15.1 which says for a backward stable algorithm the relative error $\lVert\tilde{F}(x) - F(x)\rVert/\lVert F(x)\rVert$ is small in a manner proportional to the relative condition number.
@@ -449,7 +624,7 @@ Applying this to the zero-finding we follow @doi:10.1137/1.9781611975086.
To be specific, the problem, $F$, is finding a zero of a function $f$ starting at an initial point $x_0$. The data is $(f, x_0)$, the solution is $r$ a zero of $f$.
Take the algorithm as Newton's method. Any implementation must incorporate tolerances, so this is a computational approximation to the problem. The data is the same, but technically we use $\tilde{f}$ for the function, as any computation is dependent on machine implementations. The output is $\tilde{r}$ an *approximate* zero.
For concreteness, take the algorithm as Newton's method. Any implementation must incorporate tolerances, so this is a computational approximation to the problem. The data is the same, but technically we use $\tilde{f}$ for the function, as any computation is dependent on machine implementations. The output is $\tilde{r}$ an *approximate* zero.
Suppose for sake of argument that $\tilde{f}(x) = f(x) + \epsilon$, $f$ has a continuous derivative, and $r$ is a root of $f$ and $\tilde{r}$ is a root of $\tilde{f}$. Then by linearization:
@@ -463,15 +638,22 @@ $$
$$
Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the condition number:
> The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$.
::: {.definition title="Absolute condition number"}
The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$.
:::
The error formula in Newton's method measuring the distance between the actual root and an approximation includes the derivative in the denominator, so we see large condition numbers are tied into possibly larger errors.
Now consider $g(x) = f(x) - f(\tilde{r})$. Call $f(\tilde{r})$ the residual. We have $g$ is near $f$ if the residual is small. The algorithm will solve $(g, x_0)$ with $\tilde{r}$, so with a small residual an exact solution to an approximate question will be found. Driscoll and Braun state
> The backward error in a root estimate is equal to the residual.
::: {.relationship title="Backward error and residual"}
The backward error in a root estimate is equal to the residual.
:::
Practically these two observations lead to
@@ -630,8 +812,8 @@ choices = [
"The function oscillates too much to rely on the tangent line approximation far from the zero",
"We can find an answer"
]
answ = 4
radioq(choices, answ, keep_order=true)
answer = 4
radioq(choices, answer, keep_order=true)
```
Does `find_zero` find a zero to this function starting from $0.175$?
@@ -664,6 +846,6 @@ choices = [
"The function oscillates too much to rely on the tangent line approximations far from the zero",
"We can find an answer"
]
answ = 3
radioq(choices, answ, keep_order=true)
answer = 3
radioq(choices, answer, keep_order=true)
```
quarto/derivatives/newtons_method.qmd
+551 -374
View File
File diff suppressed because it is too large. Load diff
quarto/derivatives/numeric_derivatives.qmd
+133 -96
View File
@@ -1,4 +1,4 @@
# Numeric derivatives
# Computing derivatives in Julia
{{< include ../_common_code.qmd >}}
@@ -19,10 +19,10 @@ using Roots
---
`SymPy` returns symbolic derivatives. Up to choices of simplification, these answers match those that would be derived by hand. This is useful when comparing with known answers and for seeing the structure of the answer. However, there are times we just want to work with the answer numerically. For that we have other options within `Julia`. We discuss approximate derivatives and automatic derivatives. The latter will find wide usage in these notes.
`SymPy` returns symbolic derivatives. Up to choices of simplification, these answers match those that would be derived by hand. This is useful when comparing with known answers and for seeing the structure of the answer. However, there are times we just want to work with the answer numerically. For that we have other options within `Julia`. We discuss approximate derivatives and automatic derivatives in this section. The latter will find wide usage in these notes.
### Approximate derivatives
## Approximate derivatives
By approximating the limit of the secant line with a value for a small, but positive, $h$, we get an approximation to the derivative. That is
@@ -32,24 +32,25 @@ $$
f'(x) \approx \frac{f(x+h) - f(x)}{h}.
$$
This is the forward-difference approximation. The central difference approximation looks both ways:
This is the *forward-difference approximation*. The *central difference approximation* looks to both sides of $x$:
$$
f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}.
$$
Though in general they are different, they are both approximations. The central difference is usually more accurate for the same size $h$. However, both are susceptible to round-off errors. The numerator is a subtraction of like-size numbers - a perfect opportunity to lose precision.
Though in general they are different, both are easy and fast to compute, useful approximations to the derivative. The central difference is usually more accurate for the same size $h$. However, both are susceptible to round-off errors. The numerator is a subtraction of like-size numbers---a perfect opportunity to lose precision.
As such there is a balancing act:
Due to numeric issues there is a balancing act:
* if $h$ is too big the approximation to the limit is not good.
* if $h$ is too small the round-off errors are problematic,
* if $h$ is too small the round-off errors are problematic,
* if $h$ is too big the approximation to the limit is not good.
For the forward difference $h$ values around $10^{-8}$ are typically good, for the central difference, values around $10^{-6}$ are typically good.
For the forward difference $h$ values around $10^{-8}$ are typically good, for the central difference, values around $10^{-6}$ are typically good, but these ranges aren't always the case.
##### Example
@@ -78,7 +79,7 @@ abs(factual - fapprox)
The error is about $1$ part in $100$ million.
The central difference is better here:
The central difference is better here, even with a bigger $h$:^[The [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl) and [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl) packages provide performant interfaces for differentiation based on finite differences.]
```{julia}
@@ -88,49 +89,53 @@ cdapprox = (f(c+h) - f(c-h)) / (2h)
abs(factual - cdapprox)
```
---
The [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl) and [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl) packages provide performant interfaces for differentiation based on finite differences.
### Automatic derivatives
Roughly speaking, symbolic derivatives are exact, but can be slow to compute, whereas approximate derivatives are not exact, but fast to compute. However, the widely used automatic derivatives are both exact and fast to compute.
There are some other ways to compute derivatives numerically that give much more accuracy at the expense of slightly increased computing time. Automatic differentiation is the general name for a few different approaches. These approaches promise less complexity - in some cases - than symbolic derivatives and more accuracy than approximate derivatives; the accuracy is on the order of machine precision.
Automatic differentiation (AD) is the general name for a few different approaches. We utilize forward mode automatic differentiation. The `ForwardDiff` package provides one of [several](https://juliadiff.org/) ways for `Julia` to compute automatic derivatives. `ForwardDiff` is well suited for functions encountered in these notes, which depend on at most a few variables and output no more than a few values at once.
The `ForwardDiff` package provides one of [several](https://juliadiff.org/) ways for `Julia` to compute automatic derivatives. `ForwardDiff` is well suited for functions encountered in these notes, which depend on at most a few variables and output no more than a few values at once.
The `ForwardDiff` package was loaded in this section; in general its features are available when the `CalculusWithJulia` package is loaded, as that package provides a more convenient interface. The `derivative` function is not exported by `ForwardDiff`, so its usage requires qualification. To illustrate, to find the derivative of $f(x)$ at a *point* we have this syntax:
The `ForwardDiff` package was loaded in this section; in general its features are available when the `CalculusWithJulia` package is loaded, as that package provides a more convenient interface. The `derivative` function is not exported by `ForwardDiff`, so its usage requires qualification. To illustrate, to find the derivative of $f(x)$ at a *point* we have this syntax for `ForwardDiff`:
```{julia}
ForwardDiff.derivative(f, c) # derivative is qualified by a module name
```
The `CalculusWithJulia` package defines an operator `D` which goes from finding a derivative at a point with `ForwardDiff.derivative` to defining a function which evaluates the derivative at each point. It is defined along the lines of `D(f) = x -> ForwardDiff.derivative(f,x)` in parallel to how the derivative operation for a function is defined mathematically from the definition for its value at a point.
The `CalculusWithJulia` package defines a method for `'` (a postfix operator in `Julia`) so that when applied to a `Function` object a function for computing values of the derivative, as above, is returned.^[The `CalculusWithJulia` package defines a method for `Base.adjoint(f::Function)`. In `Julia` speak this is a form of *type piracy*, as the package modifies a method (`adjoint`) for a type (`Function`) neither of which belongs to the package. As well, the meaning given does not conform with the expected generic meaning of the operation elsewhere in the `Julia` ecosystem. Admittedly this is bad form, but a useful deviation from good practice for pedagogical reasons.]
To be clear, this usage returns a function that computes the derivative of `f`:
```{julia}
f'
```
And this usage *calls* the derivative of `f` at the value stored by `c`:
```{julia}
f'(c)
```
This is the same mathematical notation that is commonly used.
Here we see the error in estimating $f'(1)$:
```{julia}
fauto = D(f)(c) # D(f) is a function, D(f)(c) is the function called on c
abs(factual - fauto)
abs(factual - f'(c))
```
In this case, it is exact.
In this case, the automatic derivative is exact.
The `D` operator is defined for most all functions in `Julia`, though, like the `diff` operator in `SymPy` there are some for which it won't work.
##### Example
For $f(x) = \sqrt{1 + \sin(\cos(x))}$ compare the difference between the forward derivative with $h=1e-8$ and that computed by `D` at $x=\pi/4$.
For $f(x) = \sqrt{1 + \sin(\cos(x))}$ compare the difference between the forward derivative with $h=1e-8$ and that computed by automatic differentiation at $x=\pi/4$.
The forward derivative is found with:
@@ -142,11 +147,11 @@ c, h = pi/4, 1e-8
fwd = (f(c+h) - f(c))/h
```
That given by `D` is:
That given by automatic differentiation is:
```{julia}
ds_value = D(f)(c)
ds_value = f'(c)
ds_value, fwd, ds_value - fwd
```
@@ -158,38 +163,25 @@ fp = diff(f(x), x)
```
```{julia}
actual = convert(Float64, fp(PI/4))
actual = float(fp(PI/4))
actual - ds_value, actual - fwd
```
#### Convenient notation
`Julia` allows the possibility of extending functions to different types. Out of the box, the `'` notation is not employed for functions, but is used for matrices. It is used in postfix position, as with `A'`. We can define it to do the same thing as `D` for functions and then, we can evaluate derivatives with the familiar `f'(x)`. This is done in `CalculusWithJulia` along the lines of `Base.adjoint(f::Function) = D(f)`.
Then, we have, for example:
```{julia}
#| hold: true
f(x) = sin(x)
f'(pi), f''(pi)
```
As expected, the automatic derivative is nearly exact and accurate up to possibly accumulated floating point differences; the forward difference is just pretty close.
##### Example
Suppose our task is to find a zero of the second derivative of $k(x) = e^{-x^2/2}$ in $[0, 10]$, a known bracket. The `D` function takes a second argument to indicate the order of the derivative (e.g., `D(f,2)`), but we use the more familiar notation:
Suppose our task is to find a zero of the second derivative of $k(x) = e^{-x^2/2}$ in $[0, 10]$, a known bracket. The second derivative is found by `k''` below, with the derivative operation being applied twice behind the scenes. With this, we have:
```{julia}
#| hold: true
k(x) = exp(-x^2/2)
find_zero(k'', 0..10)
find_zero(k'', (0, 10))
```
We pass in the function object, `k''`, and not the evaluated function.
As with plotting and other uses of functions as arguments, we pass in the function object, `k''`, and not the function evaluated at a point.
## Recap on derivatives in Julia
@@ -197,9 +189,11 @@ We pass in the function object, `k''`, and not the evaluated function.
A quick summary of the $3$ different ways for finding derivatives in `Julia` presented in these notes:
* Symbolic derivatives are found using `diff` from `SymPy`
* Automatic derivatives are found using the notation `f'` which utilizes `ForwardDiff.derivative`
* approximate derivatives at a point, `c`, for a given `h` are found with `(f(c+h)-f(c))/h`.
* Symbolic derivatives are found using `diff` from `SymPy`
* Automatic derivatives are found using the notation `f'` which utilizes `ForwardDiff.derivative`
* approximate derivatives at a point, `c`, for a given `h` are found with `(f(c+h)-f(c))/h`.
For example, here all three are computed and compared:
@@ -207,21 +201,17 @@ For example, here all three are computed and compared:
```{julia}
#| hold: true
f(x) = exp(-x)*sin(x)
f(x) = exp(-x) * sin(x)
c = pi
h = 1e-8
fp = diff(f(x),x)
fp, fp(c), f'(c), (f(c+h) - f(c))/h
Dict(:approximate=>(f(c+h) - f(c))/h, :automatic=>f'(c),
:symbolic=>fp, :symbolic_evaluated => fp(x=>c))
```
:::{.callout-note}
## Note
The use of `'` to find derivatives provided by `CalculusWithJulia` is convenient, and used extensively in these notes, but it needs to be noted that it does **not conform** with the generic meaning of `'` within `Julia`'s wider package ecosystem and may cause issue with linear algebra operations; the symbol is meant for the adjoint of a matrix.
:::
## Questions
@@ -241,7 +231,7 @@ val = (f(c+h) - f(c))/h
numericq(val)
```
Using `D` or `f'` find the value using automatic differentiation
Use `f'` find the value using automatic differentiation
```{julia}
@@ -253,44 +243,11 @@ val = f'(c)
numericq(val)
```
###### Question
Mathematically, as the value of `h` in the forward difference gets smaller the forward difference approximation gets better. On the computer, this is thwarted by floating point representation issues (in particular the error in subtracting two like-sized numbers in forming $f(x+h)-f(x)$.)
For `1e-16` what is the error (in absolute value) in finding the forward difference approximation for the derivative of $\sin(x)$ at $x=0$?
```{julia}
#| hold: true
#| echo: false
f(x) = sin(x)
h = 1e-16
c = 0
approx = (f(c+h)-f(c))/h
val = abs(cos(c) - approx)
numericq(val)
```
Repeat for $x=\pi/4$:
```{julia}
#| hold: true
#| echo: false
f(x) = sin(x)
h = 1e-16
c = pi/4
approx = (f(c+h)-f(c))/h
val = abs(cos(c) - approx)
numericq(val)
```
###### Question
Let $f(x) = x^x$. Using `D`, find $f'(3)$.
Let $f(x) = x^x$. Using automatic differentiation, find $f'(3)$.
```{julia}
@@ -304,7 +261,7 @@ numericq(val)
###### Question
Let $f(x) = \lvert 1 - \sqrt{1 + x}\rvert$. Using `D`, find $f'(3)$.
Let $f(x) = \lvert 1 - \sqrt{1 + x}\rvert$. Using automatic differentation, find $f'(3)$.
```{julia}
@@ -318,7 +275,7 @@ numericq(val)
###### Question
Let $f(x) = e^{\sin(x)}$. Using `D`, find $f'(3)$.
Let $f(x) = e^{\sin(x)}$. Using automatic differentation, find $f'(3)$.
```{julia}
@@ -371,3 +328,83 @@ fp_(h) = 3*32h^2 - 62
c = 2
numericq(fp_(2))
```
###### Question
Mathematically, as the value of `h` in the forward difference gets smaller the forward difference approximation gets better. On the computer, this is thwarted by floating point representation issues (in particular the error in subtracting two like-sized numbers in forming $f(x+h)-f(x)$.)
For `1e-16` what is the error (in absolute value) in finding the forward difference approximation for the derivative of $\sin(x)$ at $x=0$?
```{julia}
#| hold: true
#| echo: false
f(x) = sin(x)
h = 1e-16
c = 0
approx = (f(c+h)-f(c))/h
val = abs(cos(c) - approx)
numericq(val)
```
Repeat for $x=\pi/4$:
```{julia}
#| hold: true
#| echo: false
f(x) = sin(x)
h = 1e-16
c = pi/4
approx = (f(c+h)-f(c))/h
val = abs(cos(c) - approx)
numericq(val)
```
##### Question
Let $f(x) = e^{-x^2/2}$. At $c=2$ we can compare the exact answer for the derivative to the forward difference approximation for different values of `h`. For example:
```{julia}
f(x) = exp(-x^2/2)
c = 2
@syms x
exact = float(diff(f(x), x)(x=>2))
```
Whereas,
```{julia}
hs = [1/10^i for i in 0:16]
fdiffs = [(f(c+h) - f(c))/h for h in hs]
error = fdiffs .- exact
[hs error]
```
Which value of `h` provided the smallest error? Write the exponent as `i` in `1/10^i`.
```{julia}
#| echo: false
numericq(8)
```
We repeat with the central difference approximation.
```{julia}
hs = [1/10^i for i in 0:16]
cdiffs = [(f(c+h) - f(c-h))/(2h) for h in hs]
error = cdiffs .- exact
[hs error]
```
Which value of `h` provided the smallest error? Write the exponent as `i` in `1/10^i`.
```{julia}
#| echo: false
numericq(6)
```
quarto/derivatives/optimization.qmd
+213 -117
View File
@@ -30,8 +30,11 @@ For example,
The main tool is the extreme value theorem of Bolzano and Fermat's theorem about critical points, which combined say:
::: {.relationship title="Bolazano plus Fermat"}
> If the function $f(x)$ is continuous on $[a,b]$ and differentiable on $(a,b)$, then the extrema exist and must occur at either an end point or a critical point.
If the function $f(x)$ is continuous on $[a,b]$ then the extrema exist and must occur at either an end point or a critical point.
:::
@@ -52,14 +55,14 @@ With the computer we can take some shortcuts, as we will be able to graph our fu
The simplest way to investigate the maximum or minimum value of a function over a closed interval is to just graph it and look.
We began with the question of which rectangles of perimeter $20$ have the largest area? The figure shows a few different rectangles with this perimeter and their respective areas.
We began with the question of which rectangles of perimeter $20$ have the largest area? @fig-fixed-perimeter-different-shapes-give-different-areas shows a few different rectangles with this perimeter and their respective areas.
::: {#fig-fixed-perimeter-different-shapes-give-different-areas}
```{julia}
#| hold: true
#| echo: false
#| cache: true
### {{{perimeter_area_graphic}}}
#| fig-alt: Some possible rectangles that satisfy the constraint on the perimeter and their area.
gr()
function perimeter_area_graphic_graph(n)
@@ -74,11 +77,7 @@ function perimeter_area_graphic_graph(n)
plt
end
caption = """
Some possible rectangles that satisfy the constraint on the perimeter and their area.
"""
caption = ""
n = 5
anim = @animate for i=1:n
perimeter_area_graphic_graph(i-1)
@@ -90,6 +89,9 @@ plotly()
ImageFile(imgfile, caption)
```
Some possible rectangles that satisfy the constraint on the perimeter and their area.
:::
The basic mathematical approach is to find a function of a single variable to maximize or minimize. In this case we have two variables describing a rectangle: a base $b$ and height $h$. Our formulas are the area of a rectangle:
@@ -110,14 +112,17 @@ From this last one, we see that $b$ can be no bigger than $10$ and no smaller th
Maximize $A(b) = b \cdot (10 - b)$ over the interval $[0,10]$.
This is exactly the form needed to apply our theorem about the existence of extrema (a continuous function on a closed interval). Rather than solve analytically by taking a derivative, we simply graph to find the value:
This is exactly the form needed to apply our theorem about the existence of extrema (a continuous function on a closed interval). Rather than solve analytically by taking a derivative, in @fig-plot-Area-over-0-10 we simply graph to find the value.
::: {#fig-plot-Area-over-0-10}
```{julia}
Area(b) = b * (10 - b)
plot(Area, 0, 10)
```
Plot of `Area` over possible values for `b`; $[0,10]$
:::
You should see the maximum occurs at $b=5$ by symmetry, so $h=5$ as well, and the maximum area is then $25$. This gives the satisfying answer that among all rectangles of fixed perimeter, that with the largest area is a square. As well, this indicates a common result: there is often some underlying symmetry in the answer.
@@ -177,9 +182,10 @@ Here is a similar, though more complicated, example where the analytic approach
Let a "[Norman](https://en.wikipedia.org/wiki/Norman_architecture)" window consist of a rectangular window of top length $x$ and side length $y$ and a half circle on top. The goal is to maximize the area for a fixed value of the perimeter. Again, assume this perimeter is $20$ units.
This figure shows two such windows, one with base length given by $x=3$, the other with base length given by $x=4$. The one with base length $4$ seems to have much bigger area, what value of $x$ will lead to the largest area?
This figure shows two such windows, one with base length given by $x=3,$ the other with base length given by $x=4$. The one with base length $4$ seems to have much bigger area, what value of $x$ will lead to the largest area?
::: {#fig-norman-window-figure}
```{julia}
#| hold: true
#| echo: false
@@ -198,6 +204,9 @@ plot!(p, x2/2 .+ x2/2*cos.(ts), y2 .+ x2/2*sin.(ts), linetype=:polygon,
p
```
Figure of two possible Norman windows each with the same, fixed perimeter
:::
For this problem, we have two equations.
@@ -242,30 +251,37 @@ Of course both $x$ and $y$ are non-negative. The latter forces $x$ to be no more
This leaves us the calculus problem of finding an absolute maximum of a continuous function over the closed interval $[0, 20/(1+\pi/2)]$. Our theorem tells us this maximum must occur, we now proceed to find it.
We begin by simply graphing and estimating the values of the maximum and where it occurs.
We begin by simply graphing `A` in @fig-plot-area-over-0-20-over-1-plus-pi-over-2 and estimating the values of the maximum and where it occurs.
::: {#fig-plot-area-over-0-20-over-1-plus-pi-over-2}
```{julia}
#| echo: false
plot(A, 0, 20/(1+pi/2))
```
Plot of area function `A` over the range of possible values for `x`, $[0, 20/(1 + \pi/2)]$
:::
The naked eye sees that maximum value is somewhere around $27$ and occurs at $x\approx 5.6$. Clearly from the graph, we know the maximum value happens at the critical point and there is only one such critical point.
As reading the maximum from the graph is more difficult than reading a $0$ of a function, we plot the derivative using our approximate derivative.
As reading the maximum from the graph is more difficult than reading a $0$ of a function, In @fig-plot-of-derivative-of-A-over-5-point-5-to-5-point-7 we plot the derivative using our approximate derivative, confirming that the critical point is around $5.6$.
::: {#fig-plot-of-derivative-of-A-over-5-point-5-to-5-point-7}
```{julia}
#| echo: false
plot(A', 5.5, 5.7)
```
We confirm that the critical point is around $5.6$.
Plot of `A'` over $[5.5, 5.7]$
:::
#### Using `find_zero` to locate critical points.
Rather than zoom in graphically, we now use a root-finding algorithm, to find a more precise value of the zero of $A'$. We know that the maximum will occur at a critical point, a zero of the derivative. The `find_zero` function from the `Roots` package provides a non-linear root-finding algorithm based on the bisection method. The only thing to keep track of is that solving $f'(x) = 0$ means we use the derivative and not the original function.
Rather than zoom in graphically, we now use a root-finding algorithm, to find a more precise value of the zero of $A'$. $A$ is differentiable so the maximum will occur at a critical point that is a zero of the derivative. The only thing to keep track of is that solving $f'(x) = 0$ means we use the derivative and not the original function.
In a few sections we will see how to use `find_zero` to find a zero *near* an initial guess, but for now we will use `find_zero` to find a zero *between* two values which form a bracketing interval.
@@ -309,9 +325,9 @@ We could also do the above problem symbolically with the aid of `SymPy`. Here ar
A₀ = w₀ * h₀ + pi * (w₀/2)^2 / 2
Perim = 2*h₀ + w₀ + pi * w₀/2
h₁ = solve(Perim - 20, h₀)[1]
h₁ = first(solve(Perim - 20, h₀))
A₁ = A₀(h₀ => h₁)
w₁ = solve(diff(A₁,w₀) ~ 0, w₀)[1]
w₁ = first(solve(diff(A₁,w₀) ~ 0, w₀))
```
We know that `w₀` is the maximum in this example from our previous work. We shall see soon, that just knowing that the second derivative is negative at `w₀` would suffice to know this. Here we check that condition:
@@ -349,17 +365,38 @@ The figure shows a trapezoid inscribed in a circle. By adjusting the point ``P_3
nothing
```
::: {#fig-trapezoid-inscribed-in-half-circle}
```{julia}
#| hold: true
#| echo: false
function trapezoid(r)
plot(x -> sqrt(1 - x^2), -1, 1, legend=false, aspect_ratio=:equal)
plot!([-1,1,r,-r,-1], [0,0,sqrt(1-r^2), sqrt(1-r^2), 0], lw=3, color=:red)
let
gr()
function trapezoid(r)
plot(; empty_style..., aspect_ratio=:equal)
plot!([(-1.1, 0), (1.1, 0)]; arrow=true, side=:right, line=(1, :gray))
plot!([(0, -0.1), (0, 1.1)]; arrow=true, side=:right, line=(1, :gray))
plot!(x -> sqrt(1 - x^2), -1, 1; line=(2, :black))
L, R, X, Y = (-1,0), (1,0), (r, sqrt(1-r^2)), (-r, sqrt(1-r^2))
plot!([L, R, X, Y, L], line=(3, :red))
scatter!([X], marker=(3,))
annotate!([
(X..., text(L"(x,y)", :bottom, :left)),
(0, 0, text(L"(0,0)", :top, :left)),
(-1, 0, text(L"-r", :top)),
( 1, 0, text(L"r", :top))
])
end
plt = trapezoid(0.75)
plotly()
plt
end
trapezoid(0.75)
```
A trapezoid is *inscribed* in the upper-half circle of radius $r$. The trapezoid is found be connecting the points $(x,y)$ (in the first quadrant) with $(r, 0)$, $(-r,0)$, and $(-x, y)$. Find the maximum area. (The above figure has $x=0.75$ and $r=1$.)
A trapezoid inscribed in a upper-half circle with key point $(x,y)$ laying in quadrant $I$
:::
A trapezoid is *inscribed* in the upper-half circle of radius $r$. The trapezoid is found be connecting the points $(x,y)$ (in the first quadrant) with $(r, 0)$, $(-r,0)$, and $(-x, y)$. Find the maximum area. (@fig-trapezoid-inscribed-in-half-circle shows $x=0.75$ and $r=1$.)
Here the constraint is simply $r^2 = x^2 + y^2$ with $x$ and $y$ being non-negative. The area is then found through the average of the two lengths times the height. Using `height` for `y`, we have:
@@ -367,17 +404,18 @@ Here the constraint is simply $r^2 = x^2 + y^2$ with $x$ and $y$ being non-negat
```{julia}
@syms x::positive r::positive
hₜ = sqrt(r^2 - x^2)
aₜ = (2x + 2r)/2 * hₜ
possible_sols = solve(diff(aₜ, x) ~ 0, x) # possibly many solutions
height = sqrt(r^2 - x^2)
a = (2x + 2r)/2 * height
possible_sols = solve(diff(a, x) ~ 0, x) # possibly many solutions
x0 = first(possible_sols) # only solution is also found from first or [1] indexing
```
The other values of interest can be found through substitution. For example:
```{julia}
hₜ(x => x0)
height(x => x0)
```
## Trigonometry problems
@@ -389,9 +427,9 @@ Many maximization and minimization problems involve triangles, which in turn use
A ladder is to be moved through a two-dimensional hallway which has a bend and gets narrower after the bend. The hallway is $8$ feet wide then $5$ feet wide. What is the longest such ladder that can be navigated around the corner?
The figure shows a ladder of length $l_1 + l_2$ that got stuck - it was too long.
@fig-ladder-length-l1-l2 shows a ladder of length $l_1 + l_2$ that got stuck - it was too long.
::: {#fig-ladder-length-l1-l2}
```{julia}
#| hold: true
#| echo: false
@@ -401,7 +439,7 @@ let
xticks = [0,5, 15],
yticks = [0,8, 12],
line=(:blue, 2),
legend=false)
legend=false)
plot!(p, [5, 5, 15], [15, 8, 8]; line=(:blue,2))
plot!(p, [0,14.53402874075368], [12.1954981558864, 0], linewidth=3)
plot!(p, [0,5], [8,8], color=:orange)
@@ -420,6 +458,10 @@ plotly()
nothing
```
Figure illustrating top view of a ladder of some length being moved around a corner in two dimensions. The ladder in the figure would be too big, as it is stuck in this position.
:::
We approach this problem in reverse. It is easy to see when a ladder is too long. It gets stuck at some angle $\theta$. So for each $\theta$ we find that ladder length that is just too long. Then we find the minimum length of all these ladders that are too long. If a ladder is this length or more it will get stuck for some angle. However, if it is less than this length it will not get stuck. So to maximize a ladder length, we minimize a different function. Neat.
@@ -445,13 +487,16 @@ Our goal is to minimize this function for all angles between $0$ and $90$ degree
This is not a continuous function on a closed interval - it is undefined at the endpoints. That being said, a quick plot will convince us that the minimum occurs at a critical point and there is only one critical point in $(0, \pi/2)$.
::: {#fig-plot-l-over-0-pi-over-2}
```{julia}
delta = 0.2
plot(l, delta, pi/2 - delta)
#| echo: false
plot(rangeclamp(l, 40), 0, pi/2; legend=false)
```
The graph shows the minimum occurs between $0.50$ and $1.00$ radians, a bracket for the derivative. Here we find $x$ and the minimum value:
Plot of `l`, the maximum length ladder stuck at angle $\theta$, over $(0, \pi/2)$
:::
@fig-plot-l-over-0-pi-over-2 shows the minimum occurs between $0.50$ and $1.00$ radians, a bracket for the derivative. Here we find $x$ and the minimum value:
```{julia}
@@ -470,7 +515,7 @@ Ethan Hunt, a top secret spy, has a mission to chase a bad guy. Here is what we
* Ethan likes to run. He can run at $10$ miles per hour.
* He can drive a car - usually some concept car by BMW - at $30$ miles per hour, but only on the road.
* He can drive a car---usually some concept car by BMW---at $30$ miles per hour, but only on the road.
For his mission, he needs to go $10$ miles west and $5$ miles north. He can do this by:
@@ -497,30 +542,28 @@ $$
With the endpoints given by $T(0) = \sqrt{10^2 + 5^2}/10$ and $T(10) = (10 + 5)/30$.
Let's plot $T(x)$ over the interval $(0,10)$ and look:
@fig-ethan-hunts-time-over-0-10 shows a plot of $T(x)$ over the interval $(0,10)$.
::: {#fig-ethan-hunts-time-over-0-10}
```{julia}
#| echo: false
T(x) = x/30 + sqrt(5^2 + (10-x)^2)/10
```
```{julia}
plot(T, 0, 10)
```
The minimum happens way out near 8. We zoom in a bit:
Plot of $T$ over $[0,10]$.
:::
The lone minimum happens way out near 8.
We now use `find_zero` to refine our guess at the critical point using $[0, 10]$, as there is only one obvious solution:
```{julia}
plot(T, 7, 9)
```
It appears to be around $8.3$. We now use `find_zero` to refine our guess at the critical point using $[7,9]$:
```{julia}
α = find_zero(T', (7, 9))
α = find_zero(T', (0, 10))
```
Okay, got it. Around $8.23$. So is our minimum time.
@@ -540,9 +583,9 @@ sqrt(10^2 + 5^2)/10, T(α), (10+5)/30
Ahh, we see that $T(x)$ is not continuous on $[0, 10]$, as it jumps at $x=10$ down to an even smaller amount of $1/2$. It may not look as impressive as a miles-long sprint, but Mr. Hunt is advised by Benji to drive the whole way.
### Rate times time ... the origin story
### Rate times time---the origin story
::: {#fig-lhospital-image-43}
```{julia}
#| hold: true
#| echo: false
@@ -558,7 +601,7 @@ region on the side of $C$, they cover distance $a$ in time $c$, and
that on the other, on the side of $F$, distance $b$ in the same time
$c$. We ask through which point $E$ on the line $AEB$ they should pass,
so as to take the least possible time to get from $C$ to $F$? (From
http://www.ams.org/samplings/feature-column/fc-2016-05.)
[AMS.org](http://www.ams.org/samplings/feature-column/fc-2016-05).)
"""
@@ -566,15 +609,17 @@ http://www.ams.org/samplings/feature-column/fc-2016-05.)
nothing
```
![Image number $43$ from l'Hospital's calculus book (the first). A
![](./figures/fcarc-may2016-fig43-250.png){fig-alt="Image number 43 from l'Hospital's calculus book"}
Image number $43$ from l'Hospital's calculus book (the first). A
traveler leaving location $C$ to go to location $F$ must cross two
regions separated by the straight line $AEB$. We suppose that in the
region on the side of $C$, he covers distance $a$ in time $c$, and
that on the other, on the side of $F$, distance $b$ in the same time
$c$. We ask through which point $E$ on the line $AEB$ he should pass,
so as to take the least possible time to get from $C$ to $F$? (From
http://www.ams.org/samplings/feature-column/fc-2016-05.)](./figures/fcarc-may2016-fig43-250.png)
[AMS.org](http://www.ams.org/samplings/feature-column/fc-2016-05).)
:::
The last example is a modern day illustration of a problem of calculus dating back to l'Hospital. His parameterization is a bit different. Let's change his by taking two points $(0, a)$ and $(L,-b)$, with $a,b,L$ positive values. Above the $x$ axis travel happens at rate $r_0$, and below, travel happens at rate $r_1$, again, both positive. What value $x$ in $[0,L]$ will minimize the total travel time?
@@ -598,7 +643,7 @@ The answer will occur at a critical point or an endpoint, either $x=0$ or $x=L$.
The structure of `dt` is too complicated for simply calling `solve` to find the critical points. Instead we help `SymPy` out a bit. We are solving an equation of the form $a/b + c/d = 0$. These solutions will also be solutions of $(a/b)^2 - (c/d)^2=0$ or even $a^2d^2 - c^2b^2 = 0$. This follows as solutions to $u+v=0$, also solve $(u+v)\cdot(u-v)=0$, or $u^2 - v^2=0$. Setting $u=a/b$ and $v=c/d$ completes the comparison.
We can get these terms - $a$, $b$, $c$, and $d$ - as follows:
We can get these terms---$a$, $b$, $c$, and $d$---as follows:
```{julia}
@@ -620,7 +665,7 @@ p = sympy.Poly(ex, x) # a0 + a1⋅x + a2⋅x^2 + a3⋅x^3 + a4⋅x^4
p.coeffs()
```
Fourth degree polynomials can be solved. The critical points of the original equation will be among the $4$ solutions given. However, the result is complicated. The [article](http://www.ams.org/samplings/feature-column/fc-2016-05) – from which the figure came – states that "In today's textbooks the problem, usually involving a river, involves walking along one bank and then swimming across; this corresponds to setting $g=0$ in l'Hospital's example, and leads to a quadratic equation." Let's see that case, which we can get in our notation by taking $b=0$:
Fourth degree polynomials can be solved. The critical points of the original equation will be among the $4$ solutions given. However, the result is complicated. The [article](http://www.ams.org/samplings/feature-column/fc-2016-05)---from which the figure came---states that "In today's textbooks the problem, usually involving a river, involves walking along one bank and then swimming across; this corresponds to setting $g=0$ in l'Hospital's example, and leads to a quadratic equation." Let's see that case, which we can get in our notation by taking $b=0$:
```{julia}
@@ -707,22 +752,30 @@ Here the extreme value theorem doesn't technically apply, as we don't have a clo
In general, for an optimization problem of a continuous function on the interval $(a,b)$ if the right limit at $a$ and left limit at $b$ can be ruled out as candidates, the optimal value must occur at a critical point.)
So to approach this problem we first graph it over a wide interval.
So to approach this problem we first graph it over a wide interval in @fig-plot-of-x-exp-minus-x-squared-over-wide-interval.
::: {#fig-plot-of-x-exp-minus-x-squared-over-wide-interval}
```{julia}
#| echo: false
f(x) = x * exp(-x^2)
plot(f, 0, 100)
```
Plot of $f(x) = x \cdot e^{-x^2}$ over $[0, 100]$, a wide interval
:::
Clearly the action is nearer to $1$ than $100$. We try graphing the derivative near that area:
::: {#fig-plot-of-x-exp-minus-x-squared-over-interval-of-interest}
```{julia}
#| echo: false
plot(f', 0, 5)
```
Plot of the *derivative$ of $f(x) = x \cdot e^{-x^2}$ over $[0, 5]$, an interval where the values of interest lie
:::
This shows the value of interest near $0.7$ for a critical point. We use `find_zero` with $[0,1]$ as a bracket
@fig-plot-of-x-exp-minus-x-squared-over-interval-of-interest shows the value of interest near $0.7$ for a critical point. We use `find_zero` with $[0,1]$ as a bracket
```{julia}
@@ -780,13 +833,17 @@ SA(r) = SA(canheight(r), r)
This is minimized subject to the constraint that $r \geq 0$. A quick glance shows that as $r$ gets close to $0$, the can must get infinitely tall to contain that fixed volume, and would have infinite surface area as the $1/r^2$ in the first term implies. On the other hand, as $r$ goes to infinity, the height must go to $0$ to make a really flat can. Again, we would have infinite surface area, as the $r^2$ term at the end indicates. With this observation, we can rule out the endpoints as possible minima, so any minima must occur at a critical point.
We start by making a graph, making an educated guess that the answer is somewhere near a real life answer, or around $3$-$5$ cms in radius:
We start by making the graph in @fig-plot-of-surface-area-of-can-over-2-10 using an educated guess that the answer is somewhere near a real life answer, or around $3$-$5$ cms in radius.
::: {#fig-plot-of-surface-area-of-can-over-2-10}
```{julia}
#| echo: false
plot(SA, 2, 10)
```
Plot of surface area of a can over $[2, 10]$
:::
The minimum looks to be around $4$cm and is clearly between $2$cm and $6$cm. We can use `find_zero` to zero in on the value of the critical point:
@@ -848,19 +905,27 @@ numericq(val)
A rancher with $10$ meters of fence wishes to make a pen adjacent to an existing fence. The pen will be a rectangle with one edge using the existing fence. Say that has length $x$, then $10 = 2y + x$, with $y$ the other dimension of the pen. What is the maximum area that can be made?
::: {#fig-plot-of-stupid-rancher-figure-for-fencing}
```{julia}
#| hold: true
#| echo: false
p = plot(; legend=false, aspect_ratio=:equal, axis=nothing, border=:none)
let
gr()
p = plot(; legend=false, aspect_ratio=:equal, axis=nothing, border=:none)
plot!([0,10, 10, 0, 0], [0,0,10,10,0]; linewidth=3)
plot!(p, [10,14,14,10], [2, 2, 8,8]; linewidth = 1)
plot!([0,10, 10, 0, 0], [0,0,10,10,0]; linewidth=3)
plot!(p, [10,14,14,10], [2, 2, 8,8]; linewidth = 1)
annotate!(p, [(14-0.1, 5, text("x", :right)), (12,2, text("y",:bottom))])
p
annotate!(p, [(14-0.1, 5, text(L"x", :right)), (12,2, text(L"y",:bottom))])
plotly()
p
end
```
Plot of rancher's adjacent pen
:::
```{julia}
#| hold: true
#| echo: false
@@ -917,8 +982,8 @@ choices = [
"It is also 20",
"``17.888``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1054,22 +1119,31 @@ val = y(x0)
numericq(val) # 0
```
###### Question (Thanks https://www.math.ucdavis.edu/~kouba)
###### Question
A movie screen projects on a wall 20 feet high beginning 10 feet above the floor. This figure shows $\theta$ for $x=30$:
A movie screen projects on a wall 20 feet high beginning 10 feet above the floor. @fig-plot-movie-screen-projector shows $\theta$ for $x=30$:
::: {#fig-plot-movie-screen-projector}
```{julia}
#| hold: true
#| echo: false
p = plot([0, 30,30], [0,0,10], xlim=(0, 32), color=:blue, legend=false)
plot!(p, [30, 30], [10, 30], color=:blue, linewidth=4)
plot!(p, [0, 30,30,0], [0,10,30,0], color=:orange)
annotate!(p, [(x,y,l) for (x,y,l) in zip([15, 5, 31, 31], [1.5, 3.5, 5, 20], ["x=30", "θ", "10", "20"])])
let
gr()
p = plot([0, 30,30], [0,0,10], xlim=(0, 32), color=:blue, legend=false)
plot!(p, [30, 30], [10, 30], color=:blue, linewidth=4)
plot!(p, [0, 30,30,0], [0,10,30,0], color=:orange)
annotate!(p, [(x,y,l) for (x,y,l) in zip([15, 5, 31, 31], [1.5, 3.5, 5, 20], [L"x=30", L"\theta", L"10", L"20"])])
plotly()
p
end
```
What value of the largest angle $\theta$ that $x$ gives? (In degrees.)
Plot of a $20$-foot movie screen from $30$ feet away and the angle subtended
:::
What value of the largest angle $\theta$ in degrees that $x$ gives? (Thanks https://www.math.ucdavis.edu/~kouba).)
```{julia}
@@ -1107,8 +1181,8 @@ Now if $Likhood(t) = \exp(-3t) \cdot \exp(-2t) \cdot \exp(-4t), \quad 0 \leq t \
choices=["It does work and the answer is x = 2.27...",
L" $Likhood(t)$ is not continuous on $0$ to $10$",
L" $Likhood(t)$ takes its maximum at a boundary point - not a critical point"];
answ = 3;
radioq(choices, answ)
answer = 3;
radioq(choices, answer)
```
##### Question
@@ -1139,8 +1213,8 @@ choices=[
"The median, or middle number, of the values",
L"The square roots of the values squared, $(x_1^2 + \cdots + x_n^2)^2$"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1231,23 +1305,31 @@ numericq(hstar)
###### Question
Find the value $x > 0$ which minimizes the distance from the graph of $f(x) = \log_e(x) - x$ to the origin $(0,0)$.
Find the value $x > 0$ which minimizes the distance from the graph of $f(x) = \ln(x) - x$ to the origin $(0,0)$.
::: {#fig-minimize-distance-from-logx-minus-x}
```{julia}
#| hold: true
#| echo: false
f(x) = log(x) - x
p = plot(f, 0.2, 2, ylim=(-2,0.25), legend=false, linewidth=3)
plot!(p, [0,0], [-2, 0.25], color=:blue)
plot!(p, [0,2],[0,0], color=:blue)
xs = [0,1]; ys = [0, f(1)]
scatter!(p, xs,ys, color=:orange)
plot!(p, xs, ys, color=:orange, linewidth=3)
annotate!(p, [(.75, f(.5)/2, "d = $(round(sqrt(.5^2 + f(.5)^2), digits=2))")])
p
let
gr()
f(x) = log(x) - x
p = plot(f, 0.2, 2, ylim=(-2,0.25), legend=false, linewidth=3)
plot!(p, [0,0], [-2, 0.25], color=:blue)
plot!(p, [0,2],[0,0], color=:blue)
xs = [0,1/2]; ys = [0, f(1/2)]
O, P = (0,0), (1/2, f(1/2))
scatter!(p, [O, P], color=:orange)
plot!(p, [O, P], color=:orange, linewidth=3)
annotate!(p, [(.28, f(0.5)/2, text(L"d = 1.29\cdots", :left))])
plotly()
p
end
```
Plot of $f(x) = \ln(x) - x$ and the distance, $d$, of the point $(1/2, f(1/2)$ to the origin $(0,0)$
:::
```{julia}
#| hold: true
#| echo: false
@@ -1274,10 +1356,14 @@ Image number $40$ from l'Hospital's calculus book (the first calculus book). Amo
nothing
```
![Image number $40$ from l'Hospital's calculus book (the first calculus book). Among all the cones that can be inscribed in a sphere, determine which one has the largest lateral area. (From [AMS](http://www.ams.org/samplings/feature-column/fc-2016-05)).](./figures/fcarc-may2016-fig40-300.png)
::: {#fig-lhospital-image-40-optimization}
![](./figures/fcarc-may2016-fig40-300.png){fig-alt="L'Hospital image 40"}
Image number $40$ from l'Hospital's calculus book (the first calculus book). Among all the cones that can be inscribed in a sphere, determine which one has the largest lateral area. (From [AMS](http://www.ams.org/samplings/feature-column/fc-2016-05)).
:::
The figure above poses a problem about cones in spheres, which can be reduced to a two-dimensional problem. Take a sphere of radius $r=1$, and imagine a secant line of length $l$ connecting $(-r, 0)$ to another point $(x,y)$ with $y>0$. Rotating that line around the $x$ axis produces a cone and its lateral surface is given by $SA=\pi \cdot y \cdot l$. Write $SA$ as a function of $x$ and solve.
@fig-lhospital-image-40-optimization poses a problem about cones in spheres, which can be reduced to a two-dimensional problem. Take a sphere of radius $r=1$, and imagine a secant line of length $l$ connecting $(-r, 0)$ to another point $(x,y)$ with $y>0$. Rotating that line around the $x$ axis produces a cone and its lateral surface is given by $SA=\pi \cdot y \cdot l$. Write $SA$ as a function of $x$ and solve.
The largest lateral surface area is:
@@ -1306,8 +1392,8 @@ choices = ["exactly four times",
L"exactly $\pi$ times",
L"about $2.6$ times as big",
"about the same"]
answ = 3
radioq(choices, answ)
answer = 3
radioq(choices, answer)
```
###### Question
@@ -1354,8 +1440,8 @@ choices = [
"``e/a``",
"``a \\cdot e``"
]
answ=2
radioq(choices, answ)
answer=2
radioq(choices, answer)
```
###### Question
@@ -1363,26 +1449,36 @@ radioq(choices, answ)
The ladder problem has a trigonometry-free solution. We show one attributed to [Asma](http://www.mathematische-basteleien.de/ladder.htm).
::: {#fig-ladder-problem-look-ma-no-trigonometry}
```{julia}
#| hold: true
#| echo: false
plt = plot(; axis=nothing, border=:none, legend=false, aspect_ratio=:equal)
a,b = 1, 2
p = 1/2
x = a/p
plot!(plt, [0, b*(1+p), 0, 0], [0, 0, a+x, 0])
plot!(plt, [b,b,0,0],[0,a,a,0])
annotate!(plt, [
(b/2,0, text("b",:top)),
(0,a/2, text("a",:right)),
(0,a+x/2, text("x",:right)),
(b+b*p/2,0, text("bp",:top))
])
plt
let
gr()
plt = plot(; axis=nothing, border=:none, legend=false, aspect_ratio=:equal)
a,b = 1, 2
p = 1/2
x = a/p
O, P, Q = (0,0), (b*(1+p), 0), (0, a+x)
plot!(plt, [Q,O,P]; line=(1, :blue))
plot!(plt, [Q,P]; line=(3, :green))
plot!(plt, [b,b,0,0],[0,a,a,0])
annotate!(plt, [
(b/2,0, text(L"b",:top)),
(0,a/2, text(L"a",:right)),
(0,a+x/2, text(L"x",:right)),
(b+b*p/2,0, text(L"b\cdot p",:top)),
((b + b*p)/2, (a+x)/2, text(L"c", :bottom, :left))
])
plotly()
plt
end
```
Introducing a variable $p$, we get, following the above figure, the ladder of length $c$ touching the wall at $b+bp$ and $a + x$.
Figure to solve a ladder problem with Pythagorean's theorem
:::
Introducing a variable $p$, we get, following @fig-ladder-problem-look-ma-no-trigonometry, the ladder of length $c$ touching the wall at $b+bp$ and $a + x$.
Using similar triangles, we have:
quarto/derivatives/related_rates.qmd
+81 -63
View File
@@ -18,7 +18,7 @@ using SymPy
---
Related rates problems involve two (or more) unknown quantities that are related through an equation. As the two variables depend on each other, also so do their rates - change with respect to some variable which is often time. Exactly how remains to be discovered. Hence the name "related rates."
Related rates problems involve two (or more) unknown quantities that are related through an equation. As the two variables depend on each other, also so do their rates---change with respect to some variable which is often time. Exactly how remains to be discovered. Hence the name "related rates".
#### Examples
@@ -30,50 +30,51 @@ The following is a typical "book" problem:
> A *vintage* screen saver displays the outline of a $3$ cm by $2$ cm rectangle and then expands the rectangle in such a way that the $2$ cm side is expanding at the rate of $4$ cm/sec and the proportions of the rectangle never change. How fast is the area of the rectangle increasing when its dimensions are $12$ cm by $8$ cm? [Source.](http://oregonstate.edu/instruct/mth251/cq/Stage9/Practice/ratesProblems.html)
::: {#fig-growing-rects-graph}
```{julia}
#| hold: true
#| echo: false
#| cache: true
### {{{growing_rects}}}
## Secant line approaches tangent line...
gr()
function growing_rects_graph(n)
w = (t) -> 2 + 4t
h = (t) -> 3/2 * w(t)
t = n - 1
let
gr()
function growing_rects_graph(n)
w = (t) -> 2 + 4t
h = (t) -> 3/2 * w(t)
t = n - 1
w_2 = w(t)/2
h_2 = h(t)/2
w_2 = w(t)/2
h_2 = h(t)/2
w_n = w(5)/2
h_n = h(5)/2
w_n = w(5)/2
h_n = h(5)/2
plt = plot(w_2 * [-1, -1, 1, 1, -1], h_2 * [-1, 1, 1, -1, -1], xlim=(-17,17), ylim=(-17,17),
legend=false, size=fig_size)
annotate!(plt, [(-1.5, 1, "Area = $(round(Int, 4*w_2*h_2))")])
plt
plt = plot(w_2 * [-1, -1, 1, 1, -1], h_2 * [-1, 1, 1, -1, -1], xlim=(-17,17), ylim=(-17,17),
legend=false, size=fig_size)
area = round(Int, 4*w_2*h_2)
annotate!(plt, [(-1.5, 1, text(latexstring(L"Area = \$$area\$", :center)))])
plt
end
caption = ""
n=6
anim = @animate for i=1:n
growing_rects_graph(i)
end
imgfile = tempname() * ".gif"
gif(anim, imgfile, fps = 1)
plotly()
ImageFile(imgfile, caption)
end
caption = L"""
As $t$ increases, the size of the rectangle grows. The ratio of width to height is fixed. If we know the rate of change in time for the width ($dw/dt$) and the height ($dh/dt$) can we tell the rate of change of *area* with respect to time ($dA/dt$)?
"""
n=6
anim = @animate for i=1:n
growing_rects_graph(i)
end
imgfile = tempname() * ".gif"
gif(anim, imgfile, fps = 1)
plotly()
ImageFile(imgfile, caption)
```
Here we know $A = w \cdot h$ and we know some things about how $w$ and $h$ are related *and* about the rate of how both $w$ and $h$ grow in time $t$. That means that we could express this growth in terms of some functions $w(t)$ and $h(t)$, then we can figure out that the area - as a function of $t$ - will be expressed as:
As $t$ increases, the size of the rectangle grows. The ratio of width to height is fixed. If we know the rate of change in time for the width ($dw/dt$) and the height ($dh/dt$) can we tell the rate of change of *area* with respect to time ($dA/dt$)?
:::
Here we know $A = w \cdot h$ and we know some things about how $w$ and $h$ are related *and* about the rate of how both $w$ and $h$ grow in time $t$. That means that we could express this growth in terms of some functions $w(t)$ and $h(t)$, then we can figure out that the area---as a function of $t$---will be expressed as:
$$
@@ -94,7 +95,7 @@ $$
\frac{dA}{dt} = \frac{dw}{dt} h + w \frac{dh}{dt}.
$$
This relationship is true for all $t$, but the problem discusses a certain value of $t$ - when $w(t)=8$ and $h(t) = 12$. At this same value of $t$, we have $w'(t) = 4$ and so $h'(t) = 6$. Substituting these 4 values into the 4 unknowns in the formula for $A'(t)$ gives:
This relationship is true for all $t$, but the problem discusses a certain value of $t$---when $w(t)=8$ and $h(t) = 12$. At this same value of $t$, we have $w'(t) = 4$ and so $h'(t) = 6$. Substituting these 4 values into the 4 unknowns in the formula for $A'(t)$ gives:
$$
@@ -198,6 +199,7 @@ A ladder, with length $l$, is leaning against a wall. We parameterize this probl
If the ladder starts to slip away at the base, but remains in contact with the wall, express the rate of change of $h$ with respect to $t$ in terms of $db/dt$.
::: {#fig-ladder-along-wall-slipping-away}
```{julia}
#| echo: false
let
@@ -231,16 +233,13 @@ let
(b/2, h/2, text(L"L", rotation = -atand(h,b), :bottom))
])
plotly()
current()
end
```
```{julia}
#| echo: false
plotly()
nothing
```
A ladder placed on a wall is slipping away from its base
:::
We have from implicitly differentiating in $t$ the equation $l^2 = h^2 + b^2$, noting that $l$ is a constant, that:
@@ -298,9 +297,13 @@ imgfile = "figures/long-shadow-noir.png"
ImageFile(:derivatives, imgfile, caption)
```
![A man and woman walk towards the light](./figures/long-shadow-noir.png)
::: {#fig-film-noir}
![](./figures/long-shadow-noir.png){fig-alt="A man and woman walk towards the light"}
Shadows are a staple of film noir. In the photo, suppose a man and a woman walk towards a street light. As they approach the light the length of their shadow changes.
A man and woman walk towards the light
:::
Shadows are a staple of film noir. In @fig-film-noir, suppose a man and a woman walk towards a street light. As they approach the light the length of their shadow changes.
Suppose, we focus on the $5$ foot tall woman. Her shadow comes from a streetlight $12$ feet high. She is walking at $3$ feet per second towards the light. What is the rate of change of her shadow?
@@ -340,6 +343,7 @@ Solving for $l'$ gives an answer in terms of $x'$ the rate the woman is walking.
##### Example
::: {#fig-sun-setting-casts-a-shadow}
```{julia}
#| hold: true
#| echo: false
@@ -351,6 +355,10 @@ plot!(p, [25,25],[25,0], linewidth=5, color=:black)
plot!(p, [25,50], [0,0], linewidth=2, color=:black)
```
As the sun sets shadows change length
:::
The sun is setting at the rate of $1/20$ radian/min, and appears to be dropping perpendicular to the horizon, as depicted in the figure. How fast is the shadow of a $25$ meter wall lengthening at the moment when the shadow is $25$ meters long?
@@ -361,7 +369,7 @@ $$
\tan(\theta) = \frac{25}{x}
$$
of $x\tan(\theta) = 25$.
or $x\tan(\theta) = 25$.
As $t$ evolves, we know $d\theta/dt$ but what is $dx/dt$? Using implicit differentiation yields:
@@ -442,6 +450,7 @@ $$
##### Example
::: {#fig-baseball-been-berry-good-graph}
```{julia}
#| hold: true
@@ -469,15 +478,13 @@ function baseball_been_berry_good_graph(n)
plt = plot(xs, ys, legend=false, size=fig_size, xlim=(0,150), ylim=(0,15))
plot!(plt, [x(t), 100], [y(t), 0.0], color=:orange)
annotate!(plt, [(55, 4,"θ = $(round(Int, degrees)) degrees"),
(x(t), y(t), "($(round(Int, x(t))), $(round(Int, y(t))))")])
degs = round(Int, degrees)
xts, yts = round(Int, x(t)), round(Int, y(t))
annotate!(plt, [(55, 4, text(latexstring("\$\\theta = $degs\$ degrees"))),
(x(t), y(t), text(latexstring("($xts, $yts)")))])
end
caption = L"""
The flight of the ball as being tracked by a stationary outfielder. This ball will go over the head of the player. What can the player tell from the quantity $d\theta/dt$?
"""
caption = ""
n = 8
@@ -492,6 +499,9 @@ plotly()
ImageFile(imgfile, caption)
```
The flight of the ball as being tracked by a stationary outfielder. This ball will go over the head of the player. What can the player tell from the quantity $d\theta/dt$?
:::
A baseball player stands $100$ meters from home base. A batter hits the ball directly at the player so that the distance from home plate is $x(t)$ and the height is $y(t)$.
@@ -559,16 +569,20 @@ dtheta₁ = dtheta(cos(theta(t))^2 => (100 -x(t))^2/(y(t)^2 + (100-x(t))^2))
Plotting reveals some interesting things. For $v_{0y} < 10$ we have graphs that look like:
::: {#fig-dtheta-small-v}
```{julia}
plot(dtheta₁, 0, v0/5)
```
Plot of `dtheta` for smaller velocity
:::
The ball will drop in front of the player, and the change in $d\theta/dt$ is monotonic.
But let's rerun the code with $v_{0y} > 10$:
::: {#fig-dtheta-larger-v}
```{julia}
#| hold: true
v0 = 15
@@ -581,6 +595,10 @@ dtheta₁ = subs(dtheta, cos(theta(t))^2, (100 - x(t))^2/(y(t)^2 + (100 - x(t))^
plot(dtheta₁, 0, v0/5)
```
Plot of `dtheta` for larger velocity
:::
In the second case we have a different shape. The graph is not monotonic, and before the peak there is an inflection point. Without thinking too hard, we can see that the greatest change in the angle is when it is just above the head ($t=2$ has $x(t)=100$).
@@ -610,7 +628,7 @@ $$
\frac{dV}{dh} = \frac{\pi}{3} ( R(h)^2 + h \cdot 2 R \frac{dR}{dh}).
$$
We see that it depends on $R$ and the change in $R$ with respect to $h$. However, we visualize $h$ - the height - so it is better to re-express. Clearly, $dR/dh = \tan\theta$ and using $R(h) = h \tan(\theta)$ we get:
We see that it depends on $R$ and the change in $R$ with respect to $h$. However, we visualize $h$---the height---so it is better to re-express. Clearly, $dR/dh = \tan\theta$ and using $R(h) = h \tan(\theta)$ we get:
$$
@@ -626,7 +644,7 @@ How do the quantities vary in time?
For an incompressible fluid, by balancing the volume leaving with how it leaves we will have $dh/dt$ is the ratio of the cross-sectional area at bottom over that at the height of the fluid $(\pi \cdot (h_0\tan(\theta))^2) / (\pi \cdot ((h\tan\theta))^2)$ times the outward velocity of the fluid.
That is $dh/dt = (h_0/h)^2 \cdot v$. Which makes sense - larger openings ($h_0$) mean more fluid lost per unit time so the height change follows, higher levels ($h$) means the change in height is slower, as the cross-sections have more volume.
That is $dh/dt = (h_0/h)^2 \cdot v$. Which makes sense---larger openings ($h_0$) mean more fluid lost per unit time so the height change follows, higher levels ($h$) means the change in height is slower, as the cross-sections have more volume.
By [Torricelli's](http://en.wikipedia.org/wiki/Torricelli's_law) law, the out velocity follows the law $v = \sqrt{2g(h-h_0)}$. This gives:
@@ -672,8 +690,8 @@ choices = [
"The rate of change of price will increase",
"The rate of change of price will be positive and will depend on the rate of change of excess demand."
]
answ = 3
radioq(choices, answ, keep_order=true)
answer = 3
radioq(choices, answer, keep_order=true)
```
(Theoretically, when demand exceeds supply, prices increase.)
@@ -692,8 +710,8 @@ choices = [
"If the rate of change of unemployment is negative, the rate of change of wages will be negative.",
"If the rate of change of unemployment is negative, the rate of change of wages will be positive."
]
answ = 2
radioq(choices, answ, keep_order=true)
answer = 2
radioq(choices, answer, keep_order=true)
```
(Colloquially, "the rate of change of unemployment is negative" means the unemployment rate is going down, so there are fewer workers available to fill new jobs.)
@@ -713,8 +731,8 @@ L"The rate of change of pressure is always increasing by $c$",
"If volume is constant, the rate of change of pressure is proportional to the temperature",
"If volume is constant, the rate of change of pressure is proportional to the rate of change of temperature",
"If pressure is held constant, the rate of change of pressure is proportional to the rate of change of temperature"]
answ = 3
radioq(choices, answ, keep_order=true)
answer = 3
radioq(choices, answer, keep_order=true)
```
###### Question
@@ -800,8 +818,8 @@ choices = [
"``f(x) = x``",
"``f(x) = x^2``"
]
answ = 4
radioq(choices, answ, keep_order=true)
answer = 4
radioq(choices, answer, keep_order=true)
```
###### Question
@@ -833,8 +851,8 @@ choices = [
"``y = 1 - \\log(x)``",
"``y = x(2x - 1/x)``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
If $dx/dt = -1$, what is $dy/dt$?
@@ -849,6 +867,6 @@ choices = [
"``dy/dt = -2x - 1/x``",
"``dy/dt = 1``"
]
answ=1
radioq(choices, answ)
answer=1
radioq(choices, answer)
```
quarto/derivatives/symbolic_derivatives.qmd
+11 -8
View File
@@ -15,9 +15,11 @@ using TermInterface
The ability to breakdown an expression into operations and their arguments is necessary when trying to apply the differentiation rules. Such rules are applied from the outside in. Identifying the proper "outside" function is usually most of the battle when finding derivatives.
This section takes a detour into this point. **This section can be skipped; the material is not used in the sequel**.
In the following example, we provide a sketch of a framework to differentiate expressions by a chosen symbol to illustrate how the outer function drives the task of differentiation.
---
In the following example, we provide a sketch of a framework to differentiate expressions by a chosen symbol to illustrate how the outer function drives the task of differentiation; how the steps of differentiation can be automated; and how `Julia`'s multiple dispatch can be leveraged to handle the many different cases to be considered.
The `Symbolics` package provides native symbolic manipulation abilities for `Julia`, similar to `SymPy`, though without the dependence on `Python`. The `TermInterface` package, used by `Symbolics`, provides a generic interface for expression manipulation for this package that *also* is implemented for `Julia`'s expressions and symbols.
@@ -66,9 +68,9 @@ function D(ex, var=:x)
end
```
(The use of `Val` is an idiom of `Julia` allowing dispatch on certain values such as function names and numbers.)
The use of `Val` is an idiom of `Julia` allowing dispatch on certain values such as function names and numbers.
Now to develop methods for `D` for different "outside" functions and arities.
Now to develop methods for `D` for different "outside" functions and arities. We being by implementing sum, product, and quotient rules.
Addition can be unary (`:(+x)` is a valid quoting, even if it might simplify to the symbol `:x` when evaluated), *binary*, or *nary*. Here we implement the *sum rule*:
@@ -88,10 +90,10 @@ function D(::Val{:+}, ::Any, args, var)
end
```
The `args` are always held in a container, so the unary method must pull out the first one. The binary case should read as: apply `D` to each of the two arguments, and then create a quoted expression containing the sum of the results. The dollar signs interpolate into the quoting. (The "primes" are unicode notation achieved through `\prime[tab]` and not operations.) The *nary* method (which catches *any* arity besides `1` and `2`) does something similar, only using splatting to produce the sum.
The `args` are always held in a container, so the unary method must pull out the first one. The binary case should read as: apply `D` to each of the two arguments, and then create a new expression containing the sum of the results. The dollar signs interpolate into the quoting. (The "primes" are unicode notation achieved through `\prime[tab]` and not operations.) The *nary* method (which catches *any* arity besides `1` and `2`) does something similar, only using splatting to produce the sum.
Subtraction must also be implemented in a similar manner, but not for the *nary* case, as subtraction is not associative:
Subtraction is implemented in a similar manner, but there is no *nary* case; subtraction is not associative:
```{julia}
@@ -99,6 +101,7 @@ function D(::Val{:-}, ::Val{1}, args, var)
a′ = D(first(args), var)
:(-$a′)
end
function D(::Val{:-}, ::Val{2}, args, var)
a′, b′ = D.(args, var)
:($a′ - $b′)
@@ -126,17 +129,17 @@ function D(op::Val{:*}, ::Any, args, var)
end
```
The *nary* case above just peels off the first factor and then uses the binary product rule.
The *nary* case above just peels off the first factor and then recursively uses the product rule to find `b′`.
Division is only a binary operation, so here we have the *quotient rule*:
Division is only a binary operation:
```{julia}
function D(::Val{:/}, ::Val{2}, args, var)
u,v = args
u′, v′ = D(u, var), D(v, var)
:( ($u′*$v - $u*$v′)/$v^2 )
:(($u′*$v - $u*$v′)/$v^2)
end
```
quarto/derivatives/taylor_series_polynomials.qmd
+168 -132
View File
@@ -30,9 +30,16 @@ Quadratic functions are still fairly easy to work with. Is it possible to find t
More generally, for a given $n$, what would be the best polynomial of degree $n$ to approximate $f(x)$ at $c$?
We will see in this section how the Taylor polynomial answers these questions, and is the appropriate generalization of the tangent line approximation.
We will see in this section how the Taylor polynomial is the appropriate generalization of the tangent line approximation and is given by:
$$
\begin{align*}
T_n(x) &= f(c) + f'(c)(x-c) + \frac{f''(c)}{2!}(x-c)^2 + \\
&\qquad\frac{f'''(c)}{3!}(x-c)^3 + \cdots + \frac{f^{(n)}(c)}{n!}(x-c)^n.
\end{align*}
$$
::: {#fig-taylor-polynomial-animation}
```{julia}
#| hold: true
#| echo: false
@@ -66,19 +73,35 @@ imgfile = tempname() * ".gif"
gif(anim, imgfile, fps = 1)
caption = L"""
Illustration of the Taylor polynomial of degree $k$, $T_k(x)$, at $c=0$ and its graph overlaid on that of the function $1 - \cos(x)$.
"""
caption = ""
plotly()
ImageFile(imgfile, caption)
```
Illustration of the Taylor polynomial of degree $k$, $T_k(x)$, at $c=0$ and its graph overlaid on that of the function $1 - \cos(x)$.
:::
## The secant line and the tangent line
Heads up: we approach this general problem **much** more indirectly than is needed by introducing notations that are attributed to Newton and proceed from there. By leveraging `SymPy` we avoid tedious computations and *hopefully* gain some insight.
The form of the Taylor polynomial can readily be seen, as it is the polynomial of degree $n$ or less that has $n$ derivatives matching those of $f$ at $x=c$. This `SymPy` code shows equivalence for $n=5$:
```{julia}
@syms a[0:5] x c
T₅ = sum(a[i+1] * (x-c)^i for i in 0:5)
```
That is a polynomial centered at $x=c$.
These are the derivatives that would match up with $f^{(k)}(c)$ which shows $f^{(k)}(c) = k! \cdot a_k$ (or $a_k = f^{(k)}(c)/k!$) for $k$ in $0, 1, \dots, 5$:
```{julia}
[diff(T₅, x, k)(x => c) for k in 0:5]
```
---
However, in the following, we approach this general problem **much** more indirectly than is needed by introducing notations that are attributed to Newton and proceed from there. Again, by leveraging `SymPy` we avoid tedious computations and *hopefully* gain some insight.
Suppose $f(x)$ is a function which is defined in a neighborhood of $c$ and has as many continuous derivatives as we care to take at $c$.
@@ -87,7 +110,7 @@ Suppose $f(x)$ is a function which is defined in a neighborhood of $c$ and has a
We have two related formulas:
* The *secant line* connecting $(c, f(c))$ and $(c+h, f(c+h))$ for a value of $h>0$ is given in point-slope form by
* The *secant line* connecting $(c, f(c))$ and $(c+h, f(c+h))$ for a value of $h>0$ is given in point-slope form by
$$
@@ -97,14 +120,14 @@ $$
The slope is the familiar approximation to the derivative: $(f(c+h)-f(c))/h$.
* The *tangent line* to the graph of $f(x)$ at $x=c$ is described by the function
* The *tangent line* to the graph of $f(x)$ at $x=c$ is described by the function
$$
tl(x) = f(c) + f'(c) \cdot(x - c).
$$
The key is the term multiplying $(x-c)$---for the secant line this is an approximation to the related term for the tangent line. That is, the secant line approximates the tangent line, which is the linear function that best approximates the function at the point $(c, f(c))$.
The key is the slope term multiplying $(x-c)$---for the secant line this is an approximation to the related term for the tangent line. That is, the secant line approximates the tangent line, which is the linear function that best approximates the function at the point $(c, f(c))$.
This is quantified by the *mean value theorem* which states under our assumptions on $f(x)$ that there exists some $\xi$ between $x$ and $c$ for which:
@@ -135,20 +158,26 @@ $$
That is, $f(x) = f(c) + f'(c)(x-c) + f''(\xi)/2\cdot(x-c)^2$, or $f(x)-tl(x)$ is as described.)
The secant line also has an interpretation that will generalize - it is the smallest order polynomial that goes through, or *interpolates*, the points $(c,f(c))$ and $(c+h, f(c+h))$. This is obvious from the construction - as this is how the slope is derived - but from the formula itself requires showing $tl(c) = f(c)$ and $tl(c+h) = f(c+h)$. The former is straightforward, as $(c-c) = 0$, so clearly $tl(c) = f(c)$. The latter requires a bit of algebra.
The secant line also has an interpretation that will generalize---it is the smallest order polynomial that goes through, or *interpolates*, the points $(c,f(c))$ and $(c+h, f(c+h))$. This is obvious from the construction---as this is how the slope is derived---but from the formula itself requires showing $sl(c) = f(c)$ and $sl(c+h) = f(c+h)$. The former is straightforward, as $(c-c) = 0$, so clearly $sl(c) = f(c)$. The latter requires a bit of algebra.
We have:
::: {.relationship title="Linear approximation"}
> The best *linear* approximation at a point $c$ is related to the *linear* polynomial interpolating the points $c$ and $c+h$ as $h$ goes to $0$.
The best *linear* approximation at a point $c$ is related to the *linear* polynomial interpolating the points $c$ and $c+h$ as $h$ goes to $0$.
:::
This is the relationship we seek to generalize through our round about approach below:
This is the relationship we seek to generalize through our round-about approach below:
::: {.relationship title="Best approximation by a polynomial of a given degree or less"}
> The best approximation at a point $c$ by a polynomial of degree $n$ or less is related to the polynomial interpolating through the points $c, c+h, \dots, c+nh$ as $h$ goes to $0$.
The best approximation at a point $c$ by a polynomial of degree $n$ or less is related to the polynomial interpolating through the points $c, c+h, \dots, c+nh$ as $h$ goes to $0$.
:::
@@ -158,19 +187,25 @@ As in the linear case, there is flexibility in the exact points chosen for the i
---
Now, we take a small detour to define some notation. Instead of writing our two points as $c$ and $c+h,$ we use $x_0$ and $x_1$. For any set of points $x_0, x_1, \dots, x_n$, recursively define the Newton **divided differences** of $f$ inductively, as follows:
Now, we take a small detour to define some notation. Instead of writing our two points as $c$ and $c+h,$ we use $x_0$ and $x_1$. For any set of points $x_0, x_1, \dots, x_n$, recursively define
::: {.definition title="Newton divided differences"}
The divided differences of $f$ are defined inductively, as follows:
$$
\begin{align*}
f[x_0] &= f(x_0) \\
f[x_0, x_1] &= \frac{f[x_1] - f[x_0]}{x_1 - x_0}\\
\cdots &\\
f[x_0] &= f(x_0) \\
f[x_0, x_1] &= \frac{f[x_1] - f[x_0]}{x_1 - x_0}\\
f[x_0, x_1, x_2] &= \frac{f[x_1, x_2] - f[x_0, x_1]}{x_2 - x_0}\\
\vdots\qquad & \qquad\qquad \vdots\\
f[x_0, x_1, x_2, \dots, x_n] &= \frac{f[x_1, \dots, x_n] - f[x_0, x_1, x_2, \dots, x_{n-1}]}{x_n - x_0}.
\end{align*}
$$
We see the first two values look familiar, and to generate more we just take certain ratios akin to those formed when finding a secant line.
We see the first two values look familiar, and to generate more we recursively take certain ratios akin to those formed when finding a secant line.
:::
With this notation the secant line can be re-expressed as:
@@ -179,7 +214,11 @@ $$
sl(x) = f[c] + f[c, c+h] \cdot (x-c).
$$
If we think of $f[c, c+h]$ as an approximate *first* derivative, we have an even stronger parallel between a secant line $x=c$ and the tangent line at $x=c$: $tl(x) = f(c) + f'(c)\cdot (x-c)$.
If we think of $f[c, c+h]$ as an approximate *first* derivative, we have an even stronger parallel between a secant line $x=c$ and the tangent line at $x=c$:
$$
tl(x) = f(c) + f'(c)\cdot (x-c).
$$
We use `SymPy` to investigate. First we create a *recursive* function to compute the divided differences:
@@ -225,8 +264,7 @@ simplify(2ex₂)
This relationship between higher-order divided differences and higher-order derivatives generalizes. This is expressed in this [theorem](http://tinyurl.com/zjogv83):
:::{.callout-note}
## Mean value theorem for Divided differences
:::{.theorem title="Mean value theorem for divided differences"}
Suppose $m=x_0 < x_1 < x_2 < \dots < x_n=M$ are distinct points. If $f$ has $n$ continuous derivatives then there exists a value $\xi$, where $m < \xi < M$, satisfying:
@@ -242,10 +280,13 @@ This immediately applies to the above, where we parameterized by $h$: $x_0=c, x_
A proof based on Rolle's theorem appears in the appendix.
## Quadratic approximations; interpolating polynomials
## Quadratic approximations
Why the fuss? The answer comes from a result of Newton on *interpolating* polynomials. Consider a function $f$ and $n+1$ points $x_0$, $x_1, \dots, x_n$. Then an interpolating polynomial is a polynomial of least degree that goes through each point $(x_i, f(x_i))$. The [Newton form](https://en.wikipedia.org/wiki/Newton_polynomial) of such a polynomial can be written as:
Why the fuss? The answer comes from a result of Newton on *interpolating* polynomials. Consider a function $f$ and $n+1$ points $x_0$, $x_1, \dots, x_n$. Then an interpolating polynomial is a polynomial of least degree that goes through each point $(x_i, f(x_i))$.
The [Newton form](https://en.wikipedia.org/wiki/Newton_polynomial) of such a polynomial can be written as:
$$
@@ -259,20 +300,7 @@ $$
The case $n=0$ gives the value $f[x_0] = f(c)$, which can be interpreted as the slope-$0$ line that goes through the point $(c,f(c))$.
We are familiar with the case $n=1$, with $x_0=c$ and $x_1=c+h$, this becomes our secant-line formula:
$$
f[c] + f[c, c+h](x-c).
$$
As mentioned, we can verify directly that it interpolates the points $(c,f(c))$ and $(c+h, f(c+h))$. Here we let `SymPy` do the algebra:
```{julia}
p₁ = u[c] + u[c, c+h] * (x-c)
p₁(x => c) - u(c), p₁(x => c+h) - u(c+h)
```
We are familiar with the case $n=1$, with $x_0=c$ and $x_1=c+h$, as this becomes our secant-line formula/
Now for something new. Take the $n=2$ case with $x_0 = c$, $x_1 = c + h$, and $x_2 = c+2h$. Then the interpolating polynomial is:
@@ -285,7 +313,7 @@ We add the next term to our previous polynomial and simplify
```{julia}
p₂ = p₁ + u[c, c+h, c+2h] * (x-c) * (x-(c+h))
p₂ = u[c] + u[c, c+h] * (x-c) + u[c, c+h, c+2h] * (x-c) * (x-(c+h))
simplify(p₂)
```
@@ -296,7 +324,7 @@ We can check that this interpolates the three points. Notice that at $x_0=c$ and
p₂(x => c+2h) - u(c+2h)
```
Hmm, doesn't seem correct - that was supposed to be $0$. The issue isn't the math, it is that SymPy needs to be encouraged to simplify:
Hmm, doesn't seem correct---that was supposed to be $0$. The issue isn't the math, it is that SymPy needs to be encouraged to simplify:
```{julia}
@@ -320,9 +348,9 @@ $$
This is clearly related to the tangent line approximation of $f(x)$ at $x=c$, but carrying an extra quadratic term.
Here we visualize the approximations with the function $f(x) = \cos(x)$ at $c=0$.
In @fig-f-linear-quadratic-approximations we visualize the approximations with the function $f(x) = \cos(x)$ at $c=0$.
::: {#fig-f-linear-quadratic-approximations}
```{julia}
#| hold: true
f(x) = cos(x)
@@ -340,6 +368,9 @@ plot!(p, x->f(c) + fp*(x-c) + (1/2)*fpp*(x-c)^2, a, b; color=:green, alpha=0.25,
p
```
Plot of function, a linear approximation, and a quadratic approximation at $c=0$
:::
This graph illustrates that the extra quadratic term can track the curvature of the function, whereas the tangent line itself can't. So, we have a polynomial which is a "better" approximation, is it the best approximation?
@@ -347,16 +378,19 @@ The Cauchy mean value theorem, as in the case of the tangent line, will guarante
$$
f(x) - \left(f(c) + f'(c) \cdot(x-c) + \frac{1}{2}\cdot f''(c) \cdot (x-c)^2 \right) =
\begin{align*}
f(x) &- \left(f(c) + f'(c) \cdot(x-c) + \frac{1}{2}\cdot f''(c) \cdot (x-c)^2 \right) \\
&=
\frac{1}{3!}f'''(\xi) \cdot (x-c)^3.
\end{align*}
$$
In this sense, the above quadratic polynomial, called the Taylor Polynomial of degree 2, is the best *quadratic* approximation to $f$, as the difference goes to $0$ at a rate of $(x-c)^3$.
The graphs of the secant line and approximating parabola for $h=1/4$ are similar:
@fig-f-linear-quadratic-approximations-divided-differences shows the similar graphs of the secant line and the interpolating parabola for $h=1/4$.
::: {#fig-f-linear-quadratic-approximations-divided-differences}
```{julia}
#| hold: true
f(x) = cos(x)
@@ -376,7 +410,10 @@ plot!(p, x -> f0 + fd*(x-x0) + fdd * (x-x0)*(x-x1), a,b, color=:green, alpha=0.2
p
```
Though similar, the graphs are **not** identical, as the interpolating polynomials aren't the best approximations. For example, in the tangent-line graph the parabola only intersects the cosine graph at $x=0$, whereas for the secant-line graph - by definition - the parabola intersects the graph at least $2$ times and the interpolating polynomial $3$ times (at $x_0$, $x_1$, and $x_2$).
Plot of function, a linear approximation using divided differences, and a quadratic approximation using divided differences at $c=0$
:::
Though similar, the graphs in @fig-f-linear-quadratic-approximations-divided-differences are **not** identical to those in @fig-f-linear-quadratic-approximations, as the interpolating polynomials aren't the best approximations. For example, in the tangent-line graph the parabola only intersects the cosine graph at $x=0$, whereas for the secant-line graph---by definition---the parabola intersects the graph at least $2$ times and the interpolating polynomial $3$ times (at $x_0$, $x_1$, and $x_2$).
##### Example
@@ -389,9 +426,9 @@ $$
f(0) + f'(0) \cdot t + \frac{1}{2} \cdot f''(0) \cdot t^2 = 0 + 1t - \frac{t^2}{2}
$$
A graph shows the difference:
@fig-plot-log-1-plus-t-and-linear-quadratic-approximations shows the difference.
::: {#fig-plot-log-1-plus-t-and-linear-quadratic-approximations}
```{julia}
#| hold: true
f(t) = log(1 + t)
@@ -401,6 +438,9 @@ plot!(t -> t, a, b)
plot!(t -> t - t^2/2, a, b)
```
Plot of $f(t) = \ln(1+t)$ and a linear and quadratic approximation at $0$.
:::
Though we can see that the tangent line is a good approximation, the quadratic polynomial tracks the logarithm better farther from $c=0$.
@@ -417,8 +457,10 @@ By ignoring friction, the total energy is conserved giving:
$$
K = \frac{1}{2}m v^2 + mgR \cdot (1 - \cos(\theta)) =
\frac{1}{2} m R^2 (\frac{d\theta}{dt})^2 + mgR \cdot (1 - \cos(\theta)).
\begin{align*}
K &= \frac{1}{2}m v^2 + mgR \cdot (1 - \cos(\theta)) \\
&= \frac{1}{2} m R^2 (\frac{d\theta}{dt})^2 + mgR \cdot (1 - \cos(\theta)).
\end{align*}
$$
The value of $1-\cos(\theta)$ inhibits further work which would be possible were there an easier formula there. In fact, we could try the excellent approximation $1 - \theta^2/2$ from the quadratic approximation. Then we have:
@@ -490,27 +532,27 @@ f[x_0] &+ f[x_0,x_1] \cdot (x - x_0) + f[x_0, x_1, x_2] \cdot (x - x_0)\cdot(x-x
$$
and taking $x_i = c + i\cdot h$, for a given $n$, we have in the limit as $h > 0$ goes to zero that coefficients of this polynomial converge:
and taking $x_i = c + i\cdot h$, for a given $n$, in the limit as $h0$ goes to zero the coefficients of this polynomial converge.
:::{.callout-note}
## Taylor polynomial of degree $n$
:::{.definition title="Taylor polynomial of degree n"}
Suppose $f(x)$ has $n+1$ derivatives (continuous on $c$ and $x$), then
$$
T_n(x) = f(c) + f'(c)\cdot(x-c) + \frac{f''(c)}{2!}(x-c)^2 + \cdots + \frac{f^{(n)}(c)}{n!} (x-c)^n,
$$
will be the best approximation of degree $n$ or less to $f$, near $c$.
The error will be given - again by an application of the Cauchy mean value theorem:
will be the best polynomial approximation of degree $n$ or less to $f$, near $c$.
The error will be given by:
$$
\frac{1}{(n+1)!} \cdot f^{(n+1)}(\xi) \cdot (x-c)^n
$$
for some $\xi$ between $c$ and $x$.
for some $\xi$ between $c$ and $x$. (Again by an application of the Cauchy mean value theorem.)
When $c=0$, a Taylor polynomial is also called a Maclaurin polynomial.
:::
@@ -553,37 +595,24 @@ The output of `series` includes a big "Oh" term, which identifies the scale of t
:::{.callout-note}
## Note
A Taylor polynomial of degree $n$ consists of $n+1$ terms and an error term. The "Taylor series" (below) is an *infinite* collection of terms, the first $n+1$ matching the Taylor polynomial of degree $n$. The fact that series are *infinite* means care must be taken when even talking about their existence, unlike a Taylor polynomial, which is just a polynomial and exists as long as a sufficient number of derivatives are available.
A Taylor polynomial of degree $n$ consists of $n+1$ terms and has an error term. The "Taylor series" (below) is an *infinite* collection of terms, the first $n+1$ matching the Taylor polynomial of degree $n$. The fact that series are *infinite* means care must be taken when even talking about their existence, unlike a Taylor polynomial, which is just a polynomial and exists as long as a sufficient number of derivatives are available.
:::
We define a function to compute Taylor polynomials from a function. The following returns a function, not a symbolic object, using `D`, from `CalculusWithJulia`, which is based on `ForwardDiff.derivative`, to find higher-order derivatives:
##### Example: visualizing the approximations
```{julia}
#| echo: false
using CalculusWithJulia: D
function taylor_poly(f, c=0, n=2)
x -> f(c) + sum(D(f, i)(c) * (x-c)^i / factorial(i) for i in 1:n)
end
nothing
```
With a function, we can compare values. For example, here we see the difference between the Taylor polynomial and the answer for a small value of $x$:
```{julia}
#| hold: true
a = .1
f(x) = log(1+x)
Tn = taylor_poly(f, 0, 5)
Tn(a) - f(a)
```
### Plotting
Let's now visualize a function and the two approximations - the Taylor polynomial and the interpolating polynomial. We use this function to generate the interpolating polynomial as a function:
```{julia}
#| echo: false
function newton_form(f, xs)
x -> begin
tot = divided_differences(f, xs[1])
@@ -593,12 +622,15 @@ function newton_form(f, xs)
tot
end
end
nothing
```
To see a plot, we have
@fig-plot-newton-form-taylor-poly} shows a plot of $f(x) = \sin(x)$ it's Taylor at $c=0$, $T_4$, and the fourth degree interpolating polynomial at $c, c+h, c+2h, c+3h, c+4h$, with $h=1/4$. Both polynomials track the function well near $0$, but the Taylor polynomial does a better job approximating $f(x)$ farther from $0$.
::: {#fig-plot-newton-form-taylor-poly}
```{julia}
#| echo: false
f(x) = sin(x)
c, h, n = 0, 1/4, 4
int_poly = newton_form(f, [c + i*h for i in 0:n])
@@ -609,32 +641,27 @@ plot!(int_poly; color=:green, label="interpolating")
plot!(tp; color=:red, label="Taylor")
```
To get a better sense, we plot the residual differences here:
Plot of function, its Taylor polynomial, $T-4$ and a fourth-degree interpolating polynomial
:::
In @fig-plot-1-minus-cos-x-and-3-taylor-polys we make a plot of the Taylor polynomial for different sizes of $n$ for the function $f(x) = 1 - \cos(x)$:
::: {#fig-plot-1-minus-cos-x-and-3-taylor-polys}
```{julia}
d1(x) = f(x) - int_poly(x)
d2(x) = f(x) - tp(x)
plot(d1, a, b; linecolor=:blue, label="interpolating")
plot!(d2; linecolor=:green, label="Taylor")
```
The graph should be $0$ at each of the points in `xs`, which we can verify in the graph above. Plotting over a wider region shows a common phenomenon that these polynomials approximate the function near the values, but quickly deviate away:
In this graph we make a plot of the Taylor polynomial for different sizes of $n$ for the function $f(x) = 1 - \cos(x)$:
```{julia}
#| hold: true
#| echo: false
f(x) = 1 - cos(x)
a, b = -pi, pi
plot(f, a, b, linewidth=5, label="f")
plot!(taylor_poly(f, 0, 2), label="T₂")
plot!(taylor_poly(f, 0, 4), label="T₄")
plot!(taylor_poly(f, 0, 6), label="T₆")
plot(f, a, b, linewidth=5, label=L"1 - \cos(x)")
plot!(taylor_poly(f, 0, 2), label=L"T_2")
plot!(taylor_poly(f, 0, 4), label=L"T_4")
plot!(taylor_poly(f, 0, 6), label=L"T_6")
```
Plot of $f(x) = 1 - \cos(x)$ and Taylor polynomials of degree $2$, $4$, and $6$
:::
Though all are good approximations near $c=0$, as more terms are included, the Taylor polynomial becomes a better approximation over a wider range of values.
@@ -655,7 +682,11 @@ Suppose $R$ is the radius of the earth and $h$ the height above the earth assumi
$$
P = \frac{2\pi}{\sqrt{G\cdot M}} \cdot (h+R)^{3/2} = \frac{2\pi}{\sqrt{G\cdot M}} \cdot R^{3/2} \cdot (1 + h/R)^{3/2} = P_0 \cdot (1 + h/R)^{3/2},
\begin{align*}
P &= \frac{2\pi}{\sqrt{G\cdot M}} \cdot (h+R)^{3/2} \\
&= \frac{2\pi}{\sqrt{G\cdot M}} \cdot R^{3/2} \cdot (1 + h/R)^{3/2} \\
&= P_0 \cdot (1 + h/R)^{3/2},
\end{align*}
$$
where $P_0$ collects terms that involve the constants.
@@ -678,7 +709,7 @@ $$
Typically, if $h$ is much smaller than $R$ the first term is enough giving a formula like $P \approx P_0 \cdot(1 + \frac{3h}{2R})$.
A satellite phone utilizes low orbit satellites to relay phone communications. The [Iridium](http://www.kddi.com/english/business/cloud-network-voice/satellite/iridium/mobile/) system uses satellites with an elevation $h=780km$. The radius of the earth is $3,959 miles$, the mass of the earth is $5.972 × 10^{24} kg$, and the gravitational [constant](https://en.wikipedia.org/wiki/Gravitational_constant), $G$ is $6.67408 \cdot 10^{-11}$ $m^3/(kg \cdot s^2)$.
A satellite phone utilizes low orbit satellites to relay phone communications. The [Iridium](http://www.kddi.com/english/business/cloud-network-voice/satellite/iridium/mobile/) system uses satellites with an elevation $h=780km$. The radius of the earth is $3,959$ miles, the mass of the earth is $5.972 × 10^{24} kg$, and the gravitational [constant](https://en.wikipedia.org/wiki/Gravitational_constant), $G$ is $6.67408 \cdot 10^{-11}$ $m^3/(kg \cdot s^2)$.
Compare the approximate value with $1$ term to the exact value.
@@ -732,7 +763,7 @@ Prealₛ, P1ₛ, P5ₛ
We see the Taylor polynomial underestimates badly in this case. A reminder that these approximations are locally good, but may not be good on all scales. Here $h \approx 3R$. We can see from this graph of $(1+x)^{3/2}$ and its $5$th degree Taylor polynomial $T_5$ that it is a bad approximation when $x > 2$.
::: {#fig-plot-function-5th-degree-poly}
```{julia}
#| echo: false
f1(x) = (1+x)^(3/2)
@@ -741,6 +772,8 @@ plot(f1, -1, 3, linewidth=4, legend=false)
plot!(p2, -1, 3)
```
Plot of $f(x)$ and a fifth-degree Taylor polynomial
:::
---
@@ -761,7 +794,7 @@ Preal = P0 * (1 + HR)^(3/2) # in seconds
Preal, uconvert(hr, Preal) # ≈ 11.65 hours
```
We see `Preal` has the right units - the units of mass and distance cancel leaving a measure of time - but it is hard to sense how long this is. Converting to hours, helps us see the satellite orbits about twice per day.
We see `Preal` has the right units---the units of mass and distance cancel leaving a measure of time---but it is hard to sense how long this is. Converting to hours, helps us see the satellite orbits about twice per day.
##### Example: computing $\log(x)$
@@ -773,7 +806,7 @@ Where exactly does the value assigned to $\log(5)$ come from? The value needs to
But how? One can see details of a possible way [here](https://github.com/musm/Amal.jl/blob/master/src/log.jl).
First, there is usually a reduction stage. In this phase, the problem is transformed in a manner to one involving only a fixed interval of values. For this, function values of $k$ and $m$ are found so that $x = 2^k \cdot (1+m)$ *and* $\sqrt{2}/2 < 1+m < \sqrt{2}$. If these are found, then $\log(x)$ can be computed with $k \cdot \log(2) + \log(1+m)$. The first value - a multiplication - can easily be computed using pre-computed value of $\log(2)$, the second then *reduces* the problem to an interval.
First, there is usually a reduction stage. In this phase, the problem is transformed in a manner to one involving only a fixed interval of values. For this, function values of $k$ and $m$ are found so that $x = 2^k \cdot (1+m)$ *and* $\sqrt{2}/2 < 1+m < \sqrt{2}$. If these are found, then $\log(x)$ can be computed with $k \cdot \log(2) + \log(1+m)$. The first value---a multiplication---can easily be computed using a pre-computed value of $\log(2)$, the second then *reduces* the problem to an interval.
Now, for this problem a further trick is utilized, writing $s= m/(2+m)$ so that $\log(1+m)=\log(1+s)-\log(1-s)$ for some small range of $s$ values. These combined make it possible to compute $\log(x)$ for any real $x$.
@@ -786,7 +819,7 @@ To compute $\log(1\pm s)$, we can find a Taylor polynomial. Let's go out to degr
@syms s
aₗ = series(log(1 + s), s, 0, 19)
bₗ = series(log(1 - s), s, 0, 19)
a_b = (aₗ - bₗ).removeO() # remove"Oh" not remove"zero"
a_b = (aₗ - bₗ).removeO() # remove "Oh" not remove"zero"
```
This is re-expressed as $2s + s \cdot p$ with $p$ given by:
@@ -809,7 +842,7 @@ How big can the error be between this *approximations* and $\log(1+m)$? The expr
Max = (x/(2+x))(x => sqrt(2) - 1)
```
The error term is like $2/19 \cdot \xi^{19}$ which is largest at this value of $M$. Large is relative - it is really small:
The error term is like $2/19 \cdot \xi^{19}$ which is largest at this value of $M$. Large is relative---it is really small:
```{julia}
@@ -838,7 +871,7 @@ The two values differ by less than $10^{-16}$, as advertised. Re-assembling then
Δ = k * log(2) + (m - s*(m-pₗ)) - log(5)
```
The actual code is different, as the Taylor polynomial isn't used. The Taylor polynomial is a great approximation near a point, but there might be better polynomial approximations for all values in an interval. In this case there is, and that polynomial is used in the production setting. This makes things a bit more efficient, but the basic idea remains - for a prescribed accuracy, a polynomial approximation can be found over a given interval, which can be cleverly utilized to solve for all applicable values.
The actual code is different, as the Taylor polynomial isn't used. The Taylor polynomial is a great approximation near a point, but there might be better polynomial approximations for all values in an interval. In this case there is, and that polynomial is used in the production setting. This makes things a bit more efficient, but the basic idea remains---for a prescribed accuracy, a polynomial approximation can be found over a given interval, which can be cleverly utilized to solve for all applicable values.
##### Example: higher order derivatives of the inverse function
@@ -872,11 +905,11 @@ That is:
$$
b_n \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^n =
b_n \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^n \approx
(x_0 + \Delta_x) - \left( x_0 + \sum_{i=1}^{n-1} b_i \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^i \right)
$$
Solving for $b_n = g^{(n)}(y_0) / n!$ gives the formal expression:
Solving for $b_n = g^{(n)}(y_0) / n!$ gives a formal expression for $n$th derivative of the inverse function for $f(x)$, denoted $g(x)$:
$$
@@ -937,7 +970,7 @@ The `solve` function is used to identify $g^{(n)}$ represented in terms of lower
## Taylor series
Recall a *power series* has the form $\sum_{n=0}^\infty a_n (x-c)^n$. Power series have a radius of convergence ($|x - c| < r$) for which the series converges and diverges when $|x-c| > r$.
Recall a *power series* has the form $\sum_{n=0}^\infty a_n (x-c)^n$. Power series have a radius of convergence $r$ for which the series converges if $|x - c| < r$ and diverges when $|x-c| > r$.
The Taylor polynomial formula can be extended to a formal power series with through
@@ -993,8 +1026,8 @@ choices = [
"``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``",
"``\\sum_{k=0}^{10} x^n/n!``"
]
answ = 3
radioq(choices, answ)
answer = 3
radioq(choices, answer)
```
###### Question
@@ -1012,8 +1045,8 @@ choices = [
"``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``",
"``\\sum_{k=0}^{10} x^n/n!``"
]
answ = 4
radioq(choices, answ)
answer = 4
radioq(choices, answer)
```
###### Question
@@ -1031,8 +1064,8 @@ choices = [
"``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``",
"``\\sum_{k=0}^{10} x^n/n!``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1050,8 +1083,8 @@ choices = [
"``1/5!``",
"``2/15``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1071,8 +1104,8 @@ choices = [
"``x^2``",
"``x^2 \\cdot (x - x^3/3! + x^5/5!)``"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
###### Question
@@ -1094,8 +1127,8 @@ If this is true, then formally evaluating at $x=0$ gives $f(0) = a$, so $a$ is d
choices = ["``f''''(0) = e``",
"``f''''(0) = 4 \\cdot 3 \\cdot 2 e = 4! e``",
"``f''''(0) = 0``"]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
###### Question
@@ -1123,8 +1156,8 @@ yesnoq(true)
#| hold: true
#| echo: false
choices =["It is increasing", "It is decreasing", "It both increases and decreases"]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
* The maximum value of $e^x$ over $[-1,1]$ occurs at
@@ -1134,8 +1167,8 @@ radioq(choices, answ)
#| hold: true
#| echo: false
choices=["A critical point", "An end point"]
answ = 2
radioq(choices, answ)
answer = 2
radioq(choices, answer)
```
* Which theorem tells you that for a *continuous* function over *closed* interval, a maximum value will exist?
@@ -1148,8 +1181,8 @@ choices = [
"The intermediate value theorem",
"The mean value theorem",
"The extreme value theorem"]
answ = 3
radioq(choices, answ)
answer = 3
radioq(choices, answer)
```
* What is the *largest* possible value of the error:
@@ -1161,8 +1194,8 @@ radioq(choices, answ)
choices = [
"``1/6!\\cdot e^1 \\cdot 1^6``",
"``1^6 \\cdot 1 \\cdot 1^6``"]
answ = 1
radioq(choices,answ)
answer = 1
radioq(choices,answer)
```
###### Question
@@ -1182,8 +1215,8 @@ L"The function $e^x$ is increasing, so takes on its largest value at the endpoin
L"The function has a critical point at $x=1/2$",
L"The function is monotonic in $k$, so achieves its maximum at $k+1$"
]
answ = 1
radioq(choices, answ)
answer = 1
radioq(choices, answer)
```
Assuming the above is right, find the smallest value $k$ guaranteeing an error no more than $10^{-16}$.
@@ -1276,9 +1309,9 @@ Now, assuming the Newton form is correct, a [proof](http://tinyurl.com/zjogv83)
By Rolle's theorem, between any two such zeros $x_i, x_{i+1}$, $0 \leq i < n$ there must be a zero of the derivative of $h(x)$, say $\xi^1_i$. So $h'(x)$ has zeros $\xi^1_0 < \xi^1_1 < \dots < \xi^1_{n-1}$.
We visualize this with $f(x) = \sin(x)$ and $x_i = i$ for $i=0, 1, 2, 3$, The $x_i$ values are indicated with circles, the $\xi^1_i$ values indicated with squares:
We visualize this with $f(x) = \sin(x)$ and $x_i = i$ for $i=0, 1, 2, 3$ in @fig-plot-of-between-any-two-zeros-is-a-zero-of-fprime, The $x_i$ values are indicated with circles, the $\xi^1_i$ values indicated with squares.
::: {#fig-plot-of-between-any-two-zeros-is-a-zero-of-fprime}
```{julia}
#| hold: true
#| echo: false
@@ -1293,4 +1326,7 @@ scatter!(xs, h1.(xs), markersize=5)
scatter!(cps, h1.(cps), markersize=5, marker=:square)
```
Between any adjacent pair of zeros, there lies at least one relative extrema
:::
Again by Rolle's theorem, between any pair of adjacent zeros $\xi^1_i, \xi^1_{i+1}$ there must be a zero $\xi^2_i$ of $h''(x)$. So there are $n-1$ zeros of $h''(x)$. Continuing, we see that there will be $n+1-3$ zeros of $h^{(3)}(x)$, $n+1-4$ zeros of $h^{4}(x)$, $\dots$, $n+1-(n-1)$ zeros of $h^{n-1}(x)$, and finally $n+1-n$ ($1$) zeros of $h^{(n)}(x)$. Call this last zero $\xi$. It satisfies $x_0 \leq \xi \leq x_n$. Further, $0 = h^{(n)}(\xi) = f^{(n)}(\xi) - g^{(n)}(\xi)$. But $g$ is a degree $n$ polynomial, so the $n$th derivative is the coefficient of $x^n$ times $n!$. In this case we have $0 = f^{(n)}(\xi) - f[x_0, \dots, x_n] n!$. Rearranging yields the result.