What is the relative change in surface area of a sphere if the radius changes from ``r`` to ``r + dr``?
We have ``S = 4\pi r^2`` so the approximate relative change, ``dy/S`` is given, using the derivative ``dS/dr = 8\pi r``, by
```math
\frac{8\pi\cdot r\cdot dr}{4\pi r^2} = 2r\cdot dr.
```
##### Example
We are traveling ``60`` miles. At ``60`` miles an hour, we will take ``60`` minutes (or one hour). How long will it take at ``70`` miles an hour? (Assume you can't divide, but, instead, can only multiply!)
Well the answer is $60/70$ hours or $60/70 \cdot 60$ minutes. But we
can't divide, so we turn this into a multiplication problem via some algebra:
Computing the derivative can be done easily, we get for our answer:
```julia;
fp(x) = 3*x^2
c₀, Delta = 10, 0.1
approx₀ = 1000 + fp(c₀) * Delta
```
This is a relative error as a percent of:
```julia;
actual₀ = 10.1^3
(actual₀ - approx₀)/actual₀ * 100
```
The manufacturer may be interested instead in comparing the volume of the actual object to the $1$ liter target. They might use the approximate value for this comparison, which would yield:
```julia;
(1000 - approx₀)/approx₀ * 100
```
This is off by about $3$ percent. Not so bad for some applications, devastating for others.
##### Example: Eratosthenes and the circumference of the earth
[Eratosthenes](https://en.wikipedia.org/wiki/Eratosthenes) is said to have been the first person to estimate the radius (or by relation the circumference) of the earth. The basic idea is based on the difference of shadows cast by the sun. Suppose Eratosthenes sized the circumference as ``252,000`` *stadia*. Taking ``1``` stadia as ``160`` meters and the actual radius of the earth as ``6378.137`` kilometers, we can convert to see that Eratosthenes estimated the radius as ``6417``.
If Eratosthenes were to have estimated the volume of a spherical earth, what would be his approximate percentage change between his estimate and the actual?
Using ``V = 4/3 \pi r^3`` we get ``V' = 4\pi r^2``:
```julia
rₑ = 6417
rₐ = 6378.137
Δᵣ = rₑ - rₐ
Vₛ(r) = 4/3 * pi * r^3
Δᵥ = Vₛ'(rₑ) * Δᵣ
Δᵥ / Vₛ(rₑ) * 100
```
##### Example: a simple pendulum
A *simple* pendulum is comprised of a massless "bob" on a rigid "rod"
of length $l$. The rod swings back and forth making an angle $\theta$
with the perpendicular. At rest $\theta=0$, here we have $\theta$ swinging with $\lvert\theta\rvert \leq \theta_0$
for some $\theta_0$.
According to [Wikipedia](http://tinyurl.com/yz5sz7e) - and many
introductory physics book - while swinging, the angle $\theta$ varies
with time following this equation:
```math
\theta''(t) + \frac{g}{l} \sin(\theta(t)) = 0.
```
That is, the second derivative of $\theta$ is proportional to the sine
of $\theta$ where the proportionality constant involves $g$ from
gravity and the length of the "rod."
This would be much easier if the second derivative were proportional to the angle $\theta$ and not its sine.
[Huygens](http://en.wikipedia.org/wiki/Christiaan_Huygens) used the
approximation of $\sin(x) \approx x$, noted above, to say that when
the angle is not too big, we have the pendulum's swing obeying
$\theta''(t) = -g/l \cdot t$. Without getting too involved in why,
we can verify by taking two derivatives that $\theta_0\sin(\sqrt{g/l}\cdot t)$ will be a solution to this
modified equation.
With this solution, the motion is periodic with constant amplitude (assuming frictionless behaviour), as
the sine function is. More surprisingly, the period is found from $T =
2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$ - longer
"rods" take more time to swing back and forth - but does not depend
on the how wide the pendulum is swinging between (provided $\theta_0$
is not so big the approximation of $\sin(x) \approx x$ fails). This
latter fact may be surprising, though not to Galileo who discovered
it.
## Differentials
The Leibniz notation for a derivative is ``dy/dx`` indicating the
change in ``y`` as ``x`` changes. It proves convenient to decouple
this using *differentials* ``dx`` and ``dy``. What do these notations
mean? They measure change along the tangent line in same way
``\Delta_x`` and ``\Delta_y`` measure change for the function. The differential ``dy`` depends on both ``x`` and ``dx``, it being defined by ``dy=f'(x)dx``. As tangent lines locally represent a function, ``dy`` and ``dx`` are often associated with an *infinitesimal* difference.
Taking ``dx = \Delta_x``, as in the previous graphic, we can compare ``dy`` -- the change along the tangent line given by ``dy/dx \cdot dx`` -- and ``\Delta_y`` -- the change along the function given by ``f(x + \Delta_x) - f(x)``. The linear approximation, ``f(x + \Delta_x) - f(x)\approx f'(x)dx``, says that
```math
\Delta_y \approx dy; \quad \text{ when } \Delta_x = dx
```
## The error in approximation
How good is the approximation? Graphically we can see it is pretty
good for the graphs we choose, but are there graphs out there for
which the approximation is not so good? Of course. However, we can
say this (the
[Lagrange](http://en.wikipedia.org/wiki/Taylor%27s_theorem) form of a
more general Taylor remainder theorem):
> Let ``f(x)`` be twice differentiable on ``I=(a,b)``,
>``f`` is continuous on ``[a,b]``, and
> ``a < c < b``. Then for any ``x`` in ``I``, there exists some value ``\xi`` between ``c`` and ``x`` such that
The value of $\xi$, from the mean value theorem applied to $f'(x)$,
satisfies $0 < \xi < e < x$, so is in $[0,x].$
### The big (and small) "oh"
`SymPy` can find the tangent line expression as a special case of its `series` function (which implements [Taylor series](../taylor_series_polynomials.html)). The `series` function needs an expression to approximate; a variable specified, as there may be parameters in the expression; a value ``c`` for *where* the expansion is taken, with default ``0``; and a number of terms, for this example ``2`` for a constant and linear term. (There is also an optional `dir` argument for one-sided expansions.)
Here we see the answer provided for $e^{\sin(x)}$:
```julia;
@syms x
series(exp(sin(x)), x, 0, 2)
```
The expression $1 + x$ comes from the fact that `exp(sin(0))` is $1$, and the derivative `exp(sin(0)) * cos(0)` is *also* $1$. But what is the $\mathcal{O}(x^2)$?
We know the answer is *precisely* $f''(\xi)/2 \cdot x^2$ for some ``\xi``, but were we
only concerned about the scale as $x$ goes to zero
that when ``f''`` is continuous that the error when divided by ``x^2`` goes to some finite value (``f''(0)/2``). More generally, if the error divided by ``x^2`` is *bounded* as ``x`` goes to ``0``, then we say the error is "big oh" of ``x^2``.
The [big](http://en.wikipedia.org/wiki/Big_O_notation) "oh" notation,
``f(x) = \mathcal{O}(g(x))``, says that the ratio ``f(x)/g(x)`` is
bounded as ``x`` goes to ``0`` (or some other value ``c``, depending
on the context). A little "oh" (e.g., ``f(x) = \mathcal{o}(g(x))``)
would mean that the limit ``f(x)/g(x)`` would be ``0``, as
``x\rightarrow 0``, a much stronger assertion.
Big "oh" and little "oh" give us a sense of how good an approximation
is without being bogged down in the details of the exact value. As
such they are useful guides in focusing on what is primary and what is
secondary. Applying this to our case, we have this rough form of the
tangent line approximation valid for functions having a continuous second
derivative at ``c``:
```math
f(x) = f(c) + f'(c)(x-c) + \mathcal{O}((x-c)^2).
```
##### Example: the algebra of tangent line approximations
Suppose $f(x)$ and $g(x)$ are represented by their tangent lines about $c$, respectively:
The two big "Oh" terms become just one as the sum of a constant times $(x-c)^2$ plus a constant time $(x-c)^2$ is just some other constant times $(x-c)^2$. What we can read off from this is the term multiplying $(x-c)$ is just the derivative of $f(x) + g(x)$ (from the sum rule), so this too is a tangent line approximation.
Is it a coincidence that a basic algebraic operation with tangent lines approximations produces a tangent line approximation? Let's try multiplication:
The big "oh" notation just sweeps up many things including any products of it *and* the term $f'(c)\cdot g'(c) \cdot (x-c)^2$. Again, we see from the product rule that this is just a tangent line approximation for $f(x) \cdot g(x)$.
The basic mathematical operations involving tangent lines can be computed just using the tangent lines when the desired accuracy is at the tangent line level. This is even true for composition, though there the outer and inner functions may have different "$c$"s.
Knowing this can simplify the task of finding tangent line approximations of compound expressions.
For example, suppose we know that at $c=0$ we have these formula where $a \approx b$ is a shorthand for the more formal $a=b + \mathcal{O}(x^2)$:
Since $\sin(0) = 0$, we can use these to find the tangent line approximation of
```math
e^{\sin(x)} \approx e^x \approx 1 + x.
```
Note that $\sin(\exp(x))$ is approximately $\sin(1+x)$ but not approximately $1+x$, as the expansion for $\sin$ about $1$ is not simply $x$.
### The TaylorSeries package
The `TaylorSeries` packages will do these calculations in a manner similar to how `SymPy` transforms a function and a symbolic variable into a symbolic expression.
For example, we have
```julia
t = Taylor1(Float64, 1)
```
The number type and the order is specified to the constructor. Linearization is order ``1``, other orders will be discussed later. This variable can now be composed with mathematical functions and the linearization of the function will be returned:
```julia
sin(t), exp(t), 1/(1+t)
```
```julia
sin(t)/exp(t), exp(sin(t))
```
##### Example: Automatic differentiation
Automatic differentiation (forward mode) essentially uses this technique. A "dual" is introduced which has terms ``a +b\epsilon`` where ``\epsilon^2 = 0``.
The ``\epsilon`` is like ``x`` in a linear expansion, so the `a` coefficient encodes the value and the `b` coefficient reflects the derivative at the value. Numbers are treated like a variable, so their "b coefficient" is a `1`. Here then is how `0` is encoded:
```julia;
Dual(0, 1)
```
Then what is ``\(x)``? It should reflect both ``(\sin(0), \cos(0))`` the latter being the derivative of ``\sin``. We can see this is *almost* what is computed behind the scenes through:
```julia; hold=true
x = Dual(0, 1)
@code_lowered sin(x)
```
This output of `@code_lowered` can be confusing, but this simple case needn't be. Working from the end we see an assignment to a variable named `%7` of `Dual(%3, %6)`. The value of `%3` is `sin(x)` where `x` is the value `0` above. The value of `%6` is `cos(x)` *times* the value `1` above (the `xp`), which reflects the *chain* rule being used. (The derivative of `sin(u)` is `cos(u)*du`.) So this dual number encodes both the function value at `0` and the derivative of the function at `0`.)
Similarly, we can see what happens to `log(x)` at `1` (encoded by `Dual(1,1)`):
```julia; hold=true
x = Dual(1, 1)
@code_lowered log(x)
```
We can see the derivative again reflects the chain rule, it being given by `1/x * xp` where `xp` acts like `dx` (from assignments `%5` and `%4`). Comparing the two outputs, we see only the assignment to `%4` differs, it reflecting the derivative of the function.
## Questions
###### Question
What is the right linear approximation for $\sqrt{1 + x}$ near $0$?
The side length of a square is measured roughly to be $2.0$ cm. The actual length $2.2$ cm. What is the difference in area (in absolute values) as *estimated* by a tangent line approximation.
The [Birthday problem](https://en.wikipedia.org/wiki/Birthday_problem) computes the probability that in a group of ``n`` people, under some assumptions, that no two share a birthday. Without trying to spoil the problem, we focus on the calculus specific part of the problem below:
Now, use the tangent line approximation for ``\log(1 - x)`` and the sum formula for ``0 + 1 + 2 + \dots + (n-1)`` to simplify the value of ``\log(p)``:
```julia; hold=true; echo=false
choices = ["``-n(n-1)/2/365``",
"``-n(n-1)/2\\cdot 365``",
"``-n^2/(2\\cdot 365)``",
"``-n^2 / 2 \\cdot 365``"]
radioq(choices, 1, keep_order=true)
```
If ``n = 10``, what is the approximation for ``p`` (not ``\log(p)``)?
```julia; hold=true; echo=false
n=10
val = exp(-n*(n-1)/2/365)
numericq(val)
```
If ``n=100``, what is the approximation for ``p`` (not ``\log(p)``?
