daviddoji/CalculusWithJuliaNotes.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

use quarto, not Pluto to render pages

Browse Source
This commit is contained in:
jverzani
2022-07-24 16:38:24 -04:00
parent 93c993206a
commit 7b37ca828c
879 changed files with 793311 additions and 2678 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
CwJ/derivatives/newtons_method.jmd
View File

@@ -14,7 +14,7 @@ using Roots
using CalculusWithJulia.WeaveSupport
using ImplicitPlots
fig_size = (600, 400)
fig_size = (800, 600)
const frontmatter = (
title = "Newton's method",
description = "Calculus with Julia: Newton's method",
@@ -221,20 +221,15 @@ functions, convergence happens quickly.
```julia; echo=false
note("""
Newton looked at this same example in 1699 (B.T. Polyak, *Newton's
method and its use in optimization*, European Journal of Operational
Research. 02/2007; 181(3):1086-1096.) though his technique was
slightly different as he did not use the derivative, *per se*, but
rather an approximation based on the fact that his function was a
polynomial (though identical to the derivative). Raphson (1690)
proposed the general form, hence the usual name of the Newton-Raphson
method.
""")
```
!!! note
Newton looked at this same example in 1699 (B.T. Polyak, *Newton's
method and its use in optimization*, European Journal of Operational
Research. 02/2007; 181(3):1086-1096.) though his technique was
slightly different as he did not use the derivative, *per se*, but
rather an approximation based on the fact that his function was a
polynomial (though identical to the derivative). Raphson (1690)
proposed the general form, hence the usual name of the Newton-Raphson
method.
#### Examples
@@ -634,13 +629,10 @@ This convergence to ``\alpha`` will be quadratic *if*:
not necessarily a good approximation to the actual zero, $\alpha$.
```julia; echo=false
note("""
The basic tradeoff: methods like Newton's are faster than the
bisection method in terms of function calls, but are not guaranteed to
converge, as the bisection method is.
""")
```
!!! note
The basic tradeoff: methods like Newton's are faster than the
bisection method in terms of function calls, but are not guaranteed to
converge, as the bisection method is.
What can go wrong when one of these isn't the case is illustrated next:
@@ -802,8 +794,8 @@ If one step of Newton's method was used, what would be the value of $x_1$?
```julia; hold=true; echo=false
choices = ["``-2.224``", "``-2.80``", "``-0.020``", "``0.355``"]
ans = 1
radioq(choices, ans, keep_order=true)
answ = 1
radioq(choices, answ, keep_order=true)
```
###### Question
@@ -827,8 +819,8 @@ L"It must be $x_1 > \alpha$",
L"It must be $x_1 < x_0$",
L"It must be $x_0 < x_1 < \alpha$"
]
ans = 1
radioq(choices, ans)
answ = 1
radioq(choices, answ)
```
----
@@ -852,8 +844,8 @@ L"It must be $x_1 < \alpha$",
L"It must be $x_1 > x_0$",
L"It must be $\alpha < x_1 < x_0$"
]
ans = 3
radioq(choices, ans)
answ = 3
radioq(choices, answ)
```
----
@@ -866,8 +858,8 @@ L"As $f''(\xi)/2 \cdot(x-c)^2$ is non-negative, we must have $f(x) - (f(c) + f'(
L"As $f''(\xi) < 0$ it must be that $f(x) - (f(c) + f'(c)\cdot(x-c)) \geq 0$.",
L"This isn't true. The function $f(x) = x^3$ at $x=0$ provides a counterexample"
]
ans = 1
radioq(choices, ans)
answ = 1
radioq(choices, answ)
```
This question can be used to give a proof for the previous two questions, which can be answered by considering the graphs alone. Combined, they say that if a function is increasing and concave up and ``\alpha`` is a zero, then if ``x_0 < \alpha`` it will be ``x_1 > \alpha``, and for any ``x_i > \alpha``, ``\alpha <= x_{i+1} <= x_\alpha``, so the sequence in Newton's method is decreasing and bounded below; conditions for which it is guaranteed mathematically there will be convergence.
@@ -1050,8 +1042,8 @@ choices = [
"No. The initial guess is not close enough",
"No. The second derivative is too big",
L"No. The first derivative gets too close to $0$ for one of the $x_i$"]
ans = 2
radioq(choices, ans, keep_order=true)
answ = 2
radioq(choices, answ, keep_order=true)
```
@@ -1067,8 +1059,8 @@ choices = [
"No. The initial guess is not close enough",
"No. The second derivative is too big, or does not exist",
L"No. The first derivative gets too close to $0$ for one of the $x_i$"]
ans = 2
radioq(choices, ans, keep_order=true)
answ = 2
radioq(choices, answ, keep_order=true)
```
###### Question
@@ -1082,8 +1074,8 @@ choices = [
"No. The initial guess is not close enough",
"No. The second derivative is too big, or does not exist",
L"No. The first derivative gets too close to $0$ for one of the $x_i$"]
ans = 1
radioq(choices, ans, keep_order=true)
answ = 1
radioq(choices, answ, keep_order=true)
```
###### Question
@@ -1097,8 +1089,8 @@ choices = [
"No. The initial guess is not close enough",
"No. The second derivative is too big, or does not exist",
L"No. The first derivative gets too close to $0$ for one of the $x_i$"]
ans = 4
radioq(choices, ans, keep_order=true)
answ = 4
radioq(choices, answ, keep_order=true)
```
@@ -1114,8 +1106,8 @@ choices = [
"No. The initial guess is not close enough",
"No. The second derivative is too big, or does not exist",
L"No. The first derivative gets too close to $0$ for one of the $x_i$"]
ans = 3
radioq(choices, ans, keep_order=true)
answ = 3
radioq(choices, answ, keep_order=true)
```
###### Question
@@ -1177,8 +1169,8 @@ L"It doesn't fail, it converges to $0$",
L"The tangent lines for $|x| > 0.25$ intersect at $x$ values with $|x| > 0.25$",
L"The first derivative is $0$ at $1$"
]
ans = 2
radioq(choices, ans)
answ = 2
radioq(choices, answ)
```
@@ -1261,14 +1253,14 @@ So we have a means to find $y(x)$, but it is implicit.
Using `find_zero`, find the value $x$ which maximizes `y` by finding a zero of `y'`. Use this to find the point $(x,y)$ with largest $y$ value.
```julia; hold=true; echo=false
xstar = find_zero(yp, 0.5)
xstar = find_zero(findy', 0.5)
ystar = findy(xstar)
choices = ["``(-0.57735, 1.15470)``",
"``(0,0)``",
"``(0, -0.57735)``",
"``(0.57735, 0.57735)``"]
ans = 1
radioq(choices, ans)
answ = 1
radioq(choices, answ)
```
(Using automatic derivatives works for values identified with `find_zero` *as long as* the initial point has its type the same as that of `x`.)