This commit is contained in:
Steven G. Johnson 2023-04-03 08:56:01 -04:00 committed by GitHub
parent a96fdc7659
commit 3bb29c99a1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -116,7 +116,7 @@
"\n",
"In class, we considered steepest descent for $f(x) = \\kappa x_1^2 + x_2^2$ in $\\mathbb{R}^2$, and argued that for an arbitrary $x = [x_1,x_2]$ starting point and $\\kappa \\gg 1$, the steepest-descent step $x \\leftarrow x - sz$ is *approximately* $sz \\approx [x_1, x_2/\\kappa]$.\n",
"\n",
"**(a)** If $sz = [x_1, x_2/\\kappa]$ exactly, then on the next step we would have the new $x \\leftarrow x - sx = [0, (1-\\frac{1}{\\kappa} x_2)]$. However, explain why a more careful calculation shows that the new $x - sx \\approx [O(1/\\kappa), (1-\\frac{1}{\\kappa} x_2)]$, i.e. the first component is proportional to $1/\\kappa$ to leading order in $1/\\kappa$.\n",
"**(a)** If $sz = [x_1, x_2/\\kappa]$ exactly, then on the next step we would have the new $x \\leftarrow x - sz = [0, (1-\\frac{1}{\\kappa} x_2)]$. However, explain why a more careful calculation shows that the new $x - sx \\approx [O(1/\\kappa), (1-\\frac{1}{\\kappa} x_2)]$, i.e. the first component is proportional to $1/\\kappa$ to leading order in $1/\\kappa$.\n",
"\n",
"**(b)** If you start with an $x = [\\#/\\kappa, x_2]$, i.e. where the first component is proportional to $1/\\kappa$ and $\\#$ is some number of the same order of magnitude as $x_2$, show that after one steepest-descent step (for $\\kappa \\gg 1$) the $x_1$ component is *still* roughly order $1/\\kappa$ but of the opposite sign, and $x_2$ again subtracts a term roughly proportional to $x_2/\\kappa$.\n",
"\n",