This commit is contained in:
Steven G. Johnson 2023-04-05 14:23:11 -04:00 committed by GitHub
parent 3de81c7aeb
commit 7c6c9def4f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -116,7 +116,7 @@
"\n",
"In class, we considered steepest descent for $f(x) = \\kappa x_1^2 + x_2^2$ in $\\mathbb{R}^2$, and argued that for an arbitrary $x = [x_1,x_2]$ starting point and $\\kappa \\gg 1$, the steepest-descent step $x \\leftarrow x - sz$ is *approximately* $sz \\approx [x_1, x_2/\\kappa]$.\n",
"\n",
"**(a)** If $sz = [x_1, x_2/\\kappa]$ exactly, then on the next step we would have the new $x \\leftarrow x - sz = [0, (1-\\frac{1}{\\kappa}) x_2)]$. However, explain why a more careful calculation shows that the new $x - sz \\approx [O(1/\\kappa^2), x_2-\\frac{1}{\\kappa} x_2 + O(1/\\kappa^3)]$, i.e. the first component is proportional to $1/\\kappa$ to leading order in $1/\\kappa$.\n",
"**(a)** If $sz = [x_1, x_2/\\kappa]$ exactly, then on the next step we would have the new $x \\leftarrow x - sz = [0, (1-\\frac{1}{\\kappa}) x_2)]$. However, explain why a more careful calculation shows that the new $x - sz \\approx [O(1/\\kappa^2), x_2-\\frac{1}{\\kappa} x_2 + O(1/\\kappa^3)]$, i.e. the first component is proportional to $1/\\kappa^2$ to leading order in $1/\\kappa$.\n",
"\n",
"**(b)** If you start with an $x = [\\#/\\kappa^2, x_2]$, i.e. where the first component is proportional to $1/\\kappa^2$ and $\\#$ is some number of the same order of magnitude as $x_2$, show that after one steepest-descent step (for $\\kappa \\gg 1$) the $x_1$ component is *still* roughly order $1/\\kappa^2$ but of the opposite sign, and $x_2$ again subtracts a term roughly proportional to $x_2/\\kappa$.\n",
"\n",